API mesh and point cloud format?

Imagery, data, and integrations
1

For image API requests, there are fields for point cloud and mesh. Although they both provide files, it is not clear what format the files are in. The point cloud claims to be in .ply format, but no programs are able to open it. No information is provided about the mesh format.

What can I do to use these files? Thanks.

1 Like
2

Hi, I have the same question. Can someone shed some light onto what file format the API meshes are?

3

The API documentation changed since this last question, so the current status is:

sfm_cluster - { id: string, url: string } - URL to the point cloud data in JSON and compressed by zlib. See the example below.

So it is a JSON of the point cloud, but compressed. Attached a screenshot here of the example on usage.

Can you test using this and let us know where you find any obstacle?

1000000031
10000000311080Ă—2340 204 KB

4

Hi, I’m a little confused about how to decompress the point cloud data.
What should go in place of <CDN_IDENTIFIER> in the url of the curl command? I tried using the id from the sfm_cluster object but when I run this command on the data downloaded from the corresponding link the file only reads “Bad URL hash”.
I’d appreciate any guidance on where I’m going wrong

5

@silquirm : You get the actual URL in a previous response. You don’t have to do any text replacement.

Example workflow:

$ curl --header "Authorization: OAuth ${MAPILLARY_CLIENT_TOKEN}" https://graph.mapillary.com/712710037679811?fields=sfm_cluster,mesh
{
  "sfm_cluster": {
    "id": "350956967860005",
    "url": "https://scontent.fmci2-1.fna.fbcdn.net/REDACTED"
  },
  "mesh": {
    "id": "2454120511643313",
    "url": "https://scontent.fmci2-1.fna.fbcdn.net/REDACTED"
  },
  "id": "712710037679811"
}

Now, if you just want to download the point cloud, you could do something like this:

$ curl --header "Authorization: OAuth ${MAPILLARY_CLIENT_TOKEN}" https://graph.mapillary.com/${pKey}?fields=sfm_cluster \
    | jq -r '.sfm_cluster.url' \ # Get the URL from the json response
    | xargs curl \ # Get the data from the url
    | python3 -c 'import sys, zlib; print(zlib.decompress(sys.stdin.buffer.read()).decode("UTF-8"))' \ # Decompress the data
    | jq > pointcloud.ply # Write to file in a "pretty" format

Notably, you only need to know the picture id (${pKey} in the above example) and have a Mapillary client token.

As far as the mesh goes, I have no information on what format that is in (AKA, I don’t know how to read the data).

For the ply files, something I found out when I was trying to see if I could (partially) replicate the NeRF ground layer in JOSM is that the return from the Mapillary API has an additional field reference_lla which you can use to geolocate the origin of the point cloud. I have yet to figure out what units the x/y/z axis are in though.
Also, the API return may have both equirectangular and perspective as camera types, which doesn’t match with the documentation.

1 Like
7

@vorpalblade-kaart
Correction: I’m definitely able to get the sfm_cluster just fine. If I save it as a text file instead I can see all the data’s there and I can work with it in my python script as json. However, any pointcloud viewer tells me that the ply I saved using your command above is of the wrong format.

curl --header "Authorization: OAuth {MY_TOKEN}" https://graph.mapillary.com/{IMG_ID}?fields=sfm_cluster \
  | jq -r '.sfm_cluster.url' \
  | xargs curl \
  | python3 -c 'import sys, zlib; print(zlib.decompress(sys.stdin.buffer.read()).decode("UTF-8"))' \
  | jq > pointcloud.ply

I tried omitting jq from the final command and instead writing the decompressed data straight to a .ply file. Still of the wrong format (this file is ~half the size of the former, presumably that’s because the “pretty” formatting is not included(?) ).

curl --header "Authorization: OAuth {MY_TOKEN}" https://graph.mapillary.com/{IMG_ID}?fields=sfm_cluster \
  | jq -r '.sfm_cluster.url' \
  | xargs curl \
  | python3 -c 'import sys, zlib; print(zlib.decompress(sys.stdin.buffer.read()).decode("UTF-8"))' \
  > pointcloud.ply

Does the command you wrote definitely work for you? i.e. do you get a pointcloud you can actually view?
Otherwise I’m not sure what I’m doing wrong.

8

I have no clue what ply viewer you are using. Maybe it expects a specific extension, or it expects the data to be zlib compressed. Keep in mind that whatever viewer you are using may not support the opensfm ply format.

In any case, I was investigating using the point cloud to generate an aerial image in JOSM, so I was parsing the json in my code.

9

Ah I see. I’m just learning about point clouds and SfM so wasn’t aware there can be different ply formats or how one can convert between them.
I’ve done more reading and expect that the online viewer I was using (Free Online Tool to View 3D PLY Files Online - ImageToStl) for verification takes the ascii format (PLY - Polygon File Format).

Looks like MeshLab lets you convert between binary and ascii ply so I’ll test that.

Thanks for your help, I’m getting there!