SfM lat/lon not in API & Lat/lon question

#1

SfM lat/lon not in API
With the API and the Mapillary JS a Mapillary.Viewer node has two lat/lon pairs

  1. the “Structure from Motion” lat/lon (default!)
  2. the original lat/lon

I red up on the SfM system (cool!) BUT

The information I get via json calls like:
$.getJSON("https://a.mapillary.com/v3/images?client_id="+client_ID+"&"+“closeto=”+lon+","+lat+"&radius=100",.

The lat/lon information are all “original” lat/lon values, NOT the SfM lat/lon values!

Wouldn’t it be much, much more logical to use the SfM values here also?
(or am I doing something wrong?)
example:
http://geoarchief.nl/mapillery-test.html
(shows original location on map: light blue and SfM location: dark blue)

http://geoarchief.nl/mapillary.html
(the location information from the API are the dots in view, the link looks at the dot that should be below)…

Now that I am on a roll with this post. SfM repositioning the lat/lon: cool!!
But if you guys are that smart to calculate the right lat/lon, why not automaticaly correct the horizon and compass angle along with it? The last link in this post clearly shows that the compass angle is a bit off (and that my camera isn’t that good… I know… but I am in a test phase here, planing on buying a good one, but first I want the basics to go right :slight_smile: )

Lat/lon question
When the “Mapillary system” is that smart with re-positioning… is it necessary to spend (a lot of) time trying to have the EXIF data of the images set as good as possible?

Gopro Fusion Workflow
#2

Or… if this is “by design” (would like to know why :slight_smile: )

Would it be possible to expand the API adding the SfM lat/lon?

#3

Hello!

This is by design, because the response is a GeoJSON we want to maintain the original, authentic longitude and latitude. This gives the data integrity, because the computed ones, while useful for SfM aren’t real data about the original, user-uploaded images. We don’t want to provide modified data from us, but rather the exact data about the image the user owns.

However, in MapillaryJS you can get node.latLon but also node.originalLatLon which is the same typically, and node.computedLatLon. This also work with camera angle: node.ca, node.originalCa, node.computedCa. Feel free to use any of these in your custom apps. We don’t put this into the v3/images API, however.

Really, when saying “computed” we are not adjusting the images to a “true” position, but instead a “predicted” position. This can sometimes appear quite accurate, and sometimes not at all. Overall when it comes to images we try to maintain the integrity of the data the user provides as far as positional display on the map, while the SfM adjusted values are a separate arena–these are used to figure out which images you can navigate into from the current one, as well as to attempt to geolocate extracted map features in relation to the context around them.

The better the input EXIF data, the better the navigational/viewing experience in MapillaryJS, and the better the output data. Only the user can be relied upon to provide true input, with a GPS device and site survey for real context, so we are trusting the human first, and machine only after the fact, rather than machine first. We always need that original EXIF data in order to make a next step. Reliability of both humans and the machines varies, but the human who uploads is the original source, and what we always display by default.

3 Likes
#4

Thank you for the answer!

The positioning of the image on the canvas is based upon SfM latlon. I think I understand why; the transitions between the images most likely is better.

Now I position the dots in the mapillary.html from the API, and so they are the position where the user ‘says’ it is. These dots help me a lot navigating around the 360-images (thinking of placing some smaller dot’s in between to get a sort of line, a function to draw a line would be even better though :wink: ).
But when moved there the image is positioned according to the SfM not the original latlon.

I couldn’t think of a way to “bulk receive” SfM lat/lon locations, but I’m quite convinced the SfM data will improve the overall experience… (hence my feature request)

Question: maybe the “reverse” works also, is it possible to position the center of the image on the original latlon?

I’m glad I caught your attention, could you write down your thoughts about:
I have a plan to create a “version 2” for the web-uploader (or at least a good assistant) But that will (eventually…) result in “edited / smoothed” GPX/EXIF info.
Reading your post, I could conclude that Mapillary would rather have “raw location data”… but if “the dots” are relatively poorly positioned I’dd be better off with my future smoothed out version.

#5

Hi @eesger, It’s not possible to bulk receive those yet. I’ll check on it, but I know that it would mean expanding our API a lot. I also don’t see a way to center the image on the original location–essentially the only thing I can think of is to get the node.originalLatLon, node.computedLatLon, calculate the offset, then apply that to all visible points that are rendered (offset them the same ration). This isn’t a consistent offset though, but this could help.

Ideally for upload we’d allow you to do some smoothing of your own prior to upload, in a way that still retains the raw coordinates but also has your updated ones. This is what happens when you spatially edit an image on the web platform, after upload. Overall the point is that it’s good to have a papertrail of where the image was said to be, where it moved to manually, and where it was computed.

I’ve definitely used Geosetter to improve coordinates before upload, however. In a dense area of buildings and narrow pedestrian streets, such as Stone Town in Zanzibar, the GPS was very poor, but it was possible to use a system of notes on paper, memory of intersections, and one-by-one movements to put the imagery on track. https://www.mapillary.com/map/im/amiR07UXp2Va9s7-k6H9_A

I have had an idea of providing a linestring GeoJSON as the correct path, distributing the images evenly along that line, then writing the new coordinates to the EXIF, then taking to Mapillary to upload. This would be very helpful in areas where I know the sequence started and stopped at two very specific locations, and wanted to spatially distribute them evenly on a route between those. If this is what you have in mind, I think it’s a great idea. This should result probably in a closer match between original and computed coordinates.

1 Like
#6
  • Use the offset of one as a template for the rest.
    Nice idea. But no. Even in my relatively small set of images the difference between the SfM location and the original is quite erratic. And more importantly the SfM location realy does appear to be better, so that feels like doing a step backwards…

  • Raw vs computed (SfM)
    I find it quite impressive that Mapillary can calculate a better geo location based solely on the image (starting with “the rough” lat/lon given). With the small set of images I took (a few thousand) I could only find some that were less well positioned (I have this feeling that altering the forest path in OSM would improve it :wink: ). My point in essence is: when you’re that good at better calculating the location, use it! Not a little, go all the way: Reposition all the dots in Mapillary towards the calculated positions! Leave the raw data within the images (as it is now) but simply use the computed lat/lon everywhere!
    I don’t know if it is possible do a local test at the office, but the results might just be very pleasing?

  • Example:
    Rats, your LG360 does a better job at stitching the two images together then mine :open_mouth:
    Odd, clicking the arrows just wants to skip this corner? (go back and forth, you’ll see what I mean)

  • I’ve had an idea:
    You and me both, that is exactly what I mean. But in my “prefect workflow plan” this would mean that the original GPS data would be completely overwritten with “OSM smoothed data”. Resulting in the loss of the original GPS data (which would only be used as a bases). But when SfM is there anyway it could also mean that I’m wanting to build something that isn’t really needed because SfM can fix that (and the extra iteration would be a waste of time) … but because I can’t use that data the way I’dd like to… and the circle is complete.

Maybe I’m being to much of a perfectionist here, but I’m trying to get to the “the perfect workflow”. Perfect for my idea, but also fitting inside the Mapillary system as perfect as possible. When I know what would be ideal, I can understand where to do a trade-off and get practical.

So I think the question is: how important is “the paper trail” to Mapillary?'
And/or: Any chance of Mapillary “switching” towards SfM lat/lon altogether?

Maybe I’m just making a fuss over something that’s not that important and the edited/appended/styled GPX data is more valuable to Mapillary?

#7

Yes, being good at better calculations just highly varies based on the region. We definitely do not use the OSM path as a reference–this would dramatically change the licensing of Mapillary based on OSM’s ODBl license, so that has no effect. If you’re using OSM as the user before upload to adjust/“smooth” the data, that also could be questionable (I’ll have to ask). Really, in the case of GPS measurements, when you know the GPS is wrong (like very wrong), and either manually readjusting or even using a basemap as reference, you may be more correct, but it still doesn’t make the GPS coordinates actually true or precise. So in either case you have wrong data, one just looks closer to truth. What is most important is the position of the images to one another, based on the point cloud we reconstruct from the image pixels–this can be very accurate for relative position of the photos in your sequence to one another, and to other photos in the sequence, but the key is we have to scale this method globally. Nothing on our side can assume that just because the computed coordinates look better in one town or city park means we can assume it would be better worldwide. So we stick with original coordinates when it comes to the map, same as any geospatial interface like Strava, OSMAnd, etc that is displaying user created data.

Your final question is a good one: if we have a change of switching altogether to SfM lat/lon. I will ask! But I think the problem is that globally it just switches one set of problematic data for another set. Probably the idea is that if the algorithm becomes intelligent enough to truly correct everything with a massive success rate, then it could make sense.

Otherwise, I think the missing piece for you is access to the SfM coordinates via the API, which I’ll also ask about!

1 Like
#8

I don’t expect to find absolute truth in my lifetime, so I don’t mind settling for “closer to”, that’ll be a step in the right direction!

Anxiously awaiting the answers to the “I’ll ask” answers!

Best Regards,

Eesger

#9

@eesger So I think at this time we don’t see a reason or plan to switch to computed SfM locations as the default map display, because we don’t expect it to be verifiably better at all times, globally. GPS is ground truth so will stay as primary display.

For the API, we are certainly considering adding SfM coordinates as an option, but it has some complexity to it as we need to constantly recalculate these if a user is modifying image locations, whereas right now because it’s not exposed we don’t remodify it much. In short, it would be tied to a larger change in our data pipeline that we won’t do in a whim, but that probably makes sense in the long run. Can’t say when, but we’ll watch for this to become more necessary.

2 Likes