Snap to Road Mapillary

Currently one of the big sticking points for me with Mapillary is the lack of the captured Mapillary imagery not sticking to roads (you can have several separate lines for one stretch of road, very messy, and very difficult to use.

Can Mapillary implement a snap to road feature to consolidate these tracks, and then a
thumbnail preview up the top with all images at that location
or
date slider to consolidate where there is overlap on the same road with multiple captures (similar to Google)

The following website is an open library which provides snap to road implementation.

Mapillary could retain the gps tracks view in addition.

This would greatly increase usability.

2 Likes

Sometimes a Mapillary track could be the first time a road or path is being mapped. So it could be required to only snap if within certain distance.

Additionally, not all sequences are taken from the road, but on the sidewalk. If these are mapped on openstreetmap, itā€™s not easy to tell if it should snap to the road, or the sidewalk.

I think that some of the image processing, that mapillary does, actually figures out that where photos are taken in a scene, as various viewports. I wish theyā€™d use that data to tidy up areas with numerous sequences. And more sequences in an area will improve this! Which would encourage more coverage.

3 Likes

Hi

A snap to road feature is builtin a product from a competitor.
I did made some pictures on a hike, and this were snatched to road ā€œnearbyā€ 1 km away from the hiking path. Not very smart.
Also as the post above mentionend, not all roads, tracks, ways are mapped already, so a snatch to existitng road will produce more errors than inaccurate GPS.

echelon

2 Likes

Snap to road will make the map look pretier, but you donā€™t know if it will be more correct. With the precision of most GPSes it will probably be more correct most of the time, but you run the risk of making it worse. How can the algorithm know if you are driving in the center most lane or walkin on a sidewalk? Or one of the good examples @Gness mentions. It will have to make guesses and when you guess you are some times wrong.

So what is right? There is no wrong or right here. In some applications a pretty map might be preferable, but in others one that reflects the raw data.

2 Likes

I was thinking something similar, @tryl. The road map may be wrong. Thatā€™s one reason why in Open Street Maps ( OSM ) we can open the Mapillary layer to help edit those maps.

Take an example of this in High Springs, FL the other weekend. The map shows the street Iā€™m on, SE 1st Place, as continuing. You can see that it does not.

Also, I get that my Garmins are are going to all be off. But all 3 of them off that far over all at the same time? Iā€™m wondering if the map data is wrong. I havenā€™t follow up on it.

This would be a good example of snapping to the road hiding a problem.

We do have this feature in a separate, experimental tile layer. What we do is estimate which road on the OSM network a sequence corresponds to, then color that road green. This does not represent the image and sequence geometry, but does help indicate where coverage exists.

You can see it in action on any of the recent Complete the Map interfaces, such as here:

Budapest - https://mapillary.github.io/mapillary_greenhouse/ctm/budapest-2/
Istanbul - https://mapillary.github.io/mapillary_greenhouse/ctm/istanbul/

This isnā€™t available on our website yet, but weā€™ve certainly heard many requests.

I would like to invite more ideas for these two ideas then:

  1. How you would like to see the map de-cluttered, to see which roads are covered, while still being able to hover and see what images are there
  2. How you would like to see the coverage on the map indicating time of capture, so you can see where coverage is newer or older

To further elaborate, I was thinking about this blog entry: More Accurate Map Data: Improving 3D Reconstruction with Semantic Understanding - The Mapillary Blog

As you do the 3d reconstruction, you can infer where the camera was in a scene. On areas where there are lots of photos and - more importantly - a lot of sequences over time, it could correct the reported GPS coordinate of the photo with its calculated location.
turning this:

Into this:

Now if we can have those view ports converted into a visible sequence on the mapā€¦ itā€™d be fantastic and encourage further coverage of areas that already have a sequence.

This could be extended in a few different ways. Some sequences will be taken in the morning, resulting in shadows in a particular way. Those could be grouped. Do the same for differing weather conditions.

Of course, if it was possible to neutralise the weather/time of day variables, it would possibly make the data even more useful.

I hope that made sense!

1 Like

Yes! Makes sense to me. We actually have these values available in the API and MapillaryJS as well, just not used commonly on our main platform. Each image actually has originalLatLon and computedLatLon as well as the same for camera angles. Itā€™s just a question of how these should be displayed, and when they are most useful.

Do you have a recommendation for when exactly this is of use?

I kinda suspected this, as I think when viewing sequences in the app it sometimes has the map pointer not quite on the sequence line.

From my perspective, predominantly when using it to update OSM. It could also really clear up views like this:

Where thereā€™s a gps glitch because of a tunnel or bridge, instead of having a sequence all over the place - particularly as it regains its lock, itā€™ll more effectively ā€˜snapā€™ to the road/way, and in some cases, perhaps even highlight where traced aerial imagery was incorrectly offset.

It could highlight where there is deficient coverage from a particular angle too. If this updates surrounding sequences as more data is added, it really reckon it would encourage further coverage on ā€˜completedā€™ routes.

Iā€™d be hesitant to upload postprocessed gps tracks to osm directly, but Iā€™d be quite interested in using a photogrammetry processed one for routes Iā€™ve surveyed myself.

The photogrammetry enhanced gps traces could really be good for aligning aerial imagery.

This isnā€™t exactly what youā€™re looking for.

The 2 things that would save me the most time right now:

a) Flag sequences that need their direction modified. If I have 2 or more sequences right next to each other AND theyā€™re the same direction, I need to update one of those sequences.

For me, Iā€™ll throw a forward camera on the bike / car alone with another 2 cameras, one pointing left and the other right. Then I have to go back and update the left sequence to point left and the right to point right. Iā€™m horrible at this. I need a tool to help make it easy.

b) Directional coverage lacking. Let me me know what road segments have coverage in what direction.

do you mean for this to be something permanent that happens when images are being processed or something that you would switch on/off when viewing the map? the latter sounds like a better option since you could turn it on when viewing somewhere busy and turn it off for all the other places that dont need it.

as well as just snapping to the road, it would be good to have them grouped on either side of the road depending on which direction they are going. (although you would need to have different rules depending on which country and side of the road you are driving on)

another thing that might help clean up things a bit would be to group together images that were taken at same time, i.e. if someone is using 4 cameras but they are all using separate gps which would mean a lot of messy lines

a way around the problem of sidewalk images being snapped to the road might be a slider to choose how aggressive the snapping would be or how much you want them smoothed out

anyway, personally i havnt had too much of a problem with messy sequences since im mostly using mapillary through the osm editor. i find its a bit easier to see at a glance whats going on since it shows the direction of each image without having to hover over each one like you have to in the main mapillary siteā€¦

so maybe a lot of confusion could be avoided by having an option to show the image direction for all images? it would also be a much quicker solution than having to program all of the above features and calculate which roads a sequence should be snapped to and all that