Why is Mapillary breaking everything?

#16

@pkoby I have made progress with the scripts to recalculate sequences, but they are not perfect yet. Don’t be alarmed over the area being reprocessed, I’m tuning things …

#17

Okay, good to hear! It was a bit of a shock to see the map in its current state, but I trust it will return.

#18

Yes,
I was testing on a very small sample and realized that some of your sequences not giving the right results. Doing a new run right now, will check the results tomorrow (should look a whole lot better).

#19

So, things are getting better, but there is a problem in making the angle offsets into static buckets that is showing here: https://www.mapillary.com/app/?lat=38.39998688307&lng=-82.45358953513039&z=18.417242240060492&pKey=EVJ9I-OTIqVA9wM6PeK8hA&focus=map - some images get out of one bucket (offset -45-45 degrees) and into another one, so the sequence gets a false split. That is the main reason for the crossing lines you can see now. This is especially prominent with bike sequences as the offset tends to vary a bit.

Will think about how to avoid that tomorrow.

#20

Yes, I was thinking about this problem and how you might get around it.

Would it be possible to calculate some expected angle buckets? In the editor, you can normalize a sequence, which aligns all images with the direction of the associated line. If you created a bucket for images that match that forward angle, and a bucket for everything else, that might fix the issue.

I guess I’m not sure if you want to create scripts that would solve this issue for all cases, or if you’re looking to just run a refresh on my region. If the latter, I could give you more information on my techniques, and you could tailor the scripts to match that? I’m the only contributor in this area except for one sequence by JB.

For each day of the problem sequences (which would have occurred between late July and early September of 2018), I ran two cameras concurrently, one facing forward (0/360 degrees) and one to the left (at 300 degrees, I think). I interpolated location for each camera using the same GPX, so that’s why they coincide. I tended to add a slight offset of a fraction of a second so the images weren’t on top of each other, but this wasn’t always the case. Angles were calculated by averaging the look-behind angle (to the previous image), and the look-ahead angle (to the next one) for each image, except for the first and last images (first just looks ahead, last repeats second-to-last angle).

So to fix my area, you might be able to look for the images that occur within two seconds of each other (my cameras had a two-second delay), with one sequence matching the expected angles, and one with an offset. So if you start with an image of angle 0, the next one should be to the north. That image might have an angle of 45, which would put the next image 90 degrees away (to the east). And so on.

EDIT TO ADD:
I looked at some single sequences too, not concurrent with other angles. These suffer some of the same issue, though it’s not as clear why. Perhaps limiting the next image in a sequence to a distance threshold might help? Time threshold would be possible too, but there would be some pauses at lights and whatnot that would throw that off. Though a split to a new sequence in these cases wouldn’t be the worst. Again, for my area, the real issues occurred really only in August (during the q3 challenge).

Hope this information helps.

Last ditch option, you could also possible batch delete all the offending sequences from August 2018, but that would kind of be a pain to reupload. I’d get to it eventually, though.

#21

@pkoby, thanks for the details! We want to solve this problem for the generic usecase, so I will try to cluster the offsets in a time-calculated sequence in order to find the right offset-buckets (which are in this case not (-45 to +45 degrees, pointing ahead) and (-125 to -45 degrees, pointing to the left) but rather ahead and a bit more to the right. If I can find these values (let’s say assuming 4 buckets in total), we should catch most of the outliers that are creating the spiderwebs right now.

I have some other things to do today but will get back to this ASAP - this is a very good training ground for getting things right, and we want to solve this in a good way, for obvious reasons.

Thanks for the help @pkoby - I’m actually thrilled to finally get to this.

/peter

#22

So,
today I hacked this little debugging html to see the recalculated sequences (different colors per sequence so you can see how they stretch and where they cross) before putting them into the backend system. This is the result after todays work @pkoby - look MUCH better. Now I’m applying this to the system, let’s see what you think!

This is a screen of 110.000 of your images in that area :slight_smile:

#23

So,
now the area is re-rendered and re-calculated using the latest version of our repair-algos. @pkoby please let me know if this is better than before - I even see you have some streets left to cover here and there, for instance https://www.mapillary.com/app/?lat=38.41067134792331&lng=-82.45107697242031&z=15.039342308071733&pKey=TtVRdC-UJDz5bsQhr8N2Nw :wink:

The algo is not working perfectly on tight turns and varying offsets, but I think this is a good start for this iteration. we will work more on this, so expect some minor changes is some of the still falsely cut sequences, like https://www.mapillary.com/app/?lat=38.407977841798264&lng=-82.42726505175577&z=17&pKey=S1f8MVbLB53TJpCC2PwUXg - we need to think a bit more about how to do the dynamic-range offset-clustering in order to catch these variations.

I will try to get to it, but for now, let me know if this is good as a start and proof-of-concept that we are able to repair the spiderwebs form old inconsistencies regarding sequences?

/peter

#24

Well, it’s got its goods and bads.

Good:
Looks like the algorithm is handling the issues better, and catching the correct sequence more often. I haven’t seen any sequences that bounce between directions.

Bad (and unfortunately some very bad):

Here’s August 7th: https://www.mapillary.com/app/?focus=map&lat=38.40516014299351&lng=-82.45833068622551&z=15.125202321696126&pKey=onM7zUyx_z06YnL1aoH9zg&dateFrom=2018-08-07&dateTo=2018-08-08. Strangely, it seems to only have kept images with headings between 270 and 360 degrees (from both sequences).

Here’s August 9th: https://www.mapillary.com/app/?focus=map&lat=38.415416031019575&lng=-82.42215600239544&z=17&pKey=7HuQ_QhrScqZwCbzyX4rWw&dateFrom=2018-08-09&dateTo=2018-08-10. Same problem, but more directions (though mostly 90-180).

There are other dates, too. To compare, here’s August 9th complete: https://openstreetcam.org/details/1328399/953/track-info

So what works seems to work okay, not perfect, but something bad happened to some dates.
EDIT: The missing images don’t load with a cache refresh or in JOSM, either.

#25

Thanks for the feedback, will check this out tomorrow!

#26

@pkoby now I rerendered the whole area with dynamic clustering regarding the camera offset variations within a time-cut sequence, separating the 2 cameras quite nicely, please let me know what you think. Also, the 7/8th of Aug 2018 etc are back, see screenshot.

Is this something that is a good baseline?

#27

This definitely is an improvement. I’m still seeing some issues around tight corners, but again, it sounds like that’s tough for a computer to guess. There are also some weird short bits in sequences that don’t really make sense to me (e.g. here), but I don’t see any conjoined directions nor photos where there aren’t any, and those were the two big issues for me.

Looks good for now.

#28

@pkoby I improved the algorithm for this a bit more, I think things are almost perfect now. Let me now if you have any more problematic sequences in this area, I have been training on these tracks: https://www.mapillary.com/app/?focus=map&lat=38.417637810161665&lng=-82.39400830215322&z=16.11491545906358&pKey=J79odL2KVRiGcGcqbWUszQ

/peter

#29

Wow, this looks excellent! I wasn’t aware you were still working on it.

This bit was really bad, and it’s very clearly improved. Thanks!

#30

Thanks for the Feedback! I’m now trying to sort out the recalculation of tracks with more than 2 cameras with their own tracks which involves averaging the (e.g. 4) tracks and then calculating the offset against that track in order to reduce noise. When that is working, we are ready to reprocess the older imagery without shredding things.

It requires some more work but I think this is a first good step, thanks for the patience and testing so far!

/peter

#31

Yeah, no problem! I went through all of August checking my tracks, and I notice that the sequences often split at traffic lights. I assume that’s because I delete photos during processing, so there’s a jump of a number of seconds. Other than that, I find no issues.

However, it seems like August 30th snuck through unprocessed, because it’s the only day that’s still switching between the straight and left directions, and has some spiderwebbing (https://www.mapillary.com/app/?lat=38.433603895827446&lng=-82.40228698588089&z=19.069615314500293&mapStyle=mapbox_streets&pKey=TfdHsF2QcTfjmY3PYBRo3w&dateFrom=2018-08-30&dateTo=2018-08-31). Otherwise, all of August looks great!

#32

I would add a question here

Why is Mapillary support commenting only in this thread completely ignoring questions in all others?

#33

Not all questions and remarks are directed to the Mapillary team. Sometimes messages are just a chat between users.

But I agree it would be nice to see a category “Questions for developers” or “General support” when users really expect an answer from the team, and not only from other users.

#34

Peter is one of the founders.

#35

Good input, will try to push for that. If we do that I believe we need to be more stringent in not changing the topic too much, otherwise categories might fall short to describe the discussion adequately.