How to handle duplicate images

I see that mapillary_tools automatically removes duplicates, but what about the web and desktop uploaders? web version crashed on me a couple times and I had to restart loading the sequences, but couldn’t find exactly where it left off.

2 Likes

The problem is not very clear. I use mapillary desktop uploader & mapillary android app.

AFAIK the desktop uploader actually runs the tools “underneath”, so yes it will not upload an image that is within a specified distance from previous. I set mine to 1m. Looking at the code briefly the default appears to be 0.1m

I would expect the web uploader to be similar but have no experience using.

@bob3bob3, I may be misunderstanding you but that the desktop doesn’t upload if an image is .1m form another is NOT my experience. I’ve never changed any settings for that in the desktop uploader. It’s been more than happy to upload pics in the same spot. Maybe there are more criteria than that?1

@HappyMapper my experience with uploading via the web browser is that Firefox will be fine with several instances open uploading a sequence of 1,000. Chrome used to do that well, too. But awhile back it started crashing on sequences bigger then @750 or so.

I haven’t used Chrome for uploads for awhile. They may have tweaked things again to get around that.

You are correct, the web upload if it crashed will leave a sequence hanging out in the queue. If you go to Mapillary you should see something like this. I kicked off the last couple of uploads from this weekend before I left for work this morning. They’re all done uploading but I still need to publish them.

Do you see something like this for the ones that fail?

@allen to clarify my experience is that the default setting is 0.1 metres but there was also some work done to ensure that a changing view heading (rotating the camera) was not marked as a duplicate. My processing reliably marks waiting at traffic lights or if I have pulled over to (say) clean the windsacreen. I dont have to be concerned that I appear in the picture.

Perhaps the desktop uploader is different or calls the tools with duplicate suppression turned off?

@allen @bob3bob3

  • The web uploader, did NOT remove multiple images taken from the same location, when our car was parked (I forgot to remove some of these instances before uploading). Here’s an example from a web upload where we stopped to look at wildlife with the camera still rolling. The car is clearly not moving.

  • The only advanced setting I see in the desktop uploader is to manually set the offset angle (assuming that’s for side cams, rear cams, etc). My first batch of desktop uploader images are still loading… so I’ll check if the issue persists here.

  • However, it looks like it is possible to set a custom threshold for flagging duplicates using mapillary_tools. In the example they show in that link, it’s set to 0.1 meters.

Clearly it’s best practice to remove duplicates from a sequence… so my question is what’s the easiest way to remove them after upload? I don’t see an option to delete an image when I’m viewing my contributions in my individual user account.

I had a look. I wanted to grab the gps data to see if it jittered by more than the duplicate distance. Link a bit slow though.

Individual images can be removed from the sequence using the web gui. The whole sequence is also easy to remove making local data repair and re-upload an option.

Try clicking on the image you want to remove and press the 3 dots (image options) button

If the images are few, manually delete them in the sequence editor (select any image).
I apologize.

@happymapper,

You should be able to do it via the web form. If there’s a lot, well, there’s a lot. I’m not sure how much of a difference it makes.

Here’s a place where I had a few duplicates the other day. The draw bridge came up while riding down to Ortega. So I hang back and let cars pass me. Would this be the sort of dupes that you have in mind? More in one spot? I’m asking since it may not as matter as much, especially as the tools become more robust.

1 Like

I admit that was rash. Just the other day, my gopro created repetitions of images - I had to manually isolate them

I was under the impression the desktop uploader was a wrapper for mapillary_tools? Would be good to get the duplicate filter in it either way

1 Like

yea it would be great to be able to punch in some numbers like the distance and angle and see the results straight away. in the editor the photos that arent being uploaded could be shown as greyed out but you could still click on those photos to see whether you want to keep or discard them

with the tools you have to guess the settings, then load them into the editor to see what they are like and repeat that until you get it right. it can be very time consuming

I’ve just started using some software to recognise similar images and remove these before I upload them (ImageMagick). The photos from my dashcam don’t have the compass angle and this seems to stop mapillary_tools from removing the duplicates.

Takes a little while to run (maybe 1 second or so per photo) but much faster and easier than me doing it manually.

I tried this method. (maybe because I’m capturing by bike and thus the scenery changes less with each picture than by car) but if I set the sensitivity to the highest setting, it only found like 170 images on 40k images. Even tough it should have been much more. Turning up the sensitivity even slightly resulted in false positives. So this method doesn’t work well (maybe because of the bike capture, not sure)