How to properly exclude images when stopped at a red light using mapillary_tools

Contributing and equipment Command line tools
1

Hi,

I have a GoPro MAX and have been playing with extracting images from videos. Mostly this works, but when I stop at a red light a lot of useless images are being accumulated for nearly the same spot. I’m confused about the two options I believe would solve my problem:

  --cutoff_distance CUTOFF_DISTANCE
                        maximum GPS distance in meters within a sequence

and

  --duplicate_distance DUPLICATE_DISTANCE
                        max distance for two images to be considered

I would think that one of these would be the filter I need to ignore images that are close together, yet, no matter what option or distance value I put, I still get the same results.

Here is the command I’m using:

mapillary_tools --verbose video_process video2process/ sample_images/ --geotag_source "gopro_videos" --interpolate_directions --video_sample_interval 2 --duplicate_distance 5 --interpolation_use_gpx_start_time --overwrite_all_EXIF_tags

The run produced this output:

022-04-24 08:40:58,118 - DEBUG   - Parsing GoPro GPMF /tmp/tmp1nnvonsf
2022-04-24 08:40:58,230 - DEBUG   - Initial time offset for interpolation: 0.0
2022-04-24 08:40:58,230 - DEBUG   - GPX start time delta: 4:01:23.856000
2022-04-24 08:40:58,231 - DEBUG   - Final time offset for interpolation: 14483.856
2022-04-24 08:40:59,095 - INFO    -       41 images read in total
2022-04-24 08:40:59,095 - INFO    -       38 images processed and ready to be uploaded
2022-04-24 08:40:59,096 - WARNING -        3 images skipped due to MapillaryDuplicationError
2022-04-24 08:40:59,096 - INFO    - Check the image description file for details: sample_images/mapillary_image_description.json

Here are the images dragged into EasyGPS so you can see the relative positions of each image.

Snip - EasyGPS - Untitled2
Snip - EasyGPS - Untitled2740×706 27 KB

Which command should I be using to “ignore” images closer than X and how does that work exactly when you have a string of images? Is it drawing a circle around each image and ignoring any other image within the radius? If so, why is it not working as I expect?

Any help would be appreciated.

Thanks,
Rob

2

Also consider duplicate_angle (def 5degrees). Looking at the code it seems that means (for your example) <5metres AND <5degrees between the current and subsequent for the subsequent to be made a duplicate.

The duplicate test is only done from one image to the next. no circles.

GPS position output wanders about when sitting still, so I would suggest your close pattern is also due to a large bearing angle jump. Perhaps zoom up on the track to see how much wandering is going on. Then try a larger minimum angle?

Note I am theorising. I haven’t used this function of the tools for a few years

Cheers

Code snippet
def find_duplicates(
sequence: GPXSequence,
duplicate_distance: float,
duplicate_angle: float,
) → T.List[int]:
if not sequence:
return 

duplicates = []
sequence_iter = iter(sequence)
prev = next(sequence_iter)
for idx, cur in enumerate(sequence_iter):
    distance = gps_distance(
        (prev.lat, prev.lon),
        (cur.lat, cur.lon),
    )
    distance_duplicated = distance <= duplicate_distance

    if prev.angle is not None and cur.angle is not None:
        bearing_delta = diff_bearing(prev.angle, cur.angle)
        angle_duplicated = bearing_delta <= duplicate_angle
    else:
        angle_duplicated = False

    if distance_duplicated and angle_duplicated:
        duplicates.append(idx + 1)
        continue

    prev = cur

return duplicates
2 Likes
3

@bob3bob3 - That was it! I was too lazy to look at the code, but I changed line 98 to:

        if distance_duplicated: # and angle_duplicated:

and boom, I got:

2022-04-24 17:23:59,406 - INFO    -       41 images read in total
2022-04-24 17:23:59,406 - INFO    -       23 images processed and ready to be uploaded
2022-04-24 17:23:59,407 - WARNING -       18 images skipped due to MapillaryDuplicationError

And my clustered images at the red light were gone:

image
image743×700 15 KB

Time to log a bug I think… :slight_smile: Thanks for pointing me in the right direction!

Rob

4
5

Good find!

I do however vaguely remember the discussion where the angle test was added, but not why! Perhaps it was to allow for a left or right pan of a cyclist head camera as they slowed?

Plus, the default duplicate distance is 0.1metres and the sample rate 5/second. Making things too large and/or too slow seems to impair the 3D reconstruction process.

I think in the end it isn’t an bug, but that your GPS returns noisy data. Setting the angle to 359 might be your valid setting. The processing error though is a worry. That might be a mathematical divide by zero etc.

Cheers

6

Thanks to you both for raising and solving this, I ran into the same problem and also using duplicate_angle 359 helped.
I also commented over on Github, hoping this gets adjusted soon.
All the best,
Paul