Video Support Arrives to Mapillary

@allen and @tastrax with this new native video support you no longer need to run any scripts, you can just select the GoPro video file in the desktop uploader and upload it directly.

We recommend using video mode if you are driving, video time-lapse if you are biking, and photo time lapse if you are walking. Take a look a the updated recommendations at https://help.mapillary.com/hc/en-us/articles/360012674619

2 Likes

I have the impression that the video mode adapts better to changing light conditions.

Iā€™m missing a way to do post processing on the video before uploading. I want to be able to delete the first and last minutes of my video. I turn on the camera before I put on my helmet. I also want to delete some images when Iā€™m stuck at an intersection for a minute or stopped to eat something or ā€¦ .
Iā€™m trying to use the mapillary_tools to turn the video from GoPro into pictures to delete the unnecessary images. But Iā€™m stuck on creating the right command.
This command gives me images from the video with the wrong time and very bad quality. but with gps coordinates and this doesnā€™t produce 360Ā° images. but 180Ā°+
mapillary_tools video_process ā€œVIDEO_PATHā€ ā€œIMAGE_PATHā€ --geotag_source ā€œgopro_videosā€ --interpolate_directions --video_sample_interval 0.1 --geotag_source_path ā€œVIDEO_PATHā€ --overwrite_EXIF_gps_tag
Iā€™m using a gopro max with the bicycle settings as specified in the recommendations.

2 Likes

Hi @Koen_VdE - I would recommend using the GoPro Player to trim your video file. It should be pretty easy to do there and you donā€™t need to convert anything to images. You can trim the videos and then drag them into the desktop uploader to upload. Cheers!

2 Likes

And how would you delete the short lunch break (or other stops) in the middle of a video?

You would use the ā€œTrimā€ tool to break it into 2 videos, one before the lunch break, and one after the lunch break.

Also, we now automatically sample the video to one image every 3 meters, so you shouldnā€™t see a lot of images from your lunch break once to upload to Mapillary because your camera was probably stationary during that time. So another option is just to upload the whole thing and then delete selected images using the Mapillary web interface (Mapillary)

2 Likes

I would still like to use the mapillary tools to extract images first and have a little more control on what I upload. I know Iā€™m being a bit difficult but Iā€™m working in a goverment context and I need to be able to explain every step in this process. We will prefer a complete check before uploading not after uploading.

mapillary_toolsā€™s local video sampling does not handle GoPro Maxā€™s special video projection. If you have to, Iā€™d recommend try out the pre-processing in @trekview blog post Using ffmpeg to Process Raw GoPro MAX .360ā€™s into Equirectangular Projections | Trek View

Thanks for sharing! IMHO CAMM is mainly a spec for storing the telemetry data as a MP4 track, it does not mean programs have to load them into memory as its specified.

Also out of curiosity, why the memory misalignment would happen on a 8-byte double time_gps_epoch followed by a 4-byte int gps_fix_type? They look aligned.

1 Like

IMHO CAMM is mainly a spec for storing the telemetry data as a MP4 track, it does not mean programs have to load them into memory as its specified.

I am a bit surprised to read this from you. Sure, readers do not have to read all frames nor packets from a MP4 stream, nor do they have to read sequentially. But, lets be honest and realistic, the vast majority does and usually not because of laziness or something but due to simplicity of implementation. Besides, as a protocol or spec designer you usually do not want to create any pitfalls for potential users of your protocol or spec. :wink: Unless you have a very good reason to create a misaligned data structure in a stream the rule of thumb says that you should avoid misalignments. Anyway, applications that do need to read or write CAMM packets will have to access these fields at some point in time in memory, especially those that write. Sure, you can expect implementers to read or write packets field by field to or from aligned memory but this should be considered bad design since most system implementations are optimized for block reads and writes. Imposing workarounds on spec or protocol implementers is just unfair and counterproductive.

Also out of curiosity, why the memory misalignment would happen on a 8-byte double time_gps_epoch followed by a 4-byte int gps_fix_type? They look aligned.

I am sorry for perhaps creating any confusion. The misalignment happens on latitude.

Another reason to still stick with photo capturing: correction of missing or wrong GPS coordinates (happens every now and then with the GoPro Max). Currently I have a working workflow to do this with photos, adapting this to videos would take some coding effort.

Also I have the impression that the image quality is worse with videos.

1 Like

Thank you.

I will take those into advisement.

For now Iā€™m playing around with things. This is video active. Iā€™ll have to look more. Maybe I got a splash on the lens but I think the blurring feature may be hiding some of the waves ( blurring, that is ). :slight_smile:

It was a rainy day but the clouds had broken up. THe go pro 11 video ( active ) doesnā€™t quite get that sunlight. Or it doesnā€™t come through as well. Again though this isnā€™t set in a mode that Mapillary recomends, just playing to see how it well it works.

1 Like

Was the image processing toolchain also upgraded? I donā€™t remember if this happened before, but now adding GoPro photos to the Uploader shows them facing the direction of travel straight away - is this some auto interpolation, or is extra EXIF data processed?

I have a good example of this.

But it has only been uploaded 7 days ago.

And Iā€™m guessing you didnā€™t interpolate the sequence on upload manually?
Oh well, GoPros donā€™t all have a compass, so maybe this makes sense for most people

Somehow didnā€™t have GPS on for Mondayā€™s hike. Hope to have some to share here comparing some different settings.

Superb news!
I do hope however, that next to accepting current dominating formats, Mapillary will innovate in this area, as it seems to me that fixed frame rate video has quite some overhead. It seems to me that the approach of Mapillaryā€™s Android app is a strong approach still, capturing based on distance covered or curve travelled. Then instead of having separate jpgā€™s, I think HEIF would be the way to go for storage and upload.

I am curious what others think about this.

Weā€™ve been working on improving the video support feature. @Magnet do you happen to still have the GX019036.MP4 file which was giving you this trouble? Any chance you could share a link so we can see if the new method fixes the issue youā€™re describing?