Video Support Arrives to Mapillary

Hi folks,

We have a new blog post out today. Please take a look: Video Support Arrives to Mapillary.

We are happy to announce that the Desktop Uploader and mapillary_tools now have native support for uploading video files in GoPro and CAMM formats enabling denser capture and easier workflows.

Video capture allows for a much higher framerate than timelapse images and is now the recommended method of capture when you’re using moving vehicles like a car or bicycle. Video also creates more overlap between frames which enables Structure from Motion to generate better 3D reconstructions of the environment which is important to accurately estimate the location of objects. Mapillary can now process video files natively and sample them automatically for both regular and time lapse videos (for example to show one frame every 3 meters).

We would love to hear your thoughts!

  • Boris

Product Manager, Mapillary
With :heart: from Zurich, Switzerland

10 Likes

Ohh, processing video instead of pictures for SfM is a progress.
Also it gets around the loss of information while deoding images from video.,

Great

1 Like

For me is the bigger update that I’m finally able to update from the microsd card.

I will try to use video capturing within the city but outside it would drain my GoPro battery to fast.

1 Like

Super, give it a try @Magnet. Depending on which GoPro you have the battery life impact for video may not be significant. According to our friends at @trekview (https://guides.trekview.org/trek-pack/v2/camera-modes)

… the GoPro Fusion was particularly power hungry in video mode. It’s not so much of a problem for the MAX. Power consumption for a video recording is slightly greater than compared to timelapse mode due to the extra processing power required by the camera.

A GoPro MAX battery will last for around 2 hours for continuous recording in good conditions (not too cold), versus about 2.5 hours in timelapse mode.

1 Like

I gave it a try with my GoPro 11. Sadly the uploader won’t upload the first 8 minutes of a 20 min video. I extracted the gpx file with exiftools, and it seems totally fine. Do you have any idea what could be the problem?

Strange - are you receiving an error of some kind? (what exactly do you mean by “won’t upload the first 8 mins”?). Could you share a link to the file for us to take a look? (If you don’t have access to a file sharing service, PM your email and I’ll send you a onedrive folder you can upload to).

cc: @nikola @tao

1 Like

I think I might have been wrong.
I’m uploading my video right now and will give an update when it’s finished.

My statement of it losing the first 8 minutes is just based on the uploader preview.


2 Likes

The uploaded sequence will look same as the preview in DU.

It looks like the first 8 minutes were indeed lost, and it’s likely to be GPS anomaly at around 8th minute. Could you extract metadata with ExifTool by running

exiftool -b -ee3 -X GOPRO11.MP4

and share the output with us?

Sure GX019036.txt - Google Drive

Thanks @Magnet this will be super helpful. Will update here this week.

Good news, given how many people seem to be using video capture as the primary method.

Not for me though, I prefer storage management with photos, as well as no image fidelity and device/battery performance loss to video decoding.

Absolutely - both image and video capture have their place. Video is particularly good when you are moving too fast for the camera to take frequent photos (e.g. in a car)

CAMM support looks really nice. Though, after a quick look at the spec I cannot but point out a baffling slight design blunder made by Google Engineers. When you look at a type 6 packet you can see that the gps_fix_type field causes a memory alignment issue on all following fields.

case 6:
    double time_gps_epoch;
    int gps_fix_type;
    double latitude;  /* first misaligned field
                         (not on a 8 byte/octet boundry) */
    double longitude;
    float altitude;
    float horizontal_accuracy;
    float vertical_accuracy;
    float velocity_east;
    float velocity_north;
    float velocity_up;
    float speed_accuracy;
  break;

While this design surely works, it causes hiccups in modern 32 and 64 bit CPU pipelines, and on 32 bit machines probably even penalty cycles. 32 bit ARM machines are most certainly going to be affected by this. Usually, 64 bit machines can handle misaligned data types better or penalty free. It may not seem like much but on battery powered devices that caputure thousands of frames and create thousands of CAMM packets this quickly adds up. So, this is one reason why you usually want to be extra careful when designing streaming data structures. @boris Since I am not a Google customer/user, I would be glad should you want to share this info with Google Engineers.

1 Like

Interesting, thanks for sharing that @GITNE!

Google has a public tracker for the Street View API and I think if you were to report this there it would redirected to the right engineers. I think it’s best coming from you as the one who discovered the issue as well as if they have follow-up questions (though feel free to reference Mapillary).

You can file the issue at https://issuetracker.google.com/issues/new?component=316970&template=1019235

1 Like

You can file the issue at https://issuetracker.google.com/issues/new?component=316970&template=1019235

Yeah, I was not even sure where would I report something like this, so thank you for digging up this link. However, you have to have a Google account to post anything on the issue tracker, thus this is as far as I can go.

1 Like

Next pls support AZDOME and 70mai cams :slight_smile:

1 Like

Okay 3 o’clock in the morning I give up after a very very long search. Does anyone have a product link to CAMM dashcams? I have tried everything: google amazon AliExpress forums ask friends ask Twitter but now im running out of idears. It seems to be untraceable. As if the brand had never existed… :confused:

Hi @hiwi234 - sorry for the confusion, CAMM is a metadata standard for video (kind of like exif is for photos). It’s supported largely by higher-end 360 cameras like Ricoh Theta X or Insta360 Pro2. No dashcams support it yet as far as I know, however there are more manual workflows currently for various dashcams to extract images from video. We’ll look to make this process simpler in the future as well for dashcams.

1 Like

Just got my new gopro was looking here to remind myself of what format to use for uploads. IIRC in the past there was a script or three to run, correct? Did that work on MP4s? Just curious.

Anyway, I’m literally going to head out for a lunch walk and get some mapping to try uploading tonight. Cheers.

@allen Personally I use 1/2 second images rather than video as I run two cameras (zero plus 90 degrees) as per Mapillary and GoPro devices - Google Docs

I suspect a similar workflow for video if using the desktop uploader. For the command line tools try here GitHub - mapillary/mapillary_tools: Command line tools for processing and uploading Mapillary imagery