Desktop Uploader 4.0 is here! Looking for feedback

[boris]

@Aerandir14 - that’s great news! @nikola - can we pull this version in for the next desktop uploader release? The fix for videos without a GPX fix also sounds valuable.

[TumbleCow]

It looks like they don’t appear as failed either.

These tracks were uploaded at Feb 26 9:42 and march 2 11:39. (See screenshots)

My last ‘failed’ in the mapillary stream is Feb 25. (Even though it does appear that some of these ‘failed’ are visible on the map)

[boris]

@adalvit - do you know what could be happening here?

[adalvit]

@TumbleCow I see that cluster_id 401016152511972 was created on 2024-03-02 and is still pending - if you want and just for curiosity, you should find that number somewhere among the history files, and I expect it to be related to GS010023 .

As for the sequences from February, there have been some problems with object detection (likely due to file size), but from what I see they should be both in your feed and on the map. Since we don’t store original filenames, to look into it further I’d need you to look among those history files and fish out the cluster id for GS010020.

[seb]

Hello all, I have tested the Desktop uploader as well with some videos recorded by my Tesla model 3 front dash camera.
Short summary it did not work and gave me an upload failed screen.
I am not sure where I should share the complete log file. It is rather long, below is just the tail end.

[2024-03-10 15:06:04.351] [debug] [mapillary:tools] 2024-03-10 15:06:04,350 - DEBUG   - /home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-33-25-front.mp4: 2925 points extracted by parser gpx from file /tmp/mapillary_gpx/2024-03-09_17-33-25-front.gpx}

[2024-03-10 15:06:04.637] [debug] [mapillary:tools] 2024-03-10 15:06:04,636 - DEBUG   - /home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-34-26-front.mp4: 2925 points extracted by parser gpx from file /tmp/mapillary_gpx/2024-03-09_17-34-26-front.gpx}

[2024-03-10 15:06:04.722] [debug] [mapillary:tools] 2024-03-10 15:06:04,721 - DEBUG   - Validating 22 metadatas

[2024-03-10 15:06:07.761] [debug] [mapillary:tools] 2024-03-10 15:06:07,760 - INFO    - Checking upload status for 22 metadatas

[2024-03-10 15:06:08.462] [debug] [mapillary:tools] 2024-03-10 15:06:08,461 - INFO    - Check the description file for details: /tmp/mapillary_image_description.json

[2024-03-10 15:06:08.470] [debug] [mapillary:tools] 2024-03-10 15:06:08,469 - INFO    -       22 video(s) read in total

[2024-03-10 15:06:08.470] [debug] [mapillary:tools] 2024-03-10 15:06:08,469 - INFO    - 	       22 video(s) are ready to be uploaded

[2024-03-10 15:06:08.581] [debug] [mapillary:tools] process exit code 0 and signal null
[2024-03-10 15:06:08.663] [info] [services:mapillary:exif-worker] getting image description
[2024-03-10 15:06:39.878] [info] [services:observer-manager] Registering 98b74350-de93-11ee-b204-31b36a3ab0fd
[2024-03-10 15:06:39.882] [info] [power-saver-blocker] start: 1
[2024-03-10 15:06:39.899] [debug] [mapillary:tools] /tmp/.mount_mapill1Qo6fX/resources/static/mapillary_upload_cli-linux [
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_16-55-03-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_16-56-04-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_16-57-05-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_16-58-05-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_16-59-06-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-00-06-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-01-07-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-02-08-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-03-08-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-04-09-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-05-09-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-24-20-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-25-20-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-26-21-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-27-22-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-28-22-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-29-23-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-30-23-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-31-24-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-32-25-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-33-25-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_17-34-26-front.mp4',
  '/home/test/Downloads/mapillarytesla/leichhard/2024-03-09_16-55-03-front.mp4',
  '--user_name',
  'seb1204',
  '--desc_path',
  '/tmp/mapillary_image_description.json'
]
[2024-03-10 15:06:40.656] [debug] [mapillary:tools] 2024-03-10 15:06:40,655 - DEBUG   - mapillary_tools version 0.11.1
2024-03-10 15:06:40,655 - DEBUG   - CLI param: verbose: True

[2024-03-10 15:06:40.656] [debug] [mapillary:tools] 2024-03-10 15:06:40,655 - DEBUG   - CLI param: import_path: /home/test/Downloads/mapillarytesla/leichhard/2024-03-09_16-55-03-front.mp4, /home/test/Downloads/mapillarytesla/leichhard/2024-03-09_16-56-04-front.mp4, /home/test/Downloads/mapillarytesla/leichhard/2024-03-09_16-57-05-front.mp4, /home/test/Downloads/mapillarytesla/leichhard/2024-03-09_16-58-05-front.mp4, /home/test/Downloads/mapillarytesla/leichhard/2024-03-09_16-59-06-front.mp4 and 18 more
2024-03-10 15:06:40,655 - DEBUG   - CLI param: filetypes: blackvue, camm, gopro, image, video
2024-03-10 15:06:40,655 - DEBUG   - CLI param: desc_path: /tmp/mapillary_image_description.json

[2024-03-10 15:06:40.657] [debug] [mapillary:tools] 2024-03-10 15:06:40,655 - DEBUG   - CLI param: user_name: seb1204
2024-03-10 15:06:40,655 - DEBUG   - CLI param: dry_run: False

[2024-03-10 15:06:46.015] [debug] [mapillary:tools] Traceback (most recent call last):
  File "main.py", line 8, in <module>

[2024-03-10 15:06:46.016] [debug] [mapillary:tools]   File "mapillary_tools/commands/__main__.py", line 168, in main
  File "mapillary_tools/commands/upload.py", line 75, in run
  File "mapillary_tools/upload.py", line 667, in upload
  File "mapillary_tools/geotag/simple_mp4_builder.py", line 347, in transform_mp4

[2024-03-10 15:06:46.016] [debug] [mapillary:tools]   File "mapillary_tools/geotag/camm_builder.py", line 242, in _f
  File "mapillary_tools/geotag/camm_builder.py", line 97, in convert_points_to_raw_samples

[2024-03-10 15:06:46.017] [debug] [mapillary:tools] AssertionError: expected timedelta -34099 between Point(time=33.733, lat=-33.7081178, lon=151.2863442, alt=44.455, angle=None) and Point(time=30.323, lat=-33.7080992, lon=151.2863262, alt=44.9, angle=None) with timescale 10000 to be <= UINT32_MAX
[33508] Failed to execute script 'main' due to unhandled exception!

[2024-03-10 15:06:46.129] [debug] [mapillary:tools] process exit code 1 and signal null
[2024-03-10 15:06:46.135] [error] [store:modules:uploadSession] exit with code 1 and signal null
[2024-03-10 15:06:46.148] [info] [power-saver-blocker] stop: 1
[2024-03-10 15:06:46.148] [info] [services:observer-manager] Stopping 98b74350-de93-11ee-b204-31b36a3ab0fd
[2024-03-10 15:06:46.156] [error] [vue] { code: 1, message: 'exit with code 1 and signal null' }

[boris]

@seb - are you able to share a sample video file to help us debug?

[seb]

@boris, sure can. G D link. Does this work?
GPX file too. let me know once you got the file.

[boris]

Hi @seb, the links work, but the video is 1 minute (mostly stationary), where as the GPX file is a drive for many kilometers. The video and gpx file need to be for the same session (so short video, short gpx file or long video, long gpx file).

Also stationary videos aren’t a great example because most of the data will be removed (since its all shot from basically the same place).

Ideally you could either stitch your videos together using a tool like Lossless cut or edit the GPX using a tool like gpx.studio.

Let us know how it goes after that.

[TumbleCow]

I solved these last issues by re-uploading the vanished sequences in the 4GB chunks that come out of the GoPro, rather than the stitched ones.

Now, I have a different issue:

The mapillary app lists these 360-degree sequences as failed, even though they seem to appear on the map and I can view them like any other sequence.

Some sequence id’s:

• 940157890650183
• 814084563889620
• 657045256505327
• 395431593429349
• 331795202720651

(But there’s a lot more that have failed for unclear reasons)
What can I do with these?

[boris]

Yes, I’m seeing quite a few failures in your feed: Mapillary

@adalvit - any thoughts on this?

[adalvit]

So, the good new is that the sequences in this kind of state have been ingested correctly and no data was lost. The failure was in subsequent steps, like object detection - if you’re not interested in those features, you’re not missing out on anything.

The bad news is that at this time we cannot reprocess them - or, we can, but we’d run into the same failures. We are working hard on improving our infrastructure and processes, and that will solve many problems, but we still need some more months.

Since failures are usually caused by timeouts or memory limitations, my suggestion is to either reduce the max video length on the camera to obtain shorter sequences, or to reduce the resolution.

Best,
Alessandro

[enteq]

This problem doesn’t depend on files, it happens with any of my image files, with Uploader 4.2.0. So I just randomly send you one file it happened.
Opening the same file with the dialog works, and at least some time ago with X11 I never had problems, so I think it is really a problem with wayland.