Mapillary Desktop Uploader 4.6.0 is out now

Contributing and equipment
41

Really? No file size limit?

@osmplus_org try playing with MAPILLARY_TOOLS_MAX_SEQUENCE_LENGTH, MAPILLARY_TOOLS_MAX_SEQUENCE_FILESIZE, and MAPILLARY_TOOLS_MAX_SEQUENCE_PIXELS environment variables.

The whole upload process has been way too messy since the appification. Guys, we are just uploading files! How hard can it be? Please get your act together.
In my experience, only a handful of my sequences went through trouble free. Whenever I am reminded that I have to deal with these clunky upload apps I am worried that I am going to run into some new issue and then have to “take medication” in order to be able to deal with them.

Right, because it resets randomly and perhaps some other stuff, which has been known for years and never really addressed.

Exactly! This is what I have been forced to do sometimes too. Honestly, nobody should need to go through this kind of trouble in order to just upload some files.

1 Like
42

I believe there is a filesize limitation on Meta infrastructure. Sometimes we’re dealing with very large video files at Mapillary, and so this does come up. We’re actively doing work this half to improve upload and processing success rates, so stay tuned.

43

I think there are good reasons for why contributors sometimes want to upload large(r) sequences or group imagery into large(r) sequences. One reason may be for querying the API later. For example, some contributors may want to group imagery per street, per neighborhood, or city district etc into a sequence. There are surely many more reasons.

Interestingly enough, before the appification the Mapillary platform supported appending sequences via multiple uploads. I wish this feature would come back. I am not sure how much this feature had been advertised or used but it existed and solved uploading of large(r) sequences for some contributors.

1 Like
44

My current way of working is to output less interesting sections such as motorways as .mp4 files using the Insta 360 software. So far, I’ve had few problems with this format when uploading files of up to 40 GB per sequence. However, for interesting sections I prefer the Apple .mov format. Unfortunately, the uploader keeps getting out of step here. Files larger than 35 GB and more than 5 minutes long often end up in a loop for me, but I haven’t noticed any systematic reason yet, as smaller .mov files also sometimes show problems. The only thing that helps at the moment is to split the section and upload it in two sequences. I’m fighting for every pixel of image quality, but it’s probably not my Insta 360 X3 that’s limiting the image quality with Mapillary. With G, the same images look better, which gives hope that the currently rendered Mapillary image quality doesn’t have to stay that way forever.

1 Like
46

:thinking: I may be running into some file size limit on Mapillary’s side. I know there are pixel and size limits of 108 MP and 50 MB per image respectively. What is the zip and video file size limit?

Please put all kinds of limits into Mapillary Help. Please also make all limits powers of 2.

47

Technically our upload infra supports files up to 110GB, but I think the current Mapillary limit is lower. I’m not 100% sure what it is currently (@tao or @abalys might know). We have a task to increase it to 110GB.

1 Like
48

I too have often encountered problems when uploading long (and therefore large) videos. At the moment, to continue my work, I have set myself the goal of making videos in MP4 (we use the Insta Titan camera) lasting a maximum of 10 minutes. Videos like this take up a little less than 10 GB. If I don’t make longer ones locally, then with the Insta software I cut them into several parts. By doing this I have almost never had any problems.

49

Subjectively speaking, has the image quality in Mapillary improved recently? Have we missed something?

M:

G: Google Maps

52

Values ​​for a Windows 10 computer:
Through trial and error, I have determined the following Mapillary Desktop Uploader limits on my computer (50 Mbit/s upload internet access).
File format:
ProRes 422 maximum video length 3.5 minutes each (maximum file size 20GB each).
File format: H.264 maximum video length 10 minutes each (maximum file size 20GB each). If I drag a folder into the Desktop Uploader, I make sure that it is not in a parent directory on the external 1T SSD drive. OK Example: D:\Export

Values ​​for a computer running Linux:

Obviously, different limits apply to the Desktop Uploader under Linux; I’m currently researching these. And it looks really good.

Edit Update: Linux

53

According to the code, the limit in mapillary_tools was 10GB, which we just increased to 110GB in mapillary_tools - this will ship in upcoming builds of the desktop uploader as well. Thanks for the update @tao .

1 Like
54

Update: My new recommendation for uploading large Mapillary images is to use the Linux Desktop Uploader. I’m not seeing particularly high speeds, but the upload is stable enough that I can leave the computer running unattended, and I can report that I’ve never been disappointed at the end of the day. I’m currently working with two Ubuntu installations simultaneously, and scaling to additional computers is easily possible. In conclusion, it’s not speed that counts, but stability. Successfully used with up to 45 GB per upload task.

Update: four Ubuntu installations sim

IMG_0125
IMG_01251920×1175 174 KB

ultaneously

55

I fully concur with that. Speed is nice to have but if you lose any progress, cannot reliably resume or cannot complete an upload because progress resets to 0 then no amount of throughput is going to help. It is like with motorsport: You have to first finish the race before you can finish first. Nobody cares that you flew lap records during the race but did not finish. Fortunately, it looks like things have been finally solved. It is only a shame that Mapillary has not realized it themselves for so long due to lack of testing and it took about 80 posts and years to solve. But, better late than never, I guess.

3 Likes
56

Gitne is always right, he should run Mapillary.
Nevertheless five hours of work, 32.000 pictures took me 40 hours to upload.

2 Likes
57

I just recorded a video with my GoPro Hero 7 Black. Since the GoPro’s internal GPS is not so reliable, I recorded a GPS track with my phone to pair to the video. Once I uploaded the video to Mapillary with the Desktop Uploader it asked for a GPS track, so I added the one recorded with my phone. Now, the sequence is finalizing, photos are visibile but I don’t see the sequence of the GPS track matching the photos.
Ideas?

58

Gitne is always right, he should run Mapillary.

:face_with_hand_over_mouth::rofl: Thanks, I’ll take it as a complement but I do not think this would be a good idea because I am not a manager but just a software engineer. :nerd_face: I lack economic wisdom and diplomatic talk. And, I am not always right either. :wink:

Nevertheless five hours of work, 32.000 pictures took me 40 hours to upload.

:thinking: Yeah, it really should not take that long, unless you are perhaps on a cellular connection like 3G or 4G. 5G should be comparable to cable modem speeds, I guess.

59

I restarted my computer and the upload speed was normal.
The upload halted twice but I could restart it well.
But this morning :

image
image1061×396 22.8 KB

Yet I checked the GPS reception regularly.

60

GPS reception is one thing, good GPS signal is another one… I saw a lot of times a GPS signal, but the quality was too poor (all over the place).

61

I think I was to blame. I probably did not have a good look. The sky was clear.

62

@evillz - it may have just been processing which can take a day or two, how are things looking now?

63

Hi Boris, thank you for your reply. The gps track now appears under my photos once everything has been processed.

1 Like