"OSError: [Errno 28] No space left on device" when uploading

Contributing and equipment Command line tools
25

Yes, we’ll be going with uploading the selected files straight away on all platforms without an intermediary step or format. It will be available in mapillary_tools in the next release, I’d let @tao answer on when that might be, but you can already follow the progress in the GH repo.

2 Likes
26

@Richlv @GITNE @Eric_S Echo what @nikola said, we are getting rid of the tmpdir zipping process in feat: support uploading images file by file by ptpt · Pull Request #753 · mapillary/mapillary_tools · GitHub and it will be released in the next version of desktop uploader and mapillary_tools. cc @Yaro @boris

Also FYI @GITNE the offset reset issue in upload resuming should be largely solved for imagery uploading after the release too.

3 Likes
27

Hey @Richlv @GITNE @Eric_S , It’d be appreciated if you can try out Release v0.14.0b1 · mapillary/mapillary_tools · GitHub and share any feedback here

2 Likes
28

Oooh, neat. Will plain upgrade bump to that version or not?

30

Turns out, I used the downloaded version currently, so kept that approach :slight_smile:
Didn’t check the temporary file generation, but the upload itself was 3-4 times faster. Is it just a random coincidence, or is this related to parallelisation that was lost with the FB upload framework (and regained now)?

31

Glad to hear you had a good experience! Yes, the upload itself should be faster as well I believe, nice work @tao !

1 Like
32

Hello @tao
ted@ted-swiftsf314511:~$ mapillary_tools/bin/mapillary_tools --version
mapillary_tools version 0.14.0b1
I updated the tools, and the upload seems to be faster. But really often upload gets interrrupted and then the current sequence is left not uploaded. When restating the tools, the upload of the left-over sequences starts again at 0%. It’s really annoying at the moment.
Here is an example, maybe the fourth time I tried to upload folger 001_0081 with 11 sequences. As you can see, often sequences are only uploaded the some percentage and then with no error message they are left behind. When restarting, upload starts again at 0:

ted@ted-swiftsf314511:~$ mapillary_tools/bin/mapillary_tools upload “Aayi/001_0081/”
2025-07-18 07:13:32,200 - INFO - Uploading to profile “teddy73”: {‘user_upload_token’: ‘[REDACTED]’, ‘MAPSettingsUserKey’: ‘103080845264750’, ‘MAPSettingsUsername’: ‘teddy73’}
Uploading IMAGE (1/11): 28%|██████████████████████████████ | 1.12G/3.97G [35:30<2:55:11, 290kB/s]
Uploading IMAGE (1/11): 29%|██████████████████████████████▌ | 1.15G/3.97G [39:34<4:59:23, 169kB/s]Uploading IMAGE (1/11): 41%|███████████████████████████████████████████▍ | 1.61G/3.97G [49:02<26:44, 1.58MB/s]2025-07-18 08:03:04,917 - INFO - Sequence 1 has been uploaded already. Check the upload history at /home/ted/.local/share/mapillary_tools/upload_history/c7/db9ef164b7964d41c2343f15b2368e.json
Uploading IMAGE (1/11): 41%|██████████████████████████████████████████▉ | 1.61G/3.97G [49:16<1:12:13, 584kB/s]
Uploading IMAGE (3/11): 73%|████████████████████████████████████████████████████████████████████████████▋ | 2.89G/3.96G [1:08:12<05:02, 3.80MB/s]2025-07-18 09:11:31,389 - INFO - Sequence 3 has been uploaded already. Check the upload history at /home/ted/.local/share/mapillary_tools/upload_history/a7/bded76c3e6d1091768fd361faf7e6c.json
Uploading IMAGE (3/11): 73%|█████████████████████████████████████████████████████████████████████████████▍ | 2.89G/3.96G [1:08:26<25:17, 757kB/s]
Uploading IMAGE (5/11): 16%|█████████████████▏ | 633M/3.87G [08:58<01:18, 44.3MB/s]2025-07-18 09:20:43,060 - INFO - Sequence 5 has been uploaded already. Check the upload history at /home/ted/.local/share/mapillary_tools/upload_history/f1/449eb0236e5d8621329b4d07d62f16.json
2025-07-18 09:20:52,046 - INFO - Sequence 6 has been uploaded already. Check the upload history at /home/ted/.local/share/mapillary_tools/upload_history/4a/85663e0adb93c4a996548bda2a0c12.json
Uploading IMAGE (5/11): 16%|█████████████████▏ | 633M/3.87G [09:14<48:38, 1.20MB/s]
Uploading IMAGE (8/11): 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.54G/1.54G [26:14<00:00, 1.05MB/s]
Uploading IMAGE (9/11): 53%|████████████████████████████████████████████████████████▉ | 1.96G/3.69G [57:58<27:51, 1.11MB/s]2025-07-18 10:45:24,202 - INFO - Sequence 9 has been uploaded already. Check the upload history at /home/ted/.local/share/mapillary_tools/upload_history/fc/2e580779f577040135e66defa794c6.json
2025-07-18 10:45:26,128 - INFO - Sequence 10 has been uploaded already. Check the upload history at /home/ted/.local/share/mapillary_tools/upload_history/61/d600b744060b207848c808efac47bd.json
2025-07-18 10:45:26,549 - INFO - 1 image sequences uploaded
2025-07-18 10:45:26,549 - INFO - 1572.1M data in total
2025-07-18 10:45:26,549 - INFO - 1572.1M data uploaded
2025-07-18 10:45:26,549 - INFO - 1574.9s upload time
2025-07-18 10:45:26,549 - ERROR - Upload error: HTTPError: 400 Client Error: Bad Request for url: https://rupload.facebook.com/mapillary_public_uploads/mly_tools_cf4ebe29e708e54d9fc6f96f0d1842a7.jpg
2025-07-18 10:45:26,549 - ERROR - Upload error: UploadedAlreadyError:
2025-07-18 10:45:26,549 - ERROR - Upload error: HTTPError: 400 Client Error: Bad Request for url: https://rupload.facebook.com/mapillary_public_uploads/mly_tools_0533f87c354232dbe6dd6f50368e015c.jpg
2025-07-18 10:45:26,549 - ERROR - Upload error: UploadedAlreadyError:
2025-07-18 10:45:26,549 - ERROR - Upload error: HTTPError: 400 Client Error: Bad Request for url: https://rupload.facebook.com/mapillary_public_uploads/mly_tools_14a84a3fa27d89016cbafd9708b82bd7.jpg
2025-07-18 10:45:26,549 - ERROR - Upload error: UploadedAlreadyError:
2025-07-18 10:45:26,549 - ERROR - Upload error: UploadedAlreadyError:
2025-07-18 10:45:26,549 - ERROR - Upload error: HTTPError: 400 Client Error: Bad Request for url: https://rupload.facebook.com/mapillary_public_uploads/mly_tools_45c3b73fd10f3610338ad067dc32169e.jpg
2025-07-18 10:45:26,549 - ERROR - Upload error: UploadedAlreadyError:
2025-07-18 10:45:26,549 - ERROR - Upload error: UploadedAlreadyError:
Uploading IMAGE (9/11): 53%|█████████████████████████████████████████████████████████▍ | 1.96G/3.69G [58:04<51:06, 604kB/s]
ted@ted-swiftsf314511:~$
ted@ted-swiftsf314511:~$ mapillary_tools/bin/mapillary_tools upload “Aayi/001_0081/”
2025-07-18 12:13:07,268 - INFO - Uploading to profile “teddy73”: {‘user_upload_token’: ‘[REDACTED]’, ‘MAPSettingsUserKey’: ‘103080845264750’, ‘MAPSettingsUsername’: ‘teddy73’}
Uploading IMAGE (1/11): 3%|███▋ | 138M/3.97G [10:09<21:24, 3.20MB/s

33

@Teddy73 @GITNE When it’s stuck at 0% MT is actually calculating checksums for all the files. If you are uploading large files on a slow disk (especially SD card), then it’s likely appears that it’s stuck at 0% for a while.

Unfortunately we can’t skip these checksum calculations because MT needs them for upload deduplication check and resumption. I’ll try if I can make it faster, also make the progress more visible.

Another issue shown in @Teddy73’s log is HTTPError: 400 Client Error. I can’t tell what was going on. I will improve the logging so it shows more info in the verbose mode.

Thanks for sharing the feedback!

34

Good stuff @tao, maybe rather than showing “0%” we can display “Calculating checksums” to let users know what is happening.

1 Like
36

Hello @tao ,
the stuck at 0% MT is not my problem. I´m trying to describe what I mean again:
For example, I have 8 sequences to upload. First time, some squences are uploaded ok, so the progress bar gets to 100% and then the next sequence is uploaded, as it should be.
But some sequences just get to any number below 100, for example 43%, then there`s no error message at all, and the next sequence is started. The not finished sequences are not uploaded at this run at all.
Then, next time I run upload, the unfinished sequences start at 0% again and maybe finish ok this time.
So, for all 8 sequences to upload ok, I need to start upload 3 or 4 times.
Can you reproduce this error?
Three things are annoying here: No error message, no automatic retry, restart at 0%

One other bug I found, but not so annoying: Split is done at 723 pictures, then 277 pictures, then 723 pictures, then 277 pictures and so on. I guess there’s a counter for 1000 pics which is not set to 0 when another limit (something with “pixels too large”) hits.

1 Like
37

@Teddy73 @GITNE The new released v0.14 should solve the issue hopefully. Give it a try and let me know!

This particular issue is solved in #766.

2 Likes