Mapillary_tools upload errors

Contributing and equipment Command line tools
1

When I try to upload, I get network errors frequently. In fact it takes several times to upload 300 images. Sample run - I have a Wireshark trace also; it tries to reconnect but it’s as though the web host was down:

Mapillary tools 0.7.1

Uploading 305 images with valid mapillary tags (Skipping 0)
Compressing: 100%|████████████████████████████████████████████████████████████████████| 305/305 [00:16<00:00, 18.33it/s]
Uploading: 68%|████████████████████████████████████▌ | 155189248/229526817 [02:52<04:20, 285591.65it/s]
Traceback (most recent call last):
File “/home/allusers/.local/lib/python3.8/site-packages/urllib3/connection.py”, line 158, in _new_conn
conn = connection.create_connection(
File “/home/allusers/.local/lib/python3.8/site-packages/urllib3/util/connection.py”, line 80, in create_connection
raise err
File “/home/allusers/.local/lib/python3.8/site-packages/urllib3/util/connection.py”, line 70, in create_connection
sock.connect(sa)
OSError: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home/allusers/.local/lib/python3.8/site-packages/urllib3/connectionpool.py”, line 597, in urlopen
httplib_response = self._make_request(conn, method, url,
File “/home/allusers/.local/lib/python3.8/site-packages/urllib3/connectionpool.py”, line 343, in _make_request
self._validate_conn(conn)
File “/home/allusers/.local/lib/python3.8/site-packages/urllib3/connectionpool.py”, line 839, in _validate_conn
conn.connect()
File “/home/allusers/.local/lib/python3.8/site-packages/urllib3/connection.py”, line 301, in connect
conn = self._new_conn()
File “/home/allusers/.local/lib/python3.8/site-packages/urllib3/connection.py”, line 167, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f939d058250>: Failed to establish a new connection: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home/allusers/.local/lib/python3.8/site-packages/requests/adapters.py”, line 439, in send
resp = conn.urlopen(
File “/home/allusers/.local/lib/python3.8/site-packages/urllib3/connectionpool.py”, line 637, in urlopen
retries = retries.increment(method, url, error=e, _pool=self,
File “/home/allusers/.local/lib/python3.8/site-packages/urllib3/util/retry.py”, line 399, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host=‘rupload.facebook.com’, port=443): Max retries exceeded with url: /mapillary_public_uploads/mly_tools_50c65e9e6208960117ea95e8107774c5 (Caused by NewConnectionError(‘<urllib3.connection.VerifiedHTTPSConnection object at 0x7f939d058250>: Failed to establish a new connection: [Errno 101] Network is unreachable’))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home/allusers/.local/bin/mapillary_tools”, line 8, in
sys.exit(main())
File “/home/allusers/.local/lib/python3.8/site-packages/mapillary_tools/main.py”, line 93, in main
command.run(args)
File “/home/allusers/.local/lib/python3.8/site-packages/mapillary_tools/commands/upload.py”, line 112, in run
upload(
File “/home/allusers/.local/lib/python3.8/site-packages/mapillary_tools/upload.py”, line 141, in upload
uploader.upload_sequence_v4(
File “/home/allusers/.local/lib/python3.8/site-packages/mapillary_tools/uploader.py”, line 443, in upload_sequence_v4
upload_resp = service.upload(
File “/home/allusers/.local/lib/python3.8/site-packages/mapillary_tools/upload_api_v4.py”, line 60, in upload
resp = requests.post(
File “/home/allusers/.local/lib/python3.8/site-packages/requests/api.py”, line 116, in post
return request(‘post’, url, data=data, json=json, **kwargs)
File “/home/allusers/.local/lib/python3.8/site-packages/requests/api.py”, line 60, in request
return session.request(method=method, url=url, **kwargs)
File “/home/allusers/.local/lib/python3.8/site-packages/requests/sessions.py”, line 524, in request
resp = self.send(prep, **send_kwargs)
File “/home/allusers/.local/lib/python3.8/site-packages/requests/sessions.py”, line 637, in send
r = adapter.send(request, **kwargs)
File “/home/allusers/.local/lib/python3.8/site-packages/requests/adapters.py”, line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host=‘rupload.facebook.com’, port=443): Max retries exceeded with url: /mapillary_public_uploads/mly_tools_50c65e9e6208960117ea95e8107774c5 (Caused by NewConnectionError(‘<urllib3.connection.VerifiedHTTPSConnection object at 0x7f939d058250>: Failed to establish a new connection: [Errno 101] Network is unreachable’))

2 Likes
2

Disregard - still a problem but moved to a better location as a Github issue.

https://github.com/mapillary/mapillary_tools/issues/419

3

I experienced a similar problem. It went away when I updated using
python3 -m pip install --upgrade git+https://github.com/mapillary/mapillary_tools

1 Like
4

Thanks - the latest code on Github works and allows it to complete the uploads.

It’s a curious network problem; I wonder if a load balancer is dropping streams to regulate traffic? Even though it succeeds it can take 20 minutes to upload a minute’s worth of dashcam video because of timeouts and retries.

5

[Edit]

The mapillary_tools binaries are no longer available at tools (dot) mapillary (dot) com.

Binaries for Windows and Mac OS are available here

6

FYI - with my latest upload, there are many upload errors. It recovers and continues but overall progress is slow. Does something in the Mapillary infrastructure need to be reset?

7

My upload site doing BlackVue mp4’s is also having problems. Waiting for a 4000 off videos batch to complete before checking for missings via the web GUI.

Uses 0.8.1 Linux via docker.

Example debug

2022-04-25 07:05:52,658 - DEBUG - Sending upload_end via IPC: {‘total_sequence_count’: 1570, ‘sequence_idx’: 10, ‘entity_size’: 52298394, ‘import_path’: ‘to_upload/20210514BV/20210514_125339_NF.mp4’, ‘md5sum’: ‘7ccb511956c124e898f5f56d3870a596’, ‘upload_start_time’: 1650870321.714409, ‘upload_total_time’: 30.53449058532715, ‘offset’: 52298394, ‘retries’: 0, ‘upload_last_restart_time’: 1650870322.1237226, ‘upload_first_offset’: 0, ‘chunk_size’: 0, ‘upload_end_time’: 1650870352.6582131}
2022-04-25 07:06:02,685 - DEBUG - API Logging for action upload_failed_upload: {‘images’: 0, ‘sequences’: 9, ‘size’: 447.5315, ‘uploaded_size’: 447.5315, ‘speed’: 1.7194, ‘time’: 260.2831}
Traceback (most recent call last):
File “/usr/local/lib/python3.8/dist-packages/urllib3/connection.py”, line 174, in _new_conn
conn = connection.create_connection(
File “/usr/local/lib/python3.8/dist-packages/urllib3/util/connection.py”, line 72, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File “/usr/lib/python3.8/socket.py”, line 918, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known