Hi Australia 
.
This part of the forum seems very quiet?
Anyway, I’ve just started with Mapillary, running both GoPro and smart phone, in my car and today on my Vespa.
I’m actually not sure why I started, I think I stumbled on it via OSM and contributing to OpenStreetCam. Anyway, just wanted to say hello.
P.s. I’m from Melbourne.
Cheers.
I’m suprised no general topic was made before.
There is a few Australians here. I am in Adelaide.
I just drive around in the country most days (retired). Uploading is a challenge though… (Mildura at the mo)
Good to know we have a crew here. Cheers.
Nice @bob3bob3, yeah if you’re on the move, the uploading would be a pain in the butt. Do you use a dash cam/ GoPro or directly from your smartphone?
Mapillary loaned/supplied me with a BlackVue dashcam, then we put in the custom firmware for 2 or 5FPS with an associated higher picture quality. I produce roughly 500GBytes (maybe 600,000 images) of photos per month. I put them on a USB/WD disk and attach to a Linux box in Armidale NSW that has a 1TB NBN account, then remote connect to run the upload. I do this disk swap every 2-4 months.
If you look at the GUI for images later than mid August 2019 of Outback etc areas, they’ll mostly be hi qual ones from my setup.
That sounds really awesome @bob3bob3, that’s a lot of photos and I bet you get to see some fantastic places. I’ll have to check out your images.
I have roughly 90GB of images to upload, this is so slow.
Does anyone know the regional location of the upload server? Maybe I could VPN closer to it and then upload?
Cheers.
It’s almost certainly your uplink speed which is the issue. What are you using to do the upload? If using mapillary_tools try increasing the NUMBER_THREADS, I use 40 and can get up to 20MB/sec with the right connection.
My local library recently upgraded their connection from about 100kb/s to about 6MB/s which made uploading so much easier, since I’m on ADSL2+ which only gets 100kb/s upload.
I’m using the Mapillary Uploader desktop application on macOS. I have fiber to the premises with an average speed of 4.5 MB/s upload.
Maybe I need to investigate mapillary_tools, as the uploads from the desktop application are shocking.
Cheers.
I remember a post maybe a year ago that said something like the server was in Ireland? The topic was actually about an Australian server in the making. I don’t know where that is up to. Might be worth a search of posts.
The python upload code points to https://secure-upload.mapillary.com which is an alias for;
d11re41ha1wgd8.cloudfront.net at 13.226.81.76 13.226.81.101 13.226.81.56 and 13.226.81.19 I’ll be the first to admit that I don’t know if this is the actual receiving server. The API is on another host.
My setup suffers from python thread crashes so I have a 800 sec watchdog script to keep it all going. Increasing threads made that problem worse so I stuck to 4. I am using a standard (40Mbps?) NBN fibre (FTTP) connection that does maybe 40GB of upload per 12 hour night. I upload large amounts so I dont really keep track of the rate. It takes about 2 weeks of weekday evenings and weekends to upload roughly 500GBytes. I don’t watch it happen, just let it run! I am using the Linux based tools. (CLI)
I get the impression that my setup gets i/o bound somewhere near the box, as remote access back in suffers. I wouldn’t be surprised too if traffic shaping is being applied by the ISP. I cant really see how a VPN would help unless it bypasses shaping.
Thanks, @andrewharvey. I can’t use the mapillary_tools; unfortunately, I signed up with my OSM account, and I can’t log in with that, and I can’t find or work out a way to get a direct password to Mapillary itself now.
Thanks, @bob3bob3, I used a packet sniffer to find the server, it was in Ireland, however, connecting via VPN to Ireland didn’t help.
I am using a GoPro, but not via USB. I remove the sd card and transfer all the images to the hard drive before upload.
I’m now trying out Mapillary Uploader for Desktop directly from the website, and so far it seems a little faster than the desktop uploader.
@andrewharvey Hi, I’ve managed to get mapillary_tools working. It does appear to be slightly faster, but not a huge amount over the desktop application.
Does this command look ok? Can I add/ remove/ tweak any of it?
Cheers.
./mapillary_tools process_and_upload --import_path "/Volumes/Scratch_Disk/Temp/05_14" --user_name "r33za" --verbose --number_threads 10 --max_attempts 10 --advanced
That looks right, but I’d try a higher --number_threads to try and saturate the connection. 10 is low for small files with a high uplink.
I can’t tell from your example, but do you copy across these photos locally first? Running it against photos still on an SD card can be slow due to the way the mapillary_tools currently works (writes many small files to the same directory).
Thanks @andrewharvey, yes I copy the files over the computer first. I’ll try increasing the the threads a little and see how it goes.
Thanks again.
If you really want to max things out it is worth checking where the PC bottlenecks are. Since you already have finished jpgs it is unlikely to be CPU bound. The large number of subfolder writes might be causing I/O slows though. Playing with the elevator algorithm and increasing buffer sizes might help. (Assuming linux/mac) One of the primary indicators of slows is the HDD LED! A lot of continuous activity is not always good.