FTP picture upload instead of Python script?


#1

FTP clients are very common and even integrated in most operating systems. I think many users would appreciate if they can use (S)FTP instead of a python script for uploading. There are servers like proftpd that can easily authenticate against a database, also with hashed passwords.


#2

Rsync over SSH would be even better option. Faster and more secure.

With my slow 1 mbps upload speed I rsync my photos overnight to my server at hosting provider and after that run mapillary python script from there.


#3

I woudn’t even dare to ask for something ssh-based because I think auth’ing to the user database might be not so easy to implement. FTP is easy, the ususal proftpd can do and I can provide examples for configuration. But whatever standard protocol it will be, FTP, rsync, scp, Webdav, all of that would be better than having to use this proprietary python script.


#4

I vote for. It seems the only way to get my Virb XE camera into action.


#5

Whats the problem with the Python scripts and Garmin Virb XE?


#6

No, the problem is with my computer.


#7

My experience with FTP and a large amount of files is, that the transfer often stops in the middle of the process. At least the Python scripts will time out. I will not use FTP.

I think the efford would be better spend on making a rock solid HTML5 web uploader for huge abounts of files. As far as I knows, no such uploader exists. It is not easy to make, but should be doable.


#8

One feature I like with Ftp is that you can limit upload rate.


#9

One of the main advantages of FTP is the automatic resuming of broken uploads. Command-line tools like ncftp can handle unattended uploads beautifully. FTP is a very old and established protocol, older than most of the internet users today, and it is exactly for this purpose of tranferring big files.


#10

@maxim75 I do this too, my scripts I use are at https://github.com/andrewharvey/lg360-mapillary-helpers.

I use lepton as a transport compression which helps, but even then two hours of photos takes a few days to upload for me. :frowning:


#11

Uploading the pictures as they come from the camera consumes incredible long time. I am compressing all pictures (parameters can be found here https://github.com/moenk/mapillary-tools/blob/master/cmd/mapillary_uploader.bat in my uploading batch) before uploading.
Although I use that compression and I have the best internet connection available in my area it takes a whole day and night to upload the pictures from my last trip. It’d be more than a week if I wouldn’t use picture compression.


#12

I can see you both resize and compress. With 85 in compression I think that it is the resizing that makes the most difference. Have you tried to experiment with a larger image but more compression. 75 often works well for me and you may be able to double the resolution for the same file size and still don’t have noticeable compression.

The Python scripts uploades directly to Amazon, so I don’t think it will go any faster than this. In case of a broken upload, have you tried just to run the Python script again? Some people says it just resumes automatically.


#13

I thought about reducing the quality to 75% but I’m fine with uploading time like it is. I just updated the GitHub repo, time while scaling is now used for uploading to OpenStreetCam.