Backing up our contributions


#1

If I wanted to back up my two and a half million pictures I now have contributed to Mapillary how would I go about that.
I am thankful the pictures are wiped from the memory as soon as they are uploaded. That many images takes up quite a bit of space.
How can I now gather them all up for safe keeping? And then, where can I place that many pictures.


#2

You could scale down (or lower the jpeg quality) to save space, then I quess a couple of terabyte backup space will get you far.

If you’re satisfied with putting it on a portable disk it would cost you around $100. If it is a real backup solution it will be a bit more expensive (twice as much?)


#3

Besides the problem to find the best storage solution there is another problem.

The API does not provide a way to retrieve the original/raw (and unblurred) images. However there is a “Download original” button on the website and since it’s javascript you could script it anyway. However this procedure has a few drawbacks. It’s a bit more complicated than the plain usage of the “normal” API and you have to use the credentials used by Mapillary for their web applications. Obviously this is not the intended scenario and will may be blocked or extremely rate limited by Mapillary if it’s used too excessive.

Sadly this is a very frustrating situation and was discussed several times.


#4

Just copy your images to a portable hard disk before uploading them (and before the app deletes them from your phone), no api needed.


#5

micmin1972 said
““Just copy your images to a portable hard disk before uploading them (and before the app deletes them from your phone), no api needed.””

Being the quick witted person I am I thought of this option.
Unfortunately putting them on a drive before uploading won’t work for me and my way of doing things.
First off, two and a half million images have already been deleted off my memories. Getting those back takes something like an act of Congress.
Then for those from here on outward, I am usually uploading 2 phone/cameras and using the third. I may soon be adding another to the mix.
Short of running the lap top off an inverter here in the vehicle as I go along, . . .
Copying them to disk would work, if done between capture and upload.maybe I just don’t see much value in these street view sub acceptable images. I can’t see going to so much bother for them.
I do back up any real pictures I take as I am mapping to multiple storage locations.
I might take something like 100 or maybe 200 pictures in a good day. The Mapillary stitched images of everywhere I find to map take up some real serious disk space. I often go over 40,000 images in a day.


#6

Hi

A 8 TB harddrive is rather cheap these days and can store a lot of pictures, lets say 1 picture is 8 MB, so you can store 1 million pictures on a 8 TB drive. Usually the pictures taken from mobiles are less than 8 MB.
I would go with 8 TB Harddrives and a quick-change external HD case. Simplest and cheapest way of store those amount of pictures. Really. Do not even think about cloud services at all, they are far expansive in this ranges.


#7

This is a normal situation of figuring out which type of backup is suitable for you. The principle of backup is, that any copy of data may disappear at any time. Then you need another copy (the backup) to recreate your lost copy. If you have 2 copies and looses 1, you only have 1 copy. In the time you only have 1 copy you are at serious risk of loosing your data.

Even if you have 10 copies of your data, there is always the risk of loosing them all. Sooner or later you will have to settle for what is good enough.

This is complicated by that different ways of storage have different way you can loose data. E.g. an external harddrive protects you against accidential deletion, viruses and failure of your primary drive etc. But not against fire, theft, flood and the likes.

A cloud backup like CrashPlan may loose your data or go bankrupt. There are examples of people who did not find out their backup was corrupted until they tried to restore. Remember to test your backup: If you have not tried to restore you may not be able to do it.

What I do:
I have a Synology NAS with RAID 1 (2 mirrored drives) and a backup using JottaCloud (a backup company based in Norway). I use CrashPlan for some other data.

I use RAID 1 because harddrives fail. A single external harddrive may go bad at any time. Never store data on just one harddrive if you want to keep it.


#8

@JBTheMilker : Have a few extra memory cards and rotate them when you are out and about. That gives you flexibility to capture, backup and upload. That’s what I have done and that I’ve found to suit my process and needs.


#9

The problem that @JBTheMilker addressed is still concerning me.
I’m not a hacker, so retrieving already uploaded and published images without making a script or similiar to automate the process, is nothing for me.

But I also understand that a general script that goes through all contributions in the whole world, to find the ones that I have made, would take forever. So if someone could make a script that I can use by limiting the search to a specific country/region, I would be very thankful. :smile: