Some random questions - old sequences, new process

1

Hi.

So it’s taken a month to get back on (ended up having to re-route my email to a server that didn’t bin the password reset messages), but I’m uploading again. My upload backlog has grown to 750k photos in that time :slight_smile:

Some random questions about old sequences, and the new upload process:

Old sequences:

I’ve noticed a number of old sequences have all but disappeared from the map. I’ve some like this one Mapillary where a thousand images have been replaced by a handful of isolated dots in sequences of just one or two, and some like this one Mapillary where the sequence line is still there but photos are missing (see how the line goes round the roundabouts, but there aren’t any photos there any more). All of these were captured with the phone app, so uploaded and deleted - I don’t have copies. Question #1: Are these photos lost or are they still being reprocessed and will reappear?

Zip files:

From watching the new mapillary_tools at work, I see that there’s an extra step been added to uploading - the collection of files are compressed into a single temporary archive file and that’s what gets uploaded. I generate a lot of photos but only have a slow upload connection, so my batch uploader runs two threads - one getting sequences ready to upload, one uploading those prepared sequences. This runs on a Raspberry Pi accessing the photos on a NAS. The prepare stage is slow, but this doesn’t matter as it’s not the bottleneck. However, the new compress stage is part of the ‘upload’ step - so for every sequence of 500 images the uploads stop for 20 minutes while it creates a 3GB temporary file on the SD card, so the scarce resource - upload bandwidth - now lies idle for 10-20% of the time. Question #2: could the compress stage be moved to the ‘process’ step, so the ‘upload’ step can just get on with uploading?

I’m not currently using the phone app for capturing, but when I did I often filled the phone’s storage to the brim, sometimes with just a single, huge sequence. Question #3: out of curiosity, what happens with the apps now if the storage is full - presumably it won’t have space to create the archive file to upload?

Sequences being uploaded immediately before the changeover:

Question #4: I had a batch uploader running using the previous version of mapillary_tools at the time the changeover happened. A fairly random selection of 32 sequences it uploaded have not appeared on the map or my sequences view after a month - will these eventually appear, or do I re-upload?

Cheers

1 Like
2

To add: on Q2, I’ve swapped the old RaspPi 2B out for a RaspPi 4B with 8GB of RAM, with ¾ of that allocated to hold /tmp as tmpfs, and that drops the dead time used for compression from 20 mins down to 5, but it would still be nicer to eliminate it entirely and compress as part of the ‘process’ step.

3

I’ m also interested in Question #4. It happened to me, too, that I was uploading with an old version of mapillary tools, and all lost sequences before the change were released, but some of these new sequences that were uploaded during changeover with the old tools, are still “lost”, so only visible as single points in OSM ID editor. Dates of these sequences are June 16 and June 20, user name teddy73.