Python script to retrieve your images from a multicam setup

Hi !
I often use my 4 cams setup to take pictures for Mapillary/Osm. One tedious task is to copy all the pictures from the multiple sdcards, group them, create the sequence, and rename them with a timestamp in the filename.

Some scripts already exist to do this, but for only one source. With this one you can use up to 6 sources, and the script will copy the images in parallel to speed up the copy process. I reached 290 MBytes/s from 4 sources to one SSD.

It should work on Windows, Linux and OsX.
If you find some bugs, create an issue on Github, but beware, I’m not a dev, only a Python newbie :smile:

copy/paste of the doc:

Script for simultaneously copying the pictures from up to 6 cameras.
The primary goal is to copy the pictures from 4 action-cams, make groups of pictures with a cutoff time, create
a subfolder for each cam and copy the renamed pictures (timestamp added to the filename).
Let’s say you have two cams, with sd cards named “front” and “back”.

    front sdcard :
    ROOT/DCIM/YDXJ0025.jpg (timestamp is 2017-02-01_11h21mn10s)
    ROOT/DCIM/YDXJ0026.jpg (timestamp is 2017-02-01_11h21mn11s)
    ROOT/DCIM/YDXJ0027.jpg (timestamp is 2017-02-01_11h21mn12s)
    ROOT/DCIM/YDXJ0028.jpg (timestamp is 2017-02-01_11h26mn42s)
    ROOT/DCIM/YDXJ0029.jpg (timestamp is 2017-02-01_11h26mn43s)
    back sdcard :
    ROOT/DCIM/YDXJ0112.jpg (timestamp is 2017-02-01_11h21mn10s)
    ROOT/DCIM/YDXJ0113.jpg (timestamp is 2017-02-01_11h21mn11s)
    ROOT/DCIM/YDXJ0114.jpg (timestamp is 2017-02-01_11h21mn12s)
    ROOT/DCIM/YDXJ0115.jpg (timestamp is 2017-02-01_11h26mn42s)
    ROOT/DCIM/YDXJ0116.jpg (timestamp is 2017-02-01_11h26mn43s)
    with the command:
    python copy_pic.py ~/test -s front,back  -c 60
    In your test folder you will get:
    2017-02-01_11h21mn10s|
    		         |front|
    		     	       |2017-02-01_11h21mn10s-Cam_front-YDXJ0025.jpg
     			       |2017-02-01_11h21mn11s-Cam_front-YDXJ0026.jpg
        	               |2017-02-01_11h21mn12s-Cam_front-YDXJ0027.jpg
        		 |back |
    		    	       |2017-02-01_11h21mn10s-Cam_back-YDXJ0112.jpg
    			       |2017-02-01_11h21mn11s-Cam_back-YDXJ0113.jpg
    			       |2017-02-01_11h21mn12s-Cam_back-YDXJ0114.jpg
    2017-02-01_11h26mn42s|
    		         |front|
    		     	       |2017-02-01_11h26mn42s-Cam_front-YDXJ0028.jpg
    			       |2017-02-01_11h26mn43s-Cam_front-YDXJ0029.jpg
    		         |back |
    		     	       |2017-02-01_11h26mn42s-Cam_back-YDXJ0115.jpg
    			       |2017-02-01_11h26mn43s-Cam_back-YDXJ0116.jpg
3 Likes

I have made a shell script (for Mac, but it should work in Linux too), that renames videos and photos from a camera with a heading offset stored in a certain file in the root directory of the SD card. It than get the heading offset from the filename before processing and preparing for upload. The processing uses different python tools to set/adjust timestamps, set position and heading, and group them into subfolders for different sequences. I than uses the python script to upload to Mapillary.

The heading offset is stored on the SD cards, so that the script can easily make that part of the filename (VIRB0001.JPG → 0-VIRB0001.JPG), and the script for processing grabs the “0” from the filename and use it as heading offset (0 is facing front). I can that way easily make configuration for any angle offset I like, currently 0 (front) and 180 (back), but plans for 90 and 270 too, if I don’t go for 60, 120, 240, 300

Does your script copy the pictures simultaneously from your 2 cams too ? Is it available somewhere ?

BTW: Now my script should work on OsX too. Could you test it ? (don’t forget to name the sdcards)

I havn’t shared my script as I have made a lot of tweaks specific for my computer setup.

Work flow is basically this:

  1. Identify mounted camera or memory card as recognised by script
  2. Get heading offset and store variable
  3. Copy all GPX data from card
  4. Copy all FIT data from card, and convert to GPX (Garmin VIRB XE)
  5. Copy all videos and photos from card, and add heading offset to filenames
  6. Make a merged copy of GPX
  7. Match video files with GPX files from same unit
  8. Process each video with ffmpeg and Mapillary scripts to separate sequences
  9. Group photos in separate sequences
  10. Upload sequences with mapillary script (hacked to maintain sequences)

The task is divided in 3 steps having a separate shell script (import, process, upload). The plan is to set upload as a cron job, and use auto sensing (DBUS) for import, leaving only process to be manually activated.

I could do a test run of your script to verify that it runs, but I will not have time to do any stress test or bug fixing.

1 Like