# Python3 BlackVue upload **URL:** \ **Category:** Command line tools\ **Created:** [April 16, 2021, 7:56am UTC](https://forum.mapillary.com/t/python3-blackvue-upload/4775 "2021-04-16T07:56:09Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![bob3bob3](https://sea1.discourse-cdn.com/flex015/user_avatar/forum.mapillary.com/bob3bob3/32/68393_2.png) [@bob3bob3](https://forum.mapillary.com/u/bob3bob3)\ **Post date:** [April 16, 2021, 7:56am UTC](https://forum.mapillary.com/t/python3-blackvue-upload/4775/1 "2021-04-16T07:56:09Z")
Just checking First time I tried to use the tools to upload my Mapillary firmware equipped BlackVue vids in about a year - First I get an error about only supporting “Front” which it already is. ERROR: Currently, only front Blackvue videos are supported on this command. Please use video\_process command for backwards camera videos - When I comment out this test in the code it fails unable to find GPS data Warning, could not extract gps from video 20210322\_123530\_NF.mp4 due to a bytes-like object is required, not ‘str’, video will be skipped… Is anyone successfully using the firmware supplied 1CH BlackVue with the recent P3 tools? ---
**Author:** ![gpsmapper](https://sea1.discourse-cdn.com/flex015/user_avatar/forum.mapillary.com/gpsmapper/32/70818_2.png) [@gpsmapper](https://forum.mapillary.com/u/gpsmapper)\ **Post date:** [April 16, 2021, 8:01pm UTC](https://forum.mapillary.com/t/python3-blackvue-upload/4775/2 "2021-04-16T20:01:29Z")
I guess it’s designed to work with DR900 series only. Are you trying with DR900 produced video? Make sure you are using correct advance options, esp. ` --geotag_source "blackvue_videos"` ![image](https://us1.discourse-cdn.com/flex015/uploads/mapillary/original/3X/d/e/de51b3ccd339cafbbbb8b04c5d76cbec75b806b5.png) ---
**Author:** ![bob3bob3](https://sea1.discourse-cdn.com/flex015/user_avatar/forum.mapillary.com/bob3bob3/32/68393_2.png) [@bob3bob3](https://forum.mapillary.com/u/bob3bob3)\ **Post date:** [April 16, 2021, 9:45pm UTC](https://forum.mapillary.com/t/python3-blackvue-upload/4775/3 "2021-04-16T21:45:21Z")
It is a DR900, supplied by Mapillary under their camera program a couple of years back, then loaded with the custom firmware. In the past I used the now deprecated local sampling method defining the geotag source, then the Python 2 version of Direct Upload. I have had a fellow mapper uploading my mp4’s for 6 months now using doku. I have however just had the need to upload some for quick use and want to know if it is working for anyone else. The direct upload help syntax has no gps choice function, but I will try it. I am using a virtual environment. Tnx! ---
**Author:** ![danbjoseph](https://sea1.discourse-cdn.com/flex015/user_avatar/forum.mapillary.com/danbjoseph/32/68068_2.png) [@danbjoseph](https://forum.mapillary.com/u/danbjoseph)\ **Post date:** [July 1, 2021, 12:45am UTC](https://forum.mapillary.com/t/python3-blackvue-upload/4775/4 "2021-07-01T00:45:33Z")
Did you solve your issue? I am having similar issues. I have received no response from the Mapillary support email. Tao Peng has been helpful in responding on GitHub but I haven’t managed to successfully upload anything yet. [https://github.com/mapillary/mapillary\_tools/issues/394#issuecomment-871820037](https://github.com/mapillary/mapillary_tools/issues/394#issuecomment-871820037) ---
**Author:** ![bob3bob3](https://sea1.discourse-cdn.com/flex015/user_avatar/forum.mapillary.com/bob3bob3/32/68393_2.png) [@bob3bob3](https://forum.mapillary.com/u/bob3bob3)\ **Post date:** [July 1, 2021, 3:09am UTC](https://forum.mapillary.com/t/python3-blackvue-upload/4775/5 "2021-07-01T03:09:14Z")
No not as yet. I see Tao is making many many changes/fixes with master merges, so I think it’s a case of wait. ---
**Author:** ![tao](https://avatars.discourse-cdn.com/v4/letter/t/f9ae1b/32.png) [@tao](https://forum.mapillary.com/u/tao)\ **Post date:** [July 1, 2021, 8:38am UTC](https://forum.mapillary.com/t/python3-blackvue-upload/4775/6 "2021-07-01T08:38:29Z")
Hey @danbjoseph @bob3bob3 would you mind send some sample blackvue video to [support@mapillary.zendesk.com](mailto:support@mapillary.zendesk.com) so we can reproduce the issue? Thanks for trying out mapillary\_tools. ---
**Author:** ![danbjoseph](https://sea1.discourse-cdn.com/flex015/user_avatar/forum.mapillary.com/danbjoseph/32/68068_2.png) [@danbjoseph](https://forum.mapillary.com/u/danbjoseph)\ **Post date:** [July 1, 2021, 10:48am UTC](https://forum.mapillary.com/t/python3-blackvue-upload/4775/7 "2021-07-01T10:48:14Z")
sent! thank you. ---
**Author:** ![bob3bob3](https://sea1.discourse-cdn.com/flex015/user_avatar/forum.mapillary.com/bob3bob3/32/68393_2.png) [@bob3bob3](https://forum.mapillary.com/u/bob3bob3)\ **Post date:** [July 3, 2021, 10:25am UTC](https://forum.mapillary.com/t/python3-blackvue-upload/4775/8 "2021-07-03T10:25:20Z")
Sorry @tao , my email server max’s out at 32MBytes. The vid is about 50MBytes. Pls get from gdrive [https://drive.google.com/file/d/189B2MU\_otEHJCBqxgariCK2dN34Y\_yrT/view?usp=sharing](https://drive.google.com/file/d/189B2MU_otEHJCBqxgariCK2dN34Y_yrT/view?usp=sharing) Cheers ---
**Author:** ![asturksever](https://sea1.discourse-cdn.com/flex015/user_avatar/forum.mapillary.com/asturksever/32/72007_2.png) [@asturksever](https://forum.mapillary.com/u/asturksever)\ **Post date:** [November 23, 2021, 4:59pm UTC](https://forum.mapillary.com/t/python3-blackvue-upload/4775/9 "2021-11-23T16:59:10Z")
@danbjoseph Latest version of mapillary\_tools (0.8.0) supports direct Blackvue video uploads to Mapillary. Have you ever had a chance to try it? You do not need to process Blackvue videos, they will be processed on server-side. > **[Release v0.8.0 · mapillary/mapillary\_tools](https://github.com/mapillary/mapillary_tools/releases/tag/v0.8.0)** > > Features > > Immutable process and upload: do not modify images 0430c71 #276 > The command process writes all metadata a single file (image description file) instead of multiple files 081e6f6 #269 > The c... For macOS - Install mapillary\_tools package for macOS - Run below command to directly upload your Blackvue videos find /DirectoryOftheVideos | xargs -n1 ./mapillary\_tools upload - Ignore the RuntimeError and wait mapilary\_tools to upload all files from your directory - Upload summary will be popped up for each uploaded sequence Please let me know if this helps. ---
**Author:** ![danbjoseph](https://sea1.discourse-cdn.com/flex015/user_avatar/forum.mapillary.com/danbjoseph/32/68068_2.png) [@danbjoseph](https://forum.mapillary.com/u/danbjoseph)\ **Post date:** [December 10, 2021, 7:04pm UTC](https://forum.mapillary.com/t/python3-blackvue-upload/4775/10 "2021-12-10T19:04:31Z")
seems to be working. thank you. ![Screen Shot 2021-12-10 at 2.03.38 PM](https://us1.discourse-cdn.com/flex015/uploads/mapillary/original/3X/5/3/53f89ef4f098ef9553fb1de30125c72432273786.jpeg) ![Screen Shot 2021-12-10 at 2.03.23 PM](https://us1.discourse-cdn.com/flex015/uploads/mapillary/original/3X/3/7/37e4cfecc798b261d27d83014dcf3207b176e861.png) ---
**Author:** ![asturksever](https://sea1.discourse-cdn.com/flex015/user_avatar/forum.mapillary.com/asturksever/32/72007_2.png) [@asturksever](https://forum.mapillary.com/u/asturksever)\ **Post date:** [December 10, 2021, 11:48pm UTC](https://forum.mapillary.com/t/python3-blackvue-upload/4775/11 "2021-12-10T23:48:22Z")
Awesome, thanks for the update. We’re planning to support multiple file upload with single command soon. I’ll keep you posted. ---
**Author:** ![jamiebiswas](https://avatars.discourse-cdn.com/v4/letter/j/90db22/32.png) [@jamiebiswas](https://forum.mapillary.com/u/jamiebiswas)\ **Post date:** [January 25, 2022, 6:24am UTC](https://forum.mapillary.com/t/python3-blackvue-upload/4775/12 "2022-01-25T06:24:45Z")
I am writing down all of the things and code related to Python3 Blck Vue - ## Quickstart Process and upload imagery: mapillary\_tools process\_and\_upload “path/to/images/” Upload BlackVue videos: mapillary\_tools upload\_blackvue “path/to/blackvue\_videos/” ## Requirements ### User Authentication To upload images to Mapillary, an account is required and can be created [here]. When using the tools for the first time, user authentication is required. You will be prompted to enter your account credentials. ### Metadata To upload images to Mapillary, image `GPS` and `capture time` are minimally required. More information [here]. ## Installation ### Installing via Pip Python (3.6 and above) and git are required: python3 -m pip install --upgrade git+https://github.com/mapillary/mapillary\_tools If you see “Permission Denied” error, try to run the command above with `sudo` , or install it in your local [virtualenv] ### Installing on Android Devices A command line program such as Termux is required. Installation can be done without root privileges. The following commands will install Python 3, pip3, git, and all required libraries for mapillary\_tools on Termux: pkg install python git build-essential libgeos openssl libjpeg-turbo python3 -m pip install --upgrade pip wheel python3 -m pip install --upgrade git+https://github.com/mapillary/mapillary\_tools Termux must access the device’s internal storage to process and upload images. To do this, use the following command: termux-setup-storage Finally, on devices running Android 11, using a command line program, mapillary\_tools will process images very slowly if they are in shared internal storage during processing. It is advisable to first move images to the command line program’s native directory before running mapillary\_tools. For an example using Termux, if imagery is stored in the folder `Internal storage/DCIM/mapillaryimages` the following command will move that folder from shared storage to Termux: mv -v storage/dcim/mapillaryimages mapillaryimages ## Video Support To [proces). Make sure it is executable and put the downloaded binaries in your `$PATH` . You can also install `ffmpeg` with your favourite package manager. For example: On macOS, use [Homebrew]: brew install ffmpeg On Debian/Ubuntu: sudo apt install ffmpeg ## Usage ### Process The `process` command geotags images in the given directory. It extracts the required and optional metadata from image EXIF (or the other supported geotag sources), and writes all the metadata (or process errors) in an [image description] file, which will be read during [upload]. #### Examples Process all images in the directory `path/to/images/` (and its sub-directories): mapillary\_tools process “path/to/images/” Interpolate images in the directory `path/to/images/` on the GPX track read from `path/to/gpx_file.gpx` . The images are required to contain capture time in order to sort the images and interpolate them. mapillary\_tools process “path/to/images/” \ –geotag\_source “gpx” \ –geotag\_source\_path “path/to/gpx\_file.gpx” Process all images in the directory, specifying an angle offset of 90° for the camera direction and splitting images into sequences of images apart by less than 100 meters according to image `GPS` and less than 120 seconds according to image capture time. mapillary\_tools process “path/to/images/” \ –offset\_angle 90 \ –cutoff\_distance 100 \ –cutoff\_time 120 ### Upload Images Images that have been successfully processed can be uploaded with the `upload` command. #### Examples Upload all processed images in the directory `path/to/images/` to user `mly_user` for organization `mly_organization_id` . It is optional to specify `--user_name` if you have only one user [authenticated]. mapillary\_tools upload “path/to/images/” \ –user\_name “mly\_user” \ –organization\_key “mly\_organization\_id” ### Upload BlackVue Videos BlackVue videos can be uploaded with the `upload` command and will be processed on Mapillary servers. #### Examples Upload a BlackVue video with file name `video_file_name.mp4` to user `mly_user` for organization `mly_organization_id` . It is optional to specify `--user_name` if you have only one user [authenticated]. mapillary\_tools upload\_blackvue “video\_file\_name.mp4” \ –user\_name “mly\_user” \ –organization\_key “mly\_organization\_id” Upload all BlackVue videos (\*.mp4) under the folder: mapillary\_tools upload\_blackvue “path/to/blackvue\_videos/” ### Video Process Video process involves two commands: 1. `sample_video` : sample videos into images, and insert capture times to the image EXIF. Capture time is calculated based on the video start time and sampling interval. This is where `ffmpeg` is being used. 2. `process` : process (geotag) the sample images with the specified source The two commands are usually combined into a single command `video_process` . #### Examples Sample the videos located in `path/to/videos/` at the default sampling rate 2 seconds, i.e. one video frame every two seconds. Video frames will be sampled into a sub-directory `path/to/videos/mapillary_sampled_video_frames` . mapillary\_tools sample\_video “path/to/videos/” Sample the videos located in `path/to/videos/` to directory `path/to/sample_images/` at a sampling rate 0.5 seconds, i.e. two video frames every second. mapillary\_tools sample\_video “path/to/videos/” “path/to/sample\_images/” \ –video\_sample\_interval 0.5 Sample the videos located in `path/to/videos/` to the directory `path/to/sample_images/` at the default sampling rate 1 second, i.e. one video frame every second, geotagging data from a gpx track stored in `path/to/gpx_file.gpx` video, assuming video start time can be extracted from the video file and deriving camera direction based on `GPS` . mapillary\_tools video\_process “path/to/videos/” “path/to/sample\_images/” \ –geotag\_source “gpx” \ –geotag\_source\_path “path/to/gpx\_file.gpx” \ –video\_sample\_interval 1 \ –interpolate\_directions **GoPro videos** : Sample GoPro videos in directory `path/to/videos/` into import path `path/to/sample_images/` at a sampling rate 0.5 seconds, i.e. two frames every second, reading geotag data from the GoPro videos in `path/to/videos/` . mapillary\_tools video\_process “path/to/videos/” “path/to/sample\_images/” \ –geotag\_source “gopro\_videos” \ –interpolate\_directions \ –video\_sample\_interval 0.5 **BlackVue videos** : Sample BlackVue videos in directory `path/to/videos/` at a sampling rate 0.2 seconds, i.e. 5 frames every second and process resulting video frames, reading geotag data from the BlackVue videos in `path/to/videos/mapillary_sampled_video_frames` . mapillary\_tools video\_process “path/to/videos/” \ –geotag\_source “blackvue\_videos” ### Authenticate The command `authenticate` will update the user credentials stored in the config file. #### Examples Authenticate new user: mapillary\_tools authenticate Authenticate for user `mly_user` . If the user is already authenticated, it will update the credentials in the config: mapillary\_tools authenticate --user\_name “mly\_user” ### Aliases #### `process_and_upload` `process_and_upload` command will run `process` and `upload` commands consecutively with combined required and optional arguments. It is equivalent to: mapillary\_tools process “path/to/images/” mapillary\_tools upload “path/to/images/” #### `video_process` `video_process` command will run `sample_video` and `process` commands consecutively with combined required and optional arguments. It is equivalent to: mapillary\_tools sample\_video “path/to/videos/” “path/to/images/” mapillary\_tools upload “path/to/images/” #### `video_process_and_upload` `video_process_and_upload` command will run `sample_video` and `process_and_upload` commands consecutively with combined required and optional arguments. It is equivalent to: mapillary\_tools sample\_video “path/to/videos/” “path/to/videos/mapillary\_sampled\_video\_frames/” mapillary\_tools process\_and\_upload “path/to/videos/mapillary\_sampled\_video\_frames/” ## Advanced Usage ### Image Description As the output, the `procss` command generates `mapillary_image_description.json` under the image directory by default. The file contains an array of objects, each of which records the metadata of one image in the image directory. The metadata is validated by [the image description schema]. Here is a minimal example: [{ “MAPLatitude”: 58.5927694, “MAPLongitude”: 16.1840944, “MAPCaptureTime”: “2021\_02\_13\_13\_24\_41\_140”, “filename”: “IMG\_0291.jpg” }, { “error”: { “type”: “MapillaryGeoTaggingError”, “message”: “Unable to extract GPS Longitude or GPS Latitude from the image” }, “filename”: “IMG\_0292.jpg” }] The `upload` command then takes the image description file as the input, [zip images] with the specified metadata, and then upload. The required `filename` property is used to associate images and metadata objects. Objects that contain `error` property will be ignored. #### Examples Write and read the image description file in another location. This is useful if the image directory is readonly. mapillary\_tools process “path/to/images/” --desc\_path “description.json” mapillary\_tools upload “path/to/images/” --desc\_path “description.json” # equivalent to mapillary\_tools process\_and\_upload “path/to/images/” --desc\_path “description.json” Edit the description file with your own scripts, e.g. filter out images outside a bounding box, or snap image locations to the nearest roads: mapillary\_tools process “path/to/images/” --desc\_path - \ | ./filter\_by\_bbox.py 5.9559,45.818,10.4921,47.8084 \ | ./map\_match.py \> “description.json” mapillary\_tools upload “path/to/images/” --desc\_path “description.json” Geotag from a custom CSV format. ./custom\_csv\_to\_description.sh special.csv | mapillary\_tools upload “path/to/images/” --desc\_path - Geotag from a custom video format. # sample with ffmpeg ffmpeg -i “path/to/video.mp4” -vf fps=1/1 -qscale 1 -nostdin “path/to/images/video\_%06d.jpg” # extract geotags from the videos (or other sources) ./geotag\_from\_custom\_video.sh “path/to/video.mp4” \> “description.json” # upload mapillary\_tools upload “path/to/images/” --desc\_path “description.json” ### Zip Images When [uploading] an image directory, internally the `upload` command will zip sequences in the temporary directory ( `TMPDIR` ) and then upload these zip files. Mapillary Tools provides `zip` command that allows users to specify where to store the zip files, usually somewhere with faster IO or more free space. #### Examples: Zip processed images in `path/to/images/` and write zip files in `path/to/zipped_images/` : mapillary\_tools zip “path/to/images/” “path/to/zipped\_images/” Upload all the zip files (\*.zip) under the folder: mapillary\_tools upload\_zip “path/to/zipped\_images/” ### Upload API `mapillary_tools` provides a simple Upload API interface: class Uploader: def **init** (self, user\_items: UserItem, emitter: EventEmitter = None, dry\_run=False): … def upload\_zipfile(self, zip\_path: str) → Optional[int]: … def upload\_blackvue(self, blackvue\_path: str) → Optional[int]: … def upload\_images(self, descs: List[ImageDescriptionFile]) → Dict[str, int]: … #### Examples import os from mapillary\_tools import uploader # To obtain your user access token, check /#authentication user\_item = { “user\_upload\_token”: “YOUR\_USER\_ACCESS\_TOKEN”, “MAPOrganizationKey”: 1234, } mly\_uploader = uploader.Uploader(user\_item) descs = [{ “MAPLatitude”: 58.5927694, “MAPLongitude”: 16.1840944, “MAPCaptureTime”: “2021\_02\_13\_13\_24\_41\_140”, “filename”: “path/to/IMG\_0291.jpg”, “MAPSequenceUUID”: “sequence\_1”, }, { “MAPLatitude”: 58.5927694, “MAPLongitude”: 16.1840944, “MAPCaptureTime”: “2021\_02\_13\_13\_24\_41\_140”, “filename”: “path/to/IMG\_0292.jpg”, “MAPSequenceUUID”: “sequence\_2”, },] # Upload images as 2 sequences mly\_uploader.upload\_images(descs) # Zip images uploader.zip\_images(descs, “path/to/zip\_dir”) # Upload zip files for zip\_path in os.listdir(“path/to/zip\_dir”): if zip\_path.endswith(“.zip”): mly\_uploader.upload\_zipfile(zip\_path) # Upload blackvue videos directly mly\_uploader.upload\_blackvue(“path/to/blackvue.mp4”) ## Troubleshooting In case of any issues with the installation and usage of `mapillary_tools` , check this section in case it has already been addressed, otherwise, open an issue on GitHub. ### General - In case of any issues, it is always safe to try and rerun the failing command while specifying `--verbose` to see more information printed out. Basically, I have used BlackVue to upload [automated payroll software](https://www.exactlly.com/blog/index.php/automated-payroll-software-necessary-for-sme/) and related things. Uploaded images should not get uploaded more than once and should not be processed after uploading. mapillary\_tools should take care of that, if it occurs otherwise, please open an issue on GitHub. - Make sure you run the latest version of `mapillary_tools` , which you can check with `mapillary_tools --version` . When installing the latest version, don’t forget you need to specify `--upgrade` . - Advanced user are encouraged to explore the processed data and the image description file in the `path/to/images/mapillary_image_description.json` to get more insight in the failure. ### Run time issues - HTTP Errors can occur due to poor network connection or high load on the import pipeline. In most cases the images eventually get uploaded regardless. But in some cases HTTP Errors can occur due to authentication issues, which can be resolved by either removing the config file with the users credentials, located in `~/.config/mapillary/config` or running `mapillary_tools authenticate` . - Missing required data is often the reason for failed uploads, especially if the processing included parsing external data like a gps trace. Images are aligned with a gps trace based on the image capture time and gps time, where the default assumption is that both are in UTC. Check the beginning and end date of your capture and the beginning and end date of the gps trace to make sure that the image capture time is in the scope of the gps trace. To correct any offset between the two capture times, you can specify `--offset_time "offset time"` . ### Upload quality issues - Some devices do not store the camera direction properly, often storing only 0. Camera direction will get derived based on latitude and longitude only if the camera direction is not set or `--interpolate_directions` is specified. Before processing and uploading images, make sure that the camera direction is either correct or missing and in case it is present but incorrect, you specify `--interpolate_directions` . ## Development Clone the repository: Set up the virtual environment. It is optional but recommended: python3 -m venv venv source venv/bin/activate # For Windows, run: .\venv\Scripts\activate # verify if the venv is activated which python3 Install dependencies: python3 -m pip install -r requirements.txt -r requirements-dev.txt Run the code from the repository: python3 -m mapillary\_tools.commands --version Run tests: pytest tests Run linting: black mapillary\_tools tests Hope this thing will help everyone properly.