Want to get some ideas and feedback. I am creating and periodically uploading 10,000-30,000 images per day as I drive random directions in Australia.
I video_process in the vehicle and save to a USB hard disk. Then at intervals between 2 and 12 weeks I connect the drive to a PC in a fixed location and remotely run upload.
I run a virtual environment on Debian 8.
I have been getting thread crashes on various occasions, requiring 30 or more restarts for one days image.
Is anyone else running a setup similar to this? I realize a lot of the development is still in beta but it would be nice to pool ideas.
I’ll piggyback onto this thread since the title fits so well. I just got the DR900 and would like to understand an overview of the processing flow.
From your message, I assume that the first step is to apply video_process to the MP4 files and the result is separate Geo-located photos. Then upload those photos just as if they were acquired in a Smart device.
The tools do have the capability of processing and uploading with one command, but I do it separately. It is also useful to be able to delete some of the images prior to upload. (eg private places)
the sub commands are “video_process” or “video_process_and_upload” It is all written in Python and I execute from Linux. I notice some are also running under Windows.
It does take a long time to convert the mp4’s to jpg’s. On my i7 it takes about 90 minutes for 3 hours of capture at 2FPS.
There is/was also a direct mp4 upload facility that I think is close to being enabled again.
I have also perfected hot WiFi transfer of the mp4’s from the DR900 to a laptop whilst capture runs concurrently. I was concerned about popping the SD card out every day.
I setup a shell script on the Linux machine that connects to the BlackVue WiFi at 10.99.77.1. It also has the ability to do the tools process step, but is commented out. More error correction could be built in but I haven’t bothered. One concern is to repeat a partial download rather than wait for it to finish manually.
It connects at (only) 72Mbit/sec (802.11n) which means about 30-40 seconds for each one minute movie. The BlackVue setup has to be modified for WiFi always on. (Unless one wants to push the button at startup)
while :
do
# Write current list of todays files already saved to large drive
find . -name “*.mp4” | cut -c 2-26 | grep $BVDATE | sort >filesondrive.txt
# Write current list of todays files on the BlackVue SDRAM
curl http://10.99.77.1/blackvue_vod.cgi | sed 's/^n://' | sed 's/F.mp4//' | sed 's/R.mp4//' | sed 's/,s:1000000//' | sed $'s/\r//' | grep $BVDATE | sort | head -n -1 >filesonblack.txt
# Compare file lists and output the next in line to be transferred
F2G="`diff --suppress-common-lines -n filesondrive.txt filesonblack.txt| grep Record | grep -v 000000 | head -n1`"
cd Record
if [ "$F2G" = "" ] ; then
echo no-file-op
else
# Grab mp4
wget -cN http://10.99.77.1$F2G\F.mp4
# Insert background launch of mapillary_tools processing for this video
#mkdir -p /home/user/Thing/Mapillary/$F2G\F/images
#nice mapillary_tools video_process --import_path /home/user/Thing/Mapillary/$F2G\F/images/ --video_import_path /media/user/MapDisk2/Record/$F2G\F.mp4 --user_name bob3bob3 --advanced --geotag_source "blackvue_videos" --geotag_source_path /media/user/MapDisk2/Record/$F2G\F.mp4 --use_gps_start_time --interpolate_directions --video_sample_interval 0.5 --device_make "Blackvue" --device_model "DR900S-1CH" --overwrite_EXIF_gps_tag --duplicate_distance 1.0 >>/home/user/Thing/Mapillary/logs/bv$F2G\F.log 2>&1 &
fi
cd ..
sleep 5
I had the same concern and found a BlackVueDownloader. It worked, but I had visions of coming home and having it connect to WiFi while parked and I would upload all new stuff. But it takes more than a minute to upload a minute’s video clip. So for now I’ll estimate having a full SD card and dump it that way.
Off topic - I’m soldiering through installing all the conflicting requirements to get mapillary_tools running:
pip cannot import name ‘main’
Python 3 vs Python 2 conflicts, mapillary_tools needs 2.7, then a stern warning that Python 2.7 will reach end of life
Everything was resolved, but it seems more complicated than it should be.
I only have about 40cm between the BlackVue and laptop so the signal is quite good. Even at the speed I get though it is only useful if I download whilst driving. The laptop also captures a webcam and runs a 2nd GPS so it is always available for that.
I would guess that the BlackVue Wifi rate has been throttled for other reasons.
It looks like I have the first process working to convert to Geotagged JPG. I notice that when I submit the rear camera view, it is also processed. However I think those are not as useful:
Lower resolution
Most signs on the near side of the road would face the other way
The conversion script calculates the same camera heading (which is strictly correct since that’s the way it’s moving), but as a symbol on the image browse, it should face the other way? Would it combine as part of a 360 view or just confuse things?
–offset_angle OFFSET_ANGLE
offset camera angle (90 for right facing, 180 for rear
facing, -90 for left facing)
should help.
The full nmea dump is actually saved inside the mp4, including speed and bearingf The tools however seem to only decode the GPGGA sentence (that includes altitude), not GPRMC. If the WiFi download method is used there is a pseudo nmea text file available (*.gps) for each movie too.
im use this (windows10):
F:\BlackVue>mapillary_tools video_process --import_path "F:\BlackVue\Record\test\images" --video_import_path “F:\BlackVue\Record\test*.mp4” --user_name velmyshanovnyi --advanced --geotag_source “blackvue_videos” --geotag_source_path “F:\BlackVue\Record\test*.mp4” --use_gps_start_time --interpolate_directions --video_sample_interval 0.1 --device_make “Blackvue” --device_model “DR900S-1CH” --overwrite_EXIF_gps_tag --duplicate_distance 0.1
ERROR:
usage: see -h for available tools and corresponding arguments, add --advanced to see additional advanced tools and/or arguments and --version to see version.
Mapillary import tool: error: argument tool: invalid choice: ‘video_process’ (choose from ‘process’, ‘upload’, ‘process_and_upload’)
I am also concerned about popping the SD card all the time. Download by Wifi over night should be fine. But no feature like this from Blackvue or Mapillary (yet?).
I’d suspect Mapillary is working on a much wider project and would not develop/release code for such a slow xfer method. In their blog they mention custom BlackVue firmware, so they are probably using BlackVue’s SDK. Have a look at the BlackVue website and there is a note for truck/fleet operators sending event/location images to a remote server using the SDK. There are a lot of possibilities!
I am happy to assist with implementing with the forum. The code is above. I found most of it by web searching.
The method you describe are for local sampling of images.
To upload videos to Mapillary, videos are sampled into images and tagged with image GPS and capture time . Use the send_videos_for_processing command for Blackvue cameras. For all other models use Video Sampling and Upload
The recommendation for BlackVue is to send the video directly to Mapillary. Your uploads will run faster and you will reduce your local storage requirements.
BlackVue
Direct Upload (Recommended)
Upload videos located in path/to/videos directly to Mapillary. Videos are moved to path/to/videos/uploaded folder after upload. Videos that do not contain valid imagery are not uploaded to minimize bandwidth usage. Sampling is performed in the cloud so no extra disk space is required. This command supports the front camera video from the Blackvue DR900s (1-channel and 2-channel) models.
How did you make the WiFi work? The BlackVue app doesn’t work with the Mapillary firmware, and the config
file seemed to just have some salted hash for the WiFi password. Don’t think I’ve ever even seen it host an SSID while running.
I modified the config.ini text file on the SD card manually. I may have copied the ap_pw info from the standard 30fps version. onstart should be set to 1 or just use the side pushbutton. In fact try the the side button first as the default action is to leave WiFi on if in use.
