Python tool for download gpx from sequences

dalacost · October 15, 2015, 8:02pm

I’m writing a tool for download gps traces extracted from the sequences.

look the script here mapillary_tools/download_gpx_from_sequences.py at master · dalacost/mapillary_tools · GitHub

by the moment it works ok.

tell me if you would like to have a new functionality.

ligfietser · October 16, 2015, 5:35pm

Nice tool!
However, I can’t open those tracks in Basecamp/Mapsource, I think Garmin requires a stricter format of GPX, but I dont know exactly how.
I can open those tracks in JOSM, and save it as gpx, but I have to do this for each single track. With several hundreds of tracks this is not possible. Can you write/append all tracks in one single gpx file?

This way I can convert it easier to Basecamp for my Garmin GPS with tools like Javawa’s IMGfromGPX or mkgmap.

dalacost · October 16, 2015, 8:59pm

I tried with gpsbabel, but I can’t get a good output . I will add a option for paste all track in one file.

dalacost · October 17, 2015, 12:38am

hey! look my last script to join your gpx files in just one.

this scripts is a fast way to join a lot of GPX files downloaded from mapillary
if you use download_gpx_from_sequences.py , this is your second step for join
all in just one file.

use this script as:

python join_gpx_mapillary_files.py [gpx_directory] [outputfile.gpx]

ex: python join_gpx_mapillary_files.py gpx_from_sequences salida.gpx

github.com

dalacost/mapillary_tools/blob/master/python/join_gpx_mapillary_files.py

import sys
import glob, os

'''
this scripts is a fast way to join a lot of GPX files downloaded from mapillary
if you use download_gpx_from_sequences.py , this is your second step for join 
all in just one file. 

use this script as: 

python join_gpx_mapillary_files.py [gpx_directory] [outputfile.gpx]

ex: python join_gpx_mapillary_files.py gpx_from_sequences salida.gpx

by danilo@lacosox.org
15/10/2015 

'''

HEAD_FILE="<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\" ?>\n \

This file has been truncated. show original

enjoy it.

ligfietser · October 17, 2015, 7:02am

Thanks for the update Danilo,
I can now open it with Mapsource, so the structure is now a valid xml file.
But there is still one problem. It makes one big gpx track, so it connects the last point of the first file with the first point of the second file. This creates one big mess of straight lines.
Better is:

  <trk>
    <name>track 1</name>
    <trkseg>
      <trkpt lat="xx.x" lon="y.yy"/>
      <trkpt lat="xx.x" lon="y.yy"/>
    </trkseg>
  </trk>

  <trk>
    <name>track 2</name>
    <trkseg>
      <trkpt lat="xx.x" lon="y.yy"/>
      <trkpt lat="xx.x" lon="y.yy"/>
    </trkseg>
  </trk>

I have done some search and replace with

<trkseg> to <trk><trkseg> and </trkseg> to </trkseg></trk>

which works for me!

ligfietser · October 17, 2015, 7:49am

With some effort I managed to convert the gpx file with Javawa’s IMGfromGPX tool to a transparent Garmin maplayer to use on my GPS, great! Now I can see easily what streets have to be done for Mapillary

ligfietser · October 17, 2015, 10:59am

Hi Danilo,

Another issue:
It seems the bbox that I specify does not work. For instance if I use this command to get all the sequences in my area,

python download_gpx_from_sequences.py 51.9 52.4 5 6 1000

It starts to download all sequences all over the world. I also tried to edit the parameters in the py file, like

MIN_LAT_G = 51.9
MAX_LAT_G = 52.4
MIN_LON_G = 5
MAX_LON_G = 6

But this doesnt work either, again the script begins to search for the whole world.

ligfietser · October 17, 2015, 11:10am

I have found the cause, a bug in the code(line 52-54), you have ‘min-lat’, ‘max-lat’, ‘min-lon’, ‘max-lon’,
should be ‘min_lat’, ‘max_lat’, ‘min_lon’, ‘max_lon’,

dalacost · October 17, 2015, 1:23pm

I repared this issues and works better now!! thanks for your comments and support. it’s amazing tool now.

de_vries · October 17, 2015, 2:27pm

@dalacost Thanks for this, I actually wanted for a while now to have a gpx file of all sequences in my town, to use in osmand. It’s still not perfect, because osmand needs 2 minutes to render a 200.000 points gpx file, but it’s the most useful tool I have now to see unmapped places in osmand. Thanx!

edit: I’ve got a much better render time now I removed 90% of all nodes with josm.

ligfietser · October 17, 2015, 3:21pm

Thanks Danilo!
For Osmand I have now the following workflow

open gpx file in Josm
Select GPX and convert it to data
Select all tracks (ctrl-A)
Choose an existing OSM tag, which is not used in your area, in NL I can use piste:type=downhill
Save the data layer in osm format, eg map.osm
Convert map.osm to an obf osmand map with OsmandMapcreator, OsmAndMapCreator - OpenStreetMap Wiki
Copy map.obf to the osmand working directory on your device

Result in wintersport style:

ligfietser · October 29, 2015, 9:02am

@dalacost Maybe a nice improvement for your tool is to add a date from which you can start extracting the sequences. Now it will grab all sequences from an area, if I want to update this area later, I would like to see only the latest sequences that I didnt download the last time. By giving a date range you can also see which sequences are rather old and needed to be updated in the field. Something like Mapillary Explorer
with min and max date

dalacost · October 29, 2015, 12:55pm

that’s is a great idea. I will try to learn how to do that.

dalacost · November 5, 2015, 4:24am

thanks for patience, new version is finish. I added support for limit dates as you requested, and other improvements. (estimated time, and a new comfortable way to use options as unix style)

check the last code here: mapillary_tools/download_gpx_from_sequences.py at gpx_downloader · dalacost/mapillary_tools · GitHub

actually is in a pullrequest for mapillary tools

enjoy it.

ligfietser · November 5, 2015, 3:26pm

Thanks for implementing these features!
I did a test but unfortunately it failed.

My command is:
python download_gpx_from_sequences.py -r 51.9 53 4.86 7.1 -m 19999 -d1 20151030

This is the error message, what does it mean?

Traceback (most recent call last):
File “download_gpx_from_sequences.py”, line 182, in
limit_start_time = int(datetime.datetime(int(limit_start_time[:4]),int(limit_start_time[4:6]),int(limit_start_time[6:8]),0,0).strftime(‘%s’)) * 1000
ValueError: Invalid format string

dalacost · November 5, 2015, 3:52pm

i’m not sure. i can’t get your problem…

dalacost@dalacost-desktop:~/Descargas$ wget https://raw.githubusercontent.com/dalacost/mapillary_tools/gpx_downloader/python/download_gpx_from_sequences.py
--2015-11-05 12:48:47--  https://raw.githubusercontent.com/dalacost/mapillary_tools/gpx_downloader/python/download_gpx_from_sequences.py
Resolviendo raw.githubusercontent.com (raw.githubusercontent.com)... 199.27.79.133
Conectando con raw.githubusercontent.com (raw.githubusercontent.com)[199.27.79.133]:443... conectado.
Petición HTTP enviada, esperando respuesta... 200 OK
Longitud: 7881 (7,7K) [text/plain]
Grabando a: “download_gpx_from_sequences.py”

100%[==================================================================================================================================================>] 7.881       --.-K/s   en 0s      

2015-11-05 12:48:47 (904 MB/s) - “download_gpx_from_sequences.py” guardado [7881/7881]

dalacost@dalacost-desktop:~/Descargas$ python download_gpx_from_sequences.py -r 51.9 53 4.86 7.1 -m 19999 -d1 20151030
Warning: USER NOT SPECIFIED - NOT A GOOD IDEA, trying to list last gpx from all users
Getting the files list...
Result: 11 sequences in area.
Log file at gpx_from_sequences/downloaded_20151105.txt
Successfully downloaded: 20151105_y6buq8NEZfol-StqN9ZLeg.gpx 1/11 ET ...
Successfully downloaded: 20151103_VPDf7drUMahl8v19asIG4w.gpx 2/11 ET ...


dalacost@dalacost-desktop:~/Descargas$ python -V
Python 2.7.6

ligfietser · November 5, 2015, 4:55pm

Hi,
I’m using Windows 7 / 64 bits pc, maybe the date notation is different on each OS?

dalacost · November 6, 2015, 4:21am

ok, thanks for report it. I will check and back to you.

dalacost · November 6, 2015, 12:52pm

problem was resolved.
Thanks to ligfietsen for help me on this.

Richlv · December 29, 2022, 11:15am

With the Mapillary API v4, all the old tools broke.
Here’s a quick Perl script to get GPX sequence[s]. For convenience, a single image ID can be specified from each sequence.
https://www.openstreetmap.org/user/richlv/diary/400630

For my own use, I’d love to filter by user, camera and date in a bbox, but Mapillary API got so seriously nerfed with v4…

Topic		Replies	Views
Download GPX tracks Web and notifications	8	2345	November 24, 2017
Online GPX editor Contributing and equipment	6	3336	August 21, 2022
Python script for getting sequence details Contributing and equipment	2	831	December 29, 2022
Download GPX of my sequences Web and notifications	13	1639	December 29, 2022
Command Line Tools query (video process/upload) Contributing and equipment	3	686	October 12, 2019

Python tool for download gpx from sequences

Related topics