Python tool for download gpx from sequences


#1

I’m writing a tool for download gps traces extracted from the sequences.

look the script here https://github.com/dalacost/mapillary_tools/blob/master/python/download_gpx_from_sequences.py

by the moment it works ok.

tell me if you would like to have a new functionality.


Mapillary layer in OsmAnd
#2

Nice tool!
However, I can’t open those tracks in Basecamp/Mapsource, I think Garmin requires a stricter format of GPX, but I dont know exactly how.
I can open those tracks in JOSM, and save it as gpx, but I have to do this for each single track. With several hundreds of tracks this is not possible. Can you write/append all tracks in one single gpx file?

This way I can convert it easier to Basecamp for my Garmin GPS with tools like Javawa’s IMGfromGPX or mkgmap.


#3

I tried with gpsbabel, but I can’t get a good output :weary: . I will add a option for paste all track in one file.


#4

hey! look my last script to join your gpx files in just one.

this scripts is a fast way to join a lot of GPX files downloaded from mapillary
if you use download_gpx_from_sequences.py , this is your second step for join
all in just one file.

use this script as:

python join_gpx_mapillary_files.py [gpx_directory] [outputfile.gpx]

ex: python join_gpx_mapillary_files.py gpx_from_sequences salida.gpx

https://github.com/dalacost/mapillary_tools/blob/master/python/join_gpx_mapillary_files.py

enjoy it.


#5

Thanks for the update Danilo,
I can now open it with Mapsource, so the structure is now a valid xml file.
But there is still one problem. It makes one big gpx track, so it connects the last point of the first file with the first point of the second file. This creates one big mess of straight lines.
Better is:

  <trk>
    <name>track 1</name>
    <trkseg>
      <trkpt lat="xx.x" lon="y.yy"/>
      <trkpt lat="xx.x" lon="y.yy"/>
    </trkseg>
  </trk>

  <trk>
    <name>track 2</name>
    <trkseg>
      <trkpt lat="xx.x" lon="y.yy"/>
      <trkpt lat="xx.x" lon="y.yy"/>
    </trkseg>
  </trk>

I have done some search and replace with

<trkseg> to <trk><trkseg> and </trkseg> to </trkseg></trk>

which works for me!


#6

With some effort I managed to convert the gpx file with Javawa’s IMGfromGPX tool to a transparent Garmin maplayer to use on my GPS, great! Now I can see easily what streets have to be done for Mapillary :sunglasses:


#7

Hi Danilo,

Another issue:
It seems the bbox that I specify does not work. For instance if I use this command to get all the sequences in my area,

python download_gpx_from_sequences.py 51.9 52.4 5 6 1000

It starts to download all sequences all over the world. I also tried to edit the parameters in the py file, like

MIN_LAT_G = 51.9
MAX_LAT_G = 52.4
MIN_LON_G = 5
MAX_LON_G = 6

But this doesnt work either, again the script begins to search for the whole world.


#8

I have found the cause, a bug in the code(line 52-54), you have ‘min-lat’, ‘max-lat’, ‘min-lon’, ‘max-lon’,
should be ‘min_lat’, ‘max_lat’, ‘min_lon’, ‘max_lon’,


#9

I repared this issues and works better now!! thanks for your comments and support. it’s amazing tool now.


#10

@dalacost Thanks for this, I actually wanted for a while now to have a gpx file of all sequences in my town, to use in osmand. It’s still not perfect, because osmand needs 2 minutes to render a 200.000 points gpx file, but it’s the most useful tool I have now to see unmapped places in osmand. Thanx!

edit: I’ve got a much better render time now I removed 90% of all nodes with josm.


#11

Thanks Danilo!
For Osmand I have now the following workflow

  1. open gpx file in Josm
  2. Select GPX and convert it to data
  3. Select all tracks (ctrl-A)
  4. Choose an existing OSM tag, which is not used in your area, in NL I can use piste:type=downhill
  5. Save the data layer in osm format, eg map.osm
  6. Convert map.osm to an obf osmand map with OsmandMapcreator, http://wiki.openstreetmap.org/wiki/OsmAndMapCreator
  7. Copy map.obf to the osmand working directory on your device

Result in wintersport style:


#12

@dalacost Maybe a nice improvement for your tool is to add a date from which you can start extracting the sequences. Now it will grab all sequences from an area, if I want to update this area later, I would like to see only the latest sequences that I didnt download the last time. By giving a date range you can also see which sequences are rather old and needed to be updated in the field. Something like http://tangrams.github.io/mapillary-explorer
with min and max date


#13

that’s is a great idea. I will try to learn how to do that.


#14

thanks for patience, new version is finish. I added support for limit dates as you requested, and other improvements. (estimated time, and a new comfortable way to use options as unix style)

check the last code here: https://github.com/dalacost/mapillary_tools/blob/gpx_downloader/python/download_gpx_from_sequences.py

actually is in a pullrequest for mapillary tools :smile:

enjoy it.


#15

Thanks for implementing these features!
I did a test but unfortunately it failed.

My command is:
python download_gpx_from_sequences.py -r 51.9 53 4.86 7.1 -m 19999 -d1 20151030

This is the error message, what does it mean?

Traceback (most recent call last):
File “download_gpx_from_sequences.py”, line 182, in
limit_start_time = int(datetime.datetime(int(limit_start_time[:4]),int(limit_start_time[4:6]),int(limit_start_time[6:8]),0,0).strftime(’%s’)) * 1000
ValueError: Invalid format string


#16

i’m not sure. i can’t get your problem… :frowning:

dalacost@dalacost-desktop:~/Descargas$ wget https://raw.githubusercontent.com/dalacost/mapillary_tools/gpx_downloader/python/download_gpx_from_sequences.py
--2015-11-05 12:48:47--  https://raw.githubusercontent.com/dalacost/mapillary_tools/gpx_downloader/python/download_gpx_from_sequences.py
Resolviendo raw.githubusercontent.com (raw.githubusercontent.com)... 199.27.79.133
Conectando con raw.githubusercontent.com (raw.githubusercontent.com)[199.27.79.133]:443... conectado.
Petición HTTP enviada, esperando respuesta... 200 OK
Longitud: 7881 (7,7K) [text/plain]
Grabando a: “download_gpx_from_sequences.py”

100%[==================================================================================================================================================>] 7.881       --.-K/s   en 0s      

2015-11-05 12:48:47 (904 MB/s) - “download_gpx_from_sequences.py” guardado [7881/7881]

dalacost@dalacost-desktop:~/Descargas$ python download_gpx_from_sequences.py -r 51.9 53 4.86 7.1 -m 19999 -d1 20151030
Warning: USER NOT SPECIFIED - NOT A GOOD IDEA, trying to list last gpx from all users
Getting the files list...
Result: 11 sequences in area.
Log file at gpx_from_sequences/downloaded_20151105.txt
Successfully downloaded: 20151105_y6buq8NEZfol-StqN9ZLeg.gpx 1/11 ET ...
Successfully downloaded: 20151103_VPDf7drUMahl8v19asIG4w.gpx 2/11 ET ...


dalacost@dalacost-desktop:~/Descargas$ python -V
Python 2.7.6

#17

Hi,
I’m using Windows 7 / 64 bits pc, maybe the date notation is different on each OS?


#18

ok, thanks for report it. I will check and back to you.


#19

problem was resolved.
Thanks to ligfietsen for help me on this.