What’s the opinion on re-correcting images for location accuracy ? What is an acceptable drift factor ?
I’ve just uploaded a whole bunch of 360° photos taken approx 1 second apart and they are now are appearing on the map. But I’ve noticed that they are lagging on one day roughly 20 seconds from the GPS location.
I was on a bike at the time, as you can see from this image of the overhead power grid on the right and the location on the map is roughly 20 shots to the map position.
Should I remove all my images for the day and re-process them with a 20 second time addition to correct the lag ? Or just leave them ?
Has anyone removed a whole sequence successfully and republished ?
Dear J Kingsley a,
1/ You’re correct in assuming that images which are clearly placed far from their location are best removed;
To help visualise the issue, as I’m not familiar with the area, in the bottom right corner selected Mapillary satellite imagery, picked a picture-dot on an intersection and clicked forward till I found an image depicting what looks like that intersection.
2/ To concentrate the mind : picture Mapillary seems to be taken where Mapillary is placed?
3/ That’d indeed be a 25 second lag.
4/ There are other series, Mapillary looks like it was placed correctly?
5/ The pic under 4/ was taken on 2021-03-27 13:34 , the ones under 2/ are more recent, 2021-06-06 09:53;
6/ As there are only partial details on the Camera make and model : would you be geotagging the pics with the GPS track log from an external receiver? If so, how do you determine the difference between the clocks in GPS and camera, please? (For my geotagging take three photos of a screen on the GPS showing the time including seconds at start and finish of the day’s journey).
7/ Removing a sequence can be found (once you’ve picked a pic in the sequence) under the three dots in the bottom right corner : … > Editing > Edit current sequence > Delete sequence; may take a while, could be that s/o at Mapillary reviews the request; have not had problem in the past re-publishing incorrectly placed sequence.
Good luck, happy cycling, and many safe returns.
Hey @koninklijke thanks for responding - and coming up with some great comments.
The 360 camera, a modified Insta360, doesn’t naturally maintain gps location with each shot so yes I’m using the first start shot time, adding a 30 frame/second difference to set the image time of the NEXT image and then geotagging the whole sequence. It has worked ok - But as you mention the clock speed on the camera over time appeared to have lagged, this happens when the camera stays in my bag. This lag is effectively reset when I communicate to the camera with the phone - so I’m got a new ‘startup’ procedure to ensure I communicate with the camera first, then I hit the buttons on the camera to start the video recording.
Also I’ve since updated my tagging scripts to allow adding a ‘fudge’ factor in the calculation of the tagging so I can + or - seconds of the image, before I map it with the geotagging gpx file.
I now also have started to take note of your point 1, and mark a intersection when I do some initial stills and noticed if I start the app first I gain approx +4 seconds on the image to the GPS location, so I can handle the drift more easily now.
Overall the recent server updates/privacy etc - it looks to my advantage that the servers have been reset and my June images have now gone yay me. I’m waiting a few days to see what happens and see if things come back before re-submitting my changed images.
If the bad ones come back - I’ll try and remove them.
I stumbled on this thread. I’ve noticed the issue randomly with some of my x5 footage. Sometimes it is accurate and good. In rare cases either the locations and images are off and I have deleted the sequence. Interestingly for at least one it went into Street view just fine. In Mapillary it was off by 10-30 seconds. In one instance my workflow was to turn on osmtracker before starting the recording. However this caused Mapillary seemingly to start mapping points as soon as osm was logging as opposed to when I started the recording of the video.
Can you say more about connecting camera to phone first? Do you sync the camera time with the phone time before you start?
@henryu - if you are uploading a stand-alone .gpx file (for example made with osmtracker), we will assume it is correct - so if you didn’t start recording video and osmtracker at exactly the same time, you will see an offset like this. Ideally you could upload the video/images from insta360 directly to avoid this (not osmtracker).
Desktop Uploader could give a great help if it will have possibility for user to select via UI the point on track where video was actually started. Now it links start of video to start of track without shift settings and without any control where each frame is placed (like it can be checked for photo). We should use 3rd party application to use more precise gpx track in video.
Why do not include such feature into Uploader to merry video with external gpx?
Okay, I think I understand now. Mapillary does not look at the start time of the GPS timestamps. It just looks at the first point of the track and assumes that was the video start. I will remember for next time.
It appears Google looks at the actual time stamps on the GPS tack to align the start point with the video. I mean assuming you synchronize your time, which you can do, that is the way to go because you can turn the camera on and off around your study area and just leave the logger running.
@zufir - great idea - we will add “mark video start point” as a feature request for the desktop uploader.
@henryu - yes, that is correct. Most videos that require external .gpx files do not actually have GPS tracks to align to, but some do. We will add a feature request to sync to these when available to our backlog.
If I could express one wish, it would be my dream that the Mapillary Desktop Uploader could not only process GPX files, but also connect to an ODBII tracker in the future. There are countless apps that offer driving time recording for digital logbooks. These even reliably record driving paths that lead through tunnels. GPX camera recordings that lead through tunnels regularly cause major problems. Who can free us from this problem? Logbooks have long since solved this problem.
@osmplus_org - the desktop uploader runs on PC, so it probably won’t be able to connect to an ODB tracker directly, but I think you’re already able to get a .gpx file from your odb tracker, right? (if I remember this thread correctly). So then I think what’s left is the syncing I referred to in my post above (syncing the .gpx track from odb to the gps satellite time from a video like a gopro .360 file). We’ve added this to the backlog!
Just FYI, Mapillay (and other tools) have already solved the problem of syncing GPX data with video data in Releases · mapillary/mapillary_tools · GitHub by adjusting GPX timestamps with a customizable time offset.
--interpolation_use_gpx_start_time
If supplied, the first image will use the first GPX point time for interpolation, which means the image location will be interpolated to the first GPX point too.
Only works for geotagging from gpx, gopro_videos, nmea, blackvue_videos, camm.
--interpolation_offset_time INTERPOLATION_OFFSET_TIME
Time offset, in seconds, that will be added for GPX interpolation, which affects image locations. Note that it is applied after --interpolation_use_gpx_start_time.
Only works for geotagging from gpx, gopro_videos, nmea, blackvue_videos, camm. [default 0.0]
--offset_time OFFSET_TIME
Time offset, in seconds, that will be added to your image timestamps after geotagging/interpolation. [default: 0.0]
Hence, it should be easy to add the necessary UI to Desktop Uploader to enable this functionality.
The interoperability protocol for ODB II trackers with Desktop Uploader is, like for every other tracking device, GPX. Hence, no need to “reinvent the wheel”.
Starting with a GoPro Hero8, I was very dissatisfied with the general GPX situation. An ODBII app, on the other hand, showed excellent tracks, even through tunnels that curve in the mountain. I managed to convert an ODBII export into a GPX file. The app I used generates running coordinates, a start time, and then only elapsed seconds. I managed to create a GPX track from ODBII driving data. I also managed to synchronize the GPX when the video started. But the disappointment came at the first traffic light. While Mapillary waits for the green light, the GPX position in Mapillary runs ahead of the images. That was the end of my efforts. I have now switched to an Insta360, whose remote control produces stable GPX tracks. These days I solve tunnels with the JOSM editor by first importing the GPX track into JOSM, where I then distribute GPX points evenly along the tunnel course. Mapillary accepts the GPX export created with JOSM without any problems. (Google M, however, refuses.) The result isn’t perfect, but at least in regions where there are already many GPX tracks near the tunnel, you can follow my recordings in a nice, clear path through the tunnel. If I didn’t correct the track, the accumulation of GPX points thrown out by a camera at the end of the tunnel would create a spaghetti monster.
It would be fantastic to use Mapillary ODBII data. You would probably have to open a separate data channel to use GPX recordings generated alongside the camera, which might require a collaboration with a company that offers other logbook solutions. So why not link a ready-made logbook app solution to a Mapillary account?
Just as a company with a GPX tracking application always knows where its employees are, how fast they’re driving, and which road they’re currently on. I would have no problem giving Mapillary this information independently of the actual video upload. If I want to remain undisturbed, I simply unplug the ODBII adapter from my vehicle.
In addition to the images, Mapillary would receive a lot of additional road telemetry data generated by the vehicle. Mapillary contributors are committed to making on-site information available to others. The prospects for linking vehicle data and 360-degree video recordings are certainly very promising.
Good, but in the end that does not seem like a task a human should do.
Might it be possible to use the SFM results to automatically determine the offset of the GPX? I would expect that SFM results in a location track with lots of shorter relative track segments, with many gaps where SFM did not find a correlation. This fragmented track then “just” needs to be correlated with the reference GPX track, and hopefully there is one offset where it fits best.
@enteq - I think we’ll need to start with a human here, it can maybe be theoretically possible to use computer vision to help, but that is a bit of a complex problem we likely won’t be targeting for this feature. In general though, if the video file has a GPS track as well, we will also auto-align that to the GPX, so no intervention should be necessary at all in these cases.
@tao This could potentially be done indeed by examining a video stream’s creation_time field, which should and usually does denote when video recording had started. Alternatively, the video file’s modification time could also be examined by counting the video’s length backwards. Unfortunately, due to sometimes bad or incomplete firmware, camera vendors do not always pay attention that creation_time timestamps must be in UTC, like for example GoPro with the MAX, which writes local time creation_time timestamps because the MAX does not support timezone settings. So, you would need to add some local versus UTC timezone detection (or probing) code, like JOSM does when correlating JPEGs to GPX tracks.
@gitne Yes, unfortunately it feels like it’s difficult to trust the creation_time (because it may depend on the camera time which is often set by the user, particularly for older cameras). However, we should be able to extract GPS time from video streams that include GPS data and that should be accurate and can be matched to .gpx files created on phones (which also generally have accurate time because these tend to be set by the cellular networks)
I am not sure I can follow completely. Videos that include GPS data do not need GPX correlation. You are basically ready to go, except maybe for the fact that afaik GPS data for video streams is not standardized. Most often rather, videos do not have GPS data or you cannot read it (because it is in a proprietary format or is broken) but you also usually have creation_time and file modification timestamps. In other words, all you need to do is to determine the video stream’s start timestamp when you have GPX data to correlate to.
Trusting timestamp sources and matching timezones is a subsequent issue. In general, you should be able to always trust GPX timestamps (and “timezone”). The correlation logic just needs to shift the video stream’s start timestamp back or forward within 24 hours until the video’s full length fits into a GPX track or multiple tracks (for cases where the GPX tracker lost GPS signal contact etc.). Have a look at JOSM’s timezone detection and correlation logic. It basically does what I am describing.
@GITNE - yes, I’m referring to cases where the user has an external GPX which is more accurate than the GPS data in the video (for example an external GPS receiver, or a GPX created with the help of ODBII).
Thank you for the reference to JOSM timezone detection and correlation.
yay me. I’m waiting a few days to see what happens and see if things come back before re-submitting my changed images.