[šŸ›] `mapillary_tools` Extracts Incorrect `SubSecTime`

How to reproduce:

  1. Create images with SubSecTime* Exif tags
  2. Run mapillary_tools process on the files
  3. Compare the image fileā€™s SubSecTime* Exif tag values with the MAPCaptureTime values in mapillary_image_description.json
$ exiv2 -pv -g Date -g Time original.jpg
0x0132 Image        DateTime                    Ascii      20  2025:02:10 12:56:23
0x9003 Photo        DateTimeOriginal            Ascii      20  2025:02:10 12:56:23
0x9004 Photo        DateTimeDigitized           Ascii      20  2025:02:10 12:56:23
0x9010 Photo        OffsetTime                  Ascii       7  +01:00
0x9011 Photo        OffsetTimeOriginal          Ascii       7  +01:00
0x9012 Photo        OffsetTimeDigitized         Ascii       7  +01:00
0x9290 Photo        SubSecTime                  Ascii       7  626625  # mapillary_tools reads 900?
0x9291 Photo        SubSecTimeOriginal          Ascii       7  626625  # mapillary_tools reads 900?
0x9292 Photo        SubSecTimeDigitized         Ascii       7  626625  # mapillary_tools reads 900?
0x0007 GPSInfo      GPSTimeStamp                Rational    3  11/1 56/1 23/1
0x001d GPSInfo      GPSDateStamp                Ascii      11  2025:02:10
{
  "filename": "original.jpg",
  "md5sum": "7dc48ab8d7be0fa21dbc1dc1c6aad02e",
  "filesize": 573465,
  "filetype": "image",
  "MAPLatitude": 0.1234567,
  "MAPLongitude": 0.1234567,
  "MAPCaptureTime": "2025_02_10_11_56_23_900", // compare 900ms to 626.625ms!
  "MAPCompassHeading": {
    "TrueHeading": 351.336,
    "MagneticHeading": 351.336
  },
  "MAPSequenceUUID": "0",
  "MAPOrientation": 1
}

Should mapillary_tools process the GPSDateStamp & GPSTimeStamp only then in this example the MAPCaptureTime value should end in _000. Either way, something is off.

cc: @tao - could you take a quick look?

@tao A closer :face_with_monocle: look reveals that mapillary_tools does not read SubSecTime* at all and just assumes all files with identical DateTime* values (within 1 second) to basically have identical timestamps. So, it just appends _100, _200, _300, and so on for every next file with the same DateTime* value. :face_with_diagonal_mouth:

When both GPSTimeStamp and DateTimeOriginal are found in an image, mapillary_tools will prefer GPSTimeStamp. This is by design.

SubSecTime* are not designed to be applied for GPS timestamps (see EXIF Tags). Also it will cause some offset issue, here is an illustration of the potential issue:

# Assume the capture order is as follows:

Image 1:

DateTimeOriginal: 2025:02:10 12:56:23
SubSecTimeOriginal: 900000
GPSTimeStamp: 11/1 56/1 23/1

Image 2: # captured after 0.2 seconds

DateTimeOriginal: 2025:02:10 12:56:24
SubSecTimeOriginal: 100000
GPSTimeStamp: 11/1 56/1 23/1 (remained unchanged here because only 0.2 difference)

@GITNE Proposal workaround is to add your subseconds to GPSTimeStamp directly, e.g. 11/1 56/1 23.626625

The proposal fix is to sort images by both (GPSTimeStamp, DateTimeOriginal with SubSecTimeOriginal) and then interpolate sub-seconds. Similar issue here Broken image ordering for OpenCamera subsecond sequences Ā· Issue #677 Ā· mapillary/mapillary_tools Ā· GitHub see the discussion.

1 Like

I see, good catch @tao. So, mapillary_tools reads GPSTimeStamps with top precedence and compares these (only), which is generally the right thing to do. However, I think we could add a special case here. If DateTimeOriginal+OffsetTimeOriginal is equal in value to GPSDateStamp+GPSTimeStamp then mapillary_tools should read DateTimeOriginal+SubSecTimeOriginal+OffsetTimeOriginal for processing. Note that in this case you must not assume SubSecTimeOriginal == 0 should SubSecTimeOriginal be lacking. Whenever GPSTimeStamp has fraction seconds it will never be equal to DateTimeOriginal+OffsetTimeOriginal due to DateTimeOriginalā€™s 1 second limited resolution or granularity. Hence, GPSTimeStamp with subā€‘seconds should be assumed safer or closer to the truth for processing. I hope this makes sense?

Broken image ordering for OpenCamera subsecond sequences Ā· Issue #677 Ā· mapillary/mapillary_tools Ā· GitHub

I am not sure that sorting by both, GPSDateStamp+GPSTimeStamp and DateTimeOriginal+SubSecTimeOriginal+OffsetTimeOriginal, instead of by precedence only is a good strategy because GPS and DateTimeOriginal(+SubSecTimeOriginal+OffsetTimeOriginal) timestamps can deviate greatly. Maybe there is something I am unable to foresee here or I do not fully understand what you mean by ā€œto sort images by bothā€.

Alternatively, you could add an option that would enable hinting to mapillary_tools the timestamp source to use. Or, make the precedence configurable with a file (but this would be a lot of work). Anyhow, I think that for the ease of use mapillary_tools needs a sane default timestamp source precedence with proper subā€‘second support, which admittedly may be more complex than one would think at first glace. So, letā€™s call it more fashionably ā€œSmart Timestamp Source Selectionā€ (since, you know, everything has to be smart today to sell).:wink:

@GITNE Proposal workaround is to add your subseconds to GPSTimeStamp directly, e.g. 11/1 56/1 23.626625

It should be OK to do if GPSDateStamp+GPSTimeStamp and DateTimeOriginal+OffsetTimeOriginal are of equal value.

However, I think we could add a special case here. If DateTimeOriginal +OffsetTimeOriginal is equal in value to GPSDateStamp +GPSTimeStamp then mapillary_tools should read DateTimeOriginal +SubSecTimeOriginal +OffsetTimeOriginal for processing.

This is a great suggestion. We will take it into consideration. Thank you.

I am not sure that sorting by both

Sorting by both is just a better estimate for the order for the images. Ordering is more important than timestamp absolute accuracy because it affects both processing and visualization.

Anyhow, I think that for the ease of use mapillary_tools needs a sane default timestamp source precedence with proper subā€‘second support

Yeah I agree, we need this ā€œSmart Timestamp Source Seletionā€ by considering all possible clock sources. We can make it better. Before we get there, try exiftool to make sure the preferred tags are correctly updated.

1 Like