I have a large street imagery dataset that consists of cubemap images per datapoint (six separate images pointing front, back, left, right, up, down).
I’d like to contribute these to Mapillary.
AI has suggested using mapillary_tools for upload and preparing the data by simply setting the exif properties GPSLatitude and GPSLatitude to the same location for each of the six images and setting exif property GPSImgDirection to 0 for the forward facing image, 90 for right facing, 180 for back facing, etc. According to AI the images would then be correctly stitched together by Mapillary to a panorama image.
Now I would like to verify this but I couldn’t find any documentation on the topic.
Could somebody confirm that this workflow is correct?
No, what I would like is one sequence of panoramic images.
So you’re saying that in order to get a single sequence of panoramic images I need to merge the cubemap images of each datapoint into a panoramic image and then upload these?
I imagine Mapillary will process my images asynchronously before I can see the results on the map. Trial and error seems like a very inefficient approach to getting things right. Is there no documentation on which EXIF attributes need/should be set and their effect on processing? All the documentation I could find describes basic usage of mapillary_tools or the desktop client. I’m looking for something detailed like the Mapillary API docs but regarding data upload.
Someone please tell me if I am wrong, but the examples I have seen of multi camera views from the same origin do not “stitch” into one image, but can be step panned left and right on the GUI.
I suspect that pre-processing down to one panoramic image per origin would be needed.
(IMO) Also worth a mention that end usage should also be a consideration. Object detection and stepping down a street adding OSM info manually do far better with a single camera direction view, so it may even be better to upload them as separate sequences.
Any chance you could give me the EXIF data of such a set of images (that appear stitched together). I tried downloading the images you linked from Mapillary but the EXIF is gone.
I’d like to verify what properties need to be set in order to achieve the (almost) stitching together.
Building on what bob said, I’d rather not preprocess them into a single panoramic image because besides object detection doing worse I would also have to downscale the images (the panorama would have over 250 megapixels). I’d really like some opinion of the Mapillary team on this.
It seems most things have been answered during my sleep! Just a few remarks then;
I use the tools to upload EXIF annotated images from two (cellphone) cameras that face in slightly different directions. The image saved has all the regular phone EXIF data, but I don’t use the inbuilt camera GPS. I use the native GPS direction (from a UBLOX USB device) to populate all Exif.GPSInfo.GPSTrack (and all other GPS data with the default exiftool geotag options) then use the tools to set the angle offset. Since the image time is important for geo-encoding I actually name the files as a UTC date/time and “rename” for the camera processing delay, also incorporating subsecond accuracy. I have multiple uses for the images, so fully write the EXIF from the tools as well as well eg;
The other cell camera is at 270 degrees and a different model. I don’t know if Mapillary ever use the brand/model etc, but I think it good practice to include.
I vaguely remember I had problems with all GPS related EXIF fields populating with a GPX input, that worked fine with NMEA. I think it make have been speed and/or altitude missing, but did not try to hunt the problem down. My GPS sender (UBLOX) also runs at 5 posn/sec and has its movement type modified from “mostly stationery” to mobile.
I am not suggesting you use the exiftool method as such. The tools will work just as well with a --offset_angle, some method of time/image sync and the GPS track data input. You may elect to use the raw GPS stream direction or ask the tools to ‑‑interpolate_directions as already mentioned. There are a few options here more or less dependent on what you already have in the image data and what else you may want to do with it. I personally scroll through all the (day’s) images locally, so EXIF fill is needed to paste the co-ords direct to the OSM editor and display properly on Viking.
Of importance, unless something has changed recently, Mapillary does not handle up and down tilt. It kind of assumes all imagery is ground parallel and approx 1-2m above. You might have to discard these ones or once again look at pre-stitching.
I’d be interested in seeing your results, so please append to this thread so we can have a look!
I tagged each image with GPSLatitudeRef, GPSLatitude, GPSLongitudeRef, GPSLongitude and DateTimeOriginal. The four directional images of a datapoint have the same GPS and DateTime values. Along the sequence DateTime values are strictly monotonically increasing. I used –interpolate_directions together with –offset_angle = [0, 90, 180, 270]. This seems to work very well, the direction in the webviewer seems correct to me. It’s been a few days now since I’ve uploaded and so far all four sequences appear on the map and they show the controls to rotate on the spot so the four images seem correctly grouped together.
What is still missing is combined panning of all four images. I wonder if this simply takes more time or whether the problem is that my images have no overlap (as the images in a cubemap should not have any overlap). I hope that overlap is not a requirement for combined panning to work. I would rather not introduce artificial overlap in the images because it makes the images quite useless for further processing (e.g. projecting into an equirectangular panorama). Besides, it seems unnecessarily complex to match image edges via overlap if all four images have exactly the same position and a correct direction set.
Can somebody provide a definitive answer whether image overlap is needed for combined panning?
Edit: Okay, I just noticed that combined panning does in fact already work for some of the images along the sequence, I guess it just takes more processing time! It’s not as smooth as the linked example with overlapping images but it does the trick.
IME one of the giveaways for knowing that processing is still going in is the lack of morphing or smooth transition between images in the same sequence. When I viewed your test, some did that and some did not. Will view again later today.
