New Undistortion feature introduces distortion


#1

So Mapillary posted on the blog that they’ve started applying undistortion to the photos to correct for fisheye and whatnot (https://blog.mapillary.com/update/2018/10/04/mjs-undistortion.html). Great idea when/if it works.

But I noticed lately that some of my photos seemed weird. It turns out that what were perfectly fine images are now distorted after this “fix”: https://www.mapillary.com/app/?lat=38.39570889819245&lng=-82.41374516752578&z=17&focus=photo&pKey=y93MPsXiSJVTe0Fb3gAG1Q&menu=false

For comparison, here is the unprocessed version: https://s3-eu-west-1.amazonaws.com/mapillary.private.images/y93MPsXiSJVTe0Fb3gAG1Q/uploaded.jpg?AWSAccessKeyId=AKIAILU27ZWSOZX2FZ7Q&Expires=1540605404&Signature=Pilka1jqLzJwmc7kOFljW5JuJnw%3D

For reference, the camera used for this photo was a Xiaomi Yi with the setting to straighten photos turned on in-camera. I thought since this new feature uses Structure from Motion that it would recognize the lines properly and not distort it so heavily, but maybe Mapillary is using some standard undistortion per camera? And not checking/detecting the in-camera adjustment?

For comparison, here is a photo taken with a GoPro, also Undistorted (drag to look at the edges), but in this case it looks great: https://www.mapillary.com/app/?lat=38.4062428884026&lng=-82.4633746035314&z=18.713112398644466&focus=photo&pKey=jU5SOFzfyFKfxkiMtn771g&menu=false&dateFrom=2018-10-01&x=0.47044292827019496&y=0.6006917835836577&zoom=0

And the unprocessed: https://s3-eu-west-1.amazonaws.com/mapillary.private.images/jU5SOFzfyFKfxkiMtn771g/uploaded.jpg?AWSAccessKeyId=AKIAILU27ZWSOZX2FZ7Q&Expires=1540605674&Signature=qBPACJQma%2B7MdSn%2BEQbVr5xJHws%3D

In this case, it started distorted and was fixed. The GoPro was set to the photo settings that Mapillary recommends, too.

EDIT: Okay, seriously, what is this? https://www.mapillary.com/app/?lat=38.394964143634866&lng=-82.4152841493405&z=17.902020646265147&focus=photo&pKey=inFrYQs7yENm7MGfKv30jQ&x=0.5134005317081577&y=0.45625918782421904&zoom=0

Can we get a toggle to disable this mode like in the JS viewer?


#2

Hi @pkoby,
Thanks for pointing out the bad stretching. As you say, the feature is great when it works but can stretch images in weird ways when the calibration of the camera is wrong.

I’ve updated the calibration of the Xiaoyi camera used for these images and the problem is now solved for that camera.

Let me say some words of what happened and why, since similar issues can happen with other action cameras.

Action cameras use fisheye lenses. These produce distorted images in which straight lines appear bent. That is not a problem per se—some people prefer regular images, some people are fine with bent lines. However, when we want to align different images together in 3D space, it is crucial to calibrate the bending produced by the cameras.

We build a database of camera calibrations. We identify each camera using metadata fields stored in the EXIF of the image and find the appropriate calibration in the database. This allows us to align images with each other and also to display the images undistorted in the viewer.

Some action cameras, have an option to undistort the images internally. This changes the calibration needed for those images.

Your images are taken with such an option and we failed to recognize that from the EXIF attributes. The result is that the viewer undistorted the images, which were already undistorted by the camera and produced the inverted bending that you saw.

This may happen again for newer cameras and we work continuously on identifying those cases and updating the database.

Sorry for the inconvenience and thanks for warning us,
pau


#3

Hi @pau,

thanks for your explanation. I’m facing similar issues with most of the images taken with my GoPro Hero5 Session. The areas near the left and right borders are often quite distorted and streched which sometimes lead to rather weird effects.

It seems the strength of the effect changes from picture to picture, even whithin one sequence?
See
https://www.mapillary.com/map/im/UQaFLhf0-itLzAvPlMar4Q (ok)
https://www.mapillary.com/map/im/GKygcfC2dTxRMoAlqjyXiw (not ok)
https://www.mapillary.com/map/im/QS3YI7NW3SyxXDVtWUGFJQ (ok)

The three pictures are taken whithin a few seconds, Camera setings are the same for all three pictures (10MP, wide). Maybe you want to take a look into this :slight_smile:

Best Regards

Nick