A few questions about SFM

Thanks for the great project! I am interested in OpenSFM and have tried the demo. Could you also kindly provide the paper or reference of the algorithm for SFM implemented? It is difficult to understand the algorithm wholistically from the code. Thanks.

And it would be great if you could tell us what OPENSFM cant do at the moment e.g., to integrate photos taken from fisheye cameras, while what it can do is clearly demonstrated.

Another question is if the photos are taken on different hight level, e.g. taken in the Colosseum in Roma. Can OpenSFM reconstruct the 3D coordiante if the camera position and angle? if not, how to work around?

hi @peyton

OpenSfM implements what is called an incremental SfM pipeline. It was popularized by the photo tourism project and their paper has a good description of the pipeline.

Not sure what to list on what OpenSfM does not do. It actually does support fisheye cameras. What other things are you interested in?

As for the different heights, images can be taken from any height. OpenSfM only assumes ground level images for orienting the final model. If the images are taken from different heights, the final model might appear tilted with respect to the ground.

Hi Pau,

Thanks for your answer.

1: As suggested by the paper listed, fisheye images are classified as “bad images” that deteriorate the SFM performance. Does openSFM come with modifications to address this problem?

2: I am also interested in the image related problems faced by Mapillary applying OpenSFM in general. That would help me to grasp a better understanding of its functionalityies and boundaries.

3: “If the images are taken from different heights, the final model might appear tilted with respect to the ground.” Does that mean that the image paramters “camera center” would show estimated 3D coordinate?

4: Is there an extension to speed up OpenSFM? To my knowledge, some parts of the algorithm implementation can be parellized. The current implementation is time-costly in my trial experience with openSFM.

5: As the paper suggested, the GPS coordinate are mainly used for Geo registration, not for improving the SFM model. However, the github says “It also integrates external sensor (e.g. GPS, accelerometer) measurements for geographical alignment and robustness”. Could you help explain how the OpenSFM uses the GPS and accelerometer to improve the robustness of OpenSFM? I ran an experiment with modified random GPS (in EXIF) and the results turn out to be fine.

6: How do Mapillary incrementally manage newly updated photos using OpenSFM? Do we consider them as an “incremental images” to the previously built SFM model? How do we resolve the conflict of images taken from the same viewpoint but different time that the content in the image has substantially changed?

7: Does the provided GPS coordinate help (1) speed up pair-wise feature matching? (2) used as rigid or non-rigid prior for bundle adjustment?

Thank you very much Pau. Appreciate that.

Best,
Peyton