So I was able to process my own imagery using the amazing Vistas dataset to perform panoptic segmentation on the street level capture. My question is how do I get to the next step (feature extraction). Would love to get to geojson of identified features, or at a minimum identify which images contain which features.
Exciting work!
So I think roughly, what you need to do next is feed your imagery into the OpenSfM Python library, to make a 3d scene reconstruction. You will have pixels become a 3d point cloud. From there, you’ll match the label of that pixel in the original segmentation (so sidewalk, or stop sign) to the pixel in the 3d model, and start to cluster that off to mark it as a specific 3d object.
1 Like