Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Refractive COLMAP: Refractive Structure-from-Motion Revisited (2403.08640v3)

Published 13 Mar 2024 in cs.CV

Abstract: In this paper, we present a complete refractive Structure-from-Motion (RSfM) framework for underwater 3D reconstruction using refractive camera setups (for both, flat- and dome-port underwater housings). Despite notable achievements in refractive multi-view geometry over the past decade, a robust, complete and publicly available solution for such tasks is not available at present, and often practical applications have to resort to approximating refraction effects by the intrinsic (distortion) parameters of a pinhole camera model. To fill this gap, we have integrated refraction considerations throughout the entire SfM process within the state-of-the-art, open-source SfM framework COLMAP. Numerical simulations and reconstruction results on synthetically generated but photo-realistic images with ground truth validate that enabling refraction does not compromise accuracy or robustness as compared to in-air reconstructions. Finally, we demonstrate the capability of our approach for large-scale refractive scenarios using a dataset consisting of nearly 6000 images. The implementation is released as open-source at: https://cau-git.rz.uni-kiel.de/inf-ag-koeser/colmap_underwater.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. M. Shortis, “Calibration techniques for accurate measurements by underwater camera systems,” Sensors, vol. 15, no. 12, pp. 30 810–30 826, 2015. [Online]. Available: http://www.mdpi.com/1424-8220/15/12/29831
  2. J. L. Schonberger and J.-M. Frahm, “Structure-from-motion revisited,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 4104–4113.
  3. T. Treibitz, Y. Y. Schechner, and H. Singh, “Flat refractive geometry,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition CVPR 2008, 2008, pp. 1–8.
  4. G. Telem and S. Filin, “Photogrammetric modeling of underwater environments,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 65, no. 5, pp. 433–444, 2010. [Online]. Available: http://www.sciencedirect.com/science/article/B6VF4-50F9H66-1/2/d8dba566f79b0a207e13a6aa2bf3f69d
  5. A. Jordt, K. Köser, and R. Koch, “Refractive 3d reconstruction on underwater images,” Methods in Oceanography, vol. 15-16, pp. 90 – 113, 2016. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S2211122015300086
  6. M. She, D. Nakath, Y. Song, and K. Köser, “Refractive geometry for underwater domes,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 183, pp. 525–540, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S092427162100304X
  7. A. Agrawal, Y. Taguchi, and S. Ramalingam, “Analytical forward projection for axial non-central dioptric and catadioptric cameras,” in European Conference on Computer Vision.   Springer, 2010, pp. 129–143.
  8. A. Jordt-Sedlazeck and R. Koch, “Refractive calibration of underwater cameras,” in Computer Vision - ECCV 2012, ser. Lecture Notes in Computer Science, A. Fitzgibbon, S. Lazebnik, P. Pietro, Y. Sato, and C. Schmid, Eds.   Springer Berlin Heidelberg, 2012, vol. 7576, pp. 846–859.
  9. M. She, Y. Song, J. Mohrmann, and K. Köser, “Adjustment and calibration of dome port camera systems for underwater vision,” in German Conference on Pattern Recognition.   Springer, 2019, pp. 79–92.
  10. F. Chadebecq, F. Vasconcelos, R. Lacher, E. Maneas, A. Desjardins, S. Ourselin, T. Vercauteren, and D. Stoyanov, “Refractive two-view reconstruction for underwater 3d vision,” International Journal of Computer Vision, pp. 1–17, 2019.
  11. B. Elnashef and S. Filin, “A three-point solution with scale estimation ability for two-view flat-refractive underwater photogrammetry,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 198, pp. 223–237, 2023.
  12. H. Li, R. Hartley, and J.-H. Kim, “A linear approach to motion estimation using generalized camera models,” in Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, 7 2008, pp. 1 –8.
  13. X. Hu, F. Lauze, and K. S. Pedersen, “Refractive pose refinement: Generalising the geometric relation between camera and refractive interface,” International Journal of Computer Vision, vol. 131, no. 6, pp. 1448–1476, 2023.
  14. A. Jordt-Sedlazeck and R. Koch, “Refractive structure-from-motion on underwater images,” in Computer Vision (ICCV), 2011 IEEE International Conference on, 2013, pp. 57–64.
  15. F. Chadebecq, F. Vasconcelos, G. Dwyer, R. Lacher, S. Ourselin, T. Vercauteren, and D. Stoyanov, “Refractive structure-from-motion through a flat refractive interface,” in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 5315–5323.
  16. B. Elnashef and S. Filin, “Drift reduction in underwater egomotion computation by axial camera modeling,” IEEE Robotics and Automation Letters, 2023.
  17. J. L. Schönberger, E. Zheng, J.-M. Frahm, and M. Pollefeys, “Pixelwise view selection for unstructured multi-view stereo,” in European Conference on Computer Vision.   Springer, 2016, pp. 501–518.
  18. B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2021.
  19. G. Billings, R. Camilli, and M. Johnson-Roberson, “Hybrid visual slam for underwater vehicle manipulator systems,” IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 6798–6805, 2022.
  20. M. D. Grossberg and S. K. Nayar, “The raxel imaging model and ray-based calibration,” International Journal of Computer Vision, vol. 61, no. 2, pp. 119–137, 2005.
  21. T. Schops, V. Larsson, M. Pollefeys, and T. Sattler, “Why having 10,000 parameters in your camera model is better than twelve,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 2535–2544.
  22. A. Agrawal, S. Ramalingam, Y. Taguchi, and V. Chari, “A theory of multi-layer flat refractive geometry,” in CVPR, 2012.
  23. C. Kunz and H. Singh, “Hemispherical refraction and camera calibration in underwater vision,” in OCEANS 2008.   IEEE, 2008, pp. 1–7.
  24. F. Menna, E. Nocerino, F. Fassi, and F. Remondino, “Geometric and optic characterization of a hemispherical dome port for underwater photogrammetry,” Sensors, vol. 16, no. 1, 2016. [Online]. Available: http://www.mdpi.com/1424-8220/16/1/48
  25. T. Luczynski, M. Pfingsthorn, and A. Birk, “The pinax-model for accurate and efficient refraction correction of underwater cameras in flat-pane housings,” Ocean Engineering, vol. 133, pp. 9 – 22, 2017. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0029801817300434
  26. P. Sturm, “Multi-view geometry for general camera models,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1.   IEEE, 2005, pp. 206–212.
  27. P. Sturm, S. Ramalingam, and S. Lodha, “On calibration, structure from motion and multi-view geometry for generic camera models,” in Imaging Beyond the Pinhole Camera, ser. Computational Imaging and Vision, K. Daniilidis and R. Klette, Eds.   Springer, aug 2006, vol. 33.
  28. S. Ramalingam, S. K. Lodha, and P. Sturm, “A generic structure-from-motion framework,” Computer Vision and Image Understanding, vol. 103, no. 3, pp. 218–228, Sept. 2006.
  29. V. Chari and P. Sturm, “Multiple-View Geometry of the Refractive Plane,” in BMVC 2009 - 20th British Machine Vision Conference, A. Cavallaro, S. Prince, and D. C. Alexander, Eds.   London, United Kingdom: The British Machine Vision Association (BMVA), Sept. 2009, pp. 1–11. [Online]. Available: https://hal.inria.fr/inria-00434342
  30. L. Kang, L. Wu, and Y.-H. Yang, “Two-view underwater structure and motion for cameras under flat refractive interfaces,” in Computer Vision - ECCV 2012, ser. Lecture Notes in Computer Science, A. Fitzgibbon, S. Lazebnik, P. Perona, Y. Sato, and C. Schmid, Eds.   Springer Berlin / Heidelberg, 2012, vol. 7575, pp. 303–316.
  31. M. Fischler and R. Bolles, “RANdom SAmpling Consensus: a paradigm for model fitting with application to image analysis and automated cartography,” Communications of the ACM, vol. 24, no. 6, pp. 381–395, 6 1981.
  32. B. Elnashef and S. Filin, “Direct linear and refraction-invariant pose estimation and calibration model for underwater imaging,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 154, pp. 259–271, 2019.
  33. G. Hee Lee, B. Li, M. Pollefeys, and F. Fraundorfer, “Minimal solutions for pose estimation of a multi-camera system,” in Robotics Research: The 16th International Symposium ISRR.   Springer, 2016, pp. 521–538.
  34. L. Kneip and H. Li, “Efficient computation of relative pose for multi-camera systems,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 446–453.
  35. X.-S. Gao, X.-R. Hou, J. Tang, and H.-F. Cheng, “Complete solution classification for the perspective-three-point problem,” IEEE transactions on pattern analysis and machine intelligence, vol. 25, no. 8, pp. 930–943, 2003.
  36. V. Lepetit, F. Moreno-Noguer, and P. Fua, “Epnp: An accurate o (n) solution to the pnp problem,” International journal of computer vision, vol. 81, no. 2, p. 155, 2009.
  37. F. Seegräber, P. Schöntag, F. Woelk, and K. Köser, “Underwater multiview stereo using axial camera models.”
  38. D. Nistér, “An efficient solution to the five-point relative pose problem,” TPAMI, vol. 26, pp. 756–777, 2004.
  39. M. She, Y. Song, D. Nakath, and K. Köser, “Semi-hierarchical reconstruction and weak-area revisiting for robotic visual seafloor mapping,” 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2308.06147
  40. B. Elnashef and S. Filin, “Target-free calibration of flat refractive imaging systems using two-view geometry,” Optics and Lasers in Engineering, vol. 150, p. 106856, 2022.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now