Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
134 tokens/sec
GPT-4o
10 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Depth Reconstruction with Neural Signed Distance Fields in Structured Light Systems (2405.12006v1)

Published 20 May 2024 in cs.CV

Abstract: We introduce a novel depth estimation technique for multi-frame structured light setups using neural implicit representations of 3D space. Our approach employs a neural signed distance field (SDF), trained through self-supervised differentiable rendering. Unlike passive vision, where joint estimation of radiance and geometry fields is necessary, we capitalize on known radiance fields from projected patterns in structured light systems. This enables isolated optimization of the geometry field, ensuring convergence and network efficacy with fixed device positioning. To enhance geometric fidelity, we incorporate an additional color loss based on object surfaces during training. Real-world experiments demonstrate our method's superiority in geometric performance for few-shot scenarios, while achieving comparable results with increased pattern availability.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. Photogeometric structured light: A self-calibrating and multi-viewpoint framework for accurate 3d modeling. In CVPR, pages 1–8, 2008.
  2. Törf: Time-of-flight radiance fields for dynamic scene view synthesis. NeurIPS, 34, 2021.
  3. Recent progress in coded structured light as a technique to solve the correspondence problem: a survey. PR, 31(7):963–982, 1998.
  4. Improving neural implicit surfaces geometry with patch warping. In CVPR, pages 6250–6259, 2022.
  5. Depth-supervised NeRF: Fewer views and faster training for free. CVPR, pages 12872–12881, 2022a.
  6. Depth-supervised nerf: Fewer views and faster training for free. In CVPR, pages 12882–12891, 2022b.
  7. Geo-neus: Geometry-consistent neural implicit surfaces learning for multi-view reconstruction. In NeurIPS, pages 3403–3416, 2022.
  8. Unsupervised cnn for single view depth estimation: Geometry to the rescue. In ECCV, pages 740–756. Springer, 2016.
  9. A geometric perspective on structured light coding. In ECCV, pages 87–102, 2018.
  10. Stripe boundary codes for real-time structured-light range scanning of moving objects. In ICCV, pages 359–366, 2001.
  11. High-speed 3d image acquisition using coded structured light projection. In IROS, pages 925–930, 2007.
  12. Depthinspace: Exploitation and fusion of multiple video frames for structured-light depth estimation. In ICCV, pages 6039–6048, 2021.
  13. Ray tracing volume densities. SIGGRAPH, 18(3):165–174, 1984.
  14. Neural 3d mesh renderer. In CVPR, 2018.
  15. Dynamic scene shape reconstruction using a single structured light pattern. In CVPR, pages 1–8, 2008.
  16. Real-time range acquisition by adaptive structured light. IEEE TPAMI, 28(3):432–445, 2006.
  17. Build your own 3d scanner: 3d photography for beginners. In ACM siggraph courses, pages 1–94. 2009.
  18. Design and decoding of an m-array pattern for low-cost structured light 3d reconstruction systems. In ICIP, pages 2168–2172, 2013.
  19. A single-shot structured light means by encoding both color and geometrical features. PR, 54:178–189, 2016.
  20. Soft rasterizer: A differentiable renderer for image-based 3d reasoning. In ICCV, pages 7708–7717, 2019.
  21. Sparseneus: Fast generalizable neural surface reconstruction from sparse views. In ECCV, pages 210–227, 2022.
  22. Neuraludf: Learning unsigned distance fields for multi-view reconstruction of surfaces with arbitrary topologies. In CVPR, pages 20834–20843, 2023.
  23. FloatingFusion: Depth from tof and image-stabilized stereo cameras. In ECCV, 2022.
  24. NeRF: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
  25. Optimal structured light a la carte. In CVPR, pages 6248–6257, 2018.
  26. RegNeRF: Regularizing neural radiance fields for view synthesis from sparse inputs. CVPR, pages 5470–5480, 2022.
  27. Deepsdf: Learning continuous signed distance functions for shape representation. In CVPR, pages 165–174, 2019.
  28. Surface measurement by space-encoded projected beam systems. Computer graphics and image processing, 18(1):1–17, 1982.
  29. TIDE: Temporally incremental disparity estimation via pattern flow in structured light system. RA-L, 7(2):5111–5118, 2022.
  30. Connecting the Dots: Learning representations for active monocular depth estimation. In CVPR, pages 7616–7625, 2019.
  31. HyperDepth: Learning depth from structured light without matching. In CVPR, pages 5441–5450, 2016.
  32. UltraStereo: Efficient learning-based matching for active stereo systems. In CVPR, pages 6535–6544, 2017.
  33. Dense 3d reconstruction method using a single pattern for fast moving object. In ICCV, pages 1779–1786, 2009.
  34. Dense one-shot 3d reconstruction by detecting continuous regions with parallel line projection. In ICCV, pages 1911–1918, 2011.
  35. Pattern codification strategies in structured light systems. PR, 37(4):827–849, 2004.
  36. A state of the art in structured light patterns for surface profilometry. PR, 43(8):2666–2680, 2010.
  37. High-accuracy stereo depth maps using structured light. In CVPR, pages I–I, 2003.
  38. Single-photon structured light. In CVPR, pages 17865–17875, 2022.
  39. Motion-aware structured light using spatio-temporal decodable patterns. In ECCV, pages 832–845, 2012.
  40. NeuS: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. In NeurIPS, 2021.
  41. 3d shape measurement technique for multiple rapidly moving objects. Optics Express, 19(9):8539–8545, 2011.
  42. 3d absolute shape measurement of live rabbit hearts with a superfast two-frequency phase-shifting technique. Optics Express, 21(5):5822–5832, 2013.
  43. A multi-camera, multi-projector super-resolution framework for structured light. In 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission, pages 397–404. IEEE, 2011.
  44. High-speed and high-efficiency three-dimensional shape measurement based on gray-coded light. Photonics Research, 8(6):819–829, 2020.
  45. Two-frequency phase-shifting method vs. gray-coded-based method in dynamic fringe projection profilometry: A comparative review, 2022.
  46. Bungeenerf: Progressive neural radiance field for extreme multi-scale scene rendering. In ECCV, pages 106–122, 2022.
  47. Sinnerf: Training neural radiance fields on complex scenes from a single image. In ECCV, page 736–753, 2022.
  48. Continual neural mapping: Learning an implicit scene representation from sequential observations. In ICCV, pages 15782–15792, 2021.
  49. Volume rendering of neural implicit surfaces. In NeurIPS, 2021.
  50. Viewpoint-coded structured light. In CVPR, pages 1–8, 2007.
  51. Activestereonet: End-to-end self-supervised learning for active stereo systems. In ECCV, pages 784–801, 2018.
  52. High-speed three-dimensional shape measurement for dynamic scenes using bi-frequency tripolar pulse-width-modulation fringe projection. Optics and Lasers in Engineering, 51(8):953–960, 2013.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now