Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SC-NeuS: Consistent Neural Surface Reconstruction from Sparse and Noisy Views (2307.05892v1)

Published 12 Jul 2023 in cs.CV

Abstract: The recent neural surface reconstruction by volume rendering approaches have made much progress by achieving impressive surface reconstruction quality, but are still limited to dense and highly accurate posed views. To overcome such drawbacks, this paper pays special attention on the consistent surface reconstruction from sparse views with noisy camera poses. Unlike previous approaches, the key difference of this paper is to exploit the multi-view constraints directly from the explicit geometry of the neural surface, which can be used as effective regularization to jointly learn the neural surface and refine the camera poses. To build effective multi-view constraints, we introduce a fast differentiable on-surface intersection to generate on-surface points, and propose view-consistent losses based on such differentiable points to regularize the neural surface learning. Based on this point, we propose a jointly learning strategy for neural surface and camera poses, named SC-NeuS, to perform geometry-consistent surface reconstruction in an end-to-end manner. With extensive evaluation on public datasets, our SC-NeuS can achieve consistently better surface reconstruction results with fine-grained details than previous state-of-the-art neural surface reconstruction approaches, especially from sparse and noisy camera views.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. Large-scale data for multiple-view stereopsis. IJCV., 120(2):153–168, 2016.
  2. Sal: Sign agnostic learning of shapes from raw data. In IEEE CVPR, pages 2565–2574, 2020.
  3. Neural rgb-d surface reconstruction. In IEEE CVPR, pages 6290–6301, 2022.
  4. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In IEEE CVPR, pages 5470–5479, 2022.
  5. Samurai: Shape and material from unconstrained real-world arbitrary image collections. In Advances in Neural Information Processing Systems, 2022.
  6. Mvsnerf: Fast generalizable radiance field reconstruction from multi-view stereo. In IEEE CVPR, pages 14124–14133, 2021.
  7. Gaussian activated neural radiance fields for high fidelity reconstruction and pose estimation. In ECCV, pages 264–280. Springer, 2022.
  8. Improving neural implicit surfaces geometry with patch warping. In IEEE CVPR, pages 6260–6269, 2022.
  9. Depth-supervised nerf: Fewer views and faster training for free. In IEEE CVPR, pages 12882–12891, 2022.
  10. Geo-neus: Geometry-consistent neural implicit surfaces learning for multi-view reconstruction. In Advances in Neural Information Processing Systems.
  11. Large scale multi-view stereopsis evaluation. In IEEE CVPR, pages 406–413, 2014.
  12. Self-calibrating neural radiance fields. In IEEE CVPR, pages 5846–5854, 2021.
  13. Local implicit grid representations for 3d scenes. In CVPR, pages 6001–6010, 2020.
  14. Learning a multi-view stereo machine. Advances in neural information processing systems, 30, 2017.
  15. Infonerf: Ray entropy minimization for few-shot neural volume rendering. In IEEE CVPR, pages 12912–12921, 2022.
  16. Neroic: neural rendering of objects from online image collections. ACM Transactions on Graphics (TOG), 41(4):1–12, 2022.
  17. Efficient multi-view reconstruction of large-scale scenes using interest points, delaunay triangulation and graph cuts. In IEEE ICCV, pages 1–8. IEEE, 2007.
  18. Barf: Bundle-adjusting neural radiance fields. In IEEE CVPR, pages 5741–5751, 2021.
  19. Sparseneus: Fast generalizable neural surface reconstruction from sparse views. In ECCV, pages 210–227. Springer, 2022.
  20. Aslfeat: Learning local features of accurate shape and localization. In IEEE CVPR, pages 6588–6597, 2020.
  21. Gnerf: Gan-based neural radiance field without posed camera. In IEEE ICCV, pages 6351–6361, 2021.
  22. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021.
  23. Regnerf: Regularizing neural radiance fields for view synthesis from sparse inputs. In IEEE CVPR, pages 5480–5490, 2022.
  24. Unisurf: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. In IEEE ICCV, pages 5589–5599, 2021.
  25. Deepsdf: Learning continuous signed distance functions for shape representation. In IEEE CVPR, pages 165–174, 2019.
  26. Convolutional occupancy networks. In ECCV, pages 523–540. Springer, 2020.
  27. Dense depth priors for neural radiance fields from sparse input views. In IEEE CVPR, pages 12892–12901, 2022.
  28. Superglue: Learning feature matching with graph neural networks. In IEEE CVPR, pages 4937–4946, 2020.
  29. Structure-from-motion revisited. In IEEE CVPR, pages 4104–4113, 2016.
  30. Pixelwise view selection for unstructured multi-view stereo. In ECCV, pages 501–518. Springer, 2016.
  31. Photo tourism: exploring photo collections in 3d. In ACM SIGGRAPH, pages 835–846. 2006.
  32. imap: Implicit mapping and positioning in real-time. In IEEE CVPR, pages 6229–6238, 2021.
  33. Grf: Learning a general radiance field for 3d representation and rendering. In IEEE ICCV, pages 15182–15192, 2021.
  34. Sparf: Neural radiance fields from sparse and noisy poses. arXiv e-prints, pages arXiv–2211, 2022.
  35. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. Advances in Neural Information Processing Systems, 34:27171–27183, 2021.
  36. Ibrnet: Learning multi-view image-based rendering. In IEEE CVPR, pages 4690–4699, 2021.
  37. NeRF−⁣−--- -: Neural radiance fields without known camera parameters. arXiv preprint arXiv:2102.07064, 2021.
  38. Nerfingmvs: Guided optimization of neural radiance fields for indoor multi-view stereo. In IEEE CVPR, pages 5610–5619, 2021.
  39. Multi-scale geometric consistency guided multi-view stereo. In IEEE CVPR, pages 5483–5492, 2019.
  40. Pvsnet: Pixelwise visibility-aware multi-view stereo network. arXiv preprint arXiv:2007.07714, 2020.
  41. Mvsnet: Depth inference for unstructured multi-view stereo. In ECCV, pages 767–783, 2018.
  42. Blendedmvs: A large-scale dataset for generalized multi-view stereo networks. In IEEE CVPR, pages 1787–1796, 2020.
  43. Volume rendering of neural implicit surfaces. Advances in Neural Information Processing Systems, 34:4805–4815, 2021.
  44. Multiview neural surface reconstruction by disentangling geometry and appearance. Advances in Neural Information Processing Systems, 33:2492–2502, 2020.
  45. pixelnerf: Neural radiance fields from one or few images. In IEEE CVPR, pages 4578–4587, 2021.
  46. Ners: neural reflectance surfaces for sparse-view 3d reconstruction in the wild. Advances in Neural Information Processing Systems, 34:29835–29847, 2021.
  47. Relpose: Predicting probabilistic relative rotation for single objects in the wild. In ECCV, pages 592–611. Springer, 2022.
  48. Nice-slam: Neural implicit scalable encoding for slam. In IEEE CVPR, pages 12786–12796, 2022.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now