Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

GNeRP: Gaussian-guided Neural Reconstruction of Reflective Objects with Noisy Polarization Priors (2403.11899v1)

Published 18 Mar 2024 in cs.CV

Abstract: Learning surfaces from neural radiance field (NeRF) became a rising topic in Multi-View Stereo (MVS). Recent Signed Distance Function (SDF)-based methods demonstrated their ability to reconstruct accurate 3D shapes of Lambertian scenes. However, their results on reflective scenes are unsatisfactory due to the entanglement of specular radiance and complicated geometry. To address the challenges, we propose a Gaussian-based representation of normals in SDF fields. Supervised by polarization priors, this representation guides the learning of geometry behind the specular reflection and captures more details than existing methods. Moreover, we propose a reweighting strategy in the optimization process to alleviate the noise issue of polarization priors. To validate the effectiveness of our design, we capture polarimetric information, and ground truth meshes in additional reflective scenes with various geometry. We also evaluated our framework on the PANDORA dataset. Comparisons prove our method outperforms existing neural 3D reconstruction methods in reflective scenes by a large margin.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. Anisotropic ambient volume shading. IEEE Transactions on Visualization and Computer Graphics, 22(1):1015–1024, 2015.
  2. Recovery of surface orientation from diffuse polarization. IEEE Transactions on Image Processing, 15(6):1653–1664, 2006.
  3. David C Banks. Illumination in diverse codimensions. In SIGGRAPH, pp.  327–334, 1994.
  4. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. ICCV, 2021.
  5. J. Berkmann and T. Caelli. Computation of surface geometry and segmentation using covariance techniques. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp.  1114–1116, Jan 1994.
  6. Recovering non-rigid 3d shape from image streams. In CVPR, pp.  690–696, 2000.
  7. Perspective phase angle model for polarimetric 3d reconstruction. In ECCV, pp.  398–414, 2022.
  8. Multi-view 3d object detection network for autonomous driving. In CVPR, pp.  1907–1915, 2017.
  9. Micro-scale manufacture of 3d printing. In Applied Mechanics and Materials, volume 670, pp.  936–941, 2014.
  10. Edward Collett. Field guide to polarization. In SPIE, 2005.
  11. A reflectance model for computer graphics. In SIGGRAPH, 1981.
  12. Polarimetric multi-view stereo. In CVPR, pp.  1558–1567, 2017.
  13. Improving neural implicit surfaces geometry with patch warping. In CVPR, pp.  6260–6269, 2022.
  14. Pandora: Polarization-aided neural decomposition of radiance. arXiv preprint arXiv:2203.13458, 2022.
  15. Polarisation vision: overcoming challenges of working with a property of light we barely see. The Science of Nature, 105(3):1–26, 2018.
  16. Geo-neus: Geometry-consistent neural implicit surfaces learning for multi-view reconstruction. NIPS, 2022.
  17. Polarimetric normal stereo. In CVPR, pp.  682–690, 2021.
  18. Towards internet-scale multi-view stereo. In CVPR, pp.  1434–1441, 2010.
  19. Multi-view stereo: A tutorial. Foundations and Trends® in Computer Graphics and Vision, 9(1-2):1–148, 2015.
  20. Massively parallel multiview stereopsis by surface normal diffusion. In ICCV, pp.  873–881, 2015.
  21. Ref-neus: Ambiguity-reduced neural implicit surface learning for multi-view reconstruction with reflection. arXiv preprint arXiv:2303.10840, 2023.
  22. Implicit geometric regularization for learning shapes. arXiv preprint arXiv:2002.10099, 2020.
  23. Robust separation of reflection from multiple images. In CVPR, pp.  2195–2202, 2014.
  24. Sparse ellipsometry: Portable acquisition of polarimetric svbrdf and shape with unstructured flash photography. TOG, 41(4), 2022.
  25. Polarized 3d: High-quality depth sensing with polarization cues. In ICCV, pp.  3370–3378, 2015.
  26. Ray tracing volume densities. ACM SIGGRAPH computer graphics, 18(3):165–174, 1984.
  27. Separating partially-polarized diffuse and specular reflection components under unpolarized light sources. In WACV, pp.  2549–2558, 2023.
  28. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023.
  29. Nero: Neural geometry and brdf reconstruction of reflective objects from multiview images. In SIGGRAPH, 2023.
  30. Marching cubes: A high resolution 3d surface construction algorithm. In Seminal graphics: pioneering efforts that shaped the field, pp.  347–353. 1998.
  31. Nelson Max. Optical models for direct volume rendering. IEEE Transactions on Visualization and Computer Graphics, 1(2):99–108, 1995.
  32. Occupancy networks: Learning 3d reconstruction in function space. In CVPR, pp.  4460–4470, 2019.
  33. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
  34. Unisurf: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. In ICCV, pp.  5589–5599, 2021.
  35. Pixelwise view selection for unstructured multi-view stereo. In ECCV, pp.  501–518, 2016.
  36. Pixelwise view selection for unstructured multi-view stereo. In ECCV, 2016.
  37. On the computation of polarization ellipse parameters. Geophysics, 39(6):867–869, 1974.
  38. Ref-nerf: Structured view-dependent appearance for neural radiance fields. In CVPR, pp.  5481–5490, 2022.
  39. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2106.10689, 2021.
  40. Frequency principle: Fourier analysis sheds light on deep neural networks. arXiv preprint arXiv:1901.06523, 2019.
  41. Volume rendering of neural implicit surfaces. NIPS, 34:4805–4815, 2021.
  42. PhySG: Inverse rendering with spherical gaussians for physics-based material editing and relighting. In CVPR, 2021.
  43. Polarimetric multi-view inverse rendering. In ECCV, pp.  85–102, 2020.
  44. Ewa volume splatting. In VIS, pp.  29–538, 2001.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now