Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

FIRe: Fast Inverse Rendering using Directional and Signed Distance Functions (2203.16284v3)

Published 30 Mar 2022 in cs.CV and cs.GR

Abstract: Neural 3D implicit representations learn priors that are useful for diverse applications, such as single- or multiple-view 3D reconstruction. A major downside of existing approaches while rendering an image is that they require evaluating the network multiple times per camera ray so that the high computational time forms a bottleneck for downstream applications. We address this problem by introducing a novel neural scene representation that we call the directional distance function (DDF). To this end, we learn a signed distance function (SDF) along with our DDF model to represent a class of shapes. Specifically, our DDF is defined on the unit sphere and predicts the distance to the surface along any given direction. Therefore, our DDF allows rendering images with just a single network evaluation per camera ray. Based on our DDF, we present a novel fast algorithm (FIRe) to reconstruct 3D shapes given a posed depth map. We evaluate our proposed method on 3D reconstruction from single-view depth images, where we empirically show that our algorithm reconstructs 3D shapes more accurately and it is more than 15 times faster (per iteration) than competing methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. Representing 3d shapes with probabilistic directed distance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19343–19354, June 2022.
  2. A morphable model for the synthesis of 3d faces. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, 1999.
  3. Keep it SMPL: Automatic estimation of 3D human pose and shape from a single image. In European Conference on Computer Vision, 2016.
  4. Deep local shapes: Learning local sdf priors for detailed 3d reconstruction. In European Conference on Computer Vision, pages 608–625. Springer, 2020.
  5. pi-gan: Periodic implicit generative adversarial networks for 3d-aware image synthesis. In arXiv, 2020.
  6. Efficient geometry-aware 3D generative adversarial networks. In CVPR, 2022.
  7. Shapenet: An information-rich 3d model repository, 2015.
  8. Tensorf: Tensorial radiance fields. In European Conference on Computer Vision (ECCV), 2022.
  9. Learning implicit fields for generative shape modeling. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  10. Implicit functions in feature space for 3d shape reconstruction and completion. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, jun 2020.
  11. Robust reconstruction of indoor scenes. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
  12. 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In Proceedings of the European Conference on Computer Vision (ECCV), 2016.
  13. Daniel Cremers. Dynamical statistical shape priors for level set-based tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(8):1262–1273, 2006.
  14. trimesh.
  15. Fast dynamic radiance fields with time-aware neural voxels. arxiv:2205.15285, 2022.
  16. Variational principles, surface evolution, PDE’s, level set methods, and the stereo problem. IEEE TIP, 7(3):336–344, Mar. 1998.
  17. Prif: Primary ray-based implicit function. In Proceedings of the European Conference on Computer Vision (ECCV), October 2022.
  18. Justin Johnson Georgia Gkioxari, Jitendra Malik. Mesh r-cnn. Proceedings of the IEEE International Conference on Computer Vision, 2019.
  19. John C. Hart. Sphere tracing: a geometric method for the antialiased ray tracing of implicit surfaces. The Visual Computer, 12(10):527–545, Dec. 1996.
  20. Neuralodf: Learning omnidirectional distance fields for 3d shape representation, 2022.
  21. Local implicit grid representations for 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6001–6010, 2020.
  22. Adam: A method for stochastic optimization, 2014.
  23. Feature sensitive surface extraction from volume data. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’01, page 57–66, New York, NY, USA, 2001. Association for Computing Machinery.
  24. 4d shape priors for a level set segmentation of the left myocardium in spect sequences. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 92–100. Springer, 2006.
  25. Statistical shape influence in geodesic active contours. In 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002., pages 8–pp. IEEE, 2002.
  26. Photometric mesh optimization for video-aligned 3d object reconstruction. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  27. Dist: Rendering deep implicit signed distance function with differentiable sphere tracing. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  28. SMPL: A skinned multi-person linear model. ACM Transactions on Graphics, Oct. 2015.
  29. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019.
  30. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
  31. Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph., 41(4):102:1–102:15, July 2022.
  32. Shigeru Muraki. Volumetric shape description of range data using “blobby model”. In Proceedings of the 18th annual conference on Computer graphics and interactive techniques, pages 227–235, 1991.
  33. DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks. Computer Graphics Forum, 40(4), 2021.
  34. Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2020.
  35. Deepsdf: Learning continuous signed distance functions for shape representation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
  36. Pytorch: An imperative style, high-performance deep learning library, 2019.
  37. Convolutional occupancy networks. In European Conference on Computer Vision, pages 523–540. Springer, 2020.
  38. Terminerf: Ray termination prediction for efficient neural rendering. In 2021 International Conference on 3D Vision (3DV), pages 1106–1114. IEEE, 2021.
  39. Antonio Ricci. A constructive geometry for computer graphics. The Computer Journal, 16(2):157–160, 1973.
  40. 3d hair synthesis using volumetric variational autoencoders. ACM Transactions on Graphics (TOG), 37(6):1–12, 2018.
  41. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2304–2314, 2019.
  42. Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 84–93, 2020.
  43. Plenoxels: Radiance fields without neural networks. In CVPR, 2022.
  44. Light field networks: Neural scene representations with single-evaluation rendering. In Proc. NeurIPS, 2021.
  45. Scene representation networks: Continuous 3d-structure-aware neural scene representations. Advances in Neural Information Processing Systems, 32, 2019.
  46. Gradient-sdf: A semi-implicit surface representation for 3d reconstruction. arXiv preprint, 2021.
  47. Copyme3d: Scanning and printing persons in 3d. In German Conference on Pattern Recognition, pages 405–414. Springer, 2013.
  48. Neural geometric level of detail: Real-time rendering with implicit 3d shapes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11358–11367, June 2021.
  49. MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction. In Proceedings of the IEEE International Conference on Computer Vision, 2017.
  50. Pose-ndf: Modeling human pose manifolds with neural distance fields. In European Conference on Computer Vision (ECCV), October 2022.
  51. Neural-gif: Neural generalized implicit functions for animating people in clothing. In International Conference on Computer Vision (ICCV), October 2021.
  52. Volume constraints for single view reconstruction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013.
  53. Directshape: Photometric alignment of shape priors for visual vehicle pose and shape estimation. In Proceedings of the IEEE International Conference on Robotics and Automation, 2020.
  54. Unsupervised learning of probably symmetric deformable 3d objects from images in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
  55. Disn: Deep implicit surface network for high-quality single-view 3d reconstruction. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 492–502. Curran Associates, Inc., 2019.
  56. Multiview neural surface reconstruction by disentangling geometry and appearance. Advances in Neural Information Processing Systems, 33, 2020.
  57. Joint deep multi-graph matching and 3d geometry learning from inhomogeneous 2d image collections. In AAAI, 2022.
  58. i3dmm: Deep implicit 3d morphable model of human heads. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12803–12813, 2021.
  59. pixelnerf: Neural radiance fields from one or few images, 2020.
  60. A deep signed directional distance function for object shape representation. arXiv preprint arXiv:2107.11024, 2021.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now