Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Unsupervised Occupancy Learning from Sparse Point Cloud (2404.02759v1)

Published 3 Apr 2024 in cs.CV, cs.AI, cs.GR, and cs.LG

Abstract: Implicit Neural Representations have gained prominence as a powerful framework for capturing complex data modalities, encompassing a wide range from 3D shapes to images and audio. Within the realm of 3D shape representation, Neural Signed Distance Functions (SDF) have demonstrated remarkable potential in faithfully encoding intricate shape geometry. However, learning SDFs from 3D point clouds in the absence of ground truth supervision remains a very challenging task. In this paper, we propose a method to infer occupancy fields instead of SDFs as they are easier to learn from sparse inputs. We leverage a margin-based uncertainty measure to differentially sample from the decision boundary of the occupancy function and supervise the sampled boundary points using the input point cloud. We further stabilize the optimization process at the early stages of the training by biasing the occupancy function towards minimal entropy fields while maximizing its entropy at the input point cloud. Through extensive experiments and evaluations, we illustrate the efficacy of our proposed method, highlighting its capacity to improve implicit shape inference with respect to baselines and the state-of-the-art using synthetic and real data.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (98)
  1. Neural point-based graphics. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXII 16, pages 696–712. Springer, 2020.
  2. The power crust, unions of balls, and the medial axis transform. CG, 2001.
  3. Sal: Sign agnostic learning of shapes from raw data. In CVPR, 2020a.
  4. Sald: Sign agnostic learning with derivatives. In ICML, 2020b.
  5. Adi Ben-Israel. A newton-raphson method for the solution of systems of equations. In Journal of Mathematical analysis and applications, 1966.
  6. Digs: Divergence guided shape implicit neural representation for unoriented point clouds. In CVPR, 2022.
  7. The ball-pivoting algorithm for surface reconstruction. TVCG, 1999.
  8. FAUST: Dataset and evaluation for 3D mesh registration. In CVPR, 2014.
  9. Poco: Point convolution for surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6302–6314, 2022.
  10. Needrop: Self-supervised shape representation from sparse point clouds using needle dropping. In 2021 International Conference on 3D Vision (3DV), pages 940–950. IEEE, 2021.
  11. Reconstruction and representation of 3d objects with radial basis functions. In SIGGRAPH, 2001.
  12. Effective Computational Geometry for Curves and Surfaces. 2006.
  13. Deep local shapes: Learning local sdf priors for detailed 3d reconstruction. In ECCV, 2020.
  14. Efficient geometry-aware 3d generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16123–16133, 2022.
  15. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015.
  16. Unsupervised inference of signed distance functions from single sparse point clouds without learning priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023a.
  17. Gridpull: Towards scalability in learning implicit representations from 3d point clouds. In Proceedings of the ieee/cvf international conference on computer vision, pages 18322–18334, 2023b.
  18. Transformers as meta-learners for implicit neural representations. In European Conference on Computer Vision, 2022.
  19. Learning implicit fields for generative shape modeling. In CVPR, 2019.
  20. Bsp-net: Generating compact meshes via binary space partitioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
  21. Implicit feature networks for texture completion from partial 3d data. In European Conference on Computer Vision, pages 717–725. Springer, 2020.
  22. Neural unsigned distance fields for implicit function learning. In NeurIPS, 2020.
  23. Cvxnet: Learnable convex decomposition. In CVPR, 2020.
  24. Learning elementary structures for 3d shape generation and matching. In NeurIPS, 2019.
  25. Points2surf learning implicit surfaces from point clouds. In ECCV, 2020.
  26. A point set generation network for 3d object reconstruction from a single image. In CVPR, 2017.
  27. Learning shape templates with structured implicit functions. In ICCV, 2019.
  28. Local deep implicit functions for 3d shape. In CVPR, 2020.
  29. Implicit geometric regularization for learning shapes. In ICML, 2020.
  30. A papier-mâché approach to learning 3d surface generation. In CVPR, 2018.
  31. Algebraic point set surfaces. In ACM siggraph 2007 papers, pages 23–es. 2007.
  32. John C Hart. Sphere tracing: A geometric method for the antialiased ray tracing of implicit surfaces. The Visual Computer, 1996.
  33. Neural kernel surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4369–4379, 2023.
  34. Zero-shot text-guided object generation with dream fields. 2022.
  35. Neural mesh-based graphics. In European Conference on Computer Vision, pages 739–757. Springer, 2022.
  36. Local implicit grid representations for 3d scenes. In CVPR, 2020.
  37. Neural 3d mesh renderer. In CVPR, 2018.
  38. Screened poisson surface reconstruction. TOG, 2013.
  39. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4):1–14, 2023.
  40. Ravikrishna Kolluri. Provably good moving least squares. TALG, 2008.
  41. Octree guided unoriented surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16717–16726, 2023.
  42. Learning generalizable light field networks from few images. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023a.
  43. Regularizing neural radiance fields from sparse rgb-d inputs. In 2023 IEEE International Conference on Image Processing (ICIP), pages 2320–2324. IEEE, 2023b.
  44. Learning deep implicit functions for 3d shapes with dynamic code clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12840–12850, 2022.
  45. Surface reconstruction from point clouds without normals by parametrizing the gauss formula. ACM Transactions on Graphics, 42(2):1–19, 2022.
  46. Dynamic plane convolutional occupancy networks. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1829–1838, 2021.
  47. Yaron Lipman. Phase transitions, distance functions, and implicit neural representations. In ICML, 2021.
  48. Planenet: Piece-wise planar reconstruction from a single rgb image. In CVPR, 2018.
  49. Learning smooth neural functions via lipschitz regularization. arXiv preprint arXiv:2202.08345, 2022.
  50. Meshing point clouds with predicted intrinsic-extrinsic ratio guidance. In ECCV, 2020.
  51. Deep implicit moving least-squares functions for 3d reconstruction. In CVPR, 2021.
  52. Marching cubes: A high resolution 3d surface construction algorithm. In SIGGRAPH, 1987.
  53. Neural-pull: Learning signed distance functions from point clouds by learning to pull space onto surfaces. In ICML, 2021.
  54. Reconstructing surfaces for sparse point clouds with on-surface priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6315–6325, 2022a.
  55. Surface reconstruction from point clouds by learning predictive context priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6326–6337, 2022b.
  56. Moving level-of-detail surfaces. ACM Transactions on Graphics (TOG), 41(4):1–10, 2022.
  57. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4460–4470, 2019.
  58. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
  59. Few’zero level set’-shot learning of shape signed distance functions in feature space. In ECCV, 2022.
  60. Robustifying generalizable implicit shape networks with a tunable non-parametric model. In NeurIPS, 2023.
  61. Mixing-denoising generalizable occupancy networks. In 3DV, 2024.
  62. Deepcurrents: Learning implicit representations of shapes with boundaries. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18665–18675, 2022.
  63. Deepsdf: Learning continuous signed distance functions for shape representation. In CVPR, 2019.
  64. Pytorch: An imperative style, high-performance deep learning library. NeurIPS, 2019.
  65. Convolutional occupancy networks. In European Conference on Computer Vision, pages 523–540. Springer, 2020.
  66. Shape as points: A differentiable poisson solver. Advances in Neural Information Processing Systems, 34:13032–13044, 2021.
  67. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, 2017.
  68. Differentiable surface triangulation. In SIGGRAPH Asia, 2021.
  69. Octnet: Learning deep 3d representations at high resolutions. In CVPR, 2017.
  70. Kernel methods for implicit surface modeling. In NeurIPS, 2004.
  71. Structure-from-motion revisited. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
  72. Pixelwise view selection for unstructured multi-view stereo. In European Conference on Computer Vision (ECCV), 2016.
  73. Burr Settles. Active learning literature survey. 2009.
  74. Claude Elwood Shannon. A mathematical theory of communication. The Bell system technical journal, 27(3):379–423, 1948.
  75. Scene representation networks: Continuous 3d-structure-aware neural scene representations. In NeurIPS, 2019.
  76. Metasdf: Meta-learning signed distance functions. In NeurIPS, 2020a.
  77. Implicit neural representations with periodic activation functions. In NeurIPS, 2020b.
  78. Light field networks: Neural scene representations with single-evaluation rendering. In NeurIPS, 2021.
  79. Neural geometric level of detail: Real-time rendering with implicit 3d shapes. In CVPR, 2021.
  80. Sa-convonet: Sign-agnostic optimization of convolutional occupancy networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6504–6513, 2021.
  81. Octree generating networks: Efficient convolutional architectures for high-resolution 3d outputs. In ICCV, 2017.
  82. Patchnets: Patch-based generalizable deep implicit 3d shape representations. In ECCV, 2020.
  83. Learning shape abstractions by assembling volumetric primitives. In CVPR, 2017.
  84. Pixel2mesh: Generating 3d mesh models from single rgb images. In ECCV, 2018.
  85. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2106.10689, 2021a.
  86. O-cnn: Octree-based convolutional neural networks for 3d shape analysis. TOG, 2017.
  87. Metaavatar: Learning animatable clothed human models from few depth images. Advances in Neural Information Processing Systems, 34:2810–2822, 2021b.
  88. Deep geometric prior for surface reconstruction. In CVPR, 2019.
  89. Neural splines: Fitting 3d surfaces with infinitely-wide neural networks. In CVPR, 2021.
  90. Neural fields as learnable kernels for 3d reconstruction. In CVPR, 2022.
  91. Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. In NeurIPS, 2016.
  92. 3d shapenets: A deep representation for volumetric shapes. In CVPR, 2015.
  93. Volume rendering of neural implicit surfaces. Advances in Neural Information Processing Systems, 34:4805–4815, 2021.
  94. 3dias: 3d shape reconstruction with implicit algebraic surfaces. In ICCV, 2021.
  95. Tjalling J Ypma. Historical development of the newton–raphson method. SIAM review, 37(4):531–551, 1995.
  96. Learning consistency-aware unsigned distance functions progressively from raw point clouds. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
  97. Dense scene reconstruction with points of interest. ACM Transactions on Graphics (ToG), 32(4):1–8, 2013.
  98. 3d-prnn: Generating shape primitives with recurrent neural networks. In CVPR, 2017.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets