Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
173 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Generate Point Clouds with Multiscale Details from Graph-Represented Structures (2112.06433v3)

Published 13 Dec 2021 in cs.CV and cs.AI

Abstract: As details are missing in most representations of structures, the lack of controllability to more information is one of the major weaknesses in structure-based controllable point cloud generation. It is observable that definitions of details and structures are subjective. Details can be treated as structures on small scales. To represent structures in different scales at the same time, we present a graph-based representation of structures called the Multiscale Structure Graph (MSG). Given structures in multiple scales, similar patterns of local structures can be found at different scales, positions, and angles. The knowledge learned from a regional structure pattern shall be transferred to other similar patterns. An encoding and generation mechanism, namely the Multiscale Structure-based Point Cloud Generator (MSPCG) is proposed, which can simultaneously learn point cloud generation from local patterns with miscellaneous spatial properties. The proposed method supports multiscale editions on point clouds by editing the MSG. By generating point clouds from local structures and learning simultaneously in multiple scales, our MSPCG has better generalization ability and scalability. Trained on the ShapeNet, our MSPCG can generate point clouds from a given structure for unseen categories and indoor scenes. The experimental results show that our method significantly outperforms baseline methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. Learning representations and generative models for 3d point clouds. arXiv preprint arXiv:1707.02392 (2017).
  2. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203 (2013).
  3. ShapeNet: An Information-Rich 3D Model Repository. Technical Report arXiv:1512.03012 [cs.GR]. Stanford University — Princeton University — Toyota Technological Institute at Chicago.
  4. Neural ordinary differential equations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. 6572–6583.
  5. ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE.
  6. Learning elementary structures for 3d shape generation and matching. Advances in Neural Information Processing Systems 32 (2019).
  7. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition. 605–613.
  8. Generative adversarial networks. Commun. ACM 63, 11 (2020), 139–144.
  9. FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models. International Conference on Learning Representations (2019).
  10. AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).
  11. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 1025–1035.
  12. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
  13. Diederik P Kingma and Prafulla Dhariwal. 2018. Glow: Generative flow with invertible 1x1 convolutions. arXiv preprint arXiv:1807.03039 (2018).
  14. Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).
  15. Structure-Aware Shape Processing. In ACM SIGGRAPH 2014 Courses (Vancouver, Canada) (SIGGRAPH ’14). Association for Computing Machinery, New York, NY, USA, Article 13, 21 pages.
  16. StructureNet: Hierarchical Graph Networks for 3D Shape Generation. ACM Transactions on Graphics (TOG), Siggraph Asia 2019 38, 6 (2019), Article 242.
  17. PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions. European conference on computer vision (ECCV 2020) (2020).
  18. PartNet: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level 3D Object Understanding. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
  19. Conditional image generation with pixelcnn decoders. arXiv preprint arXiv:1606.05328 (2016).
  20. Tearingnet: Point cloud autoencoder to learn topology-friendly representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7453–7462.
  21. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 701–710.
  22. Waveglow: A flow-based generative network for speech synthesis. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 3617–3621.
  23. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. arXiv preprint arXiv:1706.02413 (2017).
  24. Dynamic routing between capsules. Advances in neural information processing systems 30 (2017).
  25. Natural tts synthesis by conditioning wavenet on mel spectrogram predictions. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 4779–4783.
  26. 3d point cloud generative adversarial network based on tree structured graph convolutions. In Proceedings of the IEEE International Conference on Computer Vision. 3859–3868.
  27. Canonical Capsules: Self-Supervised Capsules in Canonical Pose. Advances in Neural Information Processing Systems 34 (2021).
  28. Learning localized generative models for 3d point clouds via graph convolution. In International conference on learning representations.
  29. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).
  30. 3D ShapeNets: A Deep Representation for Volumetric Shapes. In Computer Vision and Pattern Recognition.
  31. PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows. arXiv (2019).
  32. CPCGAN: A Controllable 3D Point Cloud Generative Adversarial Network with Semantic Label Generating. Proceedings of the AAAI Conference on Artificial Intelligence 35, 4 (May 2021), 3154–3162. https://ojs.aaai.org/index.php/AAAI/article/view/16425
  33. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 206–215.
  34. 3D point capsule networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1009–1018.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now