Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Hybrid Generative and Discriminative PointNet on Unordered Point Sets (2404.12925v1)

Published 19 Apr 2024 in cs.CV

Abstract: As point cloud provides a natural and flexible representation usable in myriad applications (e.g., robotics and self-driving cars), the ability to synthesize point clouds for analysis becomes crucial. Recently, Xie et al. propose a generative model for unordered point sets in the form of an energy-based model (EBM). Despite the model achieving an impressive performance for point cloud generation, one separate model needs to be trained for each category to capture the complex point set distributions. Besides, their method is unable to classify point clouds directly and requires additional fine-tuning for classification. One interesting question is: Can we train a single network for a hybrid generative and discriminative model of point clouds? A similar question has recently been answered in the affirmative for images, introducing the framework of Joint Energy-based Model (JEM), which achieves high performance in image classification and generation simultaneously. This paper proposes GDPNet, the first hybrid Generative and Discriminative PointNet that extends JEM for point cloud classification and generation. Our GDPNet retains strong discriminative power of modern PointNet classifiers, while generating point cloud samples rivaling state-of-the-art generative approaches.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. Learning Representations and Generative Models For 3D Point Clouds. Proceedings of the 35th International Conference on Machine Learning (ICML), pages 40–49, 2018.
  2. Sharpness-aware minimization improves language model generalization. In Annual Meeting of the Association for Computational Linguistics (ACL), 2022.
  3. Jonathan T Barron. Continuously differentiable exponential linear units. arXiv preprint arXiv:1704.07483, 2017.
  4. Learning gradient fields for shape generation. In European Conference on Computer Vision, pages 364–381. Springer, 2020.
  5. When vision transformers outperform resnets without pre-training or strong data augmentations. In International Conference on Learning Representations (ICLR), 2022.
  6. Residual energy-based models for text generation. arXiv preprint arXiv:2004.11714, 2020.
  7. Implicit generation and generalization in energy-based models. In Neural Information Processing Systems (NeurIPS), 2019.
  8. Sharpness-aware minimization for efficiently improving generalization. In International Conference on Learning Representations, 2021.
  9. Multiresolution tree networks for 3d point cloud processing. In Proceedings of the European Conference on Computer Vision (ECCV), pages 103–118, 2018.
  10. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 3354–3361, 2012.
  11. Generative adversarial nets. In Neural Information Processing Systems (NeurIPS), 2014.
  12. Ffjord: Free-form continuous dynamics for scalable reversible generative models. In International Conference on Learning Representations (ICLR), 2019.
  13. Your classifier is secretly an energy based model and you should treat it like one. In International Conference on Learning Representations (ICLR), 2020.
  14. A papier-mâché approach to learning 3d surface generation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 216–224, 2018.
  15. Multi-Angle Point Cloud-VAE: Unsupervised Feature Learning for 3D Point Clouds from Multiple Angles by Joint Self-Reconstruction and Half-to-Half Prediction. Proceedings of the International Conference on Computer Vision (ICCV), 2019.
  16. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2016a.
  17. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016b.
  18. Geoffrey E Hinton. Training products of experts by minimizing contrastive divergence. Neural computation, 2002.
  19. 3d volumetric modeling with introspective neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 8481–8488, 2019.
  20. Aapo Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 2005.
  21. Learning protein structure with a differentiable simulator. In International Conference on Learning Representations, 2018.
  22. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning (ICML), 2015.
  23. On large-batch training for deep learning: Generalization gap and sharp minima. In International Conference on Learning Representations (ICLR), 2017.
  24. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.
  25. Auto-encoding variational bayes. In International Conference on Learning Representations (ICLR), 2014.
  26. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2012.
  27. Asam: Adaptive sharpness-aware minimization for scale-invariant learning of deep neural networks. In International Conference on Machine Learning (ICML), 2021.
  28. Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4558–4567, 2018.
  29. A tutorial on energy-based learning. Predicting structured data, 2006.
  30. Visualizing the Loss Landscape of Neural Nets. In Neural Information Processing Systems (NeurIPS), 2018a.
  31. SO-Net: Self-Organizing Network for Point Cloud Analysis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 9397–9406, 2018b.
  32. PointCNN: Convolution On X-Transformed Points. Proceedings of Advances in Neural Information Processing Systems (NeuralIPS), 2018c.
  33. Energy-based out-of-distribution detection. Neural Information Processing Systems (NeurIPS), 2020.
  34. Relation-Shape Convolutional Neural Network for Point Cloud Analysis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 8895–8904, 2019.
  35. Diffusion probabilistic models for 3d point cloud generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2837–2845, 2021.
  36. Rectifier nonlinearities improve neural network acoustic models. In International Conference on Machine Learning (ICML), 2013.
  37. Rectified linear units improve restricted boltzmann machines. In International Conference on Machine Learning (ICML), 2011.
  38. Learning non-convergent short-run mcmc toward energy-based model. In Neural Information Processing Systems (NeurIPS), 2019.
  39. Towards semantic maps for mobile robots. Robotics Auton. Syst., 56:915–926, 2008.
  40. Multiple 3d object tracking for augmented reality. In 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, pages 117–120, 2008.
  41. Volumetric and Multi-View CNNs for Object Classification on 3D Data. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5648–5656, 2016.
  42. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 652–660, 2017a.
  43. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. Proceedings of Advances in Neural Information Processing Systems (NeuralIPS), 2017b.
  44. Deep Hough Voting for 3D Object Detection in Point Clouds. Proceedings of the International Conference on Computer Vision (ICCV), 2019.
  45. Generative modeling by estimating gradients of the data distribution. In Neural Information Processing Systems (NeurIPS), 2019.
  46. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020.
  47. SPLATNet: Sparse Lattice Networks for Point Cloud Processing. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2530–2539, 2018.
  48. Pointgrow: Autoregressively learned point cloud generation with self-attention. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 61–70, 2020.
  49. On autoencoders and score matching for energy based models. In International Conference on Machine Learning (ICML), 2011.
  50. KPConv: Flexible and Deformable Convolution for Point Clouds. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019.
  51. SGPN: Similarity Group Proposal Network for 3D Point Cloud InstanceSegmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2569–2578, 2018.
  52. Dynamic Graph CNN for Learning on Point Clouds. ACM Transactions on Graphics (TOG), 2019.
  53. The implicit and explicit regularization effects of dropout. In International Conference on Machine Learning (ICML), 2020.
  54. Bayesian learning via stochastic gradient langevin dynamics. In International Conference on Machine Learning (ICML), 2011.
  55. Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. Advances in neural information processing systems, 29, 2016.
  56. PointConv: Deep Convolutional Networks on 3D Point Clouds. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 9622–9630, 2019.
  57. 3d shapenets: A deep representation for volumetric shapes. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1912–1920, Los Alamitos, CA, USA, 2015a. IEEE Computer Society.
  58. 3D ShapeNets: A Deep Representation for Volumetric Shapes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1912–1920, 2015b.
  59. A theory of generative convnet. In International Conference on Machine Learning (ICML), 2016.
  60. Synthesizing dynamic patterns by spatial-temporal generative convnet. In Proceedings of the ieee conference on computer vision and pattern recognition, pages 7093–7101, 2017.
  61. Learning descriptor networks for 3d shape synthesis and analysis. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8629–8638, 2018.
  62. Learning energy-based spatial-temporal generative convnets for dynamic patterns. IEEE transactions on pattern analysis and machine intelligence, 43(2):516–531, 2019.
  63. Cooperative training of descriptor and generator networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020a.
  64. Generative voxelnet: learning energy-based models for 3d shape synthesis and analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020b.
  65. Generative pointnet: Deep energy-based learning on unordered point sets for 3d generation, reconstruction and classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14976–14985, 2021a.
  66. Cooperative training of fast thinking initializer and slow thinking solver for conditional learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021b.
  67. PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019.
  68. JEM++: Improved Techniques for Training JEM. In International Conference on Computer Vision (ICCV), 2021.
  69. FoldingNet: Point Cloud Auto-encoder via Deep Grid Deformation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 206–215, 2018.
  70. PU-Net: Point Cloud Upsampling Network. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2790–2799, 2018.
  71. 3D Point-Capsule Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1009–1018, 2019.
  72. Learning energy-based generative models via coarse-to-fine expanding and sampling. In International Conference on Learning Representations ICLR, 2021.
  73. Filters, random fields and maximum entropy (frame): Towards a unified theory for texture modeling. International Journal of Computer Vision, 27(2):107–126, 1998.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets