Feature Adversarial Distillation for Point Cloud Classification (2306.14221v2)
Abstract: Due to the point cloud's irregular and unordered geometry structure, conventional knowledge distillation technology lost a lot of information when directly used on point cloud tasks. In this paper, we propose Feature Adversarial Distillation (FAD) method, a generic adversarial loss function in point cloud distillation, to reduce loss during knowledge transfer. In the feature extraction stage, the features extracted by the teacher are used as the discriminator, and the students continuously generate new features in the training stage. The feature of the student is obtained by attacking the feedback from the teacher and getting a score to judge whether the student has learned the knowledge well or not. In experiments on standard point cloud classification on ModelNet40 and ScanObjectNN datasets, our method reduced the information loss of knowledge transfer in distillation in 40x model compression while maintaining competitive performance.
- “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.
- “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” Advances in neural information processing systems, vol. 30, 2017.
- “Dynamic graph cnn for learning on point clouds,” Acm Transactions On Graphics (tog), 2019.
- “Deepgcns: Can gcns go as deep as cnns?,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019.
- “Learning inner-group relations on point clouds,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
- “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
- “Densely connected convolutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.
- “Multi-scale context aggregation by dilated convolutions,” arXiv preprint arXiv:1511.07122, 2015.
- “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
- “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020.
- “Segmenter: Transformer for semantic segmentation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
- “Pct: Point cloud transformer,” Computational Visual Media, vol. 7, no. 2, 2021.
- “Point transformer,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
- “Exploring self-attention for image recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
- “Rethinking network design and local geometry in point cloud: A simple residual mlp framework,” arXiv preprint arXiv:2202.07123, 2022.
- “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, vol. 2, no. 7, 2015.
- “Fitnets: Hints for thin deep nets,” arXiv preprint arXiv:1412.6550, 2014.
- “Relational knowledge distillation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
- “Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer,” in ICLR, 2017.
- “A task-driven scene-aware lidar point cloud coding framework for autonomous vehicles,” IEEE Transactions on Industrial Informatics, 2022.
- “An advanced lidar point cloud sequence coding scheme for autonomous driving,” in Proceedings of the 28th ACM International Conference on Multimedia, 2020.
- “Empowering knowledge distillation via open set recognition for robust 3d point cloud classification,” Pattern Recognition Letters, vol. 151, 2021.
- “Pointdistiller: Structured knowledge distillation towards efficient and compact 3d detection,” arXiv preprint arXiv:2205.11098, 2022.
- “Point-to-voxel knowledge distillation for lidar semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
- “3d shapenets: A deep representation for volumetric shapes,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015.
- “Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.