Deep Learning for 3D Point Clouds: A Survey (1912.12033v2)

Abstract: Point cloud learning has lately attracted increasing attention due to its wide applications in many areas, such as computer vision, autonomous driving, and robotics. As a dominating technique in AI, deep learning has been successfully used to solve various 2D vision problems. However, deep learning on point clouds is still in its infancy due to the unique challenges faced by the processing of point clouds with deep neural networks. Recently, deep learning on point clouds has become even thriving, with numerous methods being proposed to address different problems in this area. To stimulate future research, this paper presents a comprehensive review of recent progress in deep learning methods for point clouds. It covers three major tasks, including 3D shape classification, 3D object detection and tracking, and 3D point cloud segmentation. It also presents comparative results on several publicly available datasets, together with insightful observations and inspiring future research directions.

Deep Learning for 3D Point Clouds: A Survey

The survey paper entitled "Deep Learning for 3D Point Clouds: A Survey", authored by Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, and Mohammed Bennamoun, provides an extensive review of the state-of-the-art methodologies in the domain of deep learning techniques applied to 3D point clouds. This paper aims to catalyze future research by outlining key challenges, presenting comparative analyses across multiple datasets, and suggesting new directions for investigation.

Introduction and Background

3D point clouds have gained substantial attention for their applications across various domains such as computer vision, autonomous driving, and robotics. Unlike 2D images, 3D point clouds preserve original geometric information without discretization, retaining valuable spatial characteristics. However, the irregular and sparse nature of 3D point clouds presents unique challenges in applying deep learning methods, which have achieved remarkable success in 2D data.

The paper presents an in-depth analysis of three primary tasks in 3D point cloud processing: 3D shape classification, 3D object detection and tracking, and 3D point cloud segmentation. It also reviews multiple datasets including ModelNet, ScanObjectNN, ShapeNet, and more, detailing their attributes and utilization in the evaluation of deep learning models.

3D Shape Classification

The task of 3D shape classification involves categorizing objects into predefined classes based on their shapes. Existing methods are segmented into three main approaches: multi-view-based, volumetric-based, and point-based methods.

1. Multi-view-based Methods: These methods generate 2D projections of 3D models from multiple views and leverage 2D CNNs for feature extraction and fusion. Representative methods include MVCNN and View-GCN.
2. Volumetric-based Methods: These approaches convert point clouds into 3D voxel grids, applying 3D CNNs. Notable methods like OctNet, VoxNet, and O-CNN handle the computational efficiency challenges introduced by dense voxel representations.
3. Point-based Methods: Operating directly on raw point clouds, these methods often utilize point-wise MLPs, convolution-based, or graph-based techniques to capture local and global geometric structures. Methods like PointNet/PointNet++ and DGCNN set significant precedents in this paradigm.

Recent advancements show a trend towards improving point-based methods, focusing on hierarchical data structures and hybrid architectures integrating point-based and volumetric representations.

3D Object Detection and Tracking

The aim of 3D object detection is to identify and localize objects within 3D spaces. Techniques are broadly categorized into region proposal-based and single-shot methods.

1. Region Proposal-based Methods: Involving stages of proposal generation followed by object classification, methods are further divided into multi-view-based, segmentation-based, and frustum-based approaches. For instance, PointRCNN and Vote3Deep exemplify segmentation-based and frustum-based strategies respectively.
2. Single-shot Methods: These methods directly predict the class and location of objects in a single forward pass. Approaches like PointPillars and 3DSSD stand out for their efficiency and competitive performance.

Object tracking extends detection to sequence data, with methods like 3D-Siamese networks for real-time applications. New techniques like Point-to-Box introduce novel voting mechanisms to enhance tracking accuracy.

3D Point Cloud Segmentation

Segmentation tasks in 3D point clouds involve labeling each point according to semantic categories, object instances, or part-level descriptors. This paper discusses various approaches for segmentation:

1. Semantic Segmentation: Methods can be categorized into projection-based, discretization-based, and point-based approaches. Highlights include volumetric representation methods such as SEGCloud, and point-based networks like PointNet++.
2. Instance Segmentation: Involves not only recognizing semantic categories but also distinguishing individual object instances. Techniques are bifurcated into proposal-based methods (e.g., 3D-SIS) and proposal-free methods (e.g., SGPN).
3. Part Segmentation: Focuses on segmenting individual parts of objects. Techniques leverage both supervised and unsupervised learning mechanisms, encapsulated in methods such as PartNet and VoxSegNet.

Implications and Future Directions

The survey identifies key challenges and future research opportunities, such as:

• Enhancing long-range detection capabilities and leveraging multi-task learning.
• Ensuring computational efficiency for large-scale point cloud data.
• Addressing data imbalances for minority classes in segmentation tasks.

Advancements in multi-modal learning, integration of temporal data, and exploring new representations (e.g., sparse voxel grids), are poised to drive progress in this domain.

Conclusion

This comprehensive survey furnishes a robust synthesis of current methodologies for 3D point cloud processing via deep learning. It underscores impending research challenges and anticipates that upcoming developments will likely pivot on improving computational efficiency, exploiting multi-modal data, and refining the granularity of segmentation tasks. The field stands on the brink of transcending current limitations, potentially redefining practical applications in autonomous systems, robotics, and beyond.