ASSANet: An Anisotropic Separable Set Abstraction for Efficient Point Cloud Representation Learning (2110.10538v2)

Abstract: Access to 3D point cloud representations has been widely facilitated by LiDAR sensors embedded in various mobile devices. This has led to an emerging need for fast and accurate point cloud processing techniques. In this paper, we revisit and dive deeper into PointNet++, one of the most influential yet under-explored networks, and develop faster and more accurate variants of the model. We first present a novel Separable Set Abstraction (SA) module that disentangles the vanilla SA module used in PointNet++ into two separate learning stages: (1) learning channel correlation and (2) learning spatial correlation. The Separable SA module is significantly faster than the vanilla version, yet it achieves comparable performance. We then introduce a new Anisotropic Reduction function into our Separable SA module and propose an Anisotropic Separable SA (ASSA) module that substantially increases the network's accuracy. We later replace the vanilla SA modules in PointNet++ with the proposed ASSA module, and denote the modified network as ASSANet. Extensive experiments on point cloud classification, semantic segmentation, and part segmentation show that ASSANet outperforms PointNet++ and other methods, achieving much higher accuracy and faster speeds. In particular, ASSANet outperforms PointNet++ by $7.4$ mIoU on S3DIS Area 5, while maintaining $1.6 \times $ faster inference speed on a single NVIDIA 2080Ti GPU. Our scaled ASSANet variant achieves $66.8$ mIoU and outperforms KPConv, while being more than $54 \times$ faster.

• The paper introduces ASSANet, an extension of PointNet++ utilizing Anisotropic Separable Set Abstraction modules to improve point cloud representation learning efficiency and accuracy.
• ASSANet optimizes PointNet++'s Set Abstraction module using separable and anisotropic techniques to reduce computational cost while enhancing feature aggregation.
• Evaluations demonstrate ASSANet's superior accuracy and significantly faster inference speed compared to PointNet++, suitable for efficient point cloud processing in practice.

ASSANet: Enhancing Efficiency and Accuracy in Point Cloud Representation Learning

The paper introduces a novel architecture called ASSANet, which aims to improve the efficiency and accuracy of point cloud representation learning. The research revisits the widely recognized PointNet++ model and proposes significant modifications to address the computational inefficiencies and limitations in accuracy associated with its original design. By introducing a series of Set Abstraction (SA) module variants, the authors effectively balance computational efficiency and robustness in learning point cloud representations.

The authors identify that the computational bottleneck within PointNet++ is largely attributed to the multilevel perceptrons (MLPs) executed on neighborhood features in the Set Abstraction module. This work presents several innovations to mitigate these inefficiencies. Initially, the PreConv SA module is introduced, which processes MLPs on point features directly, substantially cutting down the computational cost. This method reduces the FLOPs by a factor approximately equal to the number of neighbors (K). Although improving speed, this variant shows a compromise in accuracy.

The Separable SA module builds on this by introducing separable MLPs—analogous to depth-wise separable convolutions in CNNs—that disentangle channel and spatial correlations, coupled with residual connections to enhance feature embedding without additional computational burden. While this adjustment provides a significant boost in inference speed, it initially shows diminished accuracy relative to the original PointNet++.

To reclaim and exceed the performance levels, the authors propose the Anisotropic Separable Set Abstraction (ASSA) module, incorporating an Anisotropic Reduction layer. This layer introduces geometric awareness by utilizing relative positions in scaling feature aggregation, treating each neighbor distinctly, thereby enhancing the network's representation capability. This anisotropic approach overcomes the inherent isotropy in conventional pooling operations.

The efficacy of the ASSANet is evidenced through comprehensive experiments spanning multiple tasks, including semantic segmentation on S3DIS, object classification on ModelNet40, and part segmentation on ShapeNetPart. The results highlight that ASSANet not only achieves superior accuracy compared to PointNet++ but also surpasses state-of-the-art methods in terms of performance symmetry between accuracy and speed. Notably, ASSANet exceeds PointNet++ by 7.4% mIoU on S3DIS Area 5 while maintaining 1.6 times faster inference speed on a GTX 2080Ti GPU. The scaled variant, ASSANet (L), presents comparable results to state-of-the-art methods like KPConv while operating at speeds over 54 times faster.

Furthermore, the research explores the implications of scaling methods, exploring the effects of increasing the network's width and depth. The findings suggest that while both scaling strategies enhance accuracy, they exert distinct impacts on computational efficiency, with diminishing returns observed as networks become wider or deeper.

In conclusion, the paper presents a meticulously crafted extension to the PointNet++ framework, offering significant practical and theoretical advancements in point cloud processing. The ASSANet architecture signifies a decisive step towards deploying point-based methods in mobile and embedded systems, where both computational efficiency and high accuracy are critical. Future research directions may involve integrating ASSANet with other point cloud processing techniques or exploring compound scaling akin to strategies used in networks like EfficientNet.