A Data-efficient Framework for Robotics Large-scale LiDAR Scene Parsing (2312.02208v1)

Abstract: Existing state-of-the-art 3D point clouds understanding methods only perform well in a fully supervised manner. To the best of our knowledge, there exists no unified framework which simultaneously solves the downstream high-level understanding tasks, especially when labels are extremely limited. This work presents a general and simple framework to tackle point clouds understanding when labels are limited. We propose a novel unsupervised region expansion based clustering method for generating clusters. More importantly, we innovatively propose to learn to merge the over-divided clusters based on the local low-level geometric property similarities and the learned high-level feature similarities supervised by weak labels. Hence, the true weak labels guide pseudo labels merging taking both geometric and semantic feature correlations into consideration. Finally, the self-supervised reconstruction and data augmentation optimization modules are proposed to guide the propagation of labels among semantically similar points within a scene. Experimental Results demonstrate that our framework has the best performance among the three most important weakly supervised point clouds understanding tasks including semantic segmentation, instance segmentation, and object detection even when limited points are labeled, under the data-efficient settings for the large-scale 3D semantic scene parsing. The developed techniques have postentials to be applied to downstream tasks for better representations in robotic manipulation and robotic autonomous navigation. Codes and models are publicly available at: https://github.com/KangchengLiu.

• The paper introduces a framework that leverages weak supervision and unsupervised region expansion to parse large-scale 3D LiDAR scenes.
• It utilizes both low-level geometric and high-level feature similarities for effective semantic and instance segmentation.
• Experimental results on ScanNet, S3DIS, and KITTI demonstrate its competitive performance with minimal labeled data.

Introduction to Data-efficient Framework

Large-scale 3D scene parsing using LiDAR is an essential component of autonomous systems, including robots and vehicles. However, the challenge of understanding point clouds rests heavily on having a large number of high-quality labeled data which is often unavailable or labor-intensive to acquire. The pursuit of a reliable framework that operates under weakly supervised learning (WSL) conditions is the goal of this work. This framework aims to understand point clouds even with limited labels while still achieving tasks such as semantic segmentation, instance segmentation, and object detection.

Methodology

In the proposed approach, an unsupervised region expansion method is introduced, leveraging local geometric properties to create initial clusters. These clusters are further processed based on both low-level geometric and high-level feature similarities, ultimately addressing the semantic side of scene parsing as well. The framework is designed to handle weak labels by optimizing the clustering process through data augmentation and a series of learning modules that guide the merging of clusters with similar semantic characteristics.

Additionally, an unsupervised learning strategy for instance segmentation is presented. This initiative facilities providing supervisory information for object detection tasks, leading to a comprehensive understanding of point clouds under weak supervision constraints.

Experimental Results

The framework's performance is verified through extensive experiments across multiple benchmarks such as ScanNet, S3DIS, and KITTI. Its achievements are then compared with fully supervised state-of-the-art approaches, demonstrating that even with very few labeled points, the proposed method can either match or significantly outperform current weakly supervised methods. The approach has been tested under a variety of weakly supervised settings and also in a completely unsupervised manner for object detection, reflecting its capability to generalize across different task requirements and scenes.

Potential Applications and Conclusions

The developed techniques hold potential for applications in areas where collecting extensive labeled data is impractical or impossible, such as robotic exploration in unknown environments or autonomous navigation in complex outdoor scenarios. The results suggest that the proposed framework can serve as a robust and versatile solution for weakly supervised 3D point cloud understanding, aiding in the advancement of autonomous systems that rely on LiDAR data. The availability of the framework's code to the public ensures that it can be utilized and further enhanced by the research community, contributing to the development of efficient learning-based methods for large-scale semantic scene parsing with limited supervision.