STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point Cloud Dataset (2203.09065v3)

Abstract: Although various 3D datasets with different functions and scales have been proposed recently, it remains challenging for individuals to complete the whole pipeline of large-scale data collection, sanitization, and annotation. Moreover, the created datasets usually suffer from extremely imbalanced class distribution or partial low-quality data samples. Motivated by this, we explore the procedurally synthetic 3D data generation paradigm to equip individuals with the full capability of creating large-scale annotated photogrammetry point clouds. Specifically, we introduce a synthetic aerial photogrammetry point clouds generation pipeline that takes full advantage of open geospatial data sources and off-the-shelf commercial packages. Unlike generating synthetic data in virtual games, where the simulated data usually have limited gaming environments created by artists, the proposed pipeline simulates the reconstruction process of the real environment by following the same UAV flight pattern on different synthetic terrain shapes and building densities, which ensure similar quality, noise pattern, and diversity with real data. In addition, the precise semantic and instance annotations can be generated fully automatically, avoiding the expensive and time-consuming manual annotation. Based on the proposed pipeline, we present a richly-annotated synthetic 3D aerial photogrammetry point cloud dataset, termed STPLS3D, with more than 16 $km^2$ of landscapes and up to 18 fine-grained semantic categories. For verification purposes, we also provide a parallel dataset collected from four areas in the real environment. Extensive experiments conducted on our datasets demonstrate the effectiveness and quality of the proposed synthetic dataset.

Exploring STPLS3D: A Comprehensive 3D Point Cloud Dataset for Photogrammetry

The paper introduces STPLS3D, a richly annotated synthetic and real aerial photogrammetry 3D point cloud dataset aimed at addressing the challenges associated with large-scale 3D data collection, annotation, and semantic segmentation. The authors propose a novel pipeline for generating synthetic 3D datasets, leveraging open geospatial data and commercial software to replicate real-world photogrammetry processes. This innovative pipeline promises to overcome the common issues of imbalanced class distribution and low-quality data samples inherent in existing datasets, while significantly reducing the manual effort and cost involved in data annotation.

Highlights of STPLS3D

The key contributions of the paper lie in the development of the STPLS3D dataset and its associated synthetic data generation pipeline, which provides several distinct advantages:

1. Comprehensive Semantic and Instance Annotation: The dataset includes over 16 square kilometers of synthetic landscapes with up to 18 fine-grained semantic categories and instance annotations, ensuring rich and diverse data coverage. This is complemented by real datasets collected from four distinct real-world sites, facilitating direct comparisons between synthetic and real datasets.
2. Automated and Scalable Data Generation: By employing a procedural approach, the pipeline automates the generation of photorealistic 3D environments, simulates UAV flight paths, and includes variations in building styles and terrains. This automation supports scalable data production that can be parallelized, lowering barriers for individuals and smaller research teams.
3. Utility for Deep Learning Applications: Experiments conducted with the dataset demonstrate its utility in enhancing deep learning models for 3D semantic segmentation. Incorporating synthetic data into training regimens leads to noticeable performance improvements on real-world data. Tools like RandLA-Net and KPConv showed marked performance gains, indicating the efficacy of STPLS3D as a training augmentation resource.
4. Comparison and Real-World Validation: A comparison with existing aerial datasets highlights STPLS3D’s broader semantic scope and area coverage, strengthening its position as a valuable resource for photogrammetry research.

Implications and Future Directions

Practically, STPLS3D could empower a variety of applications in urban planning, environmental monitoring, and autonomous navigation, where large, annotated 3D datasets are critical. The synthetic dataset enriches model training, potentially improving model generalization—a crucial aspect for deploying AI in real-world scenarios with diverse terrains and structures.

Theoretically, the paper offers a framework for creating large-scale synthetic datasets with minimal manual intervention, highlighting a potential paradigm shift in dataset generation for computer vision tasks. It paves the way for future exploration into domain adaptation and representation learning, where synthetic datasets help bridge gaps present due to domain discrepancies.

Future advancements in STPLS3D may focus on further reducing the gap between synthetic and real data, potentially through the introduction of more sophisticated procedural generation techniques or enhanced customization options for synthetic environments. Additionally, exploring the applicability of STPLS3D in other vision-related tasks such as object detection or 3D reconstructions can extend its utility beyond segmentation.

In conclusion, STPLS3D represents a significant step forward in synthesizing realistic 3D datasets that enhance both practical capabilities and theoretical understanding in aerial photogrammetry and its associated AI applications. It sets a precedent for future datasets, where automation, diversity, and quality converge to support robust machine learning development.