The ApolloScape Open Dataset for Autonomous Driving and its Application (1803.06184v4)

Abstract: Autonomous driving has attracted tremendous attention especially in the past few years. The key techniques for a self-driving car include solving tasks like 3D map construction, self-localization, parsing the driving road and understanding objects, which enable vehicles to reason and act. However, large scale data set for training and system evaluation is still a bottleneck for developing robust perception models. In this paper, we present the ApolloScape dataset [1] and its applications for autonomous driving. Compared with existing public datasets from real scenes, e.g. KITTI [2] or Cityscapes [3], ApolloScape contains much large and richer labelling including holistic semantic dense point cloud for each site, stereo, per-pixel semantic labelling, lanemark labelling, instance segmentation, 3D car instance, high accurate location for every frame in various driving videos from multiple sites, cities and daytimes. For each task, it contains at lease 15x larger amount of images than SOTA datasets. To label such a complete dataset, we develop various tools and algorithms specified for each task to accelerate the labelling process, such as 3D-2D segment labeling tools, active labelling in videos etc. Depend on ApolloScape, we are able to develop algorithms jointly consider the learning and inference of multiple tasks. In this paper, we provide a sensor fusion scheme integrating camera videos, consumer-grade motion sensors (GPS/IMU), and a 3D semantic map in order to achieve robust self-localization and semantic segmentation for autonomous driving. We show that practically, sensor fusion and joint learning of multiple tasks are beneficial to achieve a more robust and accurate system. We expect our dataset and proposed relevant algorithms can support and motivate researchers for further development of multi-sensor fusion and multi-task learning in the field of computer vision.

• The paper introduces the ApolloScape dataset, a large-scale resource with dense 3D semantic point clouds, stereoscopic videos, and precise pose information for autonomous driving research.
• The paper outlines a novel 3D-2D joint labeling pipeline that streamlines annotation and enhances semantic segmentation via active learning techniques.
• The paper demonstrates that multi-task learning and sensor fusion algorithms leveraging ApolloScape yield robust localization and improved segmentation performance.

ApolloScape: A Comprehensive Dataset for Autonomous Driving Research

The paper entitled "The ApolloScape Open Dataset for Autonomous Driving and its Application" provides an extensive overview of the ApolloScape dataset, a large-scale dataset aimed at advancing research in autonomous driving. The dataset fills a significant gap by providing comprehensive data that supports a multitude of tasks necessary for developing robust autonomous vehicles.

Key Features of ApolloScape

ApolloScape distinguishes itself by offering a more extensive and richly labeled dataset compared to existing datasets such as KITTI and Cityscapes. It provides:

• Dense 3D Semantic Point Clouds: Covering over 20 driving sites with exhaustive semantic labeling.
• Stereoscopic Driving Videos: Comprising more than 100 hours of footage, enriching temporal learning capabilities.
• High-Fidelity Pose Information: Offering precise 6DoF camera pose details, with sub-centimeter translation accuracy and sub-degree rotation accuracy.
• Diverse Environmental Conditions: Videos recorded at various daytimes, capturing different lighting conditions.
• Comprehensive Semantic Labeling: Over 144,000 images labeled at a per-pixel level for semantic understanding across 35 classes.
• Instance Segmentation and Lanemark Labeling: Extensive instance labeling for movable objects and 160,000 images annotated for lane marks.

Dataset Labeling and Development Methodologies

The dataset's scale necessitates innovative labeling methodologies, particularly given the complexity of autonomous driving environments. The authors highlight a 3D-2D joint labeling pipeline designed to streamline the annotation process. This process significantly reduces manual labeling time by employing active learning techniques and leveraging 3D semantic maps for transferring labels to 2D images via projection.

Proposed Algorithms and Applications

Alongside the dataset, the authors introduce algorithms that integrate multi-task learning and sensor fusion. These algorithms are designed to leverage the dataset's comprehensive nature to tackle tasks such as self-localization and semantic segmentation. The paper demonstrates the benefits of joint learning through the accurate and robust performance of their proposed system.

Performance and Implications

The performance metrics provided in the paper suggest ApolloScape’s potential to drive forward research in autonomous driving:

• A sensor fusion approach integrating camera data with consumer-grade GPS/IMU shows marked improvement in localization and segmentation tasks.
• The dataset's extensive scale, particularly with its task diversity and detailed labeling, offers an opportunity to explore more complex models capable of real-time inference.

Future Developments

ApolloScape's future iterations aim to expand the dataset further by incorporating additional environments and conditions, such as inclement weather, as well as increased geographical diversity. These enhancements will likely make ApolloScape even more vital for the development and testing of robust, production-ready autonomous driving systems.

Conclusion

The ApolloScape dataset stands as a substantial contribution to the field of autonomous driving research. Its comprehensive nature supports a wide range of tasks essential for autonomous vehicle development. As it evolves, ApolloScape anticipates fostering new advancements and innovations in AI-driven autonomy, making it a cornerstone resource for researchers in the field.