Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping (1910.02490v3)

Abstract: We provide an open-source C++ library for real-time metric-semantic visual-inertial Simultaneous Localization And Mapping (SLAM). The library goes beyond existing visual and visual-inertial SLAM libraries (e.g., ORB-SLAM, VINS- Mono, OKVIS, ROVIO) by enabling mesh reconstruction and semantic labeling in 3D. Kimera is designed with modularity in mind and has four key components: a visual-inertial odometry (VIO) module for fast and accurate state estimation, a robust pose graph optimizer for global trajectory estimation, a lightweight 3D mesher module for fast mesh reconstruction, and a dense 3D metric-semantic reconstruction module. The modules can be run in isolation or in combination, hence Kimera can easily fall back to a state-of-the-art VIO or a full SLAM system. Kimera runs in real-time on a CPU and produces a 3D metric-semantic mesh from semantically labeled images, which can be obtained by modern deep learning methods. We hope that the flexibility, computational efficiency, robustness, and accuracy afforded by Kimera will build a solid basis for future metric-semantic SLAM and perception research, and will allow researchers across multiple areas (e.g., VIO, SLAM, 3D reconstruction, segmentation) to benchmark and prototype their own efforts without having to start from scratch.

• The paper introduces Kimera as a groundbreaking library that advances real-time SLAM by integrating visual-inertial odometry with semantic mapping.
• Its modular design features components for VIO, robust pose graph optimization, rapid 3D meshing, and semantic fusion, enhancing accuracy and efficiency.
• Evaluations demonstrate superior pose estimation and precise semantic reconstruction, accelerating research and supporting advanced robotic applications.

Overview of "Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping"

The paper "Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping" introduces Kimera, a comprehensive C++ library aiming to facilitate real-time Simultaneous Localization and Mapping (SLAM) with a focus on metric-semantic integration. The authors present Kimera as a significant advancement in the domain by offering capabilities that extend beyond traditional approaches such as ORB-SLAM and VINS-Mono, incorporating semantic understanding alongside geometric mapping.

Core Components and Architecture

Kimera is designed to seamlessly combine visual-inertial odometry with robust metric-semantic mapping. It comprises four main components:

1. Kimera-VIO: This module provides visual-inertial odometry (VIO) using a keyframe-based approach that performs fixed-lag smoothing or full smoothing depending on the requirements. It boasts impressive performance metrics on the EuRoC dataset, powered by a structureless vision model and preintegration theories for IMU data.
2. Kimera-RPGO: The robust pose graph optimization (RPGO) module is engineered to ensure global trajectory estimation robustness by implementing modern outlier rejection methods. Kimera-RPGO demonstrates resilience to parameter variations in loop closure thresholds, showcasing a notable reduction in implementation complexity for users.
3. Kimera-Mesher: This component is dedicated to generating rapid 3D meshes that can be leveraged for immediate applications such as obstacle avoidance. It supports both per-frame and multi-frame mesh reconstructions, providing options for semantic annotation using 2D labels.
4. Kimera-Semantics: Focused on producing global and semantically annotated 3D meshes, this module employs a volumetric approach, using dense stereo methods for depth estimation and volumetric raycasting for semantic fusion.

Evaluation and Results

Kimera presents a robust benchmark performance in various aspects:

• Pose Estimation: It outperforms several state-of-the-art VIO algorithms, achieving minimal root mean squared error (RMSE) in absolute trajectory error (ATE) across benchmark scenarios on the EuRoC dataset.
• Geometric Accuracy: The 3D meshes generated by Kimera-Mesher and Kimera-Semantics are evaluated for accuracy and completeness against ground truth data. Although the multi-frame meshes are computationally efficient, the TSDF-based meshes from Kimera-Semantics provide higher precision at the cost of increased computation time.
• Semantic Integration: Performance in semantic reconstruction is examined using a photo-realistic simulator, highlighting Kimera's efficacy in preserving the accuracy of semantic segmentation when using both ground-truth data and algorithmically derived estimates.

Implications and Future Prospects

The implications of this work are multifaceted:

• Research Acceleration: By providing an open-source, modular architecture, Kimera lowers the barrier to entry for researchers aiming to develop advanced SLAM systems. The library enables rapid prototyping and experimentation without necessitating the reconstruction of foundational components.
• Application Development: In practical terms, Kimera facilitates a range of robotics applications, including autonomous navigation, where machines require a precise understanding of their environment in both geometric and semantic terms.
• Advancement in SLAM Research: The integration of semantic data into SLAM offers new avenues for research. It opens possibilities for more contextually aware robotic systems capable of interacting with environments at higher levels of abstraction.

Looking ahead, Kimera sets the groundwork for future developments in SLAM systems, encouraging further exploration into optimizing CPU-based performance and enhancing semantic segmentation techniques. As sensor technology and deep learning continue to evolve, these improvements are likely to foster more sophisticated and efficient systems in real-world robotic applications.