ORB-SLAM3: Advanced Visual-Inertial SLAM

• ORB-SLAM3 is a feature-based SLAM system that supports diverse sensor setups including monocular, stereo, and RGB-D cameras with inertial integration.
• It employs a tightly-coupled visual-inertial backend using MAP estimation to rapidly converge on metric scale and refine pose accuracy.
• Its innovative multi-map and place recognition architecture enhances long-term robustness and efficiency for real-world, dynamic, and large-scale operations.

ORB-SLAM3 is a feature-based, open-source simultaneous localization and mapping (SLAM) library supporting visual, visual-inertial, and multi-map SLAM across a range of sensor configurations, including monocular, stereo, and RGB-D cameras with both pinhole and fisheye lenses. The system advances the state of the art by introducing a tightly-integrated visual-inertial backend built on Maximum-a-Posteriori (MAP) estimation and a multi-map architecture enabling robust long-term operation across complex, large, and dynamic environments. The software is developed and maintained by the I3A group at Universidad de Zaragoza.

1. System Architecture and Core Components

ORB-SLAM3 implements a multi-threaded SLAM architecture, structured around three core concurrent threads:

• Tracking Thread: Processes each input frame to extract ORB features (Oriented FAST and Rotated BRIEF), minimizes reprojection and inertial errors, and estimates the camera pose relative to the currently active map.
• Local Mapping Thread: Handles keyframe insertion, pruning of redundant information, and local Bundle Adjustment (BA), which is either visual or visual-inertial depending on modality. For visual-inertial configurations, this thread also executes a novel IMU initialization via MAP estimation, recovering scale, gravity direction, IMU biases, and velocity.
• Loop Detection and Map Merging Thread: Runs a DBoW2-based place recognition with improved recall, performs geometric verification via direct Sim(3) or SE(3) alignment (using RANSAC and Horn’s method), and executes loop closure or merges disconnected submaps via pose-graph and local BA refinement.

A major architectural innovation in ORB-SLAM3 is the explicit exploitation of mid- and long-term data associations across all stages of the pipeline. This enables the system to incorporate information from previous sessions, widely separated frames, and even multiple maps for both localization and map optimization.

2. Visual-Inertial SLAM and IMU Initialization

A central contribution of ORB-SLAM3 is its tightly coupled, feature-based visual-inertial SLAM, which applies MAP estimation throughout—including the IMU initialization phase. The initialization pipeline proceeds as follows:

• An up-to-scale camera trajectory is first estimated via monocular or stereo visual-only BA.
• An inertial-only MAP problem is solved, leveraging the visual trajectory as a prior, to recover the scale, gravity direction (with a minimal SO(3) optimization), IMU biases, and body velocities.
• Joint visual-inertial BA is performed to refine all estimates.

This approach yields rapid convergence toward metric scale: typical scale error is reduced to below 5% within approximately 2 seconds and can reach ~1% after 15 seconds, even on sequences lacking loops. This methodology ensures robust operation in challenging cases and demonstrates efficiency across short- and long-term motions.

3. Multi-Map Representation and Place Recognition

ORB-SLAM3 introduces an "Atlas" structure, capable of hosting multiple disconnected local maps:

• When tracking is lost (e.g., due to occlusion or textureless regions), the system spawns a new map.
• It continues independent mapping until a place previously seen is recognized using a high-recall DBoW2-based place recognition system.
• Unlike prior systems, ORB-SLAM3’s place recognition does not require consecutive keyframe agreement for geometric verification. Instead, it immediately attempts direct 3D alignment (via Sim(3) or SE(3)), then searches a "welding window" of covisible keyframes to reinforce associations.
• Maps are merged in a seamless process involving pose-graph optimization and local BA refinement, enabling robust multi-session SLAM and efficient long-term operation in dynamic and large-scale environments.

This place recognition framework allows the system to reuse spatial information from all prior operations, thus enhancing both robustness and eventual global accuracy, especially during relocalization and loop closure.

4. Performance Metrics and Benchmarking Results

ORB-SLAM3 has been evaluated extensively on public benchmarks. Notable results:

• On the EuRoC MAV dataset (micro aerial vehicle with stereo-inertial sensing), stereo-inertial mode achieves RMS Absolute Trajectory Errors (ATE) of ~3.6 cm.
• On the TUM-VI room sequences (handheld, rapid motion, AR/VR scenario), the system reaches average accuracy as fine as 9 mm.
• Across all evaluated configurations (monocular, stereo, RGB-D, visual-inertial, stereo-inertial), the system demonstrates robustness equal to, or exceeding, the best existing approaches, and delivers 2–5× higher accuracy in RMS ATE than predecessors.
• Real-time performance is maintained, with the tracking thread operating at 30–40 FPS on typical hardware; local mapping and loop/merge processing run in parallel, avoiding interruption to pose estimation.

These metrics are supported by detailed per-sequence results (RMS ATE in meters and percent scale error) published in the original work.

5. Applications and Use Cases

• Augmented and Virtual Reality (AR/VR): ORB-SLAM3's precise scale estimation and robustness in rapid, unconstrained motion scenarios make it suitable for AR/VR on both handheld and head-mounted platforms.
• Robotics and Mobile Navigation: Multi-session and multi-map capabilities allow for robust operation in large, loop-rich environments and over long durations, addressing needs in service robotics and long-term environment monitoring.
• Academic and Industrial Research: As a fully modular and open-source system, ORB-SLAM3 serves as a foundation for experimentation in semantic mapping, photometric integration, and sensor-fusion research, as well as a proven technology for deployment in real products.

6. Implementation, Open Source, and Community Impact

ORB-SLAM3's source code is publicly available through the “ORB_SLAM3” GitHub repository, maintained by the I3A group at Universidad de Zaragoza (Mar de Luna 1, 50018 Zaragoza, Spain). This commitment to open access has made ORB-SLAM3 the de facto reference platform for both benchmarking and extension in new SLAM research domains (e.g., with deeper semantic integration or direct methods).

The open-source model has led to widespread adoption across both academic and industrial SLAM research and supports contributions in sensor integration, algorithmic refinement, and hardware acceleration. The practical impact is reflected in its use as a core comparison point and baseline system in numerous follow-on studies addressing the challenges of real-world, long-term, and multi-session visual-inertial SLAM.

---

ORB-SLAM3 unifies multiple sensor modalities and algorithmic advances into an accurate, scalable, and versatile SLAM solution. Its tight visual-inertial integration via MAP estimation, innovative multi-map management, and advanced place recognition set a high standard for both performance and extensibility, positioning it as a cornerstone of modern localization and mapping research.