VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator (1708.03852v1)

Abstract: A monocular visual-inertial system (VINS), consisting of a camera and a low-cost inertial measurement unit (IMU), forms the minimum sensor suite for metric six degrees-of-freedom (DOF) state estimation. However, the lack of direct distance measurement poses significant challenges in terms of IMU processing, estimator initialization, extrinsic calibration, and nonlinear optimization. In this work, we present VINS-Mono: a robust and versatile monocular visual-inertial state estimator.Our approach starts with a robust procedure for estimator initialization and failure recovery. A tightly-coupled, nonlinear optimization-based method is used to obtain high accuracy visual-inertial odometry by fusing pre-integrated IMU measurements and feature observations. A loop detection module, in combination with our tightly-coupled formulation, enables relocalization with minimum computation overhead.We additionally perform four degrees-of-freedom pose graph optimization to enforce global consistency. We validate the performance of our system on public datasets and real-world experiments and compare against other state-of-the-art algorithms. We also perform onboard closed-loop autonomous flight on the MAV platform and port the algorithm to an iOS-based demonstration. We highlight that the proposed work is a reliable, complete, and versatile system that is applicable for different applications that require high accuracy localization. We open source our implementations for both PCs and iOS mobile devices.

• The paper introduces a robust monocular VIO system that integrates precise initialization, tightly-coupled nonlinear optimization, and loop closure to ensure consistent state estimation.
• It demonstrates superior accuracy and drift correction under challenging conditions through extensive experiments on datasets like EuRoC and real-world environments.
• The real-time performance on both PC and iOS platforms highlights its potential for applications in augmented reality and autonomous drone navigation.

Overview of VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator

The paper "VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator," authored by Tong Qin, Peiliang Li, and Shaojie Shen, introduces a comprehensive solution for monocular visual-inertial odometry (VIO). This approach stands out for its integration of robust initialization procedures, tightly-coupled nonlinear optimization, and loop closure capabilities, resulting in highly accurate state estimation even under challenging conditions. The system is implemented on both PC and iOS platforms, making it accessible for various applications from augmented reality (AR) to autonomous drone navigation.

Core Contributions

The key contributions of VINS-Mono can be summarized as follows:

1. Robust Initialization: The VINS-Mono system features a robust initialization procedure that ensures reliable bootstrapping from arbitrary initial conditions. This procedure leverages a loosely-coupled sensor fusion approach to recover scale, gravity, velocity, and bias, ensuring accurate metric state estimation even under initial motion conditions.
2. Tightly-Coupled Nonlinear Optimization: The monocular VIO relies on an optimization-based approach to fuse pre-integrated IMU measurements with visual features. The tightly-coupled formulation within a sliding window framework allows for precise local pose, velocity, and orientation estimation, alongside online camera-IMU extrinsic parameter calibration and IMU bias correction.
3. Relocalization and Loop Closure: The system includes a loop detection module using DBoW2, enabling low-overhead relocalization. The relocalization is tightly integrated, leveraging previously observed features to eliminate drift and maintain global consistency. This integration extends to a global pose graph optimization that corrects accumulated drift in translation and yaw angle over long-term operations.
4. Versatility and Real-Time Performance: Demonstrating versatility, VINS-Mono has been implemented on both PCs and iOS devices. It shows real-time performance in various applications, including drone navigation and mobile AR. The iOS implementation, denoted as VINS-Mobile, even showcases superior performance compared to commercial solutions like Google Tango in certain scenarios.

Experimental Validation

The authors validate VINS-Mono through extensive experiments:

• EuRoC MAV Dataset: Compared against OKVIS, another state-of-the-art VIO system, VINS-Mono demonstrates superior accuracy in translation and consistent state estimation, particularly when loop closure is enabled. The RMSE analysis highlights VINS-Mono's robustness under varying motion patterns and illumination conditions.
• Indoor Experiments: In repetitive indoor environments, VINS-Mono effectively mitigates drift through loop closure, outperforming OKVIS and showing robustness against challenging conditions such as low light, texture-less surfaces, and reflective areas.
• Large-Scale Experiments: The system's robustness and scalability are further evidenced in mixed indoor-outdoor and extensive outdoor experiments, where it successfully maintains trajectory accuracy over long distances and durations, validated against satellite maps.
• Real-World Applications: For an onboard drone, VINS-Mono provides real-time state estimation for closed-loop control, accurately following pre-defined trajectories. The iOS application showcases its mobile adaptability, providing reliable AR experiences and robust performance comparable to specialized hardware.

Theoretical and Practical Implications

The theoretical implications of VINS-Mono lie in its robust formulation and efficient integration of visual and inertial measurements, addressing the challenges of monocular VIO systems such as scale observation and drift correction. Practically, the implementation on diverse hardware platforms demonstrates the system's adaptability and readiness for deployment in real-world applications.

Future Directions

Future research may focus on the refinement of observability analysis and online parameter calibration, particularly on consumer-grade hardware like smartphones. Additionally, exploring dense mapping and environmental perception advancements could further enhance VINS-Mono's applicability in both AR and autonomous robotics.

Conclusion

In conclusion, VINS-Mono presents a comprehensive, robust, and versatile solution for monocular visual-inertial state estimation. Its contributions to robust initialization, tightly-coupled nonlinear optimization, and efficient relocalization and loop closure make it a valuable tool for a wide range of applications requiring accurate localization. The system's open-source implementation further contributes to its impact and potential for adoption within the research community and industry.