A Self-Supervised, Differentiable Kalman Filter for Uncertainty-Aware Visual-Inertial Odometry (2203.07207v3)

Abstract: Visual-inertial odometry (VIO) systems traditionally rely on filtering or optimization-based techniques for egomotion estimation. While these methods are accurate under nominal conditions, they are prone to failure during severe illumination changes, rapid camera motions, or on low-texture image sequences. Learning-based systems have the potential to outperform classical implementations in challenging environments, but, currently, do not perform as well as classical methods in nominal settings. Herein, we introduce a framework for training a hybrid VIO system that leverages the advantages of learning and standard filtering-based state estimation. Our approach is built upon a differentiable Kalman filter, with an IMU-driven process model and a robust, neural network-derived relative pose measurement model. The use of the Kalman filter framework enables the principled treatment of uncertainty at training time and at test time. We show that our self-supervised loss formulation outperforms a similar, supervised method, while also enabling online retraining. We evaluate our system on a visually degraded version of the EuRoC dataset and find that our estimator operates without a significant reduction in accuracy in cases where classical estimators consistently diverge. Finally, by properly utilizing the metric information contained in the IMU measurements, our system is able to recover metric scene scale, while other self-supervised monocular VIO approaches cannot.

• The paper presents a novel hybrid VIO framework that unifies filtering and learning to estimate egomotion under uncertainty.
• It combines an IMU-driven process model with neural network-derived relative pose measurements for enhanced robustness in challenging conditions.
• The framework employs a self-supervised loss and online retraining to outperform traditional VIO methods and recover accurate metric scale.

The paper "A Self-Supervised, Differentiable Kalman Filter for Uncertainty-Aware Visual-Inertial Odometry" addresses the challenges faced by traditional Visual-Inertial Odometry (VIO) systems. These systems typically rely on either filtering or optimization-based techniques to estimate egomotion. While effective under normal conditions, they often struggle with issues like severe lighting changes, rapid camera movements, or low-texture environments, leading to failure.

This research introduces a novel framework combining the strengths of both learning-based systems and conventional filtering methods. The core innovation is the integration of a differentiable Kalman filter within the system, which involves two primary components:

1. IMU-Driven Process Model: This model utilizes data from an Inertial Measurement Unit (IMU) to predict the system's state over time.
2. Neural Network-Derived Relative Pose Measurement Model: This component utilizes neural networks to derive relative pose measurements, improving the robustness of the system against uncertainties.

A key advantage of this framework is the principled handling of uncertainty. The Kalman filter's differentiability allows the system to account for uncertainty during both training and testing phases. This approach contrasts with traditional methods by implementing a self-supervised loss formulation, which not only surpasses similar supervised methods in performance but also supports online retraining to adapt to new environments dynamically.

The system's effectiveness is demonstrated on a visually degraded version of the EuRoC dataset. It maintains accuracy in scenarios where classical estimators fail, showcasing its robustness. Importantly, it leverages metric information from IMU measurements to recover the metric scale of the scene—a challenge that typically hinders other self-supervised monocular VIO approaches.

In summary, this paper presents an innovative hybrid VIO system that effectively blends learning models with traditional filtering techniques, offering improved accuracy and robustness in challenging environments while maintaining the ability to recover metric scene scale.