Continuous-Time Visual-Inertial Odometry for Event Cameras (1702.07389v2)

Abstract: Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, due to the fundamentally different structure of the sensor's output, new algorithms that exploit the high temporal resolution and the asynchronous nature of the sensor are required. Recent work has shown that a continuous-time representation of the event camera pose can deal with the high temporal resolution and asynchronous nature of this sensor in a principled way. In this paper, we leverage such a continuous-time representation to perform visual-inertial odometry with an event camera. This representation allows direct integration of the asynchronous events with micro-second accuracy and the inertial measurements at high frequency. The event camera trajectory is approximated by a smooth curve in the space of rigid-body motions using cubic splines. This formulation significantly reduces the number of variables in trajectory estimation problems. We evaluate our method on real data from several scenes and compare the results against ground truth from a motion-capture system. We show that our method provides improved accuracy over the result of a state-of-the-art visual odometry method for event cameras. We also show that both the map orientation and scale can be recovered accurately by fusing events and inertial data. To the best of our knowledge, this is the first work on visual-inertial fusion with event cameras using a continuous-time framework.

• The paper introduces a novel continuous-time framework that fuses asynchronous event camera data with continuous IMU signals to improve visual-inertial odometry.
• It employs non-linear optimization and spline-based trajectory modeling to significantly reduce trajectory estimation errors in challenging conditions.
• Experimental tests on a quadrotor platform validate the method’s robustness, paving the way for advanced autonomous navigation applications.

Continuous-Time Visual-Inertial Odometry for Event Cameras

The paper "Continuous-Time Visual-Inertial Odometry for Event Cameras," authored by Elias Mueggler, Guillermo Gallego, Henri Rebecq, and Davide Scaramuzza, introduces an innovative approach to fusing sensor data from event cameras and inertial measurement units (IMUs). This work addresses the challenge of performing accurate visual-inertial odometry in environments characterized by dynamic lighting conditions and fast motion, where traditional frame-based methods often struggle.

Event cameras, such as dynamic vision sensors, capture changes in a scene asynchronously and provide high temporal resolution, making them particularly suitable for rapid, dynamic environments. The authors propose a continuous-time framework to integrate the asynchronous data from event cameras with the continuous signals from IMUs. This method seeks to leverage the complementary qualities of each sensor type: the high temporal resolution of event cameras and the reliable motion data of IMUs.

The paper details the mathematical formulation of the problem, focusing on non-linear optimization techniques to estimate the trajectory of the camera over time. Key components of the methodology include modeling the camera trajectory as a continuous-time spline and using a probabilistic sensor fusion strategy to integrate the asynchronous events and continuous IMU data. The optimization process is designed to be robust to noise and other disturbances, ensuring stable and accurate odometry.

This approach demonstrates significant improvements in localization and mapping precision compared to conventional frame-based visual odometry systems. The authors provide quantitative results from extensive real-world testing, revealing a notable reduction in trajectory estimation error under fast motion and high dynamic range conditions. The experimental setup showcases the paper’s applicability to real-world scenarios, utilizing a quadrotor environment to highlight the method's robustness and accuracy.

The theoretical implications of this research are substantial, as it paves the way for more efficient and accurate odometry systems that can function under challenging conditions where traditional systems may fail. By emphasizing a continuous-time approach, the framework aligns with the natural operational characteristics of event cameras, addressing synchronization issues inherently and leading to more precise motion estimation.

Practically, this work can enhance autonomous navigation systems in robotics, particularly in domains such as aerial vehicles, where fast and responsive odometry is critical. Furthermore, the potential integration of this methodology with other sensor systems could lead to a broader array of applications in environments that demand high-speed data processing and reduced latency.

As the field of sensor fusion and visual odometry continues to evolve, future research might explore further optimization of computational resources, the inclusion of more diverse sensory data, and enhancements in real-time processing capabilities. Expanding the applicability of continuous-time frameworks to a wider range of robotic platforms remains a promising area for subsequent investigation. The paper's contributions lay a solid foundation for further explorations into the capabilities and implementations of event-based odometry systems in dynamic environments.