Event-based Visual Inertial Velometer (2311.18189v2)

Abstract: Neuromorphic event-based cameras are bio-inspired visual sensors with asynchronous pixels and extremely high temporal resolution. Such favorable properties make them an excellent choice for solving state estimation tasks under aggressive ego motion. However, failures of camera pose tracking are frequently witnessed in state-of-the-art event-based visual odometry systems when the local map cannot be updated in time. One of the biggest roadblocks for this specific field is the absence of efficient and robust methods for data association without imposing any assumption on the environment. This problem seems, however, unlikely to be addressed as in standard vision due to the motion-dependent observability of event data. Therefore, we propose a mapping-free design for event-based visual-inertial state estimation in this paper. Instead of estimating the position of the event camera, we find that recovering the instantaneous linear velocity is more consistent with the differential working principle of event cameras. The proposed event-based visual-inertial velometer leverages a continuous-time formulation that incrementally fuses the heterogeneous measurements from a stereo event camera and an inertial measurement unit. Experiments on the synthetic dataset demonstrate that the proposed method can recover instantaneous linear velocity in metric scale with low latency.

• The paper presents a novel state estimation framework that recovers instantaneous linear velocity using event cameras and inertial measurements, bypassing traditional map updates.
• It details a rigorous normal flow computation and utilizes a cubic B-spline continuous-time model to fuse asynchronous event and inertial data efficiently.
• Experimental results on synthetic drone sequences show a dramatic reduction in absolute velocity error, outperforming conventional IMU integration and VINS-Fusion methods.

Event-based Visual Inertial Velometer

The paper, "Event-based Visual Inertial Velometer," authored by Xiuyuan Lu, Yi Zhou, and Shaojie Shen, introduces a mapping-free design for event-based visual-inertial state estimation. The researchers address the critical challenge of real-time pose tracking failure in state-of-the-art event-based visual odometry (VO) systems due to slow local map updates under aggressive motion. The proposed solution leverages neuromorphic event-based cameras and an inertial measurement unit (IMU) to estimate linear velocity, rather than position, thus aligning more closely with the differential nature of event cameras.

Key Contributions and Methodology

The primary contributions of the paper are as follows:

1. Novel State Estimator Design: The paper proposes a unique state estimation framework that utilizes the differential properties of event cameras. By focusing on the recovery of instantaneous linear velocity rather than position, the system bypasses issues related to comprehensive local map updates.
2. Rigorous Normal Flow Computation: A detailed mathematical derivation for computing normal flow from spatial-temporal gradients of event data is presented. The algorithm estimates the normal flow generated by the kinematics of the event camera, providing a robust mechanism for motion information extraction.
3. Continuous-Time Pipeline: The system employs a continuous-time formulation for event and inertial data fusion. A cubic B-spline-based parametric model is used to represent the continuous-time linear velocity. This model allows for efficient management of asynchronous event measurements and the association of these measurements with inertial data.

Experimental Results

The proposed methodology was evaluated using synthetic datasets, demonstrating strong numerical performance. The drone sequences generated via the ESIM simulator featured aggressive maneuvers, providing a rigorous testing environment. The main evaluation metric, Absolute Velocity Error (AVE), showed significant improvement for the proposed method over both conventional IMU integration and VINS-Fusion alternatives. Notably, the AVE for the proposed method in one of the more challenging sequences (Seq. 3) was 0.49 m/s, compared to 19.80 m/s for VINS-Fusion.

A detailed computational performance evaluation also revealed the efficiency of the proposed pipeline. The system achieved a state estimation update rate of up to 75 Hz, demonstrating the potential for real-time application even under demanding conditions.

Implications and Future Directions

The theoretical implications of the proposed method are substantial. By shifting focus to linear velocity estimation congruent with the properties of event cameras, the approach potentially sets a new standard for event-based visual odometry. Practically, the method provides an efficient solution for real-time applications where rapid and aggressive movements are common, such as in autonomous drones or robotic systems operating in dynamic environments.

Future developments could explore the extension of this continuous-time estimation framework to other state variables or environments beyond synthetic scenes. Furthermore, integrating this event-based velocity estimation with advanced localization and mapping systems might offer comprehensive solutions for complex navigation tasks, capitalizing on the strengths of both event-based sensors and traditional visual systems.

In summary, this paper presents a robust, efficient, and theoretically sound solution for event-based visual-inertial state estimation focusing on linear velocity. The methodology's practical effectiveness and computational performance lay a solid foundation for future explorations and advancements in the domain of state estimation using event-based sensors.