Event-based Stereo Visual Odometry (2007.15548v2)

Abstract: Event-based cameras are bio-inspired vision sensors whose pixels work independently from each other and respond asynchronously to brightness changes, with microsecond resolution. Their advantages make it possible to tackle challenging scenarios in robotics, such as high-speed and high dynamic range scenes. We present a solution to the problem of visual odometry from the data acquired by a stereo event-based camera rig. Our system follows a parallel tracking-and-mapping approach, where novel solutions to each subproblem (3D reconstruction and camera pose estimation) are developed with two objectives in mind: being principled and efficient, for real-time operation with commodity hardware. To this end, we seek to maximize the spatio-temporal consistency of stereo event-based data while using a simple and efficient representation. Specifically, the mapping module builds a semi-dense 3D map of the scene by fusing depth estimates from multiple local viewpoints (obtained by spatio-temporal consistency) in a probabilistic fashion. The tracking module recovers the pose of the stereo rig by solving a registration problem that naturally arises due to the chosen map and event data representation. Experiments on publicly available datasets and on our own recordings demonstrate the versatility of the proposed method in natural scenes with general 6-DoF motion. The system successfully leverages the advantages of event-based cameras to perform visual odometry in challenging illumination conditions, such as low-light and high dynamic range, while running in real-time on a standard CPU. We release the software and dataset under an open source licence to foster research in the emerging topic of event-based SLAM.

• The paper presents a novel event-based stereo visual odometry system that leverages asynchronous event data for accurate 6-DoF pose tracking.
• The mapping module creates a semi-dense 3D reconstruction by probabilistically fusing depth estimates from stereo event streams.
• Experimental results demonstrate the system's robustness in dynamic, low-light conditions, highlighting its practical applicability in robotics.

Event-based Stereo Visual Odometry: An Overview

The paper "Event-based Stereo Visual Odometry" presents a novel solution for achieving visual odometry by leveraging the asynchronous data captured from stereo event cameras. Event-based cameras are distinct in that each pixel operates independently, reacting to brightness changes with microsecond-level temporal resolution. These characteristics enable the capture of dynamic scenes, maintaining advantages like low latency, high dynamic range, and low power consumption, which are pivotal for high-speed applications in robotics.

Overview of the Proposed System

The proposed system integrates a parallel tracking-and-mapping methodology tailored for processing event-camera data. The system is bifurcated into two principal components:

1. Mapping Module: This module constructs a semi-dense 3D map of the scene using depth estimates derived from multiple viewpoints. It does so by capitalizing on the spatio-temporal consistency gleaned from the stereo event data. A probabilistic framework ensures robust fusion of depth information, contributing to the overall efficiency and reliability of the mapping process.
2. Tracking Module: Camera pose estimation is managed by the tracking module, which addresses a registration problem inherently posed by the data representation. The registration utilizes the consistency of event data across the stereo cameras to accurately maintain pose tracking in six degrees of freedom (6-DoF).

The system is designed to operate efficiently in real time on standard computational hardware, which is a testament to its practical applicability in real-world scenarios.

Experimental Results

The performance of the system has been rigorously evaluated using both publicly available datasets and new data recorded by the authors. Notably, the system effectively manages the challenges posed by natural environments, maintaining reliable function even in conditions of low light and high dynamic range. These results underline the system's versatility and its potential as a robust solution for real-time visual odometry.

Practical and Theoretical Implications

The implications of this work are broad within the field of robotics and computer vision. Practically, it offers a viable method for visual odometry under challenging conditions that typically pose significant difficulties for traditional frame-based sensors. Theoretically, the paper advances the understanding of how event-based representations can be utilized to resolve classical estimation problems like camera pose tracking and 3D reconstruction in a streamlined, efficient manner.

Future Directions

Future work could explore more complex integration scenarios where event-based visual odometry is combined with other sensory modalities, such as inertial measurements, to enhance robustness and accuracy. Additionally, the open-source release of their dataset and software is likely to encourage further exploration and validation, fostering continued research into event-based SLAM systems.

This paper represents a significant step in advancing the application of bio-inspired vision systems, offering pathways to overcome the limitations of conventional imaging technologies in fast-paced and high-dynamic-range environments.