The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM (1610.08336v4)

Abstract: New vision sensors, such as the Dynamic and Active-pixel Vision sensor (DAVIS), incorporate a conventional global-shutter camera and an event-based sensor in the same pixel array. These sensors have great potential for high-speed robotics and computer vision because they allow us to combine the benefits of conventional cameras with those of event-based sensors: low latency, high temporal resolution, and very high dynamic range. However, new algorithms are required to exploit the sensor characteristics and cope with its unconventional output, which consists of a stream of asynchronous brightness changes (called "events") and synchronous grayscale frames. For this purpose, we present and release a collection of datasets captured with a DAVIS in a variety of synthetic and real environments, which we hope will motivate research on new algorithms for high-speed and high-dynamic-range robotics and computer-vision applications. In addition to global-shutter intensity images and asynchronous events, we provide inertial measurements and ground-truth camera poses from a motion-capture system. The latter allows comparing the pose accuracy of ego-motion estimation algorithms quantitatively. All the data are released both as standard text files and binary files (i.e., rosbag). This paper provides an overview of the available data and describes a simulator that we release open-source to create synthetic event-camera data.

• The paper presents a comprehensive dataset and open-source simulator designed for event-based camera research with precise ground-truth poses and flexible data formats.
• It leverages asynchronous event streams, synchronous images, and inertial measurements to support robust evaluation across various motion dynamics.
• The work advances high-speed robotics and autonomous systems by addressing low latency and high dynamic range challenges in pose estimation and SLAM.

The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM

The paper, "The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM," addresses the burgeoning field of event-based cameras, particularly the Dynamic and Active-pixel Vision Sensor (DAVIS). This emerging technology promises improved performance in high-speed and high-dynamic-range robotics by offering low latency, high temporal resolution, and low data redundancy, diverging significantly from conventional frame-based cameras.

Key Contributions

The authors introduce a comprehensive dataset and simulator tailored for event-based camera research, specifically focusing on pose estimation, visual odometry, and SLAM. The datasets are designed to challenge and refine algorithms, capturing both synthetic and real-world environments with varying motion dynamics.

Dataset Composition:

• Sensor Output: Includes both asynchronous event streams and synchronous grayscale images, alongside inertial measurements.
• Ground Truth: Offers sub-millimeter precision ground-truth camera poses from a motion-capture system.
• Types of Datasets: Incorporates 6-DOF handheld motion, various scene complexities, and motorized linear slider data, ensuring robustness against diverse visual odometry and SLAM challenges.
• Format: Available in both text and rosbag formats for flexibility in processing.

Simulator and Calibration

The paper also details an open-source simulator for generating synthetic event-camera data, facilitating experimentation without physical equipment. This simulator generates events with microsecond time-resolution leveraging linear interpolation techniques.

Calibration is meticulously handled, offering intrinsic camera parameters and alignment of ground-truth poses with the camera's optical frame, ensuring users can trust the precision of their algorithms' evaluations.

Numerical Insights

The datasets cover a range of scenarios, marking significant event counts, such as 23126288 events in a rotation dataset and complex outdoor movement captures. The IMU integration further enhances these datasets by marrying visual data with motion dynamics, accommodating visual-inertial algorithm development.

Implications and Future Directions

This research opens avenues for the refinement and development of algorithms leveraging the unique properties of event-based sensors. The low latency and high dynamic range present in these datasets have potential applications in fast-moving robotics, autonomous vehicles, and real-time SLAM systems.

Future advancements may involve further reducing the inherent noise and improving event-data fusion techniques with auxiliary sensors. The integration of event-data with deep learning approaches could further the capabilities and applications of event-based cameras in complex environments.

In conclusion, the paper presents a valuable contribution to the field of computer vision and robotics, laying foundational work for subsequent research focused on time-sensitive dynamic environments. This work not only provides a substantial dataset but also a methodology that encourages continued exploration of event-based sensor applications.