Event-based, 6-DOF Camera Tracking from Photometric Depth Maps (1607.03468v2)

Abstract: Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. These cameras do not suffer from motion blur and have a very high dynamic range, which enables them to provide reliable visual information during high-speed motions or in scenes characterized by high dynamic range. These features, along with a very low power consumption, make event cameras an ideal complement to standard cameras for VR/AR and video game applications. With these applications in mind, this paper tackles the problem of accurate, low-latency tracking of an event camera from an existing photometric depth map (i.e., intensity plus depth information) built via classic dense reconstruction pipelines. Our approach tracks the 6-DOF pose of the event camera upon the arrival of each event, thus virtually eliminating latency. We successfully evaluate the method in both indoor and outdoor scenes and show that---because of the technological advantages of the event camera---our pipeline works in scenes characterized by high-speed motion, which are still unaccessible to standard cameras.

Event-based, 6-DOF Camera Tracking from Photometric Depth Maps: A Technical Overview

This paper presents a novel approach to leverage event cameras for six-degree-of-freedom (6-DOF) camera tracking using existing photometric depth maps. Event cameras, unlike traditional frame-based cameras, provide asynchronous pixel-level brightness changes with high temporal precision and dynamic range, surpassing the limitations of conventional cameras, especially under conditions of rapid motion or high dynamic range scenarios.

Core Contributions

The primary contributions of this research encompass:

1. Event-based Pose Tracking: The proposed method tracks the 6-DOF pose of an event camera with each incoming event, drastically reducing latency compared to traditional methods that provide pose estimates at slower frame rates.
2. Robust Mixture Likelihood Model: The paper introduces a likelihood function for event processing, using a mixture model to handle noise and outliers. This sensor model effectively captures the generation process of events alongside the potential disturbances and inaccuracies.
3. Posterior Distribution Approximation: The approach includes a technique for approximating the posterior distribution using exponential family distributions and conjugate priors, allowing efficient recursive Bayesian filtering despite the complexity introduced by the asynchronous nature of the events.
4. Empirical Validation: Across a series of indoor and outdoor experiments, including scenarios with significant depth variations and occlusions, the system demonstrated robust performance and accuracy even in high-speed motion conditions that typically challenge standard visual odometry systems.

Methodology

The methodology leverages Bayesian filtering principles to incorporate both motion models and a resilient measurement model. The motion model is defined through a diffusion process that accommodates high-speed movements, while the measurement model uses the contrast residuals to evaluate the fit of an event camera pose with a prior scene map. The likelihood function is represented as a normal-uniform distribution mixture to account for both accurate and spurious event data.

An exponential family distribution is utilized for posterior approximation, handling the computation efficiently with the design of an EKF-like filter weighted by event inliers. This facilitates real-time event-by-event updates and significantly improves tracking performance under fast motion scenarios.

Results and Implications

The experimental results show that the method successfully tracks 6-DOF poses both indoors and outdoors, achieving accuracy comparable to traditional, higher-resolution sensors without the drawbacks of motion blur. The computational efficiency of the algorithm further underscores its applicability in resource-constrained environments, such as mobile platforms used in AR/VR and robotic applications.

Future Directions

This research opens avenues for integrating event-based cameras into robotics and perception systems where rapid motion challenges existing visual odometry techniques. Future work might explore advancements in event camera technology, scalability to higher resolution sensors, and integration with other sensor modalities to enhance scene understanding and tracking accuracy.

Overall, the paper provides a substantial contribution to the field by refining event camera capabilities for practical use in dynamic tracking scenarios, paving the way for innovation in neuromorphic and asynchronous vision systems.