Direct Sparse Odometry (1607.02565v2)

Abstract: We propose a novel direct sparse visual odometry formulation. It combines a fully direct probabilistic model (minimizing a photometric error) with consistent, joint optimization of all model parameters, including geometry -- represented as inverse depth in a reference frame -- and camera motion. This is achieved in real time by omitting the smoothness prior used in other direct methods and instead sampling pixels evenly throughout the images. Since our method does not depend on keypoint detectors or descriptors, it can naturally sample pixels from across all image regions that have intensity gradient, including edges or smooth intensity variations on mostly white walls. The proposed model integrates a full photometric calibration, accounting for exposure time, lens vignetting, and non-linear response functions. We thoroughly evaluate our method on three different datasets comprising several hours of video. The experiments show that the presented approach significantly outperforms state-of-the-art direct and indirect methods in a variety of real-world settings, both in terms of tracking accuracy and robustness.

• The paper introduces a novel direct probabilistic model that minimizes photometric error using sparse pixel sampling, bypassing traditional feature detection.
• The paper demonstrates real-time performance and enhanced accuracy by integrating comprehensive photometric calibration in diverse conditions.
• The paper emphasizes robust initialization and efficient keyframe management to improve monocular visual odometry in dynamic environments.

Direct Sparse Odometry: A Robust Monocular Visual Odometry Framework

Overview

The paper entitled "Direct Sparse Odometry" by Jakob Engel, Vladlen Koltun, and Daniel Cremers introduces a novel formulation for visual odometry. The proposed method is a direct sparse approach that optimizes a photometric error, offering several advancements over previous feature-based (indirect) methodologies. Unlike classical direct methods, the authors present a solution that foregoes the need for pixel smoothness priors, allowing for a straightforward optimization of both geometry and camera motion in real-time. This method, referred to as Direct Sparse Odometry (DSO), includes comprehensive photometric calibration, accounting for exposure variations, lens vignetting, and non-linear response functions. Evaluations across multiple datasets demonstrate that this formulation significantly outperforms existing state-of-the-art methods in accuracy and robustness.

Key Contributions

1. Fully Direct Probabilistic Model: DSO minimizes a photometric error directly on image pixel values, accommodating a full photometric model that includes camera exposure time, lens vignetting, and non-linear response functions.
2. Sparse Data Utilization: By strategically sampling pixels throughout an image, the method ensures robustness without relying on geometric priors. This sampling enables the capture of information even from edges and weak intensity variations.
3. Real-Time Performance: The omission of smoothness priors and the adoption of efficient data sampling strategies allow DSO to achieve real-time performance, ensuring its applicability in dynamic environments seen in autonomous vehicles and UAVs.
4. Robust Initialization and Front-End Design: The authors introduce a robust front-end that accurately initializes model parameters and efficiently manages keyframe and point selections, ensuring high performance in non-convex optimization landscapes.
5. Extensive Evaluation: The method's efficacy is thoroughly substantiated through comprehensive evaluations on the TUM monoVO, EuRoC MAV, and ICL-NUIM datasets. DSO consistently demonstrates higher accuracy and robustness compared to traditional indirect methods like ORB-SLAM.

Numerical Results

• On the TUM monoVO dataset, DSO achieves lower alignment errors (e_align) and reduced rotational (e_r) and scale drift (e_s) after large loops compared to ORB-SLAM.
• For the EuRoC MAV dataset, DSO exhibits superior robustness while maintaining competitive accuracy when local loop closures in ORB-SLAM are restricted.
• Evaluations under different noise conditions reveal that DSO outperforms indirect methods under high photometric noise but is more sensitive to geometric distortions, such as those introduced by a rolling shutter.

Implications and Future Directions

Practical Implications:

• Enhanced Robustness: The ability to operate directly on raw pixel values without relying on keypoint detection makes DSO inherently more robust in environments with limited texture or repetitive patterns.
• Photometric Calibration: The comprehensive photometric calibration provides significant improvements in real-world applicability, particularly for systems where lighting conditions and camera settings dynamically change.
• Real-Time Capability: The real-time performance of DSO ensures its suitability for applications in autonomous navigation and augmented reality.

Theoretical Implications:

• Normalization in Direct Methods: The success of DSO suggests that explicit modeling of the photometric image formation process can bridge the gap between direct and indirect methods, providing robust performance without keypoint dependency.
• Non-Smooth Optimizations: The elimination of smoothness priors opens avenues for new research into other domains where similar formulations could enhance algorithm performance without added computational complexity.

Future Developments:

• Integration with IMU Data: Future work can extend DSO by integrating Inertial Measurement Unit (IMU) data to enhance robustness in scenarios with rapid motion or challenging environmental conditions.
• Rolling Shutter Modeling: Since DSO is sensitive to geometric distortions, incorporating a rolling shutter model could improve accuracy, making the method more suited to a wider range of cameras.
• Improved Photometric Priors: Learning more realistic photometric priors from real-world data could address the biases introduced by current assumptions, further improving long-term accuracy.

Conclusion

The "Direct Sparse Odometry" framework represents a significant advancement in monocular visual odometry. By leveraging a fully direct probabilistic model and omitting geometric priors, it offers robust real-time performance and improved accuracy in diverse environments. The thorough evaluations and rigorous analytical breakdown presented in the paper highlight the strengths and potential areas for enhancement of this innovative approach.