DF-VO: What Should Be Learnt for Visual Odometry? (2103.00933v1)

Abstract: Multi-view geometry-based methods dominate the last few decades in monocular Visual Odometry for their superior performance, while they have been vulnerable to dynamic and low-texture scenes. More importantly, monocular methods suffer from scale-drift issue, i.e., errors accumulate over time. Recent studies show that deep neural networks can learn scene depths and relative camera in a self-supervised manner without acquiring ground truth labels. More surprisingly, they show that the well-trained networks enable scale-consistent predictions over long videos, while the accuracy is still inferior to traditional methods because of ignoring geometric information. Building on top of recent progress in computer vision, we design a simple yet robust VO system by integrating multi-view geometry and deep learning on Depth and optical Flow, namely DF-VO. In this work, a) we propose a method to carefully sample high-quality correspondences from deep flows and recover accurate camera poses with a geometric module; b) we address the scale-drift issue by aligning geometrically triangulated depths to the scale-consistent deep depths, where the dynamic scenes are taken into account. Comprehensive ablation studies show the effectiveness of the proposed method, and extensive evaluation results show the state-of-the-art performance of our system, e.g., Ours (1.652%) v.s. ORB-SLAM (3.247%}) in terms of translation error in KITTI Odometry benchmark. Source code is publicly available at: \href{https://github.com/Huangying-Zhan/DF-VO}{DF-VO}.

• The paper introduces DF-VO, a hybrid approach that integrates deep learning with traditional multi-view geometry to enhance monocular visual odometry.
• It employs bi-directional flow consistency and scale-consistent depth alignment to robustly recover camera poses in challenging dynamic and low-texture scenes.
• Experiments on the KITTI benchmark demonstrate a reduction in translation error to 1.65% compared to 3.25% for ORB-SLAM with loop closure.

An Expert Analysis of "DF-VO: What Should Be Learnt for Visual Odometry?"

Overview

The paper "DF-VO: What Should Be Learnt for Visual Odometry?" presents an innovative approach aimed at addressing the challenges associated with monocular Visual Odometry (VO). The authors propose a hybrid system, DF-VO, which leverages the strengths of both deep learning and traditional multi-view geometry. This system is designed to enhance robustness and accuracy, especially in dynamic and low-texture environments, which are known to impede conventional VO methods.

Methodology

The framework integrates Depth and optical Flow, hence the name DF-VO. The authors incorporate a deep learning module to predict single view depths and optical flows in a self-supervised manner. By carefully sampling high-quality correspondences from dense optical flow predictions, DF-VO intends to robustly recover camera poses using geometric principles.

The novelty of the approach lies in its multi-faceted process:

1. Correspondence Sampling: High-quality 2D-2D matches are extracted using a bi-directional flow consistency check, aimed at ensuring only the best correspondences are selected. This increases the robustness against dynamic scenes and improves tracking accuracy.
2. Scale Consistency: To combat the notorious scale drift issue in monocular methods, the authors align geometrically triangulated depths with predictions from scale-consistent depth networks. This alignment is particularly crucial, as it allows DF-VO to maintain scale consistency over long sequences without imposing expensive global optimizations such as bundle adjustment.
3. Hybrid Tracking Model: The system intelligently switches between an Epipolar Geometry-based tracker and a Perspective-n-Point (PnP) tracker, depending on the scenario. This adaptability helps in efficiently resolving issues related to motion and structure degeneracy.

Results and Implications

The experimental evaluation, primarily on the KITTI Odometry benchmark, demonstrates that DF-VO outperforms state-of-the-art methods, exhibiting a notable improvement in translation error (1.652% for DF-VO compared to 3.247% for ORB-SLAM with loop closure). Such results are significant as they highlight the robustness and efficacy of incorporating learned depths and flow in traditional geometric frameworks.

The ablation studies further reinforce the effectiveness of the proposed components, such as the iterative scale recovery and local best-K correspondence selection, showcasing their contributions to the overall accuracy and robustness of the system.

Theoretical and Practical Implications

By merging deep learning insights with geometric constraints, the paper contributes to a growing body of work that seeks to enhance VO systems. The integration strategy employed by DF-VO could inform future research directions in VO, particularly in scenarios where scalability and robustness are paramount. Beyond autonomous driving, potential applications abound in domains such as augmented reality and robotics, where precise localization and mapping in dynamic environments are critical.

Future Directions

The authors suggest the possibility of integrating a local optimization module to further refine VO results. Additionally, employing multi-view stereo networks instead of single-view depth networks could enhance depth prediction accuracy. These potential developments indicate a nuanced progression towards more advanced, adaptive, and precise visual odometry systems.

In conclusion, DF-VO represents a significant step forward in monocular VO, achieved through a thoughtful integration of learning-based predictions and geometry-based motion estimation. The balance it strikes between complexity and practicality could serve as a model for future work in machine perception and visual computing.