Reducing Drift in Visual Odometry by Inferring Sun Direction Using a Bayesian Convolutional Neural Network (1609.05993v5)

Abstract: We present a method to incorporate global orientation information from the sun into a visual odometry pipeline using only the existing image stream, where the sun is typically not visible. We leverage recent advances in Bayesian Convolutional Neural Networks to train and implement a sun detection model that infers a three-dimensional sun direction vector from a single RGB image. Crucially, our method also computes a principled uncertainty associated with each prediction, using a Monte Carlo dropout scheme. We incorporate this uncertainty into a sliding window stereo visual odometry pipeline where accurate uncertainty estimates are critical for optimal data fusion. Our Bayesian sun detection model achieves a median error of approximately 12 degrees on the KITTI odometry benchmark training set, and yields improvements of up to 42% in translational ARMSE and 32% in rotational ARMSE compared to standard VO. An open source implementation of our Bayesian CNN sun estimator (Sun-BCNN) using Caffe is available at https://github. com/utiasSTARS/sun-bcnn-vo

• The paper introduces a Bayesian CNN approach for sun direction estimation that reduces visual odometry drift with translational and rotational error decreases of up to 42% and 32% respectively.
• The method employs dropout-based uncertainty quantification and integrates the sun's direction into a sliding-window bundle adjustment for more accurate 3D pose estimation.
• Quantitative experiments on the KITTI benchmark demonstrate a median vector angle error of 12° for sun direction, validating the robustness of the proposed approach.

Incorporating Sun Direction into Visual Odometry Using Bayesian CNNs

The paper introduces an innovative approach to enhancing visual odometry (VO) by integrating global orientation information derived from the sun, a method particularly useful when traditional maps or GPS data are unavailable. The authors leverage Bayesian Convolutional Neural Networks (BCNNs) to predict the sun's direction from RGB images, an advance that provides estimates alongside a measure of uncertainty, which is crucial for robust data fusion in motion estimation pipelines.

Methodology Overview

The authors employ a BCNN to infer sun direction, using a network structure inspired by GoogLeNet and trained on approximately 20,000 images. The BCNN produces a three-dimensional sun direction vector and an associated covariance matrix, utilizing dropout layers for both training and testing to obtain principled uncertainty estimates. This probabilistic approach ensures the BCNN doesn't merely output point estimates but also quantifies the confidence of each prediction, a feature the authors incorporate into a stereo VO system.

The VO process involves a sliding window approach, where visual feature tracking via stereo images is refined through bundle adjustment, integrating sun direction estimates to correct trajectory estimates. The sun's direction is treated as a fixed, known vector in global coordinates, and its estimate relative to the camera frame is refined using the BCNN-generated data.

Quantitative Results

The results demonstrate significant reductions in error accumulation within VO estimations when integrating the sun direction estimates. The BCNN achieves a median vector angle error of around 12 degrees on the KITTI odometry benchmark. When incorporated into the VO pipeline, sun direction estimates reduce translational and rotational average root mean squared error (ARMSE) by up to 42% and 32%, respectively, compared to baseline VO methods. The precision of sun direction prediction and the associated uncertainty are shown to yield improvements across sequences, underscoring the utility of incorporating even indirect global orientation cues.

Contributions

The paper makes several key contributions:

1. BCNN Application: It employs BCNNs for sun direction estimation, significantly improving over previous CNN approaches by utilizing dropout for uncertainty quantification.
2. Egomotion Pipeline Integration: It seamlessly integrates sun direction estimations and their covariances into a stereo VO pipeline, enhancing its accuracy by constraining orientation errors.
3. Full 3D Pose Estimation: The method outputs a full 3D unit-length sun direction vector, allowing its incorporation in six degrees of freedom (6-DOF) pose estimation problems.
4. Open Source Implementation: The authors provide an open-source implementation of Sun-BCNN, fostering reproducibility and further research.

Implications and Future Directions

The integration of sun-based orientation suggests a promising avenue for improving navigation systems, particularly in GPS-denied environments such as autonomous vehicles or planetary exploration. The ability to infer orientation from environmental cues like the sun could enhance the robustness and reliability of autonomous systems.

Future research could explore handling variability in environmental conditions, such as cloud cover which can obscure shadow cues. Extending the methodology to handle multiple sun estimates from cameras with varied viewpoints or developing mechanisms to learn model precisions dynamically could also improve performance further. Additionally, integrating these techniques with other localization methods can result in comprehensive navigation systems adaptable to diverse terrains and conditions.

In conclusion, the paper provides a novel strategy to reduce drift in visual odometry, utilizing advanced deep learning frameworks to introduce global orientation cues from natural landmarks, which holds significant potential for autonomous systems navigation.