Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Adaptive VIO: Deep Visual-Inertial Odometry with Online Continual Learning (2405.16754v1)

Published 27 May 2024 in cs.RO

Abstract: Visual-inertial odometry (VIO) has demonstrated remarkable success due to its low-cost and complementary sensors. However, existing VIO methods lack the generalization ability to adjust to different environments and sensor attributes. In this paper, we propose Adaptive VIO, a new monocular visual-inertial odometry that combines online continual learning with traditional nonlinear optimization. Adaptive VIO comprises two networks to predict visual correspondence and IMU bias. Unlike end-to-end approaches that use networks to fuse the features from two modalities (camera and IMU) and predict poses directly, we combine neural networks with visual-inertial bundle adjustment in our VIO system. The optimized estimates will be fed back to the visual and IMU bias networks, refining the networks in a self-supervised manner. Such a learning-optimization-combined framework and feedback mechanism enable the system to perform online continual learning. Experiments demonstrate that our Adaptive VIO manifests adaptive capability on EuRoC and TUM-VI datasets. The overall performance exceeds the currently known learning-based VIO methods and is comparable to the state-of-the-art optimization-based methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Selfvio: Self-supervised deep monocular visual–inertial odometry and depth estimation. Neural Networks, 150:119–136, 2022.
  2. Robust visual inertial odometry using a direct ekf-based approach. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 298–304, 2015.
  3. Codeslam - learning a compact, optimisable representation for dense visual slam. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2560–2568, 2018.
  4. Deep imu bias inference for robust visual-inertial odometry with factor graphs. IEEE Robotics and Automation Letters, 8(1):41–48, 2023.
  5. The euroc micro aerial vehicle datasets. The International Journal of Robotics Research, 35(10):1157–1163, 2016.
  6. Orb-slam3: An accurate open-source library for visual, visual–inertial, and multimap slam. IEEE Transactions on Robotics, 37(6):1874–1890, 2021.
  7. Rnin-vio: Robust neural inertial navigation aided visual-inertial odometry in challenging scenes. In 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 275–283, 2021.
  8. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014.
  9. Vinet: Visual-inertial odometry as a sequence-to-sequence learning problem. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, page 3995–4001. AAAI Press, 2017.
  10. The cityscapes dataset. In CVPR Workshop on the Future of Datasets in Vision. sn, 2015.
  11. On-manifold preintegration for real-time visual–inertial odometry. IEEE Transactions on Robotics, 33(1):1–21, 2017.
  12. islam: Imperative slam. arXiv preprint arXiv:2306.07894, 2023.
  13. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE conference on computer vision and pattern recognition, pages 3354–3361. IEEE, 2012.
  14. Deepvio: Self-supervised deep learning of monocular visual inertial odometry using 3d geometric constraints. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6906–6913, 2019.
  15. Visual-inertial odometry with robust initialization and online scale estimation. Sensors, 18(12):4287, 2018.
  16. Keyframe-based visual–inertial odometry using nonlinear optimization. The International Journal of Robotics Research, 34(3):314–334, 2015.
  17. High-precision, consistent ekf-based visual-inertial odometry. The International Journal of Robotics Research, 32(6):690–711, 2013.
  18. Sequential adversarial learning for self-supervised deep visual odometry. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2851–2860, 2019.
  19. Self-supervised deep visual odometry with online adaptation. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6338–6347, 2020.
  20. Generalizing to the open world: Deep visual odometry with online adaptation. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13179–13188, 2021.
  21. Tlio: Tight learned inertial odometry. IEEE Robotics and Automation Letters, 5(4):5653–5660, 2020.
  22. A multi-state constraint kalman filter for vision-aided inertial navigation. In Proceedings 2007 IEEE International Conference on Robotics and Automation, pages 3565–3572, 2007.
  23. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
  24. Vins-mono: A robust and versatile monocular visual-inertial state estimator. IEEE Transactions on Robotics, 34(4):1004–1020, 2018.
  25. A general optimization-based framework for local odometry estimation with multiple sensors, 2019.
  26. Kimera: an open-source library for real-time metric-semantic localization and mapping. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 1689–1696, 2020.
  27. The tum vi benchmark for evaluating visual-inertial odometry. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1680–1687. IEEE, 2018.
  28. Unsupervised deep visual-inertial odometry with online error correction for rgb-d imagery. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(10):2478–2493, 2020.
  29. A micro lie theory for state estimation in robotics. arXiv preprint arXiv:1812.01537, 2018.
  30. Dm-vio: Delayed marginalization visual-inertial odometry. IEEE Robotics and Automation Letters, 7(2):1408–1415, 2022.
  31. A benchmark for the evaluation of rgb-d slam systems. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pages 573–580. IEEE, 2012.
  32. Ba-net: Dense bundle adjustment network. arXiv preprint arXiv:1806.04807, 2018.
  33. DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras. Advances in neural information processing systems, 2021a.
  34. Tangent space backpropagation for 3d transformation groups. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10338–10347, 2021b.
  35. Deep patch visual odometry. arXiv preprint arXiv:2208.04726, 2022.
  36. Continual slam: Beyond lifelong simultaneous localization and mapping through continual learning. In Robotics Research, pages 19–35, Cham, 2023. Springer Nature Switzerland.
  37. Direct sparse visual-inertial odometry using dynamic marginalization. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 2510–2517, 2018.
  38. Covio: Online continual learning for visual-inertial odometry. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 2464–2473, 2023.
  39. Pypose: A library for robot learning with physics-based optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22024–22034, 2023.
  40. Deepvo: Towards end-to-end visual odometry with deep recurrent convolutional neural networks. In 2017 IEEE international conference on robotics and automation (ICRA), pages 2043–2050. IEEE, 2017.
  41. Tartanair: A dataset to push the limits of visual slam. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4909–4916. IEEE, 2020.
  42. Visual-inertial odometry based on kinematic constraints in imu frames. IEEE Robotics and Automation Letters, 7(3):6550–6557, 2022.
  43. Beyond tracking: Selecting memory and refining poses for deep visual odometry. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8575–8583, 2019.
  44. Imu data processing for inertial aided navigation: A recurrent neural network based approach. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 3992–3998, 2021.
  45. Unsupervised learning of depth and ego-motion from video. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6612–6619, 2017.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets