Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards Long Term SLAM on Thermal Imagery (2403.19885v1)

Published 28 Mar 2024 in cs.CV and cs.RO

Abstract: Visual SLAM with thermal imagery, and other low contrast visually degraded environments such as underwater, or in areas dominated by snow and ice, remain a difficult problem for many state of the art (SOTA) algorithms. In addition to challenging front-end data association, thermal imagery presents an additional difficulty for long term relocalization and map reuse. The relative temperatures of objects in thermal imagery change dramatically from day to night. Feature descriptors typically used for relocalization in SLAM are unable to maintain consistency over these diurnal changes. We show that learned feature descriptors can be used within existing Bag of Word based localization schemes to dramatically improve place recognition across large temporal gaps in thermal imagery. In order to demonstrate the effectiveness of our trained vocabulary, we have developed a baseline SLAM system, integrating learned features and matching into a classical SLAM algorithm. Our system demonstrates good local tracking on challenging thermal imagery, and relocalization that overcomes dramatic day to night thermal appearance changes. Our code and datasets are available here: https://github.com/neufieldrobotics/IRSLAM_Baseline

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Surf: Speeded up robust features. In Computer Vision–ECCV 2006: 9th European Conference on Computer Vision, Graz, Austria, May 7-13, 2006. Proceedings, Part I 9, pages 404–417. Springer, 2006.
  2. Brief: Binary robust independent elementary features. 12 2011.
  3. Orb-slam3: An accurate open-source library for visual, visual–inertial, and multimap slam. IEEE Transactions on Robotics, 37(6):1874–1890, 2021.
  4. Thermal-depth odometry in challenging illumination conditions. IEEE Robotics and Automation Letters, 2023.
  5. Appearance-based loop closure detection combining lines and learned points for low-textured environments. Auton. Robots, 46(3):451–467, mar 2022.
  6. Thermal-inertial odometry for autonomous flight throughout the night. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1122–1128. IEEE, 2019.
  7. borglab/gtsam, May 2022.
  8. Superpoint: Self-supervised interest point detection and description. In CVPR Deep Learning for Visual SLAM Workshop, 2018.
  9. D2-net: A trainable cnn for joint detection and description of local features. arXiv preprint arXiv:1905.03561, 2019.
  10. Uwit: underwater image toolbox for optical image processing and mosaicking in matlab. In Proceedings of the 2002 Interntional Symposium on Underwater Technology (Cat. No.02EX556), pages 141–145, 2002.
  11. Unified temporal and spatial calibration for multi-sensor systems. In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 1280–1286, 2013.
  12. Bags of binary words for fast place recognition in image sequences. IEEE Transactions on Robotics, 28(5):1188–1197, October 2012.
  13. Michael Grupp. evo: Python package for the evaluation of odometry and slam. https://github.com/MichaelGrupp/evo, 2017.
  14. R2d2: repeatable and reliable detector and descriptor. Proc. NeurIPS, 2019.
  15. Thermal-inertial slam for the environments with challenging illumination. IEEE Robotics and Automation Letters, 7(4):8767–8774, 2022.
  16. Design and evaluation of a generic visual slam framework for multi camera systems. IEEE Robotics and Automation Letters, 8(11):7368–7375, 2023.
  17. Robust thermal-inertial localization for aerial robots: A case for direct methods. In 2019 International Conference on Unmanned Aircraft Systems (ICUAS), pages 1061–1068. IEEE, 2019.
  18. Keyframe-based thermal–inertial odometry. Journal of Field Robotics, 37(4):552–579, 2020.
  19. Night-to-day thermal image translation for deep thermal place recognition. Intelligent Service Robotics, 16(4):403–413, 2023.
  20. David G Lowe. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60:91–110, 2004.
  21. Superthermal: Matching thermal as visible through thermal feature exploration. IEEE Robotics and Automation Letters, 6(2):2690–2697, 2021.
  22. Performance evaluation of feature detectors and descriptors beyond the visible. Journal of Intelligent & Robotic Systems, 92:33–63, 2018.
  23. GlueStick: Robust image matching by sticking points and lines together. In International Conference on Computer Vision (ICCV), 2023.
  24. Bvt-slam: A binocular visible-thermal sensors slam system in low-light environments. IEEE Sensors Journal, pages 1–1, 2023.
  25. Vins-mono: A robust and versatile monocular visual-inertial state estimator. IEEE Transactions on Robotics, 34(4):1004–1020, 2018.
  26. Orb: An efficient alternative to sift or surf. In 2011 International Conference on Computer Vision, pages 2564–2571, 2011.
  27. Srvio: Super robust visual inertial odometry for dynamic environments and challenging loop-closure conditions. IEEE Transactions on Robotics, 39(4):2878–2891, 2023.
  28. Graph-based thermal–inertial slam with probabilistic neural networks. IEEE Transactions on Robotics, 38(3):1875–1893, 2022.
  29. From coarse to fine: Robust hierarchical localization at large scale. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12708–12717, 2019.
  30. Bad slam: Bundle adjusted direct rgb-d slam. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
  31. Sparse depth enhanced direct thermal-infrared slam beyond the visible spectrum. IEEE Robotics and Automation Letters, 4(3):2918–2925, 2019.
  32. Discriminative learning of deep convolutional feature point descriptors. In Proceedings of the IEEE international conference on computer vision, pages 118–126, 2015.
  33. DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras. Advances in neural information processing systems, 2021.
  34. Disk: Learning local features with policy gradient. Advances in Neural Information Processing Systems, 33:14254–14265, 2020.
  35. S. Umeyama. Least-squares estimation of transformation parameters between two point patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(4):376–380, 1991.
  36. Visual-inertial mapping with non-linear factor recovery. IEEE Robotics and Automation Letters (RA-L) & Int. Conference on Intelligent Robotics and Automation (ICRA), 5(2):422–429, 2020.
  37. Edge-based monocular thermal-inertial odometry in visually degraded environments. IEEE Robotics and Automation Letters, 8(4):2078–2085, 2023.
  38. Lift: Learned invariant feature transform. In European conference on computer vision, pages 467–483. Springer, 2016.
  39. Robust loop closure detection based on bag of superpoints and graph verification. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3787–3793, 2019.
  40. Tp-tio: A robust thermal-inertial odometry with deep thermalpoint. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4505–4512, 2020.
  41. R2former: Unified retrieval and reranking transformer for place recognition. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19370–19380, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Colin Keil (5 papers)
  2. Aniket Gupta (9 papers)
  3. Pushyami Kaveti (11 papers)
  4. Hanumant Singh (20 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets