Papers
Topics
Authors
Recent
AI Research Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 86 tok/s
Gemini 2.5 Pro 56 tok/s Pro
GPT-5 Medium 31 tok/s Pro
GPT-5 High 33 tok/s Pro
GPT-4o 102 tok/s Pro
Kimi K2 202 tok/s Pro
GPT OSS 120B 467 tok/s Pro
Claude Sonnet 4 37 tok/s Pro
2000 character limit reached

MEM: Multi-Modal Elevation Mapping for Robotics and Learning (2309.16818v1)

Published 28 Sep 2023 in cs.RO and cs.CV

Abstract: Elevation maps are commonly used to represent the environment of mobile robots and are instrumental for locomotion and navigation tasks. However, pure geometric information is insufficient for many field applications that require appearance or semantic information, which limits their applicability to other platforms or domains. In this work, we extend a 2.5D robot-centric elevation mapping framework by fusing multi-modal information from multiple sources into a popular map representation. The framework allows inputting data contained in point clouds or images in a unified manner. To manage the different nature of the data, we also present a set of fusion algorithms that can be selected based on the information type and user requirements. Our system is designed to run on the GPU, making it real-time capable for various robotic and learning tasks. We demonstrate the capabilities of our framework by deploying it on multiple robots with varying sensor configurations and showcasing a range of applications that utilize multi-modal layers, including line detection, human detection, and colorization.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (23)
  1. J. Bresenham, “Algorithm for computer control of a digital plotter,” IBM Syst. J., vol. 4, no. 1, pp. 25–30, 1965.
  2. M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” in 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021.   IEEE, 2021, pp. 9630–9640.
  3. P. Ewen, A. Li, Y. Chen, S. Hong, and R. Vasudevan, “These maps are made for walking: Real-time terrain property estimation for mobile robots,” IEEE Robotics Autom. Lett., vol. 7, no. 3, pp. 7083–7090, 2022.
  4. P. Fankhauser, M. Bloesch, C. Gehring, M. Hutter, and R. Siegwart, “Robot-centric elevation mapping with uncertainty estimates,” Mobile Service Robotics: Proceedings of the 17th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines, pp. 433–440, 2014.
  5. P. Fankhauser, M. Bloesch, and M. Hutter, “Probabilistic terrain mapping for mobile robots with uncertain localization,” IEEE Robotics Autom. Lett., vol. 3, no. 4, pp. 3019–3026, 2018.
  6. P. Fankhauser and M. Hutter, “A Universal Grid Map Library: Implementation and Use Case for Rough Terrain Navigation,” in Robot Operating System (ROS) – The Complete Reference (Volume 1), A. Koubaa, Ed.   Springer, 2016, ch. 5.
  7. J. Frey, D. Hoeller, S. Khattak, and M. Hutter, “Locomotion policy guided traversability learning using volumetric representations of complex environments,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, oct 2022.
  8. J. Frey, M. E. Mattamala Aravena, N. Chebrolu, C. Cadena, M. Fallon, and M. Hutter, “Fast traversability estimation for wild visual navigation,” in Proceedings of Robotics: Science and System XIX.   Robotics Science & Systems Foundation, 2023, Conference Paper, robotics: Science and Systems (RSS 2023); Conference Location: Daegu, South Korea; Conference Date: July 10-14, 2023.
  9. L. Gan, Y. Kim, J. W. Grizzle, J. M. Walls, A. Kim, R. M. Eustice, and M. Ghaffari, “Multitask learning for scalable and dense multilayer bayesian map inference,” IEEE Transactions on Robotics, vol. 39, no. 1, pp. 699–717, 2023.
  10. L. Gan, R. Zhang, J. W. Grizzle, R. M. Eustice, and M. G. Jadidi, “Bayesian spatial kernel smoothing for scalable dense semantic mapping,” IEEE Robotics Autom. Lett., vol. 5, no. 2, pp. 790–797, 2020.
  11. W. V. Gansbeke, B. D. Brabandere, D. Neven, M. Proesmans, and L. V. Gool, “End-to-end lane detection through differentiable least-squares fitting,” in 2019 IEEE/CVF International Conference on Computer Vision Workshops, ICCV Workshops 2019, Seoul, Korea (South), October 27-28, 2019.   IEEE, 2019, pp. 905–913.
  12. R. Girshick, I. Radosavovic, G. Gkioxari, P. Dollár, and K. He, “Detectron,” https://github.com/facebookresearch/detectron, 2018.
  13. M. Grinvald et al., “Volumetric instance-aware semantic mapping and 3d object discovery,” IEEE Robotics Autom. Lett., vol. 4, no. 3, pp. 3037–3044, 2019.
  14. A. Hornung, K. M. Wurm, M. Bennewitz, C. Stachniss, and W. Burgard, “Octomap: an efficient probabilistic 3d mapping framework based on octrees,” Auton. Robots, vol. 34, no. 3, pp. 189–206, 2013.
  15. A. Howard et al., “Searching for mobilenetv3,” in 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019.   IEEE, 2019, pp. 1314–1324.
  16. D. Maturana et al., “Real-time semantic mapping for autonomous off-road navigation,” in Field and Service Robotics, Results of the 11th International Conference, FSR 2017, Zurich, Switzerland, 12-15 September 2017, ser. Springer Proceedings in Advanced Robotics, vol. 5.   Springer, 2017, pp. 335–350.
  17. T. Miki, L. Wellhausen, R. Grandia, F. Jenelten, T. Homberger, and M. Hutter, “Elevation mapping for locomotion and navigation using GPU,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022, Kyoto, Japan, October 23-27, 2022.   IEEE, 2022, pp. 2273–2280.
  18. R. Okuta, Y. Unno, D. Nishino, S. Hido, and Crissman, “Cupy : A numpy-compatible library for nvidia gpu calculations,” 2017.
  19. H. Oleynikova, Z. Taylor, M. Fehr, R. Siegwart, and J. I. Nieto, “Voxblox: Incremental 3d euclidean signed distance fields for on-board MAV planning,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2017, Vancouver, BC, Canada, September 24-28, 2017.   IEEE, 2017, pp. 1366–1373.
  20. Y. Pan et al., “Gem: Online globally consistent dense elevation mapping for unstructured terrain,” IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1–13, 2021.
  21. M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A. Ng, “Ros: an open-source robot operating system,” vol. 3, 01 2009.
  22. A. Rosinol, M. Abate, Y. Chang, and L. Carlone, “Kimera: an open-source library for real-time metric-semantic localization and mapping,” in 2020 IEEE International Conference on Robotics and Automation, ICRA 2020, Paris, France, May 31 - August 31, 2020.   IEEE, 2020, pp. 1689–1696.
  23. S. Thrun and et al., “Stanley: The robot that won the DARPA grand challenge,” J. Field Robotics, vol. 23, no. 9, pp. 661–692, 2006.
Citations (13)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Lightbulb On Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up