Papers
Topics
Authors
Recent
2000 character limit reached

Learning to navigate efficiently and precisely in real environments

Published 25 Jan 2024 in cs.RO and cs.CV | (2401.14349v1)

Abstract: In the context of autonomous navigation of terrestrial robots, the creation of realistic models for agent dynamics and sensing is a widespread habit in the robotics literature and in commercial applications, where they are used for model based control and/or for localization and mapping. The more recent Embodied AI literature, on the other hand, focuses on modular or end-to-end agents trained in simulators like Habitat or AI-Thor, where the emphasis is put on photo-realistic rendering and scene diversity, but high-fidelity robot motion is assigned a less privileged role. The resulting sim2real gap significantly impacts transfer of the trained models to real robotic platforms. In this work we explore end-to-end training of agents in simulation in settings which minimize the sim2real gap both, in sensing and in actuation. Our agent directly predicts (discretized) velocity commands, which are maintained through closed-loop control in the real robot. The behavior of the real robot (including the underlying low-level controller) is identified and simulated in a modified Habitat simulator. Noise models for odometry and localization further contribute in lowering the sim2real gap. We evaluate on real navigation scenarios, explore different localization and point goal calculation methods and report significant gains in performance and robustness compared to prior work.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (69)
  1. On evaluation of embodied navigation agents. arXiv preprint, 2018.
  2. Combining optimal control and learning for visual navigation in novel environments. In CoRL, 2020b.
  3. Learning to reason on uncertain topological maps. In ECCV, 2020a.
  4. Egomap: Projective mapping and structured egocentric memory for deep RL. In ECML-PKDD, 2020b.
  5. Learning to plan with uncertain topological maps. In ECCV, 2020c.
  6. Graph augmented Deep Reinforcement Learning in the GameRLand3D environment. In AAAI Workshop on Reinforcement Learning in Games, 2022.
  7. End-to-End (Instance)-Image Goal Navigation through Correspondence as an Emergent Phenomenon,. In ICLR, 2024a.
  8. Learning with a Mole: Transferable latent spatial representations for navigation without reconstruction. In ICLR, 2024b.
  9. Simultaneous localization and mapping: A survey of current trends in autonomous driving. IEEE Transactions on Intelligent Vehicles, 2017.
  10. The interactive museum tour-guide robot. In Aaai/iaai, pages 11–18, 1998.
  11. Object goal navigation using goal-oriented semantic exploration. In NeurIPS, 2020a.
  12. Learning to explore using active neural slam. In ICLR, 2020b.
  13. Semantic curiosity for active visual learning. In ECCV, 2020c.
  14. Neural topological slam for visual navigation. In CVPR, 2020d.
  15. Robustnav: Towards benchmarking robustness in embodied navigation. CoRR, 2106.04531, 2021.
  16. Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation. arXiv:2202.11742, 2022.
  17. Deep reinforcement learning of navigation in a complex and crowded environment with a limited field of view. In ICRA, 2019.
  18. Hybrid imitative planning with geometric and predictive costs in off-road environments. In ICRA, 2022.
  19. Learning whom to trust in navigation: dynamically switching between classical and neural planning. In IROS, 2023.
  20. Goal-conditioned imitation learning. In NeurIPS, 2019.
  21. PALM-E: An Embodied Multimodal Language Model. In ICML, 2023.
  22. Vtnet: Visual transformer network for object goal navigation. arXiv preprint arXiv:2105.09447, 2021.
  23. Off-dynamics reinforcement learning: Training for transfer with domain classifiers. In ICLR, 2021.
  24. Scene memory transformer for embodied agents in long-horizon tasks. In CVPR, 2019.
  25. The dynamic window approach to collision avoidance. IEEE Robotics & Automation Magazine, 4(1):23–33, 1997.
  26. Navigating to objects in the real world. Science Robotics, 8(79), 2023.
  27. Cognitive mapping and planning for visual nav. In CVPR, 2017.
  28. Mapnet: An allocentric spatial memory for mapping environments. In CVPR, 2018.
  29. Perspectives on sim2real transfer for robotics: A summary of the R: SS 2020 workshop, 2020.
  30. Inner Monologue: Embodied Reasoning through Planning with Language Models. In CoRL, 2022.
  31. Reinforcement learning with unsupervised auxiliary tasks. In ICLR, 2017.
  32. Sim2Real Predictivity: Does Evaluation in Simulation Predict Real-World Performance? IEEE Robotics and Automation Letters, 5(4):6670–6677, 2020.
  33. All-in-one: A DRL-based control switch combining state-of-the-art navigation planners. In ICRA, 2022.
  34. AI2-THOR: An Interactive 3D Environment for Visual AI. CoRR, 1712.05474, 2017.
  35. Kurt Konolige. A gradient method for realtime robot control. In IROS, 2000.
  36. RTAB-Map as an open-source lidar and visual simultaneous localization and mapping library for large-scale and long-term online operation. Journal of Field Robotics, 36(2):416–446, 2019.
  37. World model based sim2real transfer for visual navigation. In NeurIPS Robot Learning Workshop, 2023.
  38. Active Mapping and Robot Exploration: A Survey. Sensors, 21(7):2445, 2021.
  39. The marathon 2: A navigation system. In IROS, 2020.
  40. Where are we in the search for an artificial visual cortex for embodied intelligence? In arXiv:2303.18240, 2023.
  41. The office marathon: Robust navigation in an indoor office environment. In ICRA, 2010.
  42. Multi-Object Navigation with dynamically learned neural implicit representations. In ICCV, 2023.
  43. Learning to navigate in complex environments. In ICLR, 2017a.
  44. Learning to navigate in complex environments. In ICLR, 2017b.
  45. Katsuhiko Ogata. Modern Control Engineering. Prentice Hall, 2010.
  46. Neural map: Structured memory for deep reinforcement learning. In ICLR, 2018.
  47. Is mapping necessary for realistic pointgoal navigation? In CVPR, 2022.
  48. Sim-to-real transfer of robotic control with dynamics randomization. In ICRA, 2018.
  49. Habitat-matterport 3D dataset (HM3D): 1000 large-scale 3d environments for embodied AI. In NeurIPS Datasets and Benchmarks Track, 2021.
  50. A Generalist Agent. arXiv:2205.06175, 2022. arXiv: 2205.06175.
  51. Timed-elastic-bands for time-optimal point-to-point nonlinear model predictive control. In European Control Conference (ECC), 2015.
  52. An in-depth experimental study of sensor usage and visual reasoning of robots navigating in real environments. In ICRA, 2022.
  53. Habitat: A platform for embodied ai research. In ICCV, 2019.
  54. Proximal policy optimization algorithms. arXiv preprint, 2017.
  55. James A Sethian. A fast marching level set method for monotonically advancing fronts. PNAS, 93(4):1591–1595, 1996.
  56. ViKiNG: Vision-based kilometer-scale navigation with geographic hints. In RSS, 2022.
  57. ViNT: A foundation model for visual navigation. In CoRL, 2023.
  58. Sim-to-real: Learning agile locomotion for quadruped robots. In RSS, 2018.
  59. Probabilistic robotics, 2005.
  60. Bi-directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation Agents. IEEE Robotics and Automation Letters, 6(2), 2021.
  61. Rethinking sim2real: Lower fidelity simulation leads to higher sim2real transfer in navigation. In CoRL, pages 859–870, 2022.
  62. Sim-to-real strategy for spatially aware robot navigation in uneven outdoor environments. In ICRA Workshop on Releasing Robots into the Wild, 2022.
  63. Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames. In ICLR, 2019.
  64. OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNav. In arXiv:2303.07798, 2023.
  65. Success weighted by completion time: A dynamics-aware evaluation criteria for embodied navigation. In IROS, 2021.
  66. Learning robust agents for visual navigation in dynamic environments: The winning entry of igibson challenge 2021. In IROS, pages 77–83, 2022.
  67. Path planning using neural A* search. In ICML, 2021.
  68. Adversarial discriminative sim-to-real transfer of visuo-motor policies. Int. J. Robotics Res., 38(10-11), 2019.
  69. Adapting object detectors via selective cross-domain alignment. In CVPR, 2019.

Summary

  • The paper presents a novel end-to-end training pipeline with a refined motion model to bridge the sim2real gap in navigation.
  • It leverages discretized velocity commands and a second-order dynamical model to closely mimic real robot behavior.
  • Empirical results show significant gains in success metrics and robustness in navigating obstacle-rich environments.

Efficient Navigation in Real Environments: Bridging the Sim2Real Gap

This paper explores the intricate problem of efficiently training autonomous agents for navigation in real environments by minimizing the sim2real gap—discrepancies between simulated and real-world performance. The authors advocate for an approach that emphasizes the fidelity of both sensing and actuation processes in simulation to enhance the realism of agent training and thereby improve real-world performance.

Problem Context

Traditionally, the robotics community has employed model-based control solutions heavily dependent on accurate dynamics modeling and precise localization techniques for navigation tasks. However, the advent of Embodied AI has shifted the focus towards training agents within photo-realistic simulators using deep reinforcement and imitation learning, often leading to a noticeable sim2real gap owing to simplified motion models. This work addresses the gap by proposing an end-to-end training pipeline with a refined motion model that closely mirrors physical dynamics, thus ensuring smoother transitions from simulated to real-world environments.

Methodology

The paper presents a comprehensive methodology that integrates both realistic robot dynamics and noise models in the Habitat simulator. Key elements of their approach include:

  1. Velocity Command Prediction: The agent, instead of predicting discrete position commands, learns to predict discretized velocity commands (linear and angular), which are maintained through a closed-loop low-level controller on the real robot. This provides a more seamless integration into real-world navigation systems.
  2. Dynamical Modeling: A second-order dynamical model representing the real robot's behavior and its low-level controllers is integrated into the simulator. This model is derived from empirical data obtained through controlled experiments on actual robotic platforms.
  3. Sensing and Pose Estimation: The framework employs a combination of visual and Lidar sensors. It distinguishes between different localization methodologies — dead reckoning from wheel encoders and external localization with Monte Carlo techniques — to enhance the robustness of goal-oriented navigation.
  4. Recurrent Policy with Auxiliary Goal Estimation: The agent employs a recurrent neural policy with an auxiliary network that helps maintain accurate estimates of the goal location relative to its initial frame, combined with goal integration capabilities for improved navigation performance.

Experimental Results and Analysis

The paper provides an exhaustive evaluation, with both simulated experiments and real-world tests. Notably, the results demonstrate significant improvements in navigation metrics such as Success Rate (SR), Success weighted by Path Length (SPL), and Success weighted by Completion Time (SCT) when trained using the proposed approach as compared to prior models.

  • Sim2Real Transfer: The utilization of a realistic motion model during training in simulation effectively mitigates the sim2real gap, as evidenced by comparable agent performance in new environments as well as the real world.
  • Real-World Robustness: The agents exhibited robustness in avoiding collisions and efficiently navigating through complex environments with thin and finely structured obstacles, surpassing traditional map-based planning approaches in certain scenarios.
  • Localization and Goal Integration: Empirical results highlight the superior performance of models trained with static point goals combined with internal localization mechanisms, demonstrating lower dependency on external pose updates.

Implications and Future Work

This research contributes significantly to enhancing the reliability of simulated training for robotic navigation, progressively closing the sim2real gap through a refined understanding of realistic motion dynamics. Practically, it facilitates the deployment of robust, navigation-capable robots in dynamic environments with minimal tuning. Future research could explore adaptive dynamics modeling, improved noise simulation, and further integration with higher-level cognitive tasks to expand the applicability and flexibility of the approach in various robotics applications.

By addressing the challenges associated with real-world deployment post-simulation training, this work lays crucial groundwork for advancing the capabilities of autonomous systems, paving the way for more robust and versatile applications in areas such as autonomous vehicles, smart home robotics, and industrial automation.

File Document Download Save Streamline Icon: https://streamlinehq.com Markdown

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 7 tweets with 136 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free