Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Study on the Use of Simulation in Synthesizing Path-Following Control Policies for Autonomous Ground Robots (2403.18021v1)

Published 26 Mar 2024 in cs.RO

Abstract: We report results obtained and insights gained while answering the following question: how effective is it to use a simulator to establish path following control policies for an autonomous ground robot? While the quality of the simulator conditions the answer to this question, we found that for the simulation platform used herein, producing four control policies for path planning was straightforward once a digital twin of the controlled robot was available. The control policies established in simulation and subsequently demonstrated in the real world are PID control, MPC, and two neural network (NN) based controllers. Training the two NN controllers via imitation learning was accomplished expeditiously using seven simple maneuvers: follow three circles clockwise, follow the same circles counter-clockwise, and drive straight. A test randomization process that employs random micro-simulations is used to rank the ``goodness'' of the four control policies. The policy ranking noted in simulation correlates well with the ranking observed when the control policies were tested in the real world. The simulation platform used is publicly available and BSD3-released as open source; a public Docker image is available for reproducibility studies. It contains a dynamics engine, a sensor simulator, a ROS2 bridge, and a ROS2 autonomy stack the latter employed both in the simulator and the real world experiments.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. H. Choi, C. Crump, C. Duriez, A. Elmquist, G. Hager, D. Han, F. Hearl, J. Hodgins, A. Jain, F. Leve, C. Li, F. Meier, D. Negrut, L. Righetti, A. Rodriguez, J. Tan, and J. Trinkle, “On the use of simulation in robotics: Opportunities, challenges, and suggestions for moving forward,” Proceedings of the National Academy of Sciences, vol. 118, no. 1, 2021. [Online]. Available: https://www.pnas.org/content/118/1/e1907856118
  2. A. Bewley, J. Rigley, Y. Liu, J. Hawke, R. Shen, V.-D. Lam, and A. Kendall, “Learning to drive from simulation without real world labels,” in 2019 International Conference on Robotics and Automation (ICRA).   IEEE, 2019, pp. 4818–4824.
  3. B. Osiński, A. Jakubowski, P. Ziecina, P. Miłoś, C. Galias, S. Homoceanu, and H. Michalewski, “Simulation-based reinforcement learning for real-world autonomous driving,” in 2020 IEEE international conference on robotics and automation (ICRA).   IEEE, 2020, pp. 6411–6418.
  4. A. Kalapos, C. Gór, R. Moni, and I. Harmati, “Sim-to-real reinforcement learning applied to end-to-end vehicle control,” in 2020 23rd International Symposium on Measurement and Control in Robotics (ISMCR).   IEEE, 2020, pp. 1–6.
  5. N. Hamilton, P. Musau, D. M. Lopez, and T. T. Johnson, “Zero-shot policy transfer in autonomous racing: Reinforcement learning vs imitation learning,” in 2022 IEEE International Conference on Assured Autonomy (ICAA).   IEEE, 2022, pp. 11–20.
  6. A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, “CARLA: An open urban driving simulator,” in Proceedings of the 1st Annual Conference on Robot Learning, 2017, pp. 1–16.
  7. Open-Source-Robotics-Foundation, “A 3D multi-robot simulator with dynamics,” http://gazebosim.org/, accessed: 2022-03-01.
  8. O. Michel, “Cyberbotics ltd. webots: professional mobile robot simulation,” International Journal of Advanced Robotic Systems, vol. 1, no. 1, p. 5, 2004.
  9. S. Shah, D. Dey, C. Lovett, and A. Kapoor, “AirSim: High-fidelity visual and physical simulation for autonomous vehicles,” in Field and service robotics.   Springer, 2018, pp. 621–635.
  10. A. Tasora, R. Serban, H. Mazhar, A. Pazouki, D. Melanz, J. Fleischmann, M. Taylor, H. Sugiyama, and D. Negrut, “Chrono: An open source multi-physics dynamics engine,” in High Performance Computing in Science and Engineering – Lecture Notes in Computer Science, T. Kozubek, Ed.   Springer International Publishing, 2016, pp. 19–49.
  11. NVIDIA, “Isaac SDK,” 2019, available online at https://developer.nvidia.com/isaac-sdk.
  12. N. Jakobi, P. Husbands, and I. Harvey, “Noise and the reality gap: The use of simulation in evolutionary robotics,” in European Conference on Artificial Life.   Springer, 1995, pp. 704–720.
  13. A. Elmquist, A. Young, I. Mahajan, K. Fahey, A. Dashora, S. Ashokkumar, S. Caldararu, V. Freire, X. Xu, R. Serban, and D. Negrut, “A software toolkit and hardware platform for investigating and comparing robot autonomy algorithms in simulation and reality,” arXiv preprint arXiv:2206.06537, 2022.
  14. C. Rother, Z. Zhou, and J. Chen, “Development of a four-wheel steering scale vehicle for research and education on autonomous vehicle motion control,” IEEE Robotics and Automation Letters, vol. 8, no. 8, pp. 5015–5022, 2023.
  15. Y. Pan, C.-A. Cheng, K. Saigol, K. Lee, X. Yan, E. A. Theodorou, and B. Boots, “Imitation learning for agile autonomous driving,” The International Journal of Robotics Research, vol. 39, no. 2-3, pp. 286–302, 2020.
  16. J. Wang, M. T. Fader, and J. A. Marshall, “Learning-based model predictive control for improved mobile robot path following using gaussian processes and feedback linearization,” Journal of Field Robotics, 2023.
  17. X. Jin, Q. Wang, Z. Yan, and H. Yang, “Nonlinear robust control of trajectory-following for autonomous ground electric vehicles with active front steering system,” AIMS Math, vol. 8, no. 5, pp. 11 151–11 179, 2023.
  18. R. F. Benekohal and J. Treiterer, “CARSIM: Car-following model for simulation of traffic in normal and stop-and-go conditions,” Transportation research record, vol. 1194, pp. 99–111, 1988.
  19. J. E. Sierra-Garcia and M. Santos, “Combining reinforcement learning and conventional control to improve automatic guided vehicles tracking of complex trajectories,” Expert Systems, vol. 41, no. 2, p. e13076, 2024.
  20. K. Zhang, Q. Sun, and Y. Shi, “Trajectory tracking control of autonomous ground vehicles using adaptive learning mpc,” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 12, pp. 5554–5564, 2021.
  21. J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS).   IEEE, 2017, pp. 23–30.
  22. K. L. Voogd, J. P. Allamaa, J. Alonso-Mora, and T. D. Son, “Reinforcement learning from simulation to real world autonomous driving using digital twin,” IFAC-PapersOnLine, vol. 56, no. 2, pp. 1510–1515, 2023.
  23. A. Kovacs and I. Vajk, “Optimization-based model predictive tube control for autonomous ground vehicles with minimal tuning parameters,” Unmanned Systems, vol. 11, no. 01, pp. 93–108, 2023.
  24. A. Kadian, J. Truong, A. Gokaslan, A. Clegg, E. Wijmans, S. Lee, M. Savva, S. Chernova, and D. Batra, “Sim2Real predictivity: Does evaluation in simulation predict real-world performance?” IEEE Robotics and Automation Letters, vol. 5, no. 4, pp. 6670–6677, 2020.
  25. P. Akella, W. Ubellacker, and A. D. Ames, “Safety-critical controller verification via sim2real gap quantification,” in 2023 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2023, pp. 10 539–10 545.
  26. H. Zhang, S. Caldararu, and D. Negrut, “ART-Oak,” Simulation-Based Engineering Laboratory, University of Wisconsin-Madison, Tech. Rep., 2023, https://sbel.wisc.edu/technicalreports/.
  27. F. Chollet et al., “Keras,” https://keras.io, 2015.
  28. A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in PyTorch,” in NIPS 2017 Workshop Autodiff, 2017, Conference Proceedings.
  29. Project Chrono Team, “Chrono: An open source framework for the physics-based simulation of dynamic systems,” https://github.com/projectchrono/chrono, accessed: 2022-01-10.
  30. Project Chrono Development Team, “PyChrono: A Python wrapper for the Chrono multi-physics library,” https://anaconda.org/projectchrono/pychrono, accessed: 2023-01-14.
  31. S. Benatti, A. Young, A. Elmquist, J. Taves, R. Serban, D. Mangoni, A. Tasora, and D. Negrut, “Pychrono and Gym-Chrono: A deep reinforcement learning framework leveraging multibody dynamics to control autonomous vehicles and robots,” in Advances in Nonlinear Dynamics.   Springer, 2022, pp. 573–584.
  32. R. Serban, M. Taylor, D. Negrut, and A. Tasora, “Chrono::Vehicle template-based ground vehicle modeling and simulation,” International Journal of Vehicle Performance, vol. 5, no. 1, pp. 18–39, 2019.
  33. A. Elmquist, A. Young, T. Hansen, S. Ashokkumar, S. Caldararu, A. Dashora, I. Mahajan, H. Zhang, L. Fang, H. Shen, X. Xu, R. Serban, and D. Negrut, “ART/ATK: A research platform for assessing and mitigating the sim-to-real gap in robotics and autonomous vehicle engineering,” 2022. [Online]. Available: https://arxiv.org/pdf/2211.04886.pdf
  34. A. Elmquist, R. Serban, and D. Negrut, “A sensor simulation framework for training and testing robots and autonomous vehicles,” Journal of Autonomous Vehicles and Systems, vol. 1, no. 2, p. 021001, 2021.
  35. J. Taves, A. Elmquist, A. Young, R. Serban, and D. Negrut, “Synchrono: A scalable, physics-based simulation platform for testing groups of autonomous vehicles and/or robots,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2020, pp. 2251–2256.
  36. H. Zhang, S. Caldararu, A. Young, A. Ruiz, H. Unjhawala, S. Ashokkumar, I. Mahajan, N. Batagoda, L. Bakke, and D. Negrut, “Simulation Reproducibility for Different Path Following Policies,” https://github.com/uwsbel/sbel-reproducibility/tree/master/2024/IROSPathFollowing, 2024, Simulation-Based Engineering Laboratory, University of Wisconsin-Madison.
  37. H. Unjhawala, T. Hansen, H. Zhang, S. Caldraru, S. Chatterjee, L. Bakke, J. Wu, R. Serban, and D. Negrut, “An expeditious and expressive vehicle dynamics model for applications in controls and reinforcement learning,” IEEE Access, pp. 1–1, 2024.
  38. UW-Madison Simulation Based Engineering Laboratory, “Autonomy Toolkit,” http://projects.sbel.org/autonomy-toolkit/, 2022.
  39. G. Klančar and I. Škrjanc, “Tracking-error model-based predictive control for mobile robots in real time,” Robotics and autonomous systems, vol. 55, no. 6, pp. 460–469, 2007.
  40. “OSQP (Operator Splitting Quadratic Program): Open-source numerical optimization solver,” https://www.cvxpy.org/, accessed : 2023-10-21.
  41. H. Zhang, H. Unjhawala, S. Caldararu, I. Mahajan, L. Bakke, R. Serban, and D. Negrut, “Simplified 4DOF bicycle model for robotics applications,” Simulation-Based Engineering Laboratory, University of Wisconsin-Madison, Tech. Rep., 2023, https://sbel.wisc.edu/wp-content/uploads/sites/569/2023/06/TR-2023-06.pdf.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets