Papers
Topics
Authors
Recent
2000 character limit reached

Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research (2310.08710v1)

Published 12 Oct 2023 in cs.RO and cs.LG

Abstract: Simulation is an essential tool to develop and benchmark autonomous vehicle planning software in a safe and cost-effective manner. However, realistic simulation requires accurate modeling of nuanced and complex multi-agent interactive behaviors. To address these challenges, we introduce Waymax, a new data-driven simulator for autonomous driving in multi-agent scenes, designed for large-scale simulation and testing. Waymax uses publicly-released, real-world driving data (e.g., the Waymo Open Motion Dataset) to initialize or play back a diverse set of multi-agent simulated scenarios. It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training, making it suitable for modern large-scale, distributed machine learning workflows. To support online training and evaluation, Waymax includes several learned and hard-coded behavior models that allow for realistic interaction within simulation. To supplement Waymax, we benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions, where we highlight the effectiveness of routes as guidance for planning agents and the ability of RL to overfit against simulated agents.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Martín Abadi. TensorFlow: learning functions at scale. In Proceedings of the 21st ACM SIGPLAN International Conference on Functional Programming, pages 1–1, 2016.
  2. Vista 2.0: An open, data-driven simulator for multimodal sensing and policy learning for autonomous vehicles. In 2022 International Conference on Robotics and Automation (ICRA), pages 2419–2426. IEEE, 2022.
  3. ChauffeurNet: Learning to drive by imitating the best and synthesizing the worst. In Robotics: Science and Systems (RSS), 2019.
  4. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316, 2016.
  5. JAX: composable transformations of Python+NumPy programs, 2018.
  6. Openai gym, 2016.
  7. Hierarchical model-based imitation learning for planning in autonomous driving. In 2022 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 8652–8659. IEEE, 2022.
  8. nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles. arXiv preprint arXiv:2106.11810, 2021.
  9. Summit: A simulator for urban driving in massive mixed traffic. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 4023–4029. IEEE, 2020.
  10. Multipath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction. arXiv preprint arXiv:1910.05449, 2019.
  11. Learning by cheating. In Conference on Robot Learning, pages 66–75. PMLR, 2020.
  12. End-to-end driving via conditional imitation learning. In 2018 IEEE international conference on robotics and automation (ICRA), pages 4693–4700. IEEE, 2018.
  13. Causal confusion in imitation learning. Advances in Neural Information Processing Systems, 32, 2019.
  14. Carla: An open urban driving simulator. In Conference on Robot Learning, pages 1–16. PMLR, 2017.
  15. Large scale interactive motion forecasting for autonomous driving : The waymo open motion dataset. arXiv, 2021.
  16. Brax - a differentiable physics engine for large scale rigid body simulation, 2021.
  17. Densetnt: End-to-end trajectory prediction from dense goal sets. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15303–15312, 2021.
  18. Deep rl at scale: Sorting waste in office buildings with a fleet of mobile manipulators. arXiv preprint arXiv:2305.03270, 2023.
  19. Acme: A research framework for distributed reinforcement learning. arXiv preprint arXiv:2006.00979, 2020.
  20. Symphony: Learning realistic and diverse agents for autonomous driving simulation. arXiv preprint arXiv:2205.03195, 2022.
  21. Navigating occluded intersections with autonomous vehicles using deep reinforcement learning. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 2034–2039. IEEE, 2018.
  22. Learning to drive in a day. In 2019 International Conference on Robotics and Automation (ICRA), pages 8248–8254. IEEE, 2019.
  23. Drivergym: Democratising reinforcement learning for autonomous driving. arXiv preprint arXiv:2111.06889, 2021.
  24. Edouard Leurent. An environment for autonomous driving decision-making. https://github.com/eleurent/highway-env, 2018.
  25. Metadrive: Composing diverse driving scenarios for generalizable reinforcement learning. IEEE transactions on pattern analysis and machine intelligence, 2022.
  26. Learning lane graph representations for motion forecasting. In European Conference on Computer Vision, pages 541–556. Springer, 2020.
  27. Imitation is not enough: Robustifying imitation with reinforcement learning for challenging driving scenarios. In NeurIPS 2022 Machine Learning for Autonomous Driving Workshop, 2022.
  28. Systems and methods for generating synthetic sensor data via machine learning, Sept. 24 2020. US Patent App. 16/826,990.
  29. Beyond grand theft auto v for training, testing and enhancing deep learning in self driving cars. arXiv preprint arXiv:1712.01397, 2017.
  30. Pods: Policy optimization via differentiable simulation. In International Conference on Machine Learning, pages 7805–7817. PMLR, 2021.
  31. Deep learning for safe autonomous driving: Current challenges and future directions. IEEE Transactions on Intelligent Transportation Systems, 22(7):4316–4336, 2020.
  32. dm_env: A python interface for reinforcement learning environments, 2019.
  33. Sim4cv: A photo-realistic simulator for computer vision applications. International Journal of Computer Vision, 126(9):902–919, 2018.
  34. Wayformer: Motion forecasting via simple & efficient attention networks. arXiv preprint arXiv:2207.05844, 2022.
  35. Wayformer: Motion forecasting via simple and efficient attention networks, 2022.
  36. Scene transformer: A unified architecture for predicting multiple agent trajectories. arXiv preprint arXiv:2106.08417, 2021.
  37. Simulation-based reinforcement learning for real-world autonomous driving. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 6411–6418. IEEE, 2020.
  38. Praveen Palanisamy. Multi-agent connected autonomous driving using deep reinforcement learning. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1–7. IEEE, 2020.
  39. Dean A Pomerleau. Alvinn: An autonomous land vehicle in a neural network. Advances in neural information processing systems, 1, 1988.
  40. Craig Quiter. Deepdrive zero, June 2020.
  41. PRECOG: prediction conditioned on goals in visual multi-agent settings. CoRR, abs/1905.01296, 2019.
  42. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 627–635. JMLR Workshop and Conference Proceedings, 2011.
  43. Amit Sabne. Xla : Compiling machine learning for peak performance, 2020.
  44. Madras: Multi agent driving simulator. Journal of Artificial Intelligence Research, 70:1517–1555, 2021.
  45. Prioritized experience replay. arXiv preprint arXiv:1511.05952, 2015.
  46. Imagining the road ahead: Multi-agent trajectory prediction via differentiable simulation. In 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pages 720–725, 2021.
  47. Intersim: Interactive traffic simulation via explicit relation modeling. arXiv preprint arXiv:2210.14413, 2022.
  48. Block-nerf: Scalable large scene neural view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8248–8258, 2022.
  49. Multiple futures prediction. Advances in Neural Information Processing Systems, 32, 2019.
  50. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 23–30. IEEE, 2017.
  51. Congested traffic states in empirical observations and microscopic simulations. Physical review E, 62(2):1805, 2000.
  52. Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real world. arXiv preprint arXiv:2206.09889, 2022.
  53. Safetynet: Safe planning for real-world self-driving vehicles using machine-learned policies. In 2022 International Conference on Robotics and Automation (ICRA), pages 897–904. IEEE, 2022.
  54. A reinforcement learning based approach for automated lane change maneuvers. In 2018 IEEE Intelligent Vehicles Symposium (IV), pages 1379–1384. IEEE, 2018.
  55. Torcs, the open racing car simulator. Software available at http://torcs. sourceforge. net, 4(6):2, 2000.
  56. Bits: Bi-level imitation for traffic simulation. arXiv preprint arXiv:2208.12403, 2022.
  57. Smarts: Scalable multi-agent reinforcement learning training school for autonomous driving. arXiv preprint arXiv:2010.09776, 2020.
Citations (59)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now

Whiteboard

Paper to Video (Beta)

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free