2000 character limit reached
Scenario-Based Curriculum Generation for Multi-Agent Autonomous Driving (2403.17805v2)
Published 26 Mar 2024 in cs.RO, cs.LG, and cs.MA
Abstract: The automated generation of diverse and complex training scenarios has been an important ingredient in many complex learning tasks. Especially in real-world application domains, such as autonomous driving, auto-curriculum generation is considered vital for obtaining robust and general policies. However, crafting traffic scenarios with multiple, heterogeneous agents is typically considered as a tedious and time-consuming task, especially in more complex simulation environments. In our work, we introduce MATS-Gym, a Multi-Agent Traffic Scenario framework to train agents in CARLA, a high-fidelity driving simulator. MATS-Gym is a multi-agent training framework for autonomous driving that uses partial scenario specifications to generate traffic scenarios with variable numbers of agents. This paper unifies various existing approaches to traffic scenario description into a single training framework and demonstrates how it can be integrated with techniques from unsupervised environment design to automate the generation of adaptive auto-curricula. The code is available at https://github.com/AutonomousDrivingExaminer/mats-gym.
- A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, “CARLA: An open urban driving simulator,” in Proceedings of the 1st Annual Conference on Robot Learning, ser. Proceedings of Machine Learning Research, S. Levine, V. Vanhoucke, and K. Goldberg, Eds., vol. 78. PMLR, 2017, pp. 1–16. [Online]. Available: https://proceedings.mlr.press/v78/dosovitskiy17a.html
- D. J. Fremont, T. Dreossi, S. Ghosh, X. Yue, A. L. Sangiovanni-Vincentelli, and S. A. Seshia, “Scenic: A language for scenario specification and scene generation,” in Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, 2019, pp. 63–78. [Online]. Available: http://arxiv.org/abs/1809.09310
- M. Jiang, M. Dennis, J. Parker-Holder, J. Foerster, E. Grefenstette, and T. Rocktäschel, “Replay-guided adversarial environment design.” [Online]. Available: http://arxiv.org/abs/2110.02439
- E. Leurent, “An environment for autonomous driving decision-making,” publication Title: GitHub repository. [Online]. Available: https://github.com/eleurent/highway-env
- J. Bernhard, K. Esterle, P. Hart, and T. Kessler, “BARK: Open behavior benchmarking in multi-agent environments,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 6201–6208. [Online]. Available: https://ieeexplore.ieee.org/document/9341222/
- M. Althoff, M. Koschi, and S. Manzinger, “CommonRoad: Composable benchmarks for motion planning on roads,” in 2017 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2017, pp. 719–726. [Online]. Available: http://ieeexplore.ieee.org/document/7995802/
- X. Wang, H. Krasowski, and M. Althoff, “CommonRoad-RL: A configurable reinforcement learning environment for motion planning of autonomous vehicles,” in IEEE International Conference on Intelligent Transportation Systems (ITSC), 2021.
- M. Zhou, J. Luo, J. Villella, Y. Yang, D. Rusu, J. Miao et al., “SMARTS: Scalable multi-agent reinforcement learning training school for autonomous driving,” publication Title: Proceedings of the 4th Conference on Robot Learning (CoRL). [Online]. Available: https://arxiv.org/abs/2010.09776
- Q. Li, Z. Peng, Z. Xue, Q. Zhang, and B. Zhou, “Metadrive: Composing diverse driving scenarios for generalizable reinforcement learning,” arXiv preprint arXiv:2109.12674, 2021.
- D.-d. Contributors, “DI-drive: OpenDILab decision intelligence platform for autonomous driving simulation.” [Online]. Available: https://github.com/opendilab/DI-drive
- P. Palanisamy, “Multi-agent connected autonomous driving using deep reinforcement learning,” 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–7, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:207852404
- W. Ding, C. Xu, M. Arief, H. Lin, B. Li, and D. Zhao, “A survey on safety-critical driving scenario generation—a methodological perspective,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 7, pp. 6971–6988, 2023.
- R. Portelas, C. Colas, L. Weng, K. Hofmann, and P.-Y. Oudeyer, “Automatic curriculum learning for deep rl: a short survey,” in Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, ser. IJCAI’20, 2021.
- Z. Qiao, K. Muelling, J. M. Dolan, P. Palanisamy, and P. Mudalige, “Automatically generated curriculum based reinforcement learning for autonomous vehicles in urban environment,” in 2018 IEEE Intelligent Vehicles Symposium (IV), 2018, pp. 1233–1238.
- L. Anzalone, P. Barra, S. Barra, A. Castiglione, and M. Nappi, “An end-to-end curriculum learning approach for autonomous driving scenarios,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 10, pp. 19 817–19 826, 2022.
- M. Jiang, E. Grefenstette, and T. Rocktäschel, “Prioritized level replay.” [Online]. Available: http://arxiv.org/abs/2010.03934
- M. Dennis, N. Jaques, E. Vinitsky, A. Bayen, S. Russell, A. Critch, and S. Levine, “Emergent complexity and zero-shot transfer via unsupervised environment design.” [Online]. Available: http://arxiv.org/abs/2012.02096
- J. Parker-Holder, M. Jiang, M. Dennis, M. Samvelyan, J. Foerster, E. Grefenstette, and T. Rocktäschel, “Evolving curricula with regret-based environment design.” [Online]. Available: http://arxiv.org/abs/2203.01302
- I. Mediratta, M. Jiang, J. Parker-Holder, M. Dennis, E. Vinitsky, and T. Rocktäschel, “Stabilizing unsupervised environment design with a learned adversary.” [Online]. Available: http://arxiv.org/abs/2308.10797
- “ScenarioRunner — Traffic scenario definition and execution engine,” https://github.com/carla-simulator/scenario_runner, [Accessed 12-03-2024].
- J. Terry, B. Black, N. Grammel, M. Jayakumar, A. Hari, R. Sullivan et al., “PettingZoo: Gym for multi-agent reinforcement learning,” in Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, and J. W. Vaughan, Eds., vol. 34. Curran Associates, Inc., 2021, pp. 15 032–15 043. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2021/file/7ed2d3454c5eea71148b11d0c25104ff-Paper.pdf
- G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” arXiv preprint arXiv:1606.01540, 2016.
- M. Martyniak, “carla-birdeye-view,” [Accessed 12-03-2024]. [Online]. Available: https://github.com/deepsense-ai/carla-birdeye-view
- T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A next-generation hyperparameter optimization framework,” 2019.
- J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” 2017.
- C. Yu, A. Velu, E. Vinitsky, J. Gao, Y. Wang, A. Bayen, and Y. Wu, “The surprising effectiveness of ppo in cooperative multi-agent games,” Advances in Neural Information Processing Systems, vol. 35, pp. 24 611–24 624, 2022.