Can ChatGPT Enable ITS? The Case of Mixed Traffic Control via Reinforcement Learning (2306.08094v2)
Abstract: The surge in Reinforcement Learning (RL) applications in Intelligent Transportation Systems (ITS) has contributed to its growth as well as highlighted key challenges. However, defining objectives of RL agents in traffic control and management tasks, as well as aligning policies with these goals through an effective formulation of Markov Decision Process (MDP), can be challenging and often require domain experts in both RL and ITS. Recent advancements in LLMs such as GPT-4 highlight their broad general knowledge, reasoning capabilities, and commonsense priors across various domains. In this work, we conduct a large-scale user study involving 70 participants to investigate whether novices can leverage ChatGPT to solve complex mixed traffic control problems. Three environments are tested, including ring road, bottleneck, and intersection. We find ChatGPT has mixed results. For intersection and bottleneck, ChatGPT increases number of successful policies by 150% and 136% compared to solely beginner capabilities, with some of them even outperforming experts. However, ChatGPT does not provide consistent improvements across all scenarios.
- OpenAI. Gpt-4 technical report. Technical report, OpenAI, 2023.
- Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023.
- Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023.
- Google. Palm2 technical report. Technical report, Google, 2023.
- Chatgpt: A revolutionary tool for teaching and learning mathematics. Eurasia Journal of Mathematics, Science and Technology Education, 19(7):em2286, 2023.
- Chatgpt makes medicine easy to swallow: An exploratory case study on simplified radiology reports. arXiv preprint arXiv:2212.14882, 2022.
- Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
- Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022.
- Reinforcement learning: An introduction. MIT press, 2018.
- Deep reinforcement learning for intelligent transportation systems: A survey. IEEE Transactions on Intelligent Transportation Systems, 23(1):11–32, 2020.
- Deep learning for intelligent transportation systems: A survey of emerging trends. IEEE Transactions on Intelligent transportation systems, 21(8):3152–3168, 2019.
- SAE International. Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles. https://www.sae.org/standards/content/j3016_202104/, Apr 2021.
- Flow: A modular learning framework for mixed autonomy traffic. IEEE Transactions on Robotics, 38(2):1270–1286, 2021.
- Reinforcement learning for mixed autonomy intersections. In 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pages 2089–2094. IEEE, 2021.
- Learning to control and coordinate hybrid traffic through robot vehicles at complex and unsignalized intersections. arXiv preprint arXiv:2301.05294, 2023.
- Congested traffic states in empirical observations and microscopic simulations. Physical review E, 62(2):1805, 2000.
- Palm-e: An embodied multimodal language model. International Conference on Machine Learning (ICML), 2023.
- Can an embodied agent find your” cat-shaped mug”? llm-based zero-shot object navigation. arXiv e-prints, pages arXiv–2303, 2023.
- Do embodied agents dream of pixelated sheep?: Embodied decision making using language guided world modelling. In Workshop on Reincarnating Reinforcement Learning at ICLR 2023, 2023.
- Guiding pretraining in reinforcement learning with large language models. International Conference on Machine Learning (ICML), 2023.
- Foundation models for decision making: Problems, methods, and opportunities. arXiv preprint arXiv:2303.04129, 2023.
- Language instructed reinforcement learning for human-ai coordination. International Conference on Machine Learning (ICML), 2023.
- Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
- The lord of the ring road: a review and evaluation of autonomous control policies for traffic in a ring road. ACM Transactions on Cyber-Physical Systems (TCPS), 6(1):1–25, 2022.
- Empirical characterization and interpretation of hysteresis and capacity drop phenomena in freeway networks. Transportation Research Record: Journal of the Transportation Research Board, Transportation Research Board of the National Academies, Washington, DC, 2013.
- Benchmarks for reinforcement learning in mixed-autonomy traffic. In Conference on robot learning, pages 399–409. PMLR, 2018.
- OpenAI. Chatgpt: Optimizing language models for dialogue., 2023.
- Typingmind. https://www.typingmind.com. Accessed: April 2023.
- RLlib: Abstractions for distributed reinforcement learning. In International Conference on Machine Learning (ICML), 2018.