Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning to Change: Choreographing Mixed Traffic Through Lateral Control and Hierarchical Reinforcement Learning (2403.14879v1)

Published 21 Mar 2024 in cs.RO and cs.MA

Abstract: The management of mixed traffic that consists of robot vehicles (RVs) and human-driven vehicles (HVs) at complex intersections presents a multifaceted challenge. Traditional signal controls often struggle to adapt to dynamic traffic conditions and heterogeneous vehicle types. Recent advancements have turned to strategies based on reinforcement learning (RL), leveraging its model-free nature, real-time operation, and generalizability over different scenarios. We introduce a hierarchical RL framework to manage mixed traffic through precise longitudinal and lateral control of RVs. Our proposed hierarchical framework combines the state-of-the-art mixed traffic control algorithm as a high level decision maker to improve the performance and robustness of the whole system. Our experiments demonstrate that the framework can reduce the average waiting time by up to 54% compared to the state-of-the-art mixed traffic control method. When the RV penetration rate exceeds 60%, our technique consistently outperforms conventional traffic signal control programs in terms of the average waiting time for all vehicles at the intersection.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. C. Wu, A. R. Kreidieh, K. Parvate, E. Vinitsky, and A. M. Bayen, “Flow: A modular learning framework for mixed autonomy traffic,” IEEE Transactions on Robotics, vol. 38, no. 2, pp. 1270–1286, 2022.
  2. H. Wei, X. Liu, L. Mashayekhy, and K. Decker, “Mixed-autonomy traffic control with proximal policy optimization,” in 2019 IEEE Vehicular Networking Conference (VNC).   IEEE, 2019, pp. 1–8.
  3. E. Vinitsky, K. Parvate, A. Kreidieh, C. Wu, and A. Bayen, “Lagrangian control through deep-rl: Applications to bottleneck decongestion,” in IEEE International Conference on Intelligent Transportation Systems, 2018, pp. 759–765.
  4. S. Feng, X. Yan, H. Sun, Y. Feng, and H. X. Liu, “Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment,” Nature communications, vol. 12, no. 1, pp. 1–14, 2021.
  5. Z. Yan and C. Wu, “Reinforcement learning for mixed autonomy intersections,” in IEEE International Intelligent Transportation Systems Conference, 2021, pp. 2089–2094.
  6. K. Jang, E. Vinitsky, B. Chalaki, B. Remer, L. Beaver, A. A. Malikopoulos, and A. Bayen, “Simulation to scaled city: zero-shot policy transfer for traffic control via autonomous vehicles,” in ACM/IEEE International Conference on Cyber-Physical Systems, 2019, pp. 291–300.
  7. M. Villarreal, B. Poudel, J. Pan, and W. Li, “Mixed traffic control and coordination from pixels,” in IEEE International Conference on Robotics and Automation (ICRA), 2024.
  8. B. Poudel and W. Li, “Beyond simulated drivers: Evaluating the impact of real-world car-following in mixed traffic control,” arXiv preprint arXiv:2311.12261, 2023.
  9. D. Wang, W. Li, L. Zhu, and J. Pan, “Learning to control and coordinate mixed traffic through robot vehicles at complex and unsignalized intersections,” arXiv preprint arXiv:2301.05294, 2023.
  10. S. Wang, M. Shang, M. W. Levin, and R. Stern, “A general approach to smoothing nonlinear mixed traffic via control of autonomous vehicles,” Transportation Research Part C: Emerging Technologies, vol. 146, p. 103967, 2023.
  11. J. Wang, Y. Zheng, Q. Xu, J. Wang, and K. Li, “Controllability analysis and optimal controller synthesis of mixed traffic systems,” in 2019 IEEE Intelligent Vehicles Symposium (IV).   IEEE, 2019, pp. 1041–1047.
  12. M. Karimi, C. Roncoli, C. Alecsandru, and M. Papageorgiou, “Cooperative merging control via trajectory optimization in mixed vehicular traffic,” Transportation Research Part C: Emerging Technologies, vol. 116, p. 102663, 2020.
  13. H. Yang and K. Oguchi, “Intelligent vehicle control at signal-free intersection under mixed connected environment,” IET Intelligent Transport Systems, vol. 14, no. 2, pp. 82–90, 2020.
  14. L. Zhao, A. Malikopoulos, and J. Rios-Torres, “Optimal control of connected and automated vehicles at roundabouts: An investigation in a mixed-traffic environment,” IFAC-PapersOnLine, vol. 51, no. 9, pp. 73–78, 2018.
  15. D. Wang, W. Li, and J. Pan, “Large-scale mixed traffic control using dynamic vehicle routing and privacy-preserving crowdsourcing,” IEEE Internet of Things Journal, vol. 11, no. 2, pp. 1981–1989, 2024.
  16. M. Villarreal, D. Wang, J. Pan, and W. Li, “Analyzing emissions and energy efficiency in mixed traffic control at unsignalized intersections,” in IEEE Forum for Innovative Sustainable Transportation Systems (FISTS), 2024.
  17. M. Villarreal, B. Poudel, and W. Li, “Can chatgpt enable its? the case of mixed traffic control via reinforcement learning,” in IEEE International Conference on Intelligent Transportation Systems (ITSC), 2023, pp. 3749–3755.
  18. W. Schwarting, J. Alonso-Mora, and D. Rus, “Planning and decision-making for autonomous vehicles,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 1, pp. 187–210, 2018.
  19. P. Wang, C.-Y. Chan, and A. de La Fortelle, “A reinforcement learning based approach for automated lane change maneuvers,” in 2018 IEEE Intelligent Vehicles Symposium (IV).   IEEE, 2018, pp. 1379–1384.
  20. Y. Shen, W. Li, and M. C. Lin, “Inverse reinforcement learning with hybrid-weight trust-region optimization and curriculum learning for autonomous maneuvering,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 7421–7428.
  21. W. Li, D. Wolinski, and M. C. Lin, “ADAPS: Autonomous driving via principled simulations,” in IEEE International Conference on Robotics and Automation (ICRA), 2019, pp. 7625–7631.
  22. D. Chen, M. R. Hajidavalloo, Z. Li, K. Chen, Y. Wang, L. Jiang, and Y. Wang, “Deep multi-agent reinforcement learning for highway on-ramp merging in mixed traffic,” IEEE Transactions on Intelligent Transportation Systems, 2023.
  23. J. Dong, S. Chen, P. Y. J. Ha, Y. Li, and S. Labi, “A drl-based multiagent cooperative control framework for cav networks: A graphic convolution q network,” arXiv preprint arXiv:2010.05437, 2020.
  24. T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” arXiv preprint arXiv:1609.02907, 2016.
  25. V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013.
  26. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.
  27. R. Lowe, Y. I. Wu, A. Tamar, J. Harb, O. Pieter Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environments,” Advances in neural information processing systems, vol. 30, 2017.
  28. S. Gu, L. Yang, Y. Du, G. Chen, F. Walter, J. Wang, Y. Yang, and A. Knoll, “A review of safe reinforcement learning: Methods, theory and applications,” arXiv preprint arXiv:2205.10330, 2022.
  29. P. A. Lopez, M. Behrisch, L. Bieker-Walz, J. Erdmann, Y.-P. Flötteröd, R. Hilbrich, L. Lücken, J. Rummel, P. Wagner, and E. Wießner, “Microscopic traffic simulation using sumo,” in IEEE International Conference on Intelligent Transportation Systems, 2018, pp. 2575–2582.
  30. M. Treiber, A. Hennecke, and D. Helbing, “Congested traffic states in empirical observations and microscopic simulations,” Physical review E, vol. 62, no. 2, p. 1805, 2000.
  31. D. Wilkie, J. Sewall, W. Li, and M. C. Lin, “Virtualized traffic at metropolitan scales,” Frontiers in Robotics and AI, vol. 2, p. 11, 2015.
  32. W. Li, D. Wolinski, and M. C. Lin, “City-scale traffic animation using statistical learning and metamodel-based optimization,” ACM Trans. Graph., vol. 36, no. 6, pp. 200:1–200:12, 2017.
  33. L. Lin, W. Li, and L. Zhu, “Data-driven graph filter based graph convolutional neural network approach for network-level multi-step traffic prediction,” Sustainability, vol. 14, no. 24, p. 16701, 2022.
  34. K. Guo, W. Jing, L. Gao, W. Liu, W. Li, and J. Pan, “Long-term microscopic traffic simulation with history-masked multi-agent imitation learning,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets