Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

CARL: Congestion-Aware Reinforcement Learning for Imitation-based Perturbations in Mixed Traffic Control (2404.00796v2)

Published 31 Mar 2024 in cs.RO

Abstract: Human-driven vehicles (HVs) exhibit complex and diverse behaviors. Accurately modeling such behavior is crucial for validating Robot Vehicles (RVs) in simulation and realizing the potential of mixed traffic control. However, existing approaches like parameterized models and data-driven techniques struggle to capture the full complexity and diversity. To address this, in this work, we introduce CARL, a hybrid approach that combines imitation learning for close proximity car-following and probabilistic sampling for larger headways. We also propose two classes of RL-based RVs: a safety RV focused on maximizing safety and an efficiency RV focused on maximizing efficiency. Our experiments show that the safety RV increases Time-to-Collision above the critical 4-second threshold and reduces Deceleration Rate to Avoid a Crash by up to 80%, while the efficiency RV achieves improvements in throughput of up to 49%. These results demonstrate the effectiveness of CARL in enhancing both safety and efficiency in mixed traffic.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (65)
  1. USDOT’s automated vehicles comprehensive plan, 2024.
  2. U.S. Environmental Protection Agency’ green vehicle guide on self-driving vehicles, 2024.
  3. Multi-agent imitation learning for driving simulation. in 2018 ieee. In RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1534–1539.
  4. Imitating driver behavior with generative adversarial networks. In 2017 IEEE intelligent vehicles symposium (IV), pages 204–211. IEEE, 2017.
  5. A survey on autonomous vehicle control in the era of mixed-autonomy: From physics-based to ai-guided driving policy learning. Transportation research part C: emerging technologies, 125:103008, 2021.
  6. Mixed traffic control and coordination from pixels. In IEEE International Conference on Robotics and Automation (ICRA), 2024.
  7. Analyzing emissions and energy efficiency in mixed traffic control at unsignalized intersections. In IEEE Forum for Innovative Sustainable Transportation Systems (FISTS), 2024.
  8. Can chatgpt enable its? the case of mixed traffic control via reinforcement learning. In IEEE International Conference on Intelligent Transportation Systems (ITSC), pages 3749–3755, 2023.
  9. Learning to control and coordinate mixed traffic through robot vehicles at complex and unsignalized intersections. arXiv preprint arXiv:2301.05294, 2023.
  10. Large-scale mixed traffic control using dynamic vehicle routing and privacy-preserving crowdsourcing. IEEE Internet of Things Journal, 11(2):1981–1989, 2024.
  11. Endurl: Enhancing safety, stability, and efficiency of mixed traffic under real-world perturbations via reinforcement learning. arXiv preprint, 2024.
  12. Urban driving with conditional imitation learning. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 251–257. IEEE, 2020.
  13. A comprehensive comparison study of four classical car-following models based on the large-scale naturalistic driving experiment. Simulation Modelling Practice and Theory, 113:102383, 2021.
  14. Deploying traffic smoothing cruise controllers learned from trajectory data. In 2022 International Conference on Robotics and Automation (ICRA), pages 2884–2890. IEEE, 2022.
  15. Peter G Gipps. A behavioural car-following model for computer simulation. Transportation research part B: methodological, 15(2):105–111, 1981.
  16. Stefan Krauß. Microscopic modeling of traffic flow: Investigation of collision free vehicle dynamics. 1998.
  17. Traffic flow dynamics. Traffic Flow Dynamics: Data, Models and Simulation, Springer-Verlag Berlin Heidelberg, pages 983–1000, 2013.
  18. Limitations and improvements of the intelligent driver model (idm). SIAM Journal on Applied Dynamical Systems, 21(3):1862–1892, 2022.
  19. Microscopic traffic simulation using sumo. In 2018 21st international conference on intelligent transportation systems (ITSC), pages 2575–2582. IEEE, 2018.
  20. The intelligent driver model with stochasticity-new insights into traffic flow oscillations. Transportation research procedia, 23:174–187, 2017.
  21. Calibrating car-following models by using trajectory data: Methodological study. Transportation Research Record, 2088(1):148–156, 2008.
  22. A global optimization algorithm for trajectory data based car-following model calibration. Transportation Research Part C: Emerging Technologies, 68:311–332, 2016.
  23. MN Sharath and Nagendra R Velaga. Enhanced intelligent driver model for two-dimensional motion planning in mixed traffic. Transportation Research Part C: Emerging Technologies, 120:102780, 2020.
  24. Modeling car-following behavior on urban expressways in shanghai: A naturalistic driving study. Transportation research part C: emerging technologies, 93:425–445, 2018.
  25. Flow: Architecture and benchmarking for reinforcement learning in traffic control. arXiv preprint arXiv:1710.05465, 10, 2017.
  26. Dissipating stop-and-go waves in closed and open networks via deep reinforcement learning. In 2018 21st International Conference on Intelligent Transportation Systems (ITSC), pages 1475–1480. IEEE, 2018.
  27. The lord of the ring road: a review and evaluation of autonomous control policies for traffic in a ring road. ACM Transactions on Cyber-Physical Systems (TCPS), 6(1):1–25, 2022.
  28. Benchmarks for reinforcement learning in mixed-autonomy traffic. In Conference on robot learning, pages 399–409. PMLR, 2018.
  29. Piecewise constant policies for human-compatible congestion mitigation. In 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pages 2499–2505. IEEE, 2021.
  30. Capturing car-following behaviors by deep learning. IEEE Transactions on Intelligent Transportation Systems, 19(3):910–920, 2017.
  31. Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving. Transportation Research Part C: Emerging Technologies, 117:102662, 2020.
  32. Human-like autonomous car-following model with deep reinforcement learning. Transportation research part C: emerging technologies, 97:348–368, 2018.
  33. Bilateral deep reinforcement learning approach for better-than-human car following model. arXiv preprint arXiv:2203.04749, 2022.
  34. Modeling human driving behavior through generative adversarial imitation learning. IEEE Transactions on Intelligent Transportation Systems, 24(3):2874–2887, 2022.
  35. Car-following models: A multidisciplinary review. arXiv preprint arXiv:2304.07143, 2023.
  36. A survey on imitation learning techniques for end-to-end autonomous vehicles. IEEE Transactions on Intelligent Transportation Systems, 23(9):14128–14147, 2022.
  37. I-24 motion: An instrument for freeway traffic science. Transportation Research Part C: Emerging Technologies, 155:104311, 2023.
  38. Dissipation of stop-and-go waves via control of autonomous vehicles: Field experiments. Transportation Research Part C: Emerging Technologies, 89:205–221, 2018.
  39. Berthold KP Horn. Suppressing traffic flow instabilities. In 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013), pages 13–20. IEEE, 2013.
  40. Rajesh Rajamani. Vehicle dynamics and control. Springer Science & Business Media, 2011.
  41. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
  42. Hwasoo Yeo. Asymmetric microscopic driving behavior theory. University of California, Berkeley, 2008.
  43. Flow: A modular learning framework for mixed autonomy traffic. IEEE Transactions on Robotics, 38(2):1270–1286, 2021.
  44. Katja Vogel. A comparison of headway and time to collision as safety indicators. Accident analysis & prevention, 35(3):427–433, 2003.
  45. Dale F Cooper and N Ferguson. Traffic studies at t-junctions. 2. a conflict simulation record. Traffic Engineering & Control, 17(Analytic), 1976.
  46. Modelling fuel consumption and pollutant emissions based on real-world driving patterns: the hbefa approach. International journal of environment and pollution, 22(3):240–258, 2004.
  47. Extended time-to-collision measures for road traffic safety assessment. Accident Analysis & Prevention, 33(1):89–97, 2001.
  48. Preferred time-headway of highway drivers. In ITSC 2001. 2001 IEEE Intelligent Transportation Systems. Proceedings (Cat. No. 01TH8585), pages 826–829. IEEE, 2001.
  49. Learning to change: Choreographing mixed traffic through lateral control and hierarchical reinforcement learning. arXiv preprint arXiv:2403.14879, 2024.
  50. ADAPS: Autonomous driving via principled simulations. In IEEE International Conference on Robotics and Automation (ICRA), pages 7625–7631, 2019.
  51. Inverse reinforcement learning with hybrid-weight trust-region optimization and curriculum learning for autonomous maneuvering. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7421–7428, 2022.
  52. Vehicle trajectory prediction using LSTMs with spatial-temporal attention mechanisms. IEEE Intelligent Transportation Systems Magazine, 14(2):197–208, 2022.
  53. Gradient-free adversarial training against image corruption for learning-based steering. In Advances in Neural Information Processing Systems (NeurIPS), pages 26250–26263, 2021.
  54. Autojoin: Efficient adversarial training for robust maneuvering via denoising autoencoder and joint learning. 2023.
  55. Virtualized traffic at metropolitan scales. Frontiers in Robotics and AI, 2:11, 2015.
  56. City-scale traffic animation using statistical learning and metamodel-based optimization. ACM Trans. Graph., 36(6):200:1–200:12, 2017.
  57. Citywide estimation of traffic dynamics via sparse GPS traces. IEEE Intelligent Transportation Systems Magazine, 9(3):100–113, 2017.
  58. Estimating urban traffic states using iterative refinement and wardrop equilibria. IET Intelligent Transport Systems, 12(8):875–883, 2018.
  59. Efficient data collection and accurate travel time estimation in a connected vehicle environment via real-time compressive sensing. Journal of Big Data Analytics in Transportation, 1(2):95–107, 2019.
  60. Predicting station-level bike-sharing demands using graph convolutional neural network. In Transportation Research Board 98th Annual Meeting (TRB), 2019.
  61. A survey on visual traffic simulation: Models, evaluations, and applications in autonomous driving. Computer Graphics Forum, 39(1):287–308, 2020.
  62. Black-box adversarial attacks on network-wide multi-step traffic state prediction models. In 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pages 3652–3658. IEEE, 2021.
  63. Data-driven graph filter based graph convolutional neural network approach for network-level multi-step traffic prediction. Sustainability, 14(24):16701, 2022.
  64. Lasil: Learner-aware supervised imitation learning for long-term microscopic traffic simulation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
  65. Learning to control dc motor for micromobility in real time with reinforcement learning. In IEEE International Conference on Intelligent Transportation Systems (ITSC), pages 1248–1254, 2022.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets