Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedback (2306.07553v1)

Published 13 Jun 2023 in cs.CV

Abstract: Traffic Signal Control (TSC) aims to reduce the average travel time of vehicles in a road network, which in turn enhances fuel utilization efficiency, air quality, and road safety, benefiting society as a whole. Due to the complexity of long-horizon control and coordination, most prior TSC methods leverage deep reinforcement learning (RL) to search for a control policy and have witnessed great success. However, TSC still faces two significant challenges. 1) The travel time of a vehicle is delayed feedback on the effectiveness of TSC policy at each traffic intersection since it is obtained after the vehicle has left the road network. Although several heuristic reward functions have been proposed as substitutes for travel time, they are usually biased and not leading the policy to improve in the correct direction. 2) The traffic condition of each intersection is influenced by the non-local intersections since vehicles traverse multiple intersections over time. Therefore, the TSC agent is required to leverage both the local observation and the non-local traffic conditions to predict the long-horizontal traffic conditions of each intersection comprehensively. To address these challenges, we propose DenseLight, a novel RL-based TSC method that employs an unbiased reward function to provide dense feedback on policy effectiveness and a non-local enhanced TSC agent to better predict future traffic conditions for more precise traffic control. Extensive experiments and ablation studies demonstrate that DenseLight can consistently outperform advanced baselines on various road networks with diverse traffic flows. The code is available at https://github.com/junfanlin/DenseLight.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control. In AAAI, volume 34, pages 3414–3421, 2020.
  2. Bidirectional spatial-temporal adaptive transformer for urban traffic flow forecasting. TNNLS, 2022.
  3. Multi-agent deep reinforcement learning for large-scale traffic signal control. TITS, 21(3):1086–1095, 2019.
  4. Self-organizing traffic lights: A realistic simulation. In Advances in applied self-organizing systems, pages 45–55. Springer, 2013.
  5. Designing reinforcement learning agents for traffic signal control with the right goals: a time-loss based approach. In ITSC, pages 1412–1418. IEEE, 2021.
  6. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  7. Addressing function approximation error in actor-critic methods. In ICML, pages 1587–1596. PMLR, 2018.
  8. Urban traffic light control via active multi-agent communication and supply-demand modeling. TKDE, pages 1–1, 2021.
  9. Traffic signal control with adaptive online-learning scheme using multiple-model neural networks. TNNLS, 2022.
  10. Network-scale traffic signal control via multiagent reinforcement learning with deep spatiotemporal attentive network. IEEE trans cybern, 2021.
  11. Scoot-a traffic responsive method of coordinating signals. Publication of: Transport and Road Research Laboratory, LR 1014 Monograph, Jan 1981.
  12. Traffic signal timing manual. United States. Federal Highway Administration, 2008.
  13. Deep learning. nature, 521(7553):436–444, 2015.
  14. An integrated reinforcement learning and centralized programming approach for online taxi dispatching. TNNLS, 2021.
  15. Maxband: A versatile program for setting signals on arteries and triangular networks. Transportation Research Record 795, pages 40–46, 1981.
  16. Hierarchically learned view-invariant representations for cross-view action recognition. IEEE Transactions on Circuits and Systems for Video Technology, 29(8):2416–2430, 2018.
  17. Global temporal representation based cnns for infrared action recognition. IEEE Signal Processing Letters, 25(6):848–852, 2018.
  18. Deep image-to-video adaptation and fusion networks for action recognition. IEEE Transactions on Image Processing, 29:3168–3182, 2019.
  19. Dynamic spatial-temporal representation learning for traffic flow prediction. TITS, 22(11):7169–7183, 2020.
  20. Semantics-aware adaptive knowledge distillation for sensor-to-vision action recognition. IEEE Transactions on Image Processing, 30:5573–5588, 2021.
  21. Tcgl: Temporal contrastive graph for self-supervised video representation learning. IEEE Transactions on Image Processing, 31:1978–1993, 2022.
  22. Causal reasoning meets visual representation learning: A prospective study. Machine Intelligence Research, pages 1–27, 2022.
  23. Cross-modal causal relational reasoning for event-level visual question answering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
  24. Probabilistic regularized extreme learning for robust modeling of traffic flow forecasting. TNNLS, 2020.
  25. Scats-application and field comparison with a transyt optimised fixed time system. In International Conference on Road Traffic Signalling, number 207, 1982.
  26. A real-time traffic signal control system: architecture, algorithms, and analysis. Transportation Research Part C: Emerging Technologies, 9(6):415–432, 2001.
  27. Human-level control through deep reinforcement learning. nature, 518(7540):529–533, 2015.
  28. Attendlight: Universal attention-based reinforcement learning model for traffic signal control. NeurIPS, 33:4079–4090, 2020.
  29. Curiosity-driven exploration by self-supervised prediction. In ICML, pages 2778–2787. PMLR, 2017.
  30. LA Prashanth and Shalabh Bhatnagar. Reinforcement learning with function approximation for traffic signal control. TITS, 12(2):412–421, 2010.
  31. QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning. In ICML, volume 80, pages 4295–4304. PMLR, 2018.
  32. Traffic engineering. Pearson/Prentice Hall, 2004.
  33. High-dimensional continuous control using generalized advantage estimation. In ICLR, 2016.
  34. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
  35. QTRAN: Learning to factorize with transformation for cooperative multi-agent reinforcement learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, ICML, volume 97 of PMLR, pages 5887–5896. PMLR, 09–15 Jun 2019.
  36. Policy gradient methods for reinforcement learning with function approximation. NeurIPS, 12, 1999.
  37. Cooperative deep reinforcement learning for large-scale traffic grid signal control. IEEE trans cybern, 50(6):2687–2700, 2019.
  38. Brian D Taylor. Rethinking traffic congestion. Access Magazine, 1(21):8–16, 2002.
  39. Elise Van der Pol and Frans A Oliehoek. Coordinated deep reinforcement learners for traffic light control. NeurIPS, 2016.
  40. Deep reinforcement learning with double q-learning. In AAAI, volume 30, 2016.
  41. Pravin Varaiya. Max pressure control of a network of signalized intersections. Transportation Research Part C: Emerging Technologies, 36:177–195, 2013.
  42. Attention is all you need. NeurIPS, 30, 2017.
  43. Large-scale traffic signal control using a novel multiagent reinforcement learning. IEEE transactions on cybernetics, 51(1):174–187, 2020.
  44. Synchronous spatiotemporal graph transformer: A new framework for traffic data prediction. TNNLS, 2022.
  45. Urban regional function guided traffic flow prediction. Information Sciences, 634:308–320, 2023.
  46. Q-learning. Machine learning, 8(3):279–292, 1992.
  47. Intellilight: A reinforcement learning approach for intelligent traffic light control. In CKDDM, pages 2496–2505, 2018.
  48. Presslight: Learning max pressure control to coordinate traffic signals in arterial network. In CKDDM, pages 1290–1298, 2019.
  49. Colight: Learning network-level cooperation for traffic signal control. In CIKM, pages 1913–1922, 2019.
  50. Tianshou: A highly modularized deep reinforcement learning library. arXiv preprint arXiv:2107.14171, 2021.
  51. Efficient pressure: Improving efficiency for signalized intersections. arXiv preprint arXiv:2112.02336, 2021.
  52. Hierarchically and cooperatively learning traffic signal control. In AAAI, volume 35, pages 669–677, 2021.
  53. Optimized structure of the traffic flow forecasting model with a deep learning approach. TNNLS, 28(10):2371–2381, 2016.
  54. Metalight: Value-based meta-reinforcement learning for traffic signal control. In AAAI, volume 34, pages 1153–1160, 2020.
  55. Zheng Zeng. Graphlight: Graph-based reinforcement learning for traffic signal control. In ICCCS, pages 645–650. IEEE, 2021.
  56. Air pollution and health risks due to vehicle traffic. Science of the total Environment, 450:307–316, 2013.
  57. Cityflow: A multi-agent reinforcement learning environment for large scale city traffic scenario. In W3C, pages 3620–3624, 2019.
  58. Expression is enough: Improving traffic signal control with advanced traffic state representation. arXiv preprint arXiv:2112.10107, 2021.
  59. Learning phase competition for traffic signal control. In CIKM, pages 1963–1972, 2019.
  60. Hybrid-order representation learning for electricity theft detection. IEEE Transactions on Industrial Informatics, 19(2):1248–1259, 2022.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now