Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
164 tokens/sec
GPT-4o
10 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DataLight: Offline Data-Driven Traffic Signal Control (2303.10828v2)

Published 20 Mar 2023 in cs.LG and cs.AI

Abstract: Reinforcement learning (RL) has emerged as a promising solution for addressing traffic signal control (TSC) challenges. While most RL-based TSC systems typically employ an online approach, facilitating frequent active interaction with the environment, learning such strategies in the real world is impractical due to safety and risk concerns. To tackle these challenges, this study introduces an innovative offline data-driven approach, called DataLight. DataLight employs effective state representations and reward function by capturing vehicular speed information within the environment. It then segments roads to capture spatial information and further enhances the spatially segmented state representations with sequential modeling. The experimental results demonstrate the effectiveness of DataLight, showcasing superior performance compared to both state-of-the-art online and offline TSC methods. Additionally, DataLight exhibits robust learning capabilities concerning real-world deployment issues. The code is available at https://github.com/LiangZhang1996/DataLight.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp.  3414–3421, 2020.
  2. Decision transformer: Reinforcement learning via sequence modeling. Advances in neural information processing systems, 34:15084–15097, 2021.
  3. PRGLight: A novel traffic light control framework with pressure-based-reinforcement learning and graph neural network. In IJCAI 2021 Reinforcement Learning for Intelligent Transportation Systems (RL4ITS) Workshop, 2021.
  4. Self-organizing traffic lights: A realistic simulation. In Advances in applied self-organizing systems, pp.  45–55. Springer, 2013.
  5. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pp. 1126–1135. PMLR, 2017.
  6. A minimalist approach to offline reinforcement learning. Advances in neural information processing systems, 34:20132–20145, 2021.
  7. Off-policy deep reinforcement learning without exploration. In International conference on machine learning, pp. 2052–2062. PMLR, 2019.
  8. Eigensubspace of temporal-difference dynamics and how it improves value approximation in reinforcement learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp.  573–589. Springer, 2023.
  9. Generative adversarial imitation learning. Advances in neural information processing systems, 29, 2016.
  10. The SCOOT on-line traffic signal optimisation technique. Traffic Engineering & Control, 23(4), 1982.
  11. Offline reinforcement learning as one big sequence modeling problem. Advances in neural information processing systems, 34:1273–1286, 2021.
  12. Traffic signal timing manual. Technical report, United States. Federal Highway Administration, 2008.
  13. Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation. Advances in neural information processing systems, 29, 2016.
  14. Stabilizing off-policy Q-learning via bootstrapping error reduction. Advances in Neural Information Processing Systems, 32, 2019.
  15. Conservative Q-learning for offline reinforcement learning. Advances in Neural Information Processing Systems, 33:1179–1191, 2020.
  16. Lowrie, P. SCATS: A traffic responsive method of controlling urban traffic. Sales information brochure published by Roads & Traffic Authority, Sydney, Australia, 1990.
  17. AttendLight: Universal attention-based reinforcement learning model for traffic signal control. Advances in Neural Information Processing Systems, 33:4079–4090, 2020.
  18. Reinforcement learning: An introduction. MIT press, 2018.
  19. The green wave model of two-dimensional traffic: Transitions in the flow properties and in the geometry of the traffic jam. Physica A: Statistical Mechanics and its Applications, 231(4):515–533, 1996.
  20. Varaiya, P. Max pressure control of a network of signalized intersections. Transportation Research Part C: Emerging Technologies, 36:177–195, 11 2013. doi: 10.1016/j.trc.2013.08.014.
  21. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  22. Graph attention networks. stat, 1050:20, 2017.
  23. Dueling network architectures for deep reinforcement learning. In International conference on machine learning, pp. 1995–2003. PMLR, 2016.
  24. IntelliLight: A reinforcement learning approach for intelligent traffic light control. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.  2496–2505, 2018.
  25. PressLight: Learning max pressure control to coordinate traffic signals in arterial network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.  1290–1298, 2019a.
  26. CoLight: Learning network-level cooperation for traffic signal control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp.  1913–1922, 2019b.
  27. A survey on traffic signal control methods. arXiv preprint arXiv:1904.08117, 2019c.
  28. Behavior regularized offline reinforcement learning. arXiv preprint arXiv:1911.11361, 2019.
  29. Learning traffic signal control from demonstrations. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp.  2289–2292, 2019.
  30. Hierarchically and cooperatively learning traffic signal control. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.  669–677, 2021.
  31. MetaLight: Value-based meta-reinforcement learning for traffic signal control. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp.  1153–1160, 2020.
  32. CityFlow: A multi-agent reinforcement learning environment for large scale city traffic scenario. In The World Wide Web Conference, pp.  3620–3624, 2019.
  33. Expression might be enough: Representing pressure and demand for reinforcement learning based traffic signal control. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp.  26645–26654. PMLR, 17–23 Jul 2022.
  34. Leveraging queue length and attention mechanisms for enhanced traffic signal control optimization. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp.  141–156. Springer, 2023.
  35. Learning phase competition for traffic signal control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp.  1963–1972, 2019a.
  36. Diagnosing reinforcement learning for traffic signal control. arXiv preprint arXiv:1905.04716, 2019b.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets