Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

MTLight: Efficient Multi-Task Reinforcement Learning for Traffic Signal Control (2404.00886v1)

Published 1 Apr 2024 in cs.AI

Abstract: Traffic signal control has a great impact on alleviating traffic congestion in modern cities. Deep reinforcement learning (RL) has been widely used for this task in recent years, demonstrating promising performance but also facing many challenges such as limited performances and sample inefficiency. To handle these challenges, MTLight is proposed to enhance the agent observation with a latent state, which is learned from numerous traffic indicators. Meanwhile, multiple auxiliary and supervisory tasks are constructed to learn the latent state, and two types of embedding latent features, the task-specific feature and task-shared feature, are used to make the latent state more abundant. Extensive experiments conducted on CityFlow demonstrate that MTLight has leading convergence speed and asymptotic performance. We further simulate under peak-hour pattern in all scenarios with increasing control difficulty and the results indicate that MTLight is highly adaptable.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. Traffic light control in non-stationary environments based on multi agent q-learning. In ITSC. IEEE, 2011.
  2. Holonic multi-agent system for traffic signals control. Engineering Applications of Artificial Intelligence, 2013.
  3. Reinforcement learning-based multi-agent system for network traffic signal control. IET Intelligent Transport Systems, 2010.
  4. A geometric perspective on optimal representations for reinforcement learning. Advances in neural information processing systems, 32, 2019.
  5. Rich Caruana. Multitask learning. Machine learning, 1997.
  6. Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control. In AAAI, 2020.
  7. Stephen Chiu. Adaptive traffic signal control using fuzzy logic. In Proceedings of the Intelligent Vehicles92 Symposium. IEEE, 1992.
  8. Self-organizing traffic control via fuzzy logic. In IEEE Conference on Decision and Control. IEEE, 1993.
  9. Multi-agent deep reinforcement learning for large-scale traffic signal control. ITS, 2019.
  10. Self-organizing traffic lights: A realistic simulation. In Advances in applied self-organizing systems. Springer, 2013.
  11. Distributed w-learning: Multi-policy optimization in self-organizing systems. In self-adaptive and self-organizing systems. IEEE, 2009.
  12. Multiagent reinforcement learning for integrated network of adaptive traffic signal controllers (marlin-atsc): methodology and large-scale application on downtown toronto. IEEE TITS, 2013.
  13. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML. PMLR, 2017.
  14. Bayesian reinforcement learning: A survey. Foundations and Trends® in Machine Learning, 2015.
  15. Exploiting interpretable patterns for flow prediction in dockless bike sharing systems. IEEE Transactions on Knowledge and Data Engineering, 2020.
  16. Urban traffic light control via active multi-agent communication and supply-demand modeling. IEEE Transactions on Knowledge and Data Engineering, 2021.
  17. Spatio-temporal capsule-based reinforcement learning for mobility-on-demand coordination. IEEE Transactions on Knowledge and Data Engineering, 2020.
  18. Modellight: Model-based meta-reinforcement learning for traffic signal control. arXiv preprint arXiv:2111.08067, 2021.
  19. Scoot-a traffic responsive method of coordinating signals. Technical report, 1981.
  20. Reinforcement learning with unsupervised auxiliary tasks. arXiv preprint arXiv:1611.05397, 2016.
  21. Dynamic lane traffic signal control with group attention and multi-timescale reinforcement learning. IJCAI, 2021.
  22. Learning to delay in ride-sourcing systems: a multi-agent deep reinforcement learning framework. IEEE Transactions on Knowledge and Data Engineering, 2020.
  23. Traffic signal timing manual. Technical report, United States. Federal Highway Administration, 2008.
  24. Maximum pressure controller for stabilizing queues in signalized arterial networks. Transportation Research Record, 2014.
  25. Multiagent reinforcement learning for urban traffic control using coordination graphs. In ECML-PKDD. Springer, 2008.
  26. Adaptive auxiliary task weighting for reinforcement learning. Advances in neural information processing systems, 2019.
  27. Michael L Littman. Markov games as a framework for multi-agent reinforcement learning. In Machine learning proceedings. Elsevier, 1994.
  28. Urban flow pattern mining based on multi-source heterogeneous data fusion and knowledge graph embedding. IEEE Transactions on Knowledge and Data Engineering, 2021.
  29. PR Lowrie. Scats, sydney co-ordinated adaptive traffic system: A traffic responsive method of controlling urban traffic. 1990.
  30. On the effect of auxiliary tasks on representation dynamics. In International Conference on Artificial Intelligence and Statistics. PMLR, 2021.
  31. An experimental review of reinforcement learning algorithms for adaptive traffic signal control. In Autonomic road transport support systems. Springer, 2016.
  32. Human-level control through deep reinforcement learning. Nature, 2015.
  33. Generalization in multitask deep neural classifiers: a statistical physics approach. Advances in Neural Information Processing Systems, 2019.
  34. Traffic signal control based on reinforcement learning with graph convolutional neural nets. In ITSC. IEEE, 2018.
  35. Zero-shot task generalization with multi-task deep reinforcement learning. In ICML. PMLR, 2017.
  36. Attendlight: Universal attention-based reinforcement learning model for traffic signal control. arXiv preprint arXiv:2010.05772, 2020.
  37. Spatio-temporal meta learning for urban traffic prediction. IEEE Transactions on Knowledge and Data Engineering, 2020.
  38. Time critic policy gradient methods for traffic signal control in complex and congested scenarios. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019.
  39. Traffic engineering. Pearson/Prentice Hall, 2004.
  40. Sebastian Ruder. An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098, 2017.
  41. Multi-agent generative adversarial imitation learning. Advances in neural information processing systems, 2018.
  42. Scat: System control analysis and training simulator. In Human Detection and Diagnosis of System Failures. Springer, 1981.
  43. Combinatorial optimization meets reinforcement learning: Effective taxi order dispatching at large-scale. IEEE Transactions on Knowledge and Data Engineering, 2021.
  44. Asynchronous deep reinforcement learning for the mobile robot navigation with supervised auxiliary tasks. In International Conference on Robotics and Automation Engineering (ICRAE), pp.  68–72. IEEE, 2017.
  45. Elise Van der Pol and Frans A Oliehoek. Coordinated deep reinforcement learners for traffic light control. NeurIPS, 2016.
  46. Pravin Varaiya. The max-pressure controller for arbitrary networks of signalized intersections. In Advances in Dynamic Network Modeling in Complex Transportation Systems. Springer, 2013.
  47. Deep learning for spatio-temporal data mining: A survey. IEEE transactions on knowledge and data engineering, 2020.
  48. Fo Vo Webster. Traffic signal settings. Technical report, 1958.
  49. FV Webster. Traffic signals. Road research technical paper, 1966.
  50. Intellilight: A reinforcement learning approach for intelligent traffic light control. In SIGKDD, 2018.
  51. Presslight: Learning max pressure control to coordinate traffic signals in arterial network. In SIGKDD, 2019a.
  52. Colight: Learning network-level cooperation for traffic signal control. In CIKM, 2019b.
  53. Learning traffic signal control from demonstrations. In CIKM, 2019.
  54. Hierarchically and cooperatively learning traffic signal control. In AAAI, 2021.
  55. Macar: Urban traffic light control via active multi-agent communication and action rectification. In IJCAI, 2020.
  56. Metalight: Value-based meta-reinforcement learning for traffic signal control. In AAAI, 2020.
  57. Periodic weather-aware lstm with event mechanism for parking behavior prediction. IEEE Transactions on Knowledge and Data Engineering, 2021.
  58. Cityflow: A multi-agent reinforcement learning environment for large scale city traffic scenario. In WWW, 2019.
  59. Planlight: Learning to optimize traffic signal control with planning and iterative policy improvement. IEEE Access, 2020a.
  60. Generalight: Improving environment generalization of traffic signal control via meta reinforcement learning. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 2020b.
  61. Yu Zhang and Qiang Yang. A survey on multi-task learning. IEEE Transactions on Knowledge and Data Engineering, 2021.
  62. Learning hierarchical features from deep generative models. In ICML. PMLR, 2017.
  63. Learning phase competition for traffic signal control. In CIKM, 2019a.
  64. Diagnosing reinforcement learning for traffic signal control. arXiv, 2019b.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now