Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
166 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Antifragile Perimeter Control: Anticipating and Gaining from Disruptions with Reinforcement Learning (2402.12665v1)

Published 20 Feb 2024 in eess.SY and cs.SY

Abstract: The optimal operation of transportation networks is often susceptible to unexpected disruptions, such as traffic incidents and social events. Many established control strategies rely on mathematical models that struggle to cope with real-world uncertainties, leading to a significant decline in effectiveness when faced with substantial disruptions. While previous research works have dedicated efforts to improving the robustness or resilience of transportation systems against disruptions, this paper applies the cutting-edge concept of antifragility to better design a traffic control strategy for urban road networks. Antifragility sets itself apart from robustness and resilience as it represents a system's ability to not only withstand stressors, shocks, and volatility but also thrive and enhance performance in the presence of such adversarial events. Hence, modern transportation systems call for solutions that are antifragile. In this work, we propose a model-free deep Reinforcement Learning (RL) scheme to control a two-region urban traffic perimeter network. The system exploits the learning capability of RL under disruptions to achieve antifragility. By monitoring the change rate and curvature of the traffic state with the RL framework, the proposed algorithm anticipates imminent disruptions. An additional term is also integrated into the RL algorithm as redundancy to improve the performance under disruption scenarios. When compared to a state-of-the-art model predictive control approach and a state-of-the-art RL algorithm, our proposed method demonstrates two antifragility-related properties: (a) gradual performance improvement under disruptions of constant magnitude; and (b) increasingly superior performance under growing disruptions.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (80)
  1. A functional form with a physical meaning for the macroscopic fundamental diagram. Transportation Research Part B: Methodological 137, 119–132. doi:10.1016/j.trb.2018.10.013.
  2. Disentangling the city traffic rhythms: A longitudinal analysis of MFD patterns over a year. Transportation Research Part C: Emerging Technologies 126, 103065. doi:10.1016/j.trc.2021.103065.
  3. A case study of Zurich’s two-layered perimeter control , 8 p.doi:10.3929/ETHZ-B-000206987.
  4. Approximative Network Partitioning for MFDs from Stationary Sensor Data. Transportation Research Record 2673, 94–103. doi:10.1177/0361198119843264.
  5. CasADi – A software framework for nonlinear optimization and optimal control. Mathematical Programming Computation 11, 1–36. doi:10.1007/s12532-018-0139-4.
  6. Traffic signal optimization through discrete and continuous reinforcement learning with robustness analysis in downtown Tehran. Advanced Engineering Informatics 38, 639–655. doi:10.1016/j.aei.2018.08.002.
  7. The Concept of Antifragility and its Implications for the Practice of Risk Analysis. Risk Analysis 35, 476–483. doi:10.1111/risa.12279.
  8. Antifragile control systems: The case of an oscillator-based network model of urban road traffic dynamics. arXiv preprint arXiv:2210.10460 .
  9. Antifragile Control Systems: The Case of an Anti-Symmetric Network Model of the Tumor-Immune-Drug Interactions. Symmetry 14, 2034. doi:10.3390/sym14102034.
  10. Antifragile Control Systems: The case of mobile robot trajectory tracking in the presence of uncertainty.
  11. Model Predictive Control Design: New Trends and Tools, in: Proceedings of the 45th IEEE Conference on Decision and Control, pp. 6678–6683. doi:10.1109/CDC.2006.377490.
  12. Antifragility as a design criterion for modelling dynamic systems. Systems Research and Behavioral Science 37, 23–37. doi:10.1002/sres.2574.
  13. The relationship between congestion levels and accidents URL: https://trid.trb.org/view/680981. number: MD-03-SP 208B46,.
  14. Data efficient reinforcement learning and adaptive optimal perimeter control of network traffic dynamics. Transportation Research Part C: Emerging Technologies 142, 103759. doi:10.1016/j.trc.2022.103759.
  15. Toward A Thousand Lights: Decentralized Deep Reinforcement Learning for Large-Scale Traffic Signal Control. Proceedings of the AAAI Conference on Artificial Intelligence 34, 3414–3421. doi:10.1609/aaai.v34i04.5744.
  16. Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control. IEEE Transactions on Intelligent Transportation Systems 21, 1086–1095. doi:10.1109/TITS.2019.2901791.
  17. Urban gridlock: Macroscopic modeling and mitigation approaches. Transportation Research Part B: Methodological 41, 49–62. doi:10.1016/j.trb.2006.03.001.
  18. An analytical approximation for the macroscopic fundamental diagram of urban traffic. Transportation Research Part B: Methodological 42, 771–781. doi:10.1016/j.trb.2008.06.008.
  19. MPC: Current practice and challenges. Control Engineering Practice 20, 328–342. doi:10.1016/j.conengprac.2011.12.004.
  20. Emergence of Antifragility by Optimum Postdisruption Restoration Planning of Infrastructure Networks. Journal of Infrastructure Systems 23, 04017024. doi:10.1061/(ASCE)IS.1943-555X.0000380.
  21. Federal Statistical Office, 2020. Mobilität und Verkehr: Panorama (in German/French only). 16704292, Neuchâtel. URL: https://dam-api.bfs.admin.ch/hub/api/dam/assets/16704292/master.
  22. Towards the development of intelligent transportation systems, in: ITSC 2001. 2001 IEEE Intelligent Transportation Systems. Proceedings (Cat. No.01TH8585), pp. 1206–1211. doi:10.1109/ITSC.2001.948835.
  23. Perimeter Control and Route Guidance of Multi-Region MFD Systems With Boundary Queues Using Colored Petri Nets. IEEE Transactions on Intelligent Transportation Systems 23, 12977–12999. doi:10.1109/TITS.2021.3119017.
  24. Resilience in Intelligent Transportation Systems (ITS). Transportation Research Part C: Emerging Technologies 100, 318–329. doi:10.1016/j.trc.2019.01.014.
  25. Clockwise hysteresis loops in the Macroscopic Fundamental Diagram: An effect of network instability. Transportation Research Part B: Methodological 45, 643–655. doi:10.1016/j.trb.2010.11.006.
  26. Dynamic optimal congestion pricing in multi-region urban networks by application of a Multi-Layer-Neural network. Transportation Research Part C: Emerging Technologies 134, 103485. doi:10.1016/j.trc.2021.103485.
  27. Existence of urban-scale macroscopic fundamental diagrams: Some experimental findings. Transportation Research Part B: Methodological 42, 759–770. doi:10.1016/j.trb.2008.02.002.
  28. Optimal Perimeter Control for Two Urban Regions With Macroscopic Fundamental Diagrams: A Model Predictive Approach. IEEE Transactions on Intelligent Transportation Systems 14, 348–359. doi:10.1109/TITS.2012.2216877.
  29. Applications of Deep Learning in Intelligent Transportation Systems. Journal of Big Data Analytics in Transportation 2, 115–145. doi:10.1007/s42421-020-00020-1.
  30. Sustainable, safe, smart—three key elements of Singapore’s evolving transport policies. Transport Policy 27, 20–31. doi:10.1016/j.tranpol.2012.11.017.
  31. Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey. IEEE Transactions on Intelligent Transportation Systems 23, 11–32. doi:10.1109/TITS.2020.3008612.
  32. The impact of incidents on macroscopic fundamental diagrams. Proceedings of the Institution of Civil Engineers: Transport 168, 396–405. doi:10.1680/tran.13.00026.
  33. Antifragility analysis and measurement framework for systems of systems. International Journal of Disaster Risk Science 4, 159–168. doi:10.1007/s13753-013-0017-7.
  34. The impact of flexibility and redundancy on improving supply chain resilience to disruptions. International Journal of Production Research 60, 1992–2020. doi:10.1080/00207543.2021.1883759.
  35. Exploiting the fundamental diagram of urban networks for feedback-based gating. Transportation Research Part B: Methodological 46, 1393–1403. doi:10.1016/j.trb.2012.06.008.
  36. Antifragility Predicts the Robustness and Evolvability of Biological Networks through Multi-Class Classification with a Convolutional Neural Network. Entropy 22, 986. doi:10.3390/e22090986.
  37. Routing Strategies Based on Macroscopic Fundamental Diagram. Transportation Research Record 2315, 1–10. doi:10.3141/2315-01.
  38. How Well Do Reinforcement Learning Approaches Cope With Disruptions? The Case of Traffic Signal Control. IEEE Access 11, 36504–36515. doi:10.1109/ACCESS.2023.3266644.
  39. Enhancing model-based feedback perimeter control with data-driven online adaptive optimization. Transportation Research Part B: Methodological 96, 26–45. doi:10.1016/j.trb.2016.10.011.
  40. Deep Reinforcement Learning: An Overview. ArXiv:1701.07274.
  41. Continuous control with deep reinforcement learning. doi:10.48550/arXiv.1509.02971.
  42. Rapid development of modular and sustainable nonlinear model predictive control solutions. Control Engineering Practice 60, 51–62. doi:10.1016/j.conengprac.2016.12.009.
  43. Heterogeneous Innovation and the Antifragile Economy .
  44. Using GPS Data to Gain Insight into Public Transport Travel Time Variability. Journal of Transportation Engineering 136, 623–631. doi:10.1061/(ASCE)TE.1943-5436.0000126.
  45. Playing Atari with Deep Reinforcement Learning .
  46. Resilience, robustness, and antifragility: Towards an appreciation of distinct organizational responses to adversity. International Journal of Management Reviews 24, 181–187. doi:10.1111/ijmr.12289.
  47. Regularizing Action Policies for Smooth Control with Reinforcement Learning, in: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 1810–1816. doi:10.1109/ICRA48506.2021.9561138.
  48. Deep learning methods in transportation domain: a review. IET Intelligent Transport Systems 12, 998–1004. doi:10.1049/iet-its.2018.0064.
  49. Cordon control with spatially-varying metering rates: A Reinforcement Learning approach. Transportation Research Part C: Emerging Technologies 98, 358–369. doi:10.1016/j.trc.2018.12.007.
  50. Investigating the interaction of factors for implementing additive manufacturing to build an antifragile supply chain: TISM-MICMAC approach. Operations Management Research 15, 567–588. doi:10.1007/s12063-022-00259-7.
  51. A survey of industrial model predictive control technology. Control engineering practice 11, 733–764.
  52. Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures, in: 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp. 3559–3566. doi:10.1109/ITSC.2019.8917451.
  53. Estimating network travel time reliability with network partitioning. Transportation Research Part C: Emerging Technologies 112, 46–61. doi:10.1016/j.trc.2020.01.013.
  54. Deterministic Policy Gradient Algorithms .
  55. Hierarchical control for stochastic network traffic with reinforcement learning. Transportation Research Part B: Methodological 167, 196–216. doi:10.1016/j.trb.2022.12.001.
  56. Reinforcement Learning, second edition: An Introduction. MIT Press.
  57. Antifragile: Things That Gain from Disorder. volume 3. Random House.
  58. ’Antifragility’ as a mathematical idea. Nature 494, 430–430. doi:10.1038/494430e.
  59. Mathematical definition, mapping, and detection of (anti)fragility. Quantitative Finance 13, 1677–1689. doi:10.1080/14697688.2013.800219.
  60. Working with Convex Responses: Antifragility from Finance to Oncology. Entropy 25, 343. doi:10.3390/e25020343.
  61. Resilience in Transportation Systems. Procedia - Social and Behavioral Sciences 48, 3441–3450. doi:10.1016/j.sbspro.2012.06.1308.
  62. Robust Deep Reinforcement Learning for Traffic Signal Control. Journal of Big Data Analytics in Transportation 2, 263–274. doi:10.1007/s42421-020-00029-6.
  63. A graph-based model to measure structural redundancy for supply chain resilience. International Journal of Production Research 57, 6385–6404. doi:10.1080/00207543.2019.1566666.
  64. Evaluating resilience in urban transportation systems for sustainability: A systems-based Bayesian network model. Transportation Research Part C: Emerging Technologies 121, 102840. doi:10.1016/j.trc.2020.102840.
  65. Short-term traffic forecasting: Where we are and where we’re going. Transportation Research Part C: Emerging Technologies 43, 3–19. doi:10.1016/j.trc.2014.01.005.
  66. An empirical analysis of macroscopic fundamental diagrams for sendai road networks. Interdisciplinary Information Sciences 21, 49–61. doi:10.4036/iis.2015.49.
  67. CTRL: Cooperative Traffic Tolling via Reinforcement Learning, in: Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Association for Computing Machinery, New York, NY, USA. pp. 3545–3554. doi:10.1145/3511808.3557112.
  68. CoLight: Learning Network-level Cooperation for Traffic Signal Control, in: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, ACM, Beijing China. pp. 1913–1922. doi:10.1145/3357384.3357902.
  69. Multi-Agent Reinforcement Learning for Traffic Signal Control: Algorithms and Robustness Analysis, in: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–7. doi:10.1109/ITSC45102.2020.9294623.
  70. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical Programming 106, 25–57. doi:10.1007/s10107-004-0559-y.
  71. Multi-scale Perimeter Control Approach in a Connected-Vehicle Environment. Transportation Research Procedia 23, 101–120. doi:10.1016/j.trpro.2017.05.007.
  72. Equilibrium Analysis and Route Guidance in Large-scale Networks with MFD Dynamics. Transportation Research Procedia 9, 185–204. doi:10.1016/j.trpro.2015.07.011.
  73. Breaking the Deadly Triad with a Target Network, in: Proceedings of the 38th International Conference on Machine Learning, PMLR. pp. 12621–12631.
  74. Modeling and optimization of multimodal urban networks with limited parking and dynamic pricing. Transportation Research Part B: Methodological 83, 36–58. doi:10.1016/j.trb.2015.10.008.
  75. A dynamic cordon pricing scheme combining the Macroscopic Fundamental Diagram and an agent-based traffic model. Transportation Research Part A: Policy and Practice 46, 1291–1303. doi:10.1016/j.tra.2012.05.006.
  76. Model-free perimeter metering control for two-region urban networks using deep reinforcement learning. Transportation Research Part C: Emerging Technologies 124, 102949. doi:10.1016/j.trc.2020.102949.
  77. Scalable multi-region perimeter metering control for urban networks: A multi-agent deep reinforcement learning approach. Transportation Research Part C: Emerging Technologies 148, 104033. doi:10.1016/j.trc.2023.104033.
  78. Resilience of Transportation Systems: Concepts and Comprehensive Review. IEEE Transactions on Intelligent Transportation Systems 20, 4262–4276. doi:10.1109/TITS.2018.2883766.
  79. Big Data Analytics in Intelligent Transportation Systems: A Survey. IEEE Transactions on Intelligent Transportation Systems 20, 383–398. doi:10.1109/TITS.2018.2815678.
  80. A deep reinforcement learning framework for delay management with passenger re-routing, in: 9th International Conference on Railway Operations Modelling and Analysis.
Citations (1)

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com