Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SLS-BRD: A system-level approach to seeking generalised feedback Nash equilibria (2404.03809v3)

Published 4 Apr 2024 in math.OC, cs.SY, and eess.SY

Abstract: This work proposes a policy learning algorithm for seeking generalised feedback Nash equilibria (GFNE) in $N_P$-player noncooperative dynamic games. We consider linear-quadratic games with stochastic dynamics and design a best-response dynamics in which players update and broadcast a parametrisation of their state-feedback policies. Our approach leverages the System Level Synthesis (SLS) framework to formulate each player's update rule as the solution to a robust optimisation problem. Under certain conditions, rates of convergence to a feedback Nash equilibrium can be established. The algorithm is showcased in exemplary problems ranging from the decentralised control of unstable systems to competition in oligopolistic markets.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. T. Başar and G. J. Olsder, Dynamic noncooperative game theory. SIAM, 1998.
  2. F. Facchinei and C. Kanzow, “Generalized Nash equilibrium problems,” Annals of Operations Res., vol. 175, no. 1, pp. 177–211, 2010.
  3. C. Daskalakis, P. W. Goldberg, and C. H. Papadimitriou, “The complexity of computing a Nash equilibrium,” Communications of the ACM, vol. 52, no. 2, pp. 89–97, 2009.
  4. Cambridge University Press, 2011.
  5. J. S. Shamma and G. Arslan, “Dynamic fictitious play, dynamic gradient play, and distributed convergence to Nash equilibria,” IEEE Transactions on Automatic Control, vol. 50, no. 3, pp. 312–327, 2005.
  6. J. Hofbauer and S. Sorin, “Best response dynamics for continuous zero-sum games,” Discrete and Continuous Dynamical Systems, vol. 6, no. 1, pp. 215–224, 2006.
  7. J. R. Marden, G. Arslan, and J. S. Shamma, “Joint strategy fictitious play with inertia for potential games,” IEEE Transactions on Automatic Control, vol. 54, no. 2, pp. 208–220, 2009.
  8. A. Cortés and S. Martínez, “Self-triggered best-response dynamics for continuous games,” IEEE Transactions on Automatic Control, vol. 60, no. 4, pp. 1115–1120, 2014.
  9. S. Grammatico, “Dynamic control of agents playing aggregative games with coupling constraints,” IEEE Transactions on Automatic Control, vol. 62, no. 9, pp. 4537–4548, 2017.
  10. B. Swenson, R. Murray, and S. Kar, “On best-response dynamics in potential games,” SIAM Journal on Control and Optimization, vol. 56, no. 4, pp. 2734–2767, 2018.
  11. A. Jafari, A. Greenwald, D. Gondek, and G. Ercal, “On no-regret learning, fictitious play, and nash equilibrium,” in ICML, vol. 1, pp. 226–233, 2001.
  12. Cambridge university press, 2006.
  13. G. J. Gordon, A. Greenwald, and C. Marks, “No-regret learning in convex games,” in Proceedings of the 25th international conference on Machine learning, pp. 360–367, 2008.
  14. J. Hartline, V. Syrgkanis, and E. Tardos, “No-regret learning in bayesian games,” Advances in Neural Information Processing Systems, vol. 28, 2015.
  15. A. Celli, A. Marchesi, G. Farina, and N. Gatti, “No-regret learning dynamics for extensive-form correlated equilibrium,” Advances in Neural Information Processing Systems, vol. 33, pp. 7722–7732, 2020.
  16. G. Belgioioso, D. Liao-McPherson, M. H. de Badyn, S. Bolognani, R. S. Smith, J. Lygeros, and F. Dörfler, “Online feedback equilibrium seeking,” arXiv preprint arXiv:2210.12088, 2022.
  17. G. Scutari, S. Barbarossa, and D. Palomar, “Potential games: A framework for vector power control problems with coupled constraints,” in 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, vol. 4, pp. IV–IV, 2006.
  18. W. Saad, Z. Han, H. V. Poor, and T. Basar, “Game-theoretic methods for the smart grid: An overview of microgrid systems, demand-side management, and smart grid communications,” IEEE signal processing magazine, vol. 29, no. 5, pp. 86–105, 2012.
  19. S. Maghsudi and S. Stańczak, “Channel selection for network-assisted d2d communication via no-regret bandit learning with calibrated forecasting,” IEEE Transactions on Wireless Communications, vol. 14, no. 3, pp. 1309–1322, 2015.
  20. J. F. Fisac, E. Bronstein, E. Stefansson, D. Sadigh, S. S. Sastry, and A. D. Dragan, “Hierarchical game-theoretic planning for autonomous vehicles,” in 2019 International conference on robotics and automation (ICRA), pp. 9590–9596, IEEE, 2019.
  21. R. Spica, E. Cristofalo, Z. Wang, E. Montijano, and M. Schwager, “A real-time game theoretic planner for autonomous two-player drone racing,” IEEE Transactions on Robotics, vol. 36, no. 5, pp. 1389–1403, 2020.
  22. Y. Chen and J. W. Vaughan, “A new understanding of prediction markets via no-regret learning,” in Proceedings of the 11th ACM conference on Electronic commerce, pp. 189–198, 2010.
  23. M. Braverman, J. Mao, J. Schneider, and M. Weinberg, “Selling to a no-regret buyer,” in Proceedings of the 2018 ACM Conference on Economics and Computation, pp. 523–538, 2018.
  24. D. Aussel, L. Brotcorne, S. Lepaul, and L. von Niederhäusern, “A trilevel model for best response in energy demand-side management,” European Journal of Operational Research, vol. 281, no. 2, pp. 299–315, 2020.
  25. J. Engwerda, W. Van Den Broek, and J. M. Schumacher, “Feedback Nash equilibria in uncertain infinite time horizon differential games,” in Proceedings of the 14th International Symposium of Mathematical Theory of Networks and Systems, MTNS 2000, pp. 1–6, Unknown Publisher, 2000.
  26. D. Vrabie and F. Lewis, “Integral reinforcement learning for online computation of feedback Nash strategies of nonzero-sum differential games,” in 49th IEEE Conference on Decision and Control (CDC), pp. 3066–3071, IEEE, 2010.
  27. P. V. Reddy and G. Zaccour, “Feedback Nash equilibria in linear-quadratic difference games with constraints,” IEEE Transactions on Automatic Control, vol. 62, no. 2, pp. 590–604, 2017.
  28. P. V. Reddy and G. Zaccour, “Open-loop and feedback Nash equilibria in constrained linear–quadratic dynamic games played over event trees,” Automatica, vol. 107, pp. 162–174, 2019.
  29. G. Kossioris, M. Plexousakis, A. Xepapadeas, A. de Zeeuw, and K.-G. Mäler, “Feedback Nash equilibria for non-linear differential games in pollution control,” Journal of Economic Dynamics and Control, vol. 32, no. 4, pp. 1312–1331, 2008.
  30. R. Kamalapurkar, J. R. Klotz, and W. E. Dixon, “Concurrent learning-based approximate feedback-Nash equilibrium solution of n-player nonzero-sum differential games,” IEEE/CAA journal of Automatica Sinica, vol. 1, no. 3, pp. 239–247, 2014.
  31. D. Fridovich-Keil, E. Ratner, L. Peters, A. D. Dragan, and C. J. Tomlin, “Efficient iterative linear-quadratic approximations for nonlinear multi-player general-sum differential games,” in 2020 IEEE international conference on robotics and automation (ICRA), pp. 1475–1481, IEEE, 2020.
  32. D. Fridovich-Keil, V. Rubies-Royo, and C. J. Tomlin, “An iterative quadratic method for general-sum differential games with feedback linearizable dynamics,” in 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 2216–2222, IEEE, 2020.
  33. A. Tanwani and Q. Zhu, “Feedback Nash equilibrium for randomly switching differential–algebraic games,” IEEE Transactions on Automatic Control, vol. 65, no. 8, pp. 3286–3301, 2019.
  34. Z. Wang, R. Spica, and M. Schwager, “Game theoretic motion planning for multi-robot racing,” in Distributed Autonomous Robotic Systems: The 14th International Symposium, pp. 225–238, Springer, 2019.
  35. F. Laine, D. Fridovich-Keil, C.-Y. Chiu, and C. Tomlin, “The computation of approximate generalized feedback nash equilibria,” SIAM Journal on Optimization, vol. 33, no. 1, pp. 294–318, 2023.
  36. J. Anderson, J. C. Doyle, S. H. Low, and N. Matni, “System level synthesis,” Annual Reviews in Control, vol. 47, pp. 364–393, 2019.
  37. I. L. Glicksberg, “A further generalization of the kakutani fixed theorem, with application to Nash equilibrium points,” Proceedings of the American Mathematical Society, vol. 3, no. 1, pp. 170–174, 1952.
  38. E. K. Ryu and S. Boyd, “Primer on monotone operator methods,” Appl. comput. math, vol. 15, no. 1, pp. 3–43, 2016.
  39. E. K. Ryu and W. Yin, Large-scale convex optimization: algorithms & analyses via monotone operators. Cambridge University Press, 2022.
  40. T. Başar, “Stochastic differential games and intricacy of information structures,” in Dynamic Games in Economics, pp. 23–49, Springer, 2014.
  41. Prentice hall Upper Saddle River, NJ, 1998.
  42. A. Nayyar and T. Başar, “Dynamic stochastic games with asymmetric information,” in 2012 IEEE 51st IEEE conference on decision and control (CDC), pp. 7145–7150, IEEE, 2012.
  43. Cambridge University Press, 2017.
  44. A. Skopalik and B. Vöcking, “Inapproximability of pure Nash equilibria,” in Proceedings of the fortieth annual ACM symposium on Theory of computing, pp. 355–364, 2008.
  45. S. P. Boyd and L. Vandenberghe, Convex optimization. Cambridge University Press, 2004.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now