2000 character limit reached
Generalizing Better Response Paths and Weakly Acyclic Games
Published 26 Mar 2024 in cs.GT and econ.TH | (2403.18086v1)
Abstract: Weakly acyclic games generalize potential games and are fundamental to the study of game theoretic control. In this paper, we present a generalization of weakly acyclic games, and we observe its importance in multi-agent learning when agents employ experimental strategy updates in periods where they fail to best respond. While weak acyclicity is defined in terms of path connectivity properties of a game's better response graph, our generalization is defined using a generalized better response graph. We provide sufficient conditions for this notion of generalized weak acyclicity in both two-player games and $n$-player games. To demonstrate that our generalization is not trivial, we provide examples of games admitting a pure Nash equilibrium that are not generalized weakly acyclic. The generalization presented in this work is closely related to the recent theory of satisficing paths, and the counterexamples presented here constitute the first negative results in that theory.
- P. Frihauf, M. Krstic, and T. Basar, “Nash equilibrium seeking in noncooperative games,” IEEE Transactions on Automatic Control, vol. 57, no. 5, pp. 1192–1207, 2011.
- F. Salehisadaghiani and L. Pavel, “Distributed Nash equilibrium seeking: A gossip-based algorithm,” Automatica, vol. 72, pp. 209–216, 2016.
- M. Ye and G. Hu, “Distributed Nash equilibrium seeking by a consensus based approach,” IEEE Transactions on Automatic Control, vol. 62, no. 9, pp. 4811–4818, 2017.
- D. Gadjov and L. Pavel, “A passivity-based approach to Nash equilibrium seeking over networks,” IEEE Transactions on Automatic Control, vol. 64, no. 3, pp. 1077–1092, 2018.
- H. P. Young, “The evolution of conventions,” Econometrica: Journal of the Econometric Society, pp. 57–84, 1993.
- I. Milchtaich, “Congestion games with player-specific payoff functions,” Games and economic behavior, vol. 13, no. 1, pp. 111–124, 1996.
- H. P. Young, Individual strategy and social structure: An evolutionary theory of institutions. Princeton University Press, 1998.
- H. P. Young, Strategic learning and its limits. OUP Oxford, 2004.
- A. Fabrikant, A. D. Jaggard, and M. Schapira, “On the structure of weakly acyclic games,” Theory of Computing Systems, vol. 53, no. 1, pp. 107–122, 2013.
- D. Monderer and L. S. Shapley, “Potential games,” Games and Economic Behavior, vol. 14, no. 1, pp. 124–143, 1996.
- J. R. Marden, G. Arslan, and J. S. Shamma, “Cooperative control and potential games,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 39, no. 6, pp. 1393–1407, 2009.
- N. Li and J. R. Marden, “Designing games for distributed optimization,” IEEE Journal of Selected Topics in Signal Processing, vol. 7, no. 2, pp. 230–242, 2013.
- R. Gopalakrishnan, J. R. Marden, and A. Wierman, “Potential games are necessary to ensure pure Nash equilibria in cost sharing games,” Mathematics of Operations Research, vol. 39, no. 4, pp. 1252–1296, 2014.
- B. Yongacoglu, G. Arslan, and S. Yüksel, “Satisficing paths and independent multiagent reinforcement learning in stochastic games,” SIAM Journal on Mathematics of Data Science, vol. 5, no. 3, pp. 745–773, 2023.
- J. R. Marden, H. P. Young, G. Arslan, and J. S. Shamma, “Payoff-based dynamics for multiplayer weakly acyclic games,” SIAM Journal on Control and Optimization, vol. 48, no. 1, pp. 373–396, 2009.
- G. Arslan and S. Yüksel, “Decentralized Q-learning for stochastic teams and games,” IEEE Transactions on Automatic Control, vol. 62, no. 4, pp. 1545–1558, 2016.
- B. Swenson, C. Eksin, S. Kar, and A. Ribeiro, “Distributed inertial best-response dynamics,” IEEE Transactions on Automatic Control, vol. 63, no. 12, pp. 4294–4300, 2018.
- D. Foster and H. P. Young, “Regret testing: Learning to play Nash equilibrium without knowing you have an opponent,” Theoretical Economics, vol. 1, no. 3, pp. 341–367, 2006.
- F. Germano and G. Lugosi, “Global Nash convergence of Foster and Young’s regret testing,” Games and Economic Behavior, vol. 60, no. 1, pp. 135–154, 2007.
- H. P. Young, “Learning by trial and error,” Games and Economic Behavior, vol. 65, no. 2, pp. 626–643, 2009.
- B. S. Pradelski and H. P. Young, “Learning efficient Nash equilibria in distributed systems,” Games and Economic behavior, vol. 75, no. 2, pp. 882–897, 2012.
- G. C. Chasparis, A. Arapostathis, and J. S. Shamma, “Aspiration learning in coordination games,” SIAM Journal on Control and Optimization, vol. 51, no. 1, pp. 465–490, 2013.
- J. R. Marden, H. P. Young, and L. Y. Pao, “Achieving Pareto optimality through distributed learning,” SIAM Journal on Control and Optimization, vol. 52, no. 5, pp. 2753–2770, 2014.
- J. R. Marden, “Selecting efficient correlated equilibria through distributed learning,” Games and Economic Behavior, vol. 106, pp. 114–133, 2017.
- Z. Hu, M. Zhu, P. Chen, and P. Liu, “On convergence rates of game theoretic reinforcement learning algorithms,” Automatica, vol. 104, pp. 90–101, 2019.
- B. Yongacoglu, G. Arslan, and S. Yüksel, “Decentralized learning for optimality in stochastic dynamic teams and games with local control and global state information,” IEEE Transactions on Automatic Control, vol. 67, no. 10, pp. 5230–5245, 2022.
- J. Gaveau, C. J. Le Martret, and M. Assaad, “Performance analysis of trial and error algorithms,” IEEE Transactions on Parallel and Distributed Systems, vol. 31, no. 6, pp. 1343–1356, 2020.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.