Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash 99 tok/s
Gemini 2.5 Pro 60 tok/s Pro
GPT-5 Medium 32 tok/s
GPT-5 High 27 tok/s Pro
GPT-4o 102 tok/s
GPT OSS 120B 461 tok/s Pro
Kimi K2 227 tok/s Pro
2000 character limit reached

Learning Macroeconomic Policies through Dynamic Stackelberg Mean-Field Games (2403.12093v4)

Published 14 Mar 2024 in econ.TH and cs.AI

Abstract: Macroeconomic outcomes emerge from individuals' decisions, making it essential to model how agents interact with macro policy via consumption, investment, and labor choices. We formulate this as a dynamic Stackelberg game: the government (leader) sets policies, and agents (followers) respond by optimizing their behavior over time. Unlike static models, this dynamic formulation captures temporal dependencies and strategic feedback critical to policy design. However, as the number of agents increases, explicitly simulating all agent-agent and agent-government interactions becomes computationally infeasible. To address this, we propose the Dynamic Stackelberg Mean Field Game (DSMFG) framework, which approximates these complex interactions via agent-population and government-population couplings. This approximation preserves individual-level feedback while ensuring scalability, enabling DSMFG to jointly model three core features of real-world policymaking: dynamic feedback, asymmetry, and large scale. We further introduce Stackelberg Mean Field Reinforcement Learning (SMFRL), a data-driven algorithm that learns the leader's optimal policies while maintaining personalized responses for individual agents. Empirically, we validate our approach in a large-scale simulated economy, where it scales to 1,000 agents (vs. 100 in prior work) and achieves a fourfold increase in GDP over classical economic methods and a nineteenfold improvement over the static 2022 U.S. federal income tax policy.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. Bayesian analysis of dsge models. Econometric reviews, 26(2-4):113–172, 2007.
  2. AI and Macroeconomic Modeling: Deep Reinforcement Learning in an RBC Model, March 2023.
  3. Optimal incentives to mitigate epidemics: a stackelberg mean field game approach. SIAM Journal on Control and Optimization, 60(2):S294–S322, 2022.
  4. Mean field stackelberg games: Aggregation of delayed instructions. SIAM Journal on Control and Optimization, 53(4):2237–2266, 2015.
  5. Mean field games in a stackelberg problem with an informed major player. arXiv preprint arXiv:2311.05229, 2023.
  6. Real wage rigidities and the new keynesian model. Journal of money, credit and banking, 39:35–65, 2007.
  7. The green solow model. Journal of Economic Growth, 15:127–153, 2010.
  8. Deep learning for principal-agent mean field games. arXiv preprint arXiv:2110.01127, 2021.
  9. Deep Reinforcement Learning in a Monetary Model, January 2023.
  10. Analyzing Micro-Founded General Equilibrium Models with Many Agents using Deep Reinforcement Learning, February 2022.
  11. AI-driven Prices for Externalities and Sustainability in Production Markets, January 2023.
  12. Econometric theory and methods, volume 5. Oxford University Press New York, 2004.
  13. A machine learning method for stackelberg mean field games. arXiv preprint arXiv:2302.10440, 2023.
  14. Is independent learning all you need in the starcraft multi-agent challenge? arXiv preprint arXiv:2011.09533, 2020.
  15. Dutt, A. K. Aggregate demand, aggregate supply and economic growth. International review of applied economics, 20(3):319–336, 2006.
  16. Mean-field leader-follower games with terminal state constraint. SIAM Journal on Control and Optimization, 58(4):2078–2113, 2020.
  17. Gabaix, X. A behavioral new keynesian model. American Economic Review, 110(8):2271–2327, 2020.
  18. Gali, J. How well does the is-lm model fit postwar us data? The Quarterly Journal of Economics, 107(2):709–738, 1992.
  19. Optimization frameworks and sensitivity analysis of stackelberg mean-field games. arXiv preprint arXiv:2210.04110, 2022.
  20. Hicks, J. Is-lm: an explanation. Journal of post Keynesian economics, 3(2):139–154, 1980.
  21. Solving Heterogeneous General Equilibrium Economic Models with Deep Reinforcement Learning. URL http://arxiv.org/abs/2103.16977.
  22. Optimal Monetary Policy Using Reinforcement Learning. URL https://papers.ssrn.com/abstract=4025682.
  23. Mean field stackelberg games: State feedback equilibrium. IFAC-PapersOnLine, 53(2):2237–2242, 2020.
  24. Emergent Bartering Behaviour in Multi-Agent Reinforcement Learning, May 2022.
  25. Econometric methods. 1963.
  26. Human-centred mechanism design with Democratic AI. 6(10):1398–1407. ISSN 2397-3374. doi: 10.1038/s41562-022-01383-x. URL https://www.nature.com/articles/s41562-022-01383-x.
  27. Kuriksha, A. An Economy of Neural Networks: Learning from Heterogeneous Experiences, October 2021.
  28. Growth and convergence in a multi-country empirical stochastic solow model. Journal of applied Econometrics, 12(4):357–392, 1997.
  29. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015.
  30. Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems, 30, 2017.
  31. Taxai: A dynamic economic simulator and benchmark for multi-agent reinforcement learning. arXiv preprint arXiv:2309.16307, 2023.
  32. Deep Reinforcement Learning for Optimal Investment and Saving Strategy Selection in Heterogeneous Profiles: Intelligent Agents working towards retirement, June 2022.
  33. Political economics and macroeconomic policy. Handbook of macroeconomics, 1:1397–1482, 1999.
  34. The public policy primer: Managing the policy process. 2010.
  35. Theoretical and empirical analysis of relieff and rrelieff. Machine learning, 53:23–69, 2003.
  36. Rui and Shi. Learning from zero: How to make consumption-saving decisions in a stochastic environment with an AI algorithm, February 2022.
  37. Saez, E. Using elasticities to derive optimal income tax rates. The review of economic studies, 68(1):205–229, 2001.
  38. Sch, A. A. O. Intelligence in the economy: Emergent behaviour in international trade modelling with reinforcement learning. 2021.
  39. Politico-economic models of macroeconomic policy: A review of the empirical evidence. Political business cycles, pp.  239–275, 1988.
  40. Shi, R. A. Can an AI agent hit a moving target. arXiv preprint arXiv, 2110, 2021.
  41. Deterministic policy gradient algorithms. In International conference on machine learning, pp.  387–395. Pmlr, 2014.
  42. Shocks and frictions in us business cycles: A bayesian dsge approach. American economic review, 97(3):586–606, 2007.
  43. Tilbury, C. Reinforcement learning in macroeconomic policy design: A new frontier? arXiv preprint arXiv:2206.08781, 2022.
  44. Building a Foundation for Data-Driven, Interpretable, and Robust Policy Design using the AI Economist. URL http://arxiv.org/abs/2108.02904.
  45. Vedung, E. Public policy and program evaluation. Routledge, 2017.
  46. The emergence of division of labor through decentralized social sanctioning. URL http://arxiv.org/abs/2208.05568.
  47. Mean field multi-agent reinforcement learning. In International conference on machine learning, pp.  5571–5580. PMLR, 2018.
  48. The AI Economist: Taxation policy design via two-level deep multiagent reinforcement learning. 8(18):eabk2607. doi: 10.1126/sciadv.abk2607. URL https://www.science.org/doi/10.1126/sciadv.abk2607.
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Ai Generate Text Spark Streamline Icon: https://streamlinehq.com

Paper Prompts

Sign up for free to create and run prompts on this paper using GPT-5.

List Streamline Icon: https://streamlinehq.com Explore 10 Community Prompts
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets