Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deep Backward and Galerkin Methods for the Finite State Master Equation (2403.04975v2)

Published 8 Mar 2024 in math.OC and stat.ML

Abstract: This paper proposes and analyzes two neural network methods to solve the master equation for finite-state mean field games (MFGs). Solving MFGs provides approximate Nash equilibria for stochastic, differential games with finite but large populations of agents. The master equation is a partial differential equation (PDE) whose solution characterizes MFG equilibria for any possible initial distribution. The first method we propose relies on backward induction in a time component while the second method directly tackles the PDE without discretizing time. For both approaches, we prove two types of results: there exist neural networks that make the algorithms' loss functions arbitrarily small, and conversely, if the losses are small, then the neural networks are good approximations of the master equation's solution. We conclude the paper with numerical experiments on benchmark problems from the literature up to dimension 15, and a comparison with solutions computed by a classical method for fixed initial distributions.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Y. Achdou and M. Laurière. Mean field games and applications: Numerical aspects. Mean Field Games, page 249, 2020.
  2. Extensions of the deep galerkin method. Applied Mathematics and Computation, 430:127287, 2022.
  3. R. J. Aumann. Markets with a continuum of traders. Econometrica, 32:39–50, 1964.
  4. A numerical scheme for a mean field game in some queueing systems based on Markov chain approximation method. SIAM Journal on Control and Optimization, 56, 2017.
  5. E. Bayraktar and A. Cohen. Analysis of a finite state many player game using its master equation. SIAM Journal on Control and Optimization, 56(5):3538–3568, 2018.
  6. A neural network approach to high-dimensional optimal switching problems with jumps in energy markets, 2022.
  7. Mean field games and mean field type control theory. SpringerBriefs in Mathematics. Springer, New York, 2013.
  8. C. Bertucci and A. Cecchin. Mean field games master equations: from discrete to continuous state space. arXiv e-prints, page arXiv:2207.03191, July 2022.
  9. F. Black and M. Scholes. The pricing of options and corporate liabilities. J. Polit. Econ., 81(3):637–654, 1973.
  10. Connecting GANs, MFGs, and OT. arXiv preprint arXiv:2002.04112, 2020.
  11. The master equation and the convergence problem in mean field games, volume 201 of Annals of Mathematics Studies. Princeton University Press, 2019.
  12. R. Carmona and F. Delarue. Probabilistic theory of mean field games with applications. I, volume 83 of Probability Theory and Stochastic Modelling. Springer, Cham, 2018.
  13. R. Carmona and M. Laurière. Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games I: The ergodic case. SIAM Journal on Numerical Analysis, 59(3):1455–1485, 2021.
  14. R. Carmona and M. Laurière. Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games: II—the finite horizon case. The Annals of Applied Probability, 32(6):4065–4105, 2022.
  15. A. Cecchin and G. Pelino. Convergence, fluctuations and large deviations for finite state mean field games via the master equation. Stochastic Processes and their Applications, 129(11):4510 – 4555, 2019.
  16. A. Cohen and E. Zell. Analysis of the Finite-State Ergodic Master Equation. Applied Mathematics and Optimization, 87, 2023.
  17. The complexity of computing a nash equilibrium. Communications of the ACM, 52(2):89–97, 2009.
  18. Deep learning for population-dependent controls in mean field control problems. Accepted for an extended abstract to AAMAS’23 (arXiv preprint arXiv:2306.04788), 2023.
  19. From the master equation to mean field game limit theory: a central limit theorem. Electron. J. Probab., 24, 2019.
  20. From the master equation to mean field game limit theory: Large deviations and concentration of measure. Annals of Probability, 48(1):211–263, 2020.
  21. Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Commun. Math. Stat., 5(4):349–380, 2017.
  22. L. C. Evans. Partial differential equations, volume 19 of Graduate Studies in Mathematics. American Mathematical Society, Providence, RI, second edition, 2010.
  23. Efficient and accurate estimation of lipschitz constants for deep neural networks. Advances in Neural Information Processing Systems, 32, 2019.
  24. J.-P. Fouque and Z. Zhang. Deep learning methods for mean field control problems with delay. Frontiers in Applied Mathematics and Statistics, 6(11), 2020.
  25. Deepsets and their derivative networks for solving symmetric PDEs. Journal of Scientific Computing, 91(2):63, 2022.
  26. Numerical resolution of McKean-Vlasov FBSDEs using neural networks. Methodology and Computing in Applied Probability, 24(4):2557–2586, 2022.
  27. Rate of convergence for particle approximation of PDEs in wasserstein space. Journal of Applied Probability, 59(4):992–1008, 2022.
  28. Discrete time, finite state space mean field games. J. Math. Pures Appl. (9), 93(3):308–328, 2010.
  29. Continuous time finite state mean field games. Appl. Math. Optim., 68(1):99–143, 2013.
  30. Regularisation of neural networks by enforcing Lipschitz continuity. Mach. Learn., 110(2):393–416, 2021.
  31. Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks. Neural Networks, 3:551–560, 1990.
  32. R. Hu and M. Laurière. Recent developments in machine learning methods for stochastic control and games. arXiv preprint arXiv:2303.10257, 2023.
  33. Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle. Commun. Inf. Syst., 6(3):221–251, 2006.
  34. Deep backward schemes for high-dimensional nonlinear PDEs. Math. Comp., 89(324):1547–1579, 2020.
  35. P. Knabner and L. Angermann. Numerical methods for elliptic and parabolic partial differential equations, volume 44 of Texts in Applied Mathematics. Springer, Cham, extended edition, 2021. With contributions by Andreas Rupp.
  36. V. N. Kolokoltsov and A. Bensoussan. Mean-field-game model for botnet defense in cyber-security. Appl. Math. Optim., 74(3):669–692, 2016.
  37. Jeux à champ moyen. II. Horizon fini et contrôle optimal. C. R. Math. Acad. Sci. Paris, 343(10):679–684, 2006.
  38. Mean field games. Jpn. J. Math., 2(1):229–260, 2007.
  39. M. Laurière. Numerical methods for mean field games and mean field type control. In Mean field games, volume 78 of Proc. Sympos. Appl. Math., pages 221–282. Amer. Math. Soc., Providence, RI, 2021.
  40. P.-L. Lions. Estimées nouvelles pour les équations quasilinéaires. Seminar in Applied Mathematics at the Collège de France., 2014.
  41. J. Nash. Non-cooperative games. Annals of mathematics, pages 286–295, 1951.
  42. Training robust neural networks using lipschitz bounds. IEEE Control Systems Letters, 6:121–126, 2021.
  43. Generalization in mean field games by learning master policies. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 9413–9421, 2022.
  44. Neural networks-based backward scheme for fully nonlinear PDEs. Partial Differ. Equ. Appl., 2(1):Paper No. 16, 24, 2021.
  45. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019.
  46. A machine learning framework for solving high-dimensional mean field game and mean field control problems. Proceedings of the National Academy of Sciences, 117(17):9183–9193, 2020.
  47. D. Schmeidler. Equilibrium points of nonatomic games. J. Statist. Phys., 7:295–300, 1973.
  48. S. Shalev-Shwartz and Y. Wexler. Minimizing the maximal loss: How and why. In Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 793–801. PMLR, 2016.
  49. J. Sirignano and K. Spiliopoulos. DGM: a deep learning algorithm for solving partial differential equations. J. Comput. Phys., 375:1339–1364, 2018.
  50. P. Tseng. An incremental gradient(-projection) method with momentum term and adaptive stepsize rule. SIAM J. Optim., 8(2):506–531, 1998.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets