Convergence of fictitious play for fully coupled FBSDEs in finite-player stochastic differential games

Abstract: In this article we investigate the theoretical convergence properties of the fictitious-play approximation procedure applied to coupled FBSDE systems for finite-player non-zero-sum stochastic differential games. Under one set of assumptions, the convergence is shown to be geometric. Under an additional structural assumption, the geometric convergence rate further improves to a super-exponential rate in a special class of games. To the best of our knowledge, this provides the first convergence analysis of fictitious play for fully coupled FBSDEs. A numerical experiment with a linear-quadratic interbank borrowing and lending problem confirms the geometric convergence.

• The paper establishes geometric convergence of fictitious play in fully coupled FBSDE systems that model finite-player stochastic differential games.
• It employs an iterative fixed-point scheme where each agent solves a parametrized control problem, achieving super-exponential convergence under symmetric or linear-quadratic assumptions.
• Empirical results from interbank models confirm the theoretical convergence rates and illustrate the method's scalability in high-dimensional settings.

Convergence of Fictitious Play for Fully Coupled FBSDEs in Finite-Player Stochastic Differential Games

Problem Formulation and Background

The paper addresses the convergence of fictitious play applied to fully coupled systems of forward-backward stochastic differential equations (FBSDEs) arising in finite-player nonzero-sum stochastic differential games (SDGs). In this setting, $N$ players interact through controlled stochastic dynamics, each minimizing an individual cost functional dependent on the joint evolution and strategies. The dynamic programming principle translates the Nash equilibrium computation into solving a coupled system of Hamilton-Jacobi-Bellman (HJB) PDEs or, equivalently, a Nash FBSDE system.

Classically, only specific structures such as linear-quadratic games admit tractable solutions, and grid-based numerical methods suffer from the curse of dimensionality, limiting practical computation. Existing works for mean-field games and decoupled FBSDEs show some success, but for fully coupled, finite-player FBSDE systems, direct convergence results of iterative approximation procedures such as fictitious play were lacking.

Fictitious Play Algorithm in FBSDEs

Fictitious play is employed as an iterative best-response scheme: at each step, each agent solves a single-player stochastic control problem, treating the policies of the other agents as frozen. In the dynamic programming framework, this corresponds to solving a parametrized system of single-agent FBSDEs. The main complexity arises when the forward and backward components are fully coupled, i.e., the future values naturally affect the controlled dynamics, and vice versa, for all agents. The authors implement fictitious play in this setting as a fixed-point scheme over the joint space of Markov policies.

Main Theoretical Results

The central results establish conditions that ensure geometric convergence of the fictitious play iterates to a closed-loop Nash equilibrium for general nonzero-sum stochastic differential games with fully coupled FBSDEs. Furthermore, in a special class of games, the geometric rate sharpens to super-exponential convergence under additional structural assumptions—for instance, when the best-response mapping decouples from the opponents’ controls, as often seen in symmetric or linear-quadratic settings.

The analysis accommodates both finite and unbounded spatial domains, treating cases with gradient bounds and smallness conditions on model coefficients or time horizons. The proofs involve a detailed decomposition of iteration errors into control, value, and strategy mismatches, with careful handling of the coupled stochastic and PDE structures. Notably, the authors show that all error components contract at a geometric rate under mild regularity and Lipschitz-continuity assumptions on the system coefficients, Hamiltonians, and best-response mappings.

Implications

Practical Consequences

The convergence guarantees justify the use of iterative best-response solvers in high-dimensional and fully coupled stochastic differential games. For practitioners, this means that robust, scalable numerical methods—including those leveraging deep learning for the approximation of FBSDEs—can be confidently applied to relevant multi-agent domains, such as interbank lending, systemic risk, resource management, or multi-agent mechanical systems, as long as the modeling assumptions align with those analyzed.

The numerical experiments, carried out on a multi-player linear-quadratic interbank model, confirm geometric convergence rates up to machine precision for both small ($N=2$) and larger ($N=20$) agent populations. Empirically, the rate of convergence improves as the player coupling weakens, matching theoretical predictions. This validates the feasibility of applying the developed convergence theory in realistic, higher-dimensional stochastic games.

Theoretical Understanding

From a theoretical perspective, the results fill a crucial gap regarding iterative methods for coupled Nash FBSDEs, going significantly beyond prior work restricted to decoupled systems. The analyses clarify the impact of game structure—such as gradient boundedness versus smallness of coefficients—on convergence and open future directions for exploring existence, uniqueness, and regularity of Nash equilibria in even more general classes of games.

Furthermore, the methodology developed, especially the careful treatment of the stopping-time and Markov feedback policies within the coupled stochastic system, may serve as a blueprint for convergence analyses in related domains, such as mean-field control, multi-population games, or robust multi-agent reinforcement learning algorithms relying on FBSDE solvers.

Future Developments

Potential future directions include:

• Extending the convergence analysis to games with non-Markovian and path-dependent coefficients.
• Developing explicit error bounds for deep learning-based FBSDE solvers within the fictitious play framework.
• Analyzing games with state-constrained controls or common noise, where forward-backward coupling and boundary conditions become more intricate.
• Investigating analogous convergence properties in the asymptotic regime as $N \to \infty$, connecting with mean-field games.

Conclusion

The paper rigorously establishes geometric—and under further conditions, super-exponential—convergence of fictitious play for fully coupled FBSDE systems characterizing finite-player stochastic differential games. The results provide both the theoretical underpinning and practical validation for the use of modern, scalable algorithms in multi-agent continuous-time stochastic control. This advances the mathematical toolbox available for high-dimensional strategic decision-making under uncertainty and interfaces naturally with burgeoning machine learning methodologies in stochastic game theory.

Reference: "Convergence of fictitious play for fully coupled FBSDEs in finite-player stochastic differential games" (2607.08861)