Coalitional Mean-Field Games

• Coalitional Mean-Field Games are frameworks where large populations are partitioned into teams that cooperate internally while competing externally through the mean-field of states.
• They use mathematical formulations such as team-Nash equilibria, FBSDEs, and fixed-point iterations to derive decentralized strategies and guarantee ε-approximate equilibria.
• Applications span economics, robotics, and supply chains, offering insights into robust control, risk-sharing, and computational methods including deep reinforcement learning.

A coalitional mean-field type game is a class of large-scale stochastic dynamic games in which the agent population is partitioned into several coalitions (teams or populations); agents within each coalition cooperate to optimize a joint objective, while the coalitions interact competitively or non-cooperatively. The interaction among agents is mediated via the empirical distribution—mean-field—of states, which typically influences both the agents’ state dynamics and cost functionals. This framework generalizes classical mean-field games (MFG, fully non-cooperative) and mean-field type control (MFTC, fully cooperative) to accommodate hierarchical, mixed, and block-structured coalition architectures, enabling the study and design of decentralized strategies in systems with multi-level cooperation and competition.

1. Mathematical Formulation and Coalitional Structures

Let $N$ agents be partitioned into $m$ disjoint coalitions, indexed by $i=1,\dots,m$. Agents within a coalition coordinate to minimize a social (team) cost, while coalitions themselves compete non-cooperatively (team Nash). In the finite-agent model, the empirical state distribution within each coalition $i$ is

$$\mu_t^{(i),N} = \frac1{N^{(i)}}\sum_{j=1}^{N^{(i)}}\delta_{X_t^{i,j}},$$

and the joint mean-field is $Z_t^N = (\mu_t^{(1),N}, ..., \mu_t^{(m),N})$.

A generic agent $j$ in coalition $i$ evolves as:

$$dX_t^{i,j} = b_i\bigl(t, X_t^{i,j}, Z_t^N, U_t^{i,j}\bigr)dt + \sigma_i\bigl(t, X_t^{i,j}, Z_t^N\bigr)dW_t^{i,j},$$

with corresponding stage cost $c^{(i)}(X_t^{i,j}, U_t^{i,j}, Z_t^N)$. The central objective for coalition $m$0 is the team-averaged expected cost, conditioned on all (possibly time-inhomogeneous) policies across the coalitions.

In the infinite-population ($m$1) mean-field limit, the empirical measures converge to deterministic flows $m$2, and the game is reduced to a deterministic dynamic game on the space of probability measures.

The table below encapsulates the taxonomy:

Structure	Within-Coalition Interaction	Between-Coalition Interaction
MFG	Non-cooperative	Non-cooperative
MFTC	Fully cooperative	Fully cooperative
Coalitional MFG	Cooperative (Team)	Competitive (Game/Team Nash)
Mixed	Some coalitions cooperate	Others behave non-cooperatively

(Jianhui et al., 2022, Subramanian et al., 2023, Fujii, 2019)

2. Equilibrium Concepts: Team Nash and Mixed Equilibria

The standard equilibrium for coalitional mean-field type games is a team-Nash equilibrium, defined as a profile $m$3 of coalition-level strategies such that, for each coalition $m$4:

$m$5

where $m$6 denotes the expected cumulative (team-averaged) cost, given all strategies, and “unilateral deviation” means all agents of coalition $m$7 jointly deviate to another identical team policy.

In the mean-field (limit) regime, this equilibrium notion is refined as:

• Mean-Field Markov-Perfect Equilibrium (MF-MPE): An equilibrium in Markov strategies where each coalition’s prescription (mapping mean-field to action distributions) is optimal, given opponents’ prescriptions and consistent with the evolution of the mean-field (Subramanian et al., 2023).
• Asymptotic Mixed-Equilibrium-Optima (AMEO): In LQ models with two teams, AMEO specifies a limit in which each team’s cost is asymptotically optimal with respect to unilateral team deviations (Jianhui et al., 2022).

When some coalitions are fully cooperative and others are not, the equilibrium is a coupled FBSDE solution involving both MFTC and MFG components (Fujii, 2019).

3. Solution Methodologies and Fixed-Point Characterizations

Coalitional MFTGs admit solution methodologies rooted in stochastic control, dynamic programming, and FBSDE theory.

• Pontryagin’s Maximum Principle for McKean–Vlasov Dynamics: The optimality conditions reduce to solving, for each coalition, a system of coupled forward-backward stochastic differential equations (FBSDEs) of McKean–Vlasov type, reflecting the dependence on both the agent’s state and the mean-field (Fujii, 2019).
• Dynamic Programming and Bellman Equations: The problem is reduced to a $m$8-player stochastic game in the space of mean-field distributions, with value functions $m$9 satisfying coupled Bellman equations. Optimal prescriptions are constructed via backward induction (Subramanian et al., 2023).
• Linear-Quadratic (LQ) Analysis: In LQ-Gaussian settings, the FBSDEs and Bellman equations decouple into a tractable set of coupled matrix Riccati ODEs (for second moments and feedback gains), with existence and uniqueness under standard convexity and controllability conditions (Jianhui et al., 2022, Barreiro-Gomez et al., 2019).
• Consistency (Fixed-Point) Iteration: A forward–backward scheme is deployed:
  1. Fix candidate mean-field flows.
  2. Solve best-response (dynamic programming or FBSDE) for each coalition.
  3. Update mean-field trajectories induced by the new policies.
  4. Iterate until convergence (Subramanian et al., 2023, Fujii, 2019).

In tabular finite state/action settings, fixed-point iteration over discretized mean-field spaces enables Nash Q-learning or deep RL approaches to compute equilibria (Shao et al., 2024).

4. Theoretical Properties and Approximation Guarantees

The principal theoretical guarantee is that the mean-field limit equilibrium (coalitional MFTG equilibrium) serves as an $i=1,\dots,m$0-approximate team-Nash equilibrium for the corresponding finite-population game:

• Approximate Nash Guarantees: Under suitable regularity (Lipschitz) and convexity conditions, the error $i=1,\dots,m$1 in cost vanishes as $i=1,\dots,m$2:

$i=1,\dots,m$3

(Subramanian et al., 2023, Shao et al., 2024, Fujii, 2019).

• Risk-Sharing and Robustness: In risk-sensitive and robust MFTGs, coalition formation expands the domain of well-posedness (larger allowable risk-aversion parameters) due to population risk-sharing. The risk-sensitive Riccati condition improves under full cooperation:

$i=1,\dots,m$4

compared to the individual condition, with robustness against disturbances realized through a duality between risk sensitivity and adversarial design (Barreiro-Gomez et al., 2019).

• Information Structure: These games typically feature non-classical information structure, with agents observing their own state and the aggregate mean-field but not the states of others (Subramanian et al., 2023).

5. Computational Aspects and Reinforcement Learning Methods

Classical approaches are limited by the curse of dimensionality inherent in representing measures over high-dimensional spaces. Recent advances address computation as follows:

• Quantization-Based Nash Q-Learning: The mean-field simplex is discretized to form a finite state/action game among central coalition-players, and Nash Q-learning is used to converge to fixed-point Q-values, with convergence guarantees as discretization error vanishes. See (Shao et al., 2024) for detailed algorithmic structure.

• Deep Reinforcement Learning (DRL): To scale to large mean-field dimensions (tested up to $i=1,\dots,m$5), actor–critic methods (e.g., DDPG) approximate both central value functions and optimal policies over the mean-field state space, with empirical performance closely tracking Nash fixed points (Shao et al., 2024).
• Backward Induction/Sampling: For finite-horizon settings, backward dynamic programming (on discretized mean-field state space) and sampling-based RL (e.g., fitted Q-iteration) are used to approximate policies without explicit enumeration of all mean-field states (Subramanian et al., 2023).
• Gaussian Approximation: In very large populations, multinomial mean-field transitions may be approximated by Gaussian processes, reducing sample complexity and variance in simulation-based solvers (Subramanian et al., 2023).

6. Applications, Interpretations, and Open Problems

Coalitional mean-field games structure arises naturally in economic, engineering, and multi-agent systems:

• Duopoly and Oligopoly Markets: Competing firms (coalitions), each controlling a network of agents/outlets that internally collaborate, fit this paradigm (Jianhui et al., 2022).
• Multi-Agent Robotics: Robotic swarms or sensor networks arranged in competing groups, where inter-team adversarial behavior coexists with intra-team cooperation.
• Supply Chains and Alliances: Large consortia forming internal alliances (coalitions) and competing externally capture several large-scale industrial settings.
• Systemic Risk and Robust Control: Coalition formation enhances robustness and admissible risk exposure, especially in stochastic systems with regime-switching and jump disturbances (Barreiro-Gomez et al., 2019).

Notable limitations and open questions include extending results beyond LQ-Gaussian models to general utilities and non-Gaussian noise, accommodating more than two coalitions (block structure $i=1,\dots,m$6 and above), and incorporating major–minor agent hierarchies or common noise (Jianhui et al., 2022, Fujii, 2019).

7. Notational and Conceptual Unification

Coalitional mean-field game models unify and generalize the following paradigms:

Paradigm	Coalition Matrix $i=1,\dots,m$7	Within-Coalition	Between-Coalition	Reference
MFG	$i=1,\dots,m$8	Non-coop	Non-coop	(Jianhui et al., 2022, Fujii, 2019)
MFTC	$i=1,\dots,m$9	Coop	Coop	(Jianhui et al., 2022, Fujii, 2019)
Mixed	Block $i$0/$i$1	Coop (in block)	Non-coop (across)	(Jianhui et al., 2022, Fujii, 2019, Subramanian et al., 2023)

The solution theory—forward–backward SDEs (or dynamic programming on measure space), Riccati equations (in LQ models), and fixed-point iteration—provides a systematic paradigm for decentralized, scalable control in large systems with both cooperative and adversarial components.

References:

(Jianhui et al., 2022, Subramanian et al., 2023, Fujii, 2019, Barreiro-Gomez et al., 2019, Shao et al., 2024)