Stationary Mean Field Game Equilibria

• Stationary mean field games are defined as equilibrium models with time-invariant agent distributions where each agent optimizes its objective while interacting with the aggregate population state.
• The framework leverages controlled Markov processes and fixed-point theorems to characterize and compute equilibria, ensuring the existence and robustness of solutions.
• Applications span economics, engineering, and social sciences, providing computational and theoretical insights into large-scale dynamic interactions and essential equilibrium properties.

A stationary mean field game (MFG) is a mathematical framework describing equilibrium configurations of a continuum of rational agents who optimize individual objectives while interacting through the collective distribution over a (static) state space. Agents evolve according to controlled stochastic processes, and the aggregate effect of the population feeds back into each agent's optimization. In the stationary (time-homogeneous) setting, attention focuses on characterizing and analyzing equilibria where the joint law of agents' states and the population's distribution are invariant in time. Stationary MFGs serve as tractable models for dynamic games involving large populations, with broad application in economics, engineering, and social sciences.

1. Core Model Structures and Formal Definitions

A stationary MFG is defined by:

• A state space $S = \{1, \dots, S\}$ and action space $A = \{1, \dots, A\}$ (finite case), or a continuous state/action domain.
• The population distribution is represented by $m \in P(S)$, the simplex of probability measures on $S$.
• Each agent selects a (possibly mixed) stationary Markov control $\pi$ mapping states to distributions over $A$; the set of all such controls is $\Pi^s$, with $D^s$ the set of deterministic strategies (functions $d: S \to A$).

The controlled dynamics are specified by a family of transition rates $Q_{\cdot,\cdot,a}(m)$ for each action $a$ and population configuration $m$, and the running reward $r(i,a,m)$ depends on the agent's current state, action, and population distribution. Fixing a discount factor $\beta \in (0,1)$, the controlled Markov process for an agent using $\pi$ under stationary law $m$ has generator

$Q_{ij}^\pi(m) = \sum_{a=1}^A Q_{ija}(m) \pi_i(a).$

The agent's expected discounted payoff from initial distribution $x_0$ is

$$V_{x_0}(\pi, m) = \mathbb{E}^{\pi,m}_{x_0} \left[ \int_0^\infty e^{-\beta t} r_{X_t, a_t}(m)\,dt \right].$$

Definition (Stationary Mean Field Equilibrium, SMFE):

A pair $(m, \pi) \in P(S) \times \Pi^s$ is a stationary MFE if:

1. Invariance: $m$ is invariant under $Q^\pi(m)$ (i.e., $m^T Q^\pi(m) = 0$).
2. Optimality: $\pi$ is optimal given $m$: $V_{x_0}(\pi, m) \geq V_{x_0}(\pi', m)$ for all $\pi'$. (Neumann, 2020).

2. Semi-Explicit Solution Structure in Finite State/Action Models

The set of all stationary equilibria in finite settings admits a semi-explicit characterization:

• For fixed $m$, the set of optimal stationary strategies is the convex hull of optimal deterministic strategies:

$$D(m) = \left\{ d \in D^s : V(d,m)=\max_{d'\in D^s} V(d',m) \right\}.$$

Thus, any equilibrium $\pi$ must be in $\operatorname{co} D(m)$.

• The equilibrium fixed-point seeks $m$ for which there exists $\pi \in \operatorname{co} D(m)$ such that $m^T Q^\pi(m)=0$.
• Alternatively, define the equilibrium correspondence

$$\mathcal{F}(m) = \left\{ m' \in P(S) : \exists \pi \in \operatorname{co} D(m), m'^T Q^\pi(m)=0 \right\}$$

and look for $m$ such that $m \in \mathcal{F}(m)$.

This formulation allows for a computational approach: enumerate all deterministic strategies $d^1, \ldots, d^K$ in $D(m)$, compute their stationary laws, and solve finite-dimensional linear equations in $m$. The convex structure arises due to non-uniqueness of optimal controls for generic population distributions (Neumann, 2020, Neumann, 2019).

3. Notion and Characterization of Essential Equilibria

A central property in stationary MFGs is essentiality, capturing robustness of equilibria under model perturbations.

Definition (Essential Equilibrium):

A stationary MFE $(m, \pi)$ of $(Q, r)$ is essential if, for any $\varepsilon > 0$, there exists $\delta > 0$ such that any perturbed game $(Q', r')$ with $d((Q,r), (Q',r')) < \delta$ (in the uniform sup-norm over all relevant arguments) admits a stationary MFE $(m', \pi')$ within $\varepsilon$ of $(m, \pi)$.

This concept ensures the equilibrium survives small misspecifications in transition rates or rewards. The main results are:

• Genericity: The set of finite-state/action games for which every stationary equilibrium is essential is a residual set (dense $G_\delta$) in the metric space of all games—so essentiality is generic ((Neumann, 2020), Theorem 3.1).
• Uniqueness $\Longrightarrow$ Essentiality: If a game admits a unique SMFE, it is automatically essential (Corollary 3.2).
• Deterministic–Stationary Criterion: A deterministic equilibrium $(m, d)$ is essential if $d$ is the unique maximizer in $D(m)$ and $m$ is a stable fixed point for $m^T Q^d(m)=0$.

Consequently, in most parameter regimes, stationary equilibria are structurally stable to parametric perturbations (Neumann, 2020).

4. Existence, Computation, and Algorithmic Strategies

Key existence results rely on fixed-point theorems. For finite state/action games (with Lipschitz/continuous data), the best response and invariance correspondences are nonempty, convex, and upper semicontinuous, so Kakutani's or Brouwer's theorem yields existence of SMFE (Neumann, 2019).

Algorithmic outline for finite S, A:

1. Partition $P(S)$ by optimal deterministic policy regions: For each deterministic strategy, determine the region of population configurations where it is optimal.
2. Compute stationary distributions: For each deterministic policy $d$, solve $m^T Q^d(m)=0$, $\sum m = 1$.
3. Enumerate mixed equilibria: In regions with multiple optimal strategies, construct convex combinations of their stationary laws, and solve for fixed-point conditions.
4. Reduce to finite-dimensional nonlinear problems: For each region, solve for $m$ as a fixed point of a convex-valued set map, often leading to a finite set of solutions (Neumann, 2019, Neumann, 2020).

5. Structural Results, Robustness, and Applications

The residuality of essential equilibria indicates

• Parameterized families of models will almost always avoid nonessential equilibria, except on knife-edge sets of measure zero.
• Small perturbations of transition rates or rewards preserve the existence and local uniqueness of equilibria, provided the conditions above hold (Neumann, 2020).
• Mixed stationary equilibria emerge only at points where multiple deterministic controls are simultaneously optimal, supporting a nontrivial convex structure in the equilibrium set.

Representative applications include:

• Consumer choice models with switching and congestion costs: Essentiality holds except along codimension-1 parameter curves (Neumann, 2020).
• Botnet defense and corruption models: Deterministic equilibria become essential if only one action is optimal at equilibrium distribution.
• Multi-agent Markov games and stochastic binary-action MFGs: Existence, explicit comparative statics, and uniqueness under monotonicity or positive externality assumptions (Huang et al., 2021).

6. Essential Equilibria: Main Theorems and Genericity Table

Main Theorems (Neumann, 2020):

Theorem	Summary of Condition	Conclusion about Essentiality
Genericity (Thm 3.1)	All stationary equilibria essential is residual/dense	Generic games have robust SMFE
Uniqueness ⇒ Essential (Cor)	Unique stationary equilibrium	Singleton is essential
Det-stationary (Thm 4.3)	Deterministic $(m,d)$, unique maximizer, stable fixed pt	$(m, d)$ is essential

The formal results provide sharp topological and computational guarantees for the robustness and structure of stationary mean field equilibria in finite settings.

---

References:

For foundational results, proofs, and computational details, see (Neumann, 2020, Neumann, 2019), and further illustrative examples and applications as referenced in these works.