Linear Programming Mean Field Equilibria

• Linear programming mean field equilibria is a framework that uses infinite-dimensional optimization over occupation measures to characterize Nash equilibria in mean field games.
• It employs constrained optimization techniques, including primal-dual and fixed point methods, to handle various control problems such as singular, jump-diffusion, and reflected dynamics.
• This approach facilitates existence proofs, efficient numerical methods, and robust equilibrium selection in complex stochastic control and PDE-based settings.

Linear programming mean field equilibria refer to the characterization and computation of mean field game (MFG) equilibria by means of infinite-dimensional linear programming over measures (also called occupation measures), and closely related primal-dual and fixed-point methods. These approaches encode the Nash equilibrium conditions or relaxed optimality problems in the language of constrained optimization, thereby providing powerful tools for existence proofs, numerical methods, and connections to stochastic control, PDEs, operator theory, and equilibrium selection. The field covers a broad methodological spectrum: from adjoint and forward-backward ODE/SDE systems in linear-quadratic settings, to occupation-measure linear programming formulations for controlled diffusions, jump-diffusions, games with singular or impulsive controls, and robust MFGs under model uncertainty.

1. Linear Programming Formulation and Occupation Measures

In canonical formulations, a representative agent's optimal control problem, possibly in the presence of a mean-field interaction, is lifted to an optimization over measures that record the occupation of the state-action(-stopping time) space. For a diffusion process with general control and/or singular controls, the occupation measure encodes the long-run fraction of time spent in each state-action pair (and, for singular or impulse controls, alongside random measures over time-space-control).

The general structure is:

• Objective functional: Linear in the measures (e.g., $\int f(x, a) m(dx, da) + \int g(x) p(dx)$).
• Dynamics constraint: For all sufficiently smooth test functions $u$, a linear constraint is imposed, e.g.:

$$\int u(T, x) \nu(dx) = \int u(0, x) m_0(dx) + \int_{[0, T] \times X \times A} (\partial_t u + L_t u)(t, x, a) m(dt, dx, da) + \text{boundary/reflection terms}$$

where $L_t$ is the controlled generator (possibly including jump or reflection contributions).

• Relaxed (randomized) controls: The control at each time and state is allowed to be a probability measure over actions, leading to compactness.

This abstraction is crucial in settings with non-smooth controls (as in singular or impulse control), boundary constraints (as with reflections), or when working with possibly non-Markovian or non-regular dynamics (Dumitrescu et al., 2020, Cohen et al., 11 Apr 2024, Liang et al., 28 Aug 2025).

2. Existence and Equivalence of Linear Programming Equilibria

Existence of equilibria in the LP formulation fundamentally relies on convexity, linearity, and compactness. By formulating the control problem over the space of occupation measures (typically endowed with the weak topology), one ensures compactness of admissible measures via tightness or Lyapunov-based arguments.

In the context of mean field games, the equilibrium mapping (which assigns to any candidate population measure $\nu$ the set of occupation measures optimal in response to $\nu$) is set-valued, convex, and compact-valued under appropriate conditions. Upper hemicontinuity arises due to the linear structure and regular dependence—ensured, for example, by boundedness or continuity of coefficients—making the Kakutani–Glicksberg–Fan fixed point theorem applicable (Cohen et al., 11 Apr 2024, Dumitrescu et al., 2020, Liang et al., 28 Aug 2025).

Furthermore, there is a rigorous equivalence between the LP (occupation measure) formulation and the controlled martingale (weak solution) or relaxed control approach (Dumitrescu et al., 2020, Cohen et al., 11 Apr 2024, Liang et al., 28 Aug 2025), bridging the measure or occupation-based perspective with the pathwise stochastic analysis.

3. Adjoint Equation, Banach Fixed Point, and Forward-Backward ODE Structures in LQ Mean Field Games

In linear-quadratic mean field games, two analytical approaches are prevalent:

• Adjoint Equation Approach: Rather than solving a coupled system of HJB and FP equations via dynamic programming, one applies the stochastic maximum principle resulting in adjoint processes (see (Bensoussan et al., 2014)). The optimal control is recovered via:

$$u_t = -R_t^{-1} B_t^* p_t \quad \text{where} \quad p_t = \mathbb{E}[w_t | \mathcal{F}_t]$$

with $w_t$ the solution of a backward adjoint ODE.

• Forward-Backward ODE/SDE System: The expected state and adjoint variables satisfy coupled deterministic forward-backward ODEs. The optimal mean-field is characterized as the fixed point in a suitable Hilbert space—most effectively obtained by demonstrating contraction of the mean field mapping using a carefully chosen norm:

$$\sqrt{T}\|\varphi\|_T \|\bar{A}\|_T(1+\|\mathcal{S}\|_T) + \|\mathcal{S}\|_T < 1$$

which ensures uniqueness of the fixed point by the Banach fixed point theorem (Bensoussan et al., 2014).

• Riccati Equations: These often arise in the characterization of the adjoint process, especially in higher-dimensional or non-symmetric settings. Notably, nonsymmetric Riccati equations appear, with unique solvability conditions independent of control coefficients.

These approaches allow for the explicit computation of equilibrium strategies, characterize cases of uniqueness and non-existence, and offer advantages over dynamic programming in higher-dimensional, coupled, or non-symmetric problems.

4. Linear Programming in Mean Field Games with Singular, Jump, and Reflected Dynamics

For singular controls, harvesting models, or systems with jumps and reflections, occupation measure linear programming remains applicable, provided the key structural assumption that the dynamics (generators) and reward/cost are linear in the measures.

• Queueing and Harvesting Models: For queueing-inspired models (minimization under long-term average cost), the optimal relaxed singular controls exist and are fully described via measures over state and control; similar results hold for downward-only (harvesting) controls, with equilibrium characterized through Lyapunov function arguments (Cohen et al., 11 Apr 2024).
• Reflected Jump-Diffusion Models: For controlled reflective jump-diffusions (e.g., inventory systems), the LP formulation utilizes occupation measures (terminal, running, and reflection) and imposes a linear constraint arising from Itô’s formula with both jump and reflection corrections. The existence of solutions, equivalence with weak relaxed control formalisms, and fixed point existence for LP-MFG equilibria are established under mild regularity and continuity of coefficients (Liang et al., 28 Aug 2025).

5. Primal-Dual Characterization and Strong Duality

Strong duality results connect the LP occupation measure approach to dual formulations involving subsolutions of Hamilton-Jacobi-BeLLMan (HJB) equations. In continuous-time MFGs with measurable coefficients, for any fixed mean-field flow, the agent problem’s LP over measures admits a dual consisting of a maximization problem over smooth subsolutions. The dual solution, when it exists, provides both necessary and sufficient optimality conditions for Nash equilibria, even in the absence of uniqueness or convexity of the Hamiltonian, or absence of classical HJB solutions (Guo et al., 2 Mar 2025).

This duality underpins a complete characterization of all Nash equilibria for MFGs and reveals the structural relation between occupation measures, HJB subsolutions, and strong duality in infinite-dimensional LP.

6. Selection among Multiple Equilibria, Set Values, and Robustness

Linear programming frameworks provide valuable tools for addressing nonuniqueness and equilibrium selection:

• Selection by Regularization: When MFGs have multiple equilibria (e.g., via degeneracy in FBSDEs in LQ-MFGs), techniques akin to adding “vanishing viscosity” (zero noise or N-player limits) select among equilibria via entropy solutions of associated PDEs. This has direct analogies with regularization in degenerate LP, with implications for “robust” or “stable” solution selection (Delarue et al., 2018).
• Set Value Formulation: Rather than identifying a single equilibrium (or value), one may consider the set of all possible equilibrium values—leading to the concept of the “set value” of the MFG. This set is characterized via dynamic programming principles and proven to be stable under appropriate regularity conditions. Set value analysis can be implemented via convex hulls or set-valued BeLLMan/LP formulations (Iseri et al., 2021).
• Robustness to Model Uncertainty: In robust mean-field Markov games under model uncertainty, the agent solves a max-min problem over occupation measures (and transition kernels). Dynamic programming recursions yield value functions as the solution to a linear-convex game over measures, with equilibrium concepts extending to robust or ambiguity-averse formulations (Langner et al., 15 Oct 2024).

7. Algorithmic and Practical Implications

The LP formulation, due to its convexity and reliance on measures rather than pathwise controls or distributions, is especially amenable to discretization, numerical linear programming, and primal-dual iteration schemes (e.g., projected gradient descent, interior point methods). Notably, for mean-field games where both the dynamics and cost are linear in the measure (the "linear mean-field game"), the problem reduces naturally to a finite-dimensional LP (or a generalized Nash equilibrium problem) (Saldi, 2023, Guo et al., 2022). In such settings, classical and modern optimization algorithms apply out-of-the-box, providing scalable and flexible pathways for computation in large-scale, high-dimensional, or non-unique equilibrium environments.

Algorithmic developments include:

• Value/Policy Iteration and Q-learning schemes: Recursive methods aligned with the LP occupation measure constraints.
• Primal-dual or potential-reduction interior point methods: Suitable for solving generalized Nash equilibrium (GNEP) formulations.
• Bandit and regret-minimization meta-learning: Particularly effective for learning correlated, coarse correlated, or Nash equilibria in mean-field regimes, leveraging LP formulations of equilibrium constraints.

These approaches are robust to discontinuities, lack of convexity, nonstandard dynamics (jumps, reflections, singular controls), and even pathwise degeneracy.

---

Summary Table: Key LP MFG Structural Elements

Feature	Typical Mathematical Formulation	Reference Example
Occupation measure constraint	$$\int u(T,x)\nu(dx) = \int u(0,x)m_0(dx) + \int(\partial_t u + Lu) m +$$ boundary	(Dumitrescu et al., 2020, Liang et al., 28 Aug 2025)
Objective function (in LP)	$\int f(x, a) m(dx, da) + \int g(x) p(dx)$	(Dumitrescu et al., 2020, Cohen et al., 11 Apr 2024)
Equivalence to relaxed control/martingale	Construction mapping from LP measures to weak (martingale) solutions and conversely	(Dumitrescu et al., 2020, Cohen et al., 11 Apr 2024)
Primal-dual (strong duality)	LP in occupation measures $\Longleftrightarrow$ dual over HJB subsolutions	(Guo et al., 2 Mar 2025)
Existence of equilibrium (fixed point)	Kakutani–Glicksberg–Fan fixed point in convex compact set	(Cohen et al., 11 Apr 2024, Liang et al., 28 Aug 2025)

---

• Linear-Quadratic Mean Field Games: adjoint, Riccati, and fixed point theory (Bensoussan et al., 2014)
• Linear Programming Approaches: occupation measure, relaxed control, martingale equivalence (Dumitrescu et al., 2020, Cohen et al., 11 Apr 2024, Liang et al., 28 Aug 2025)
• Primal-Dual Characterization and Strong Duality (Guo et al., 2 Mar 2025)
• Robust Mean Field Games under Model Uncertainty (Langner et al., 15 Oct 2024)
• Algorithmic/Computational Techniques: Value iteration, interior point methods, primal-dual methods (Saldi, 2023, Guo et al., 2022, Anahtarci et al., 2019).

This integrated framework enables rigorous analysis, existence and characterization results, and efficient computation for mean field game equilibria—even in technically challenging settings involving non-standard dynamics, relaxed controls, multiple equilibria, or robustness requirements.