Optimal Coarse Correlated Equilibria in Mean Field Games: Linear Programming and No-Regret Learning

Abstract: We introduce optimal coarse correlated equilibria for continuous-time mean field games. A coarse correlated equilibrium is a randomized recommendation scheme from which no player can gain by ignoring the recommendation and switching to an alternative strategy. The problem is as follows: a moderator selects, among all mean-field coarse correlated equilibria, one that optimizes a prescribed performance criterion, which may differ from the representative player's objective. After formulating the problem, we develop a linear programming (LP) formulation, prove the existence of optimal LP coarse correlated equilibria, and relate the LP characterization to the original probabilistic setting. Building on this characterization, we design a no-regret primal-dual algorithm, based on an equivalent Lagrangian formulation of the external-regret constraint, for learning such equilibria. We provide explicit convergence rates for the learning algorithm, and numerical examples illustrate the method.

• The paper introduces a linear programming formulation for optimal coarse correlated equilibria in mean field games, providing rigorous existence and structural proofs.
• It proposes a no-regret primal-dual algorithm with explicit nonasymptotic convergence rates, demonstrated through mean-field flocking and emission abatement models.
• The research bridges external-regret learning with convex analysis, enabling practical neural implementations for coordinating large-scale continuous-time agent systems.

Optimal Coarse Correlated Equilibria in Mean Field Games: LP and No-Regret Learning

Introduction and Motivation

The paper "Optimal Coarse Correlated Equilibria in Mean Field Games: Linear Programming and No-Regret Learning" (2606.20062) advances the analysis of equilibrium concepts beyond the classical Nash paradigm in mean field games (MFG). Motivated by both theoretical considerations and practical applications—where a social planner or moderator can issue recommendations to an infinite population of agents with negligible individual power—the authors focus on mean-field analogues of coarse correlated equilibria (CCE). Unlike Nash or Aumann correlated equilibria, CCE allow a central correlation device to recommend randomized strategies, which agents may choose to ignore without the benefit of seeing the recommended realization ("ex ante" deviation).

The contributions of the paper are twofold. First, it provides a functional-analytic and occupation-measure-based linear programming (LP) framework for optimal MFG-CCEs, characterizing existence and structure, and relating to both the classical probabilistic and martingale formulations of mean-field control. Second, it gives a constructive, implementable no-regret learning algorithm—framed as a primal-dual gradient scheme—for synthesizing optimal CCEs, including explicit nonasymptotic convergence rates. The approach is illustrated via numerical experiments on canonical continuous-time, continuous-state models.

Coarse Correlated Equilibria and Formulations

The probabilistic formulation defines a mean-field CCE as a pair $(\lambda, \mu)$, with $\lambda$ an $\mathbb{F}$-progressive random process (the recommended strategy), and $\mu$ a flow of probability measures encoding both the state and action, which may be random due to the moderator's lottery. The two defining properties are:

• Consistency: At each $t$, the distribution $\mu_t$ equals the conditional law of $(X_t, \lambda_t)$ given the whole flow $\mu$, enforcing that $\mu$'s own distribution is self-consistent under $\lambda$.
• Optimality: The representative agent cannot reduce their expected cost by ignoring the recommendation and adopting any admissible $\lambda$0-adapted deviation $\lambda$1 instead.

The moderator (correlation device) aims to optimize a potentially distinct objective $\lambda$2 over the set of all (mean-field) CCE. This selection problem is crucial given the potential multiplicity or abundance of CCEs.

A central innovation is the translation of this probabilistic problem into an LP over occupation measures, following the path laid by Kurtz, Stockbridge, and recent MFG literature. Here, CCE are characterized as (possibly randomized) measures on the space of state-action occupation flows, satisfying linear martingale equations (for the dynamics), and an infinite family of linear inequalities (for external regret against deviations). The problem is lifted to a convex and compact set, enabling application of convex analytic machinery.

Existence, LP Formulation, and Equivalence

The authors prove that under standard regularity and compactness assumptions (especially on the action set and controlled coefficients), optimal LP-CCEs exist for general performance criteria. The existence argument leverages convexity and tightness properties—inheritances of the occupation measure formalism—together with the continuity of the LP representation of costs.

A key technical step is the analysis of the relation between the LP and probabilistic formulations. Under convexity-of-relaxation or $\lambda$3-independence of the drift, they establish that any LP-CCE can be represented as a probabilistic mean-field CCE with matching moderator value, and (under additional hypotheses) the set of optimal CCEs coincides in both frameworks.

No-Regret Primal-Dual Algorithm and Convergence Analysis

To address practical computation and learning of LP-CCEs, the authors recast the selection problem as a constrained optimization:

$\lambda$4

where $\lambda$5 is the moderator's objective functional ($\lambda$6 denotes a correlated occupation measure), and $\lambda$7 is external regret—the maximum gain a representative agent could realize by switching, ex ante, to a fixed deviation strategy.

This yields a convex-concave saddle-point problem amenable to primal-dual approaches. The paper proposes a regularized iterative scheme using Bregman divergence, alternating between:

• Primal updates: Over LP-CCE occupation measures, minimizing a Bregman-regularized Lagrangian (objective plus lagrange-penalized regret).
• Dual updates: Adjusting the regret multiplier in response to observed regret in the current iterate.

A rigorous convergence proof is provided. The authors derive that averaged iterates converge to a saddle point of the Lagrangian, i.e., an optimal LP-CCE, with the averaged external regret and duality gap decaying as $\lambda$8. This rate is optimal among (stochastic) no-regret algorithms for similar variational inequalities.

Parametrization and Neural Implementation

Recognizing the infinite-dimensional nature of the occupation measure optimization, the authors introduce a finite-dimensional parametrization scheme compatible with practical computation. The primal variables are neural network weights for randomized Markovian policies and parameters controlling the correlation device (the moderator's randomization). The induced occupation flow is then constructed by integrating the Fokker–Planck equations corresponding to these parametrized policies.

Algorithmic updates are performed via (stochastic) gradient descent (Adam optimizer) for neural parameters and projected gradient ascent for the regret multipliers, with cost and regret terms estimated via Monte Carlo and PDE integration. This design allows scalable application to complex continuous-time agent models.

Numerical Illustrations

Mean-Field Flocking System

In a flocking control model with quadratic costs and state/mean interaction, the algorithm rapidly converges to the mean-field Nash solution with negligible residual external regret. The computed CCE, as expected in this setting, does not exploit additional correlation—matching classic linear-quadratic MFG theoretical predictions.

Figure 1: Approximation of the solution to the mean-field flocking system. The average cost produced by the learning algorithm matches the mean-field Nash value, with regret remaining consistently negative.

Emission Abatement Game

In an LQ emission abatement model, the approach computes explicit equilibria where the CCE achieves higher individual payoff than the Nash equilibrium, but with lower aggregate emission abatement—a nontrivial trade-off. When the moderator's objective shifts to maximizing terminal abatement rather than payoff, the learning process produces equilibria that indeed prioritize the desired aggregate outcome, at the cost of individual payoff.

Figure 2: Computed optimal CCE, external regret, and terminal abatement values in the emission abatement game, showing improved agent payoffs relative to Nash, with regret maintained below zero.

Figure 3: Approximation of a CCE that maximizes terminal emission abatement, trading off individual payoff for higher aggregate performance.

Notably, these results demonstrate the capability of the approach to interpolate between agent-centric and centrally-coordinated optima beyond the Nash regime, depending on the moderator's prescribed objective.

Implications and Future Directions

The establishment of an LP-based existence and learning framework for MFG-CCEs enables both theoretical analysis and practical computation of correlated equilibria that can outperform Nash in terms of welfare or public good provision. The saddle-point algorithm, coupled with neural parametrization and occupation measure representation, opens the way for applying these ideas to large-scale, continuous-time, and continuous-state-agent populations, potentially informing mechanism design, Stackelberg control, or decentralized incentive structures.

Theoretically, this work connects mean-field game analysis with the broader convex-analytic and external-regret learning literature, and suggests further exploration of equilibrium refinement, uniqueness, and the convergence of other dynamics (e.g., fictitious play) in mean-field contexts.

Practically, the framework paves the way for training coordination devices—realizable as neural networks or algorithmic moderators—that robustly steer populations toward designated global objectives under decentralized compliance, a relevant problem in economics, regulation, and engineered systems.

Conclusion

This paper delivers a comprehensive variational and algorithmic theory for selecting and computing optimal coarse correlated equilibria in continuous-time mean-field games. By leveraging a linear programming occupation-measure formalism and a provably convergent primal-dual no-regret algorithm, the authors demonstrate both theoretical guarantees and practical efficacy in computing CCEs optimized for arbitrary moderator objectives. The approach is validated on canonical models, with the ability to synthesize equilibria unattainable by Nash, highlighting both mathematical depth and applied potential in large-population strategic models.