Mean-Field Stochastic LQR Control

• MF-SLQR is a framework that generalizes classical LQR by incorporating mean-field terms in both state dynamics and cost, addressing large-population systems.
• The technique employs coupled Riccati equations to derive optimal linear state feedback that penalizes deviations and the ensemble mean concurrently.
• Computational approaches using semidefinite programming reformulate the MF-SLQR problem into LMIs, ensuring stability and enabling efficient numerical solutions.

Mean-Field Stochastic Linear-Quadratic Regulator (MF-SLQR)

The mean-field stochastic linear-quadratic regulator (MF-SLQR) is a framework in which optimal control is sought for systems governed by linear stochastic differential or difference equations, with cost functionals and/or dynamics involving both the state and its mean (i.e., average over an ensemble or population). MF-SLQR arises in large-population systems, collective control, and stochastic games, and generalizes the classical stochastic LQR by penalizing the variance as well as the mean and allowing the dynamics to be influenced by mean-field terms. The archetype MF-SLQR problem features coupled Riccati equations, mean-field SDEs (or difference equations), and admits optimal controls in the form of linear state feedbacks that depend simultaneously on the instantaneous state and its (conditional or unconditional) mean.

1. Problem Formulation and System Structure

A canonical continuous-time, infinite-horizon MF-SLQR problem consists of the dynamics

$$dX(t) = [A X(t) + \bar{A} \mathbb{E}[X(t)] + B u(t) + \bar{B} \mathbb{E}[u(t)]]\,dt + [C X(t) + \bar{C} \mathbb{E}[X(t)] + D u(t) + \bar{D} \mathbb{E}[u(t)]]\,dW(t),$$

where $X(t) \in \mathbb{R}^n$ is the state, $u(t) \in \mathbb{R}^m$ is the control, and all coefficients are constant matrices. The cost for an admissible control $u(\cdot)$ is

$$J(x;u) = \mathbb{E} \int_0^\infty \left[ X(t)^\top Q X(t) + \mathbb{E}[X(t)]^\top \bar{Q} \mathbb{E}[X(t)] + u(t)^\top R u(t) + \mathbb{E}[u(t)]^\top \bar{R} \mathbb{E}[u(t)] \right] dt,$$

where $Q,\bar{Q} \in \mathbb{S}^n$, $R,\bar{R}\in\mathbb{S}^m$ with $Q\ge0,\ Q+\bar Q\ge0,\ R>0,\ R+\bar R>0$ to ensure coercivity and admissibility (Huang et al., 2012).

This extends naturally to discrete-time settings, finite-horizon cases, and systems with regime-switching or random coefficients, always with the characterizing feature that both the state evolution and the cost depend on the instantaneous mean of the state and/or control (Elliott et al., 2013, Ahmadova et al., 2022, Mei et al., 1 Jan 2025).

2. Solvability Conditions and Riccati Equations

The optimal control law for MF-SLQR is governed by a system of two coupled algebraic Riccati equations (AREs) in the infinite-horizon case, or difference/differential Riccati equations in discrete/finite-horizon settings. For the continuous-time infinite-horizon case, these are: $$\begin{aligned} 0 &= PA + A^\top P + C^\top P C + Q - (PB + C^\top P D) (R + D^\top P D)^{-1} (B^\top P + D^\top P C), \ 0 &= \Pi (A+\bar A) + (A+\bar A)^\top \Pi + (C+\bar C)^\top P (C+\bar C) + Q+\bar Q \ &\quad - \left[\Pi (B+\bar B) + (C+\bar C)^\top P (D+\bar D)\right] [R+\bar R + (D+\bar D)^\top P (D+\bar D)]^{-1} \ &\quad \times [(B+\bar B)^\top \Pi + (D+\bar D)^\top P (C+\bar C)]. \end{aligned}$$ Solvability requires mean-field $L^2$-stabilizability of both the ODE and SDE pairs and the uniform positive-definiteness of the input weighting matrices (Huang et al., 2012, Elliott et al., 2013). In the discrete-time context, analogous coupled backward recursions appear.

The coupled AREs are equivalent to the feasibility and optimality conditions of a semidefinite programming (SDP) problem involving two block matrix LMIs (Huang et al., 2012). This reduction is constructive for both theoretical analysis and numerical solution.

Extensions to regime-switching systems lead to a set of $X(t) \in \mathbb{R}^n$0 coupled AREs (fluctuation and mean for each regime) with additional terms involving the generator of the Markov chain (Mei et al., 1 Jan 2025).

3. Optimal Control Law: Feedback Representation

Upon solvability of the Riccati system, the unique optimal control law is linear in the state and its mean: $X(t) \in \mathbb{R}^n$1 where: $X(t) \in \mathbb{R}^n$2

$X(t) \in \mathbb{R}^n$3

This structure is preserved in discrete time, where the gains are recursively indexed by the time step (Elliott et al., 2013, Ahmadova et al., 2022). If the coefficients include regime-switching, the gains themselves are indexed by the current regime, switching according to the underlying Markov process (Mei et al., 1 Jan 2025).

The mean-field LQR thus realizes a two-layer feedback—the first acting on the deviation from the mean (fluctuation), the second acting on the mean itself—fully characterizing the optimal closed-loop control in terms of the Riccati solutions.

4. Analysis of Stability and Value Function

The resulting closed-loop mean-field SDE is exponentially mean-square stable when the coupled AREs are solvable with $X(t) \in \mathbb{R}^n$4 and the stabilizability conditions are met. The minimal value of the cost is quadratic in the initial state: $X(t) \in \mathbb{R}^n$5 where $X(t) \in \mathbb{R}^n$6 is the Riccati solution associated with the mean (Huang et al., 2012, Ahmadova et al., 2022).

In the presence of regime switching, exponential $X(t) \in \mathbb{R}^n$7-stability of the controlled system is still tied to the regime-indexed Riccati solution (Mei et al., 1 Jan 2025).

SDP-based analysis provides a certificate of both stability and optimality, as the feasibility of the LMI system is equivalent to the existence of a stabilizing Riccati solution (Huang et al., 2012).

5. Computational Aspects and SDP/LMI Reformulation

Analytically, the Riccati equations can be challenging to solve, especially in high-dimensional settings or when regime switching, random coefficients, or delay are present. The SDP/LMI equivalence provides a practical route for solution: $X(t) \in \mathbb{R}^n$8 This reframes the Riccati system as a convex optimization problem, accessible to modern semidefinite programming solvers. The complementary slackness (zero-Schur-complement condition) yields the Riccati equations themselves (Huang et al., 2012).

For mean-field LQRs with regime switching, the computational framework is extended to several coupled LMIs, one per regime (Mei et al., 1 Jan 2025).

6. Special Cases and Research Extensions

The MF-SLQR theory specializes to:

• Classical LQR: $X(t) \in \mathbb{R}^n$9 recovers the usual single Riccati equation and purely decentralized feedback (Huang et al., 2012).
• Discrete-Time MF-SLQ: Discrete analogues with coupled difference Riccati recursions, ensuring mean-square stability under similar conditions (Elliott et al., 2013, Ahmadova et al., 2022).
• Regime Switching and Random Environments: Systems with coefficients modulated by exogenous Markov chains; the optimal controller switches gains in synchrony with the current regime (Mei et al., 1 Jan 2025), or with random coefficients driven by external filtrations.
• Delayed and Large-Scale Social Optimization: MF-SLQR with input and output delays, distributed agents, and delayed mean-field couplings, leading to anticipated FBSDEs and decentralized laws (Nie et al., 2023).
• Adaptive and Learning-Based MF-SLQR: Extensions for parameter uncertainty, learning, or partially model-free implementations, integrating data-driven estimation with Riccati-based policy synthesis.

Research directions include well-posedness under weaker stabilizability, extension to partial observation, game-theoretic settings (mean-field games), and applications in decentralized control of networks.

---

References

• (Huang et al., 2012) "A Linear-Quadratic Optimal Control Problem for Mean-Field Stochastic Differential Equations in Infinite Horizon"
• (Elliott et al., 2013) "Discrete Time Mean-Field Stochastic Linear-Quadratic Optimal Control Problems"
• (Ahmadova et al., 2022) "Mean-field type discrete stochastic linear quadratic optimal control problems"
• (Mei et al., 1 Jan 2025) "Linear-Quadratic Optimal Control for Mean-Field Stochastic Differential Equations in Infinite-Horizon with Regime Switching"
• (Nie et al., 2023) "Linear-Quadratic Delayed Mean-Field Social Optimization"