---
title: Mean-Field Stochastic LQR Controller
url: https://www.emergentmind.com/topics/mean-field-stochastic-lqr-mf-slqr-controller
type: topic
---

# Mean-Field Stochastic LQR Controller

A mean-field stochastic linear quadratic regulator (MF-SLQR) controller generalizes the classical LQR approach to systems with both individual-state and mean-field (population-averaged) interactions under stochastic uncertainty, and further constrains state fluctuations to address risk. Such controllers are crucial in high-dimensional multi-agent networks, power grids, and large-scale coupled stochastic systems, particularly when low-probability, high-impact events must be systematically attenuated rather than averaged away as in the risk-neutral setting.

## 1. Problem Formulation: Dynamics, Cost, and Risk Constraint

Consider $n$ exchangeable agents indexed by $i$ with individual state $x_t^i \in \mathbb{R}^{d_x}$ and control $u_t^i \in \mathbb{R}^{d_u}$. Each agent's dynamics incorporate both local and mean-field coupling:
\[
x_{t+1}^i = A\,x_t^i + B\,u_t^i + \bar{A}\,\bar{x}_t + \bar{B}\,\bar{u}_t + w_t^i,
\]
where $\bar{x}_t = \frac{1}{n}\sum_{j=1}^n x_t^j$, $\bar{u}_t = \frac{1}{n}\sum_{j=1}^n u_t^j$, and $w_t^i$ is an i.i.d. zero-mean noise sequence. In the $n\to\infty$ mean-field limit, agent states decompose into fluctuation and mean components:
\[
\begin{aligned}
&\tilde{x}^i_{t+1} = A\,\tilde{x}_t^i + B\,\tilde{u}_t^i + \tilde{w}_t^i, \\
&\bar{x}_{t+1} = (A+\bar{A})\,\bar{x}_t + (B+\bar{B})\,\bar{u}_t + \bar{w}_t,
\end{aligned}
\]
where $\tilde{x}_t^i = x_t^i - \bar{x}_t$, $\tilde{u}_t^i = u_t^i - \bar{u}_t$.

The infinite-horizon average quadratic cost per agent (risk-neutral) is
\[
J = \limsup_{T\to\infty} \frac{1}{T} \mathbb{E}\!\left[ \sum_{t=0}^{T-1} \left( (x_t^i)^\top Q x_t^i + (u_t^i)^\top R u_t^i + \bar{x}_t^\top \bar{Q} \bar{x}_t + \bar{u}_t^\top \bar{R} \bar{u}_t \right) \right].
\]
To control rare but impactful fluctuations, a variance-type risk constraint is imposed. Define
\[
d_t^i = \left( (x_t^i)^\top Q x_t^i - \mathbb{E}\left[(x_t^i)^\top Q x_t^i | h_t^i \right] \right)^2,
\]
where $h_t^i$ denotes agent $i$'s history. The per-player time-averaged variance is
\[
J_c = \frac{1}{n}\sum_{i=1}^n \limsup_{T\to\infty} \frac{1}{T} \mathbb{E}\left[ \sum_{t=0}^{T-1} d_t^i \right].
\]
The MF-SLQR controller seeks to
\[
\begin{aligned}
&\min_{\{u_t^i\}}~~ J \\
&\text{s.t.}~~ (1),~~ J_c \leq \Gamma~,
\end{aligned}
\]
with risk budget $\Gamma > 0$ [2307.07129].

## 2. Lagrangian Dual Formulation and Decomposition

A Lagrange multiplier $\lambda \geq 0$ combines the nominal and risk costs:
\[
\mathcal{L} = J + \lambda (J_c - \Gamma).
\]
Owing to orthogonal mean–fluctuation decomposition, $\mathcal{L}$ decouples into two independent infinite-horizon LQR objectives:
\[
\mathcal{L} = \underbrace{\limsup_{T \to \infty} \frac{1}{T} \mathbb{E} \left[ \sum_t \tilde{x}_t^{i\top} Q_\lambda \tilde{x}_t^i + \tilde{u}_t^{i\top} R_\lambda \tilde{u}_t^i \right]}_{\text{fluctuation-LQR}}
+
\underbrace{\limsup_{T \to \infty} \frac{1}{T} \mathbb{E} \left[ \sum_t \bar{x}_t^\top \mathcal{Q}_{\bar\lambda} \bar{x}_t + \bar{u}_t^\top \mathcal{R}_{\bar\lambda} \bar{u}_t \right]}_{\text{mean-LQR}},
\]
where
\[
Q_\lambda = Q + \frac{4\lambda}{n} Q M_2 Q, \quad \mathcal{Q}_{\bar\lambda} = Q + \bar{Q} + 4\lambda Q M_2 Q, \quad R_\lambda = R,~\mathcal{R}_{\bar\lambda} = R + \bar{R}
\]
and $M_2 = \limsup_{T\to\infty} \frac{1}{T} \mathbb{E} \sum_{t=0}^{T-1}\left(x_t^\top Q x_t - \mathbb{E}[x_t^\top Q x_t]\right)^2$. The dependence on $\lambda$ drives risk sensitivity [2307.07129].

## 3. Coupled Riccati Equations and Controller Synthesis

Both mean and fluctuation subsystems admit closed-form LQR solutions. The optimal value of $\lambda = \lambda^*$ is enforced via a primal–dual algorithm.

#### Riccati Equations
\[
\begin{aligned}
P &= Q_\lambda + A^\top P A - A^\top P B (R + B^\top P B)^{-1} B^\top P A, \\
\mathcal{P} &= \mathcal{Q}_{\bar\lambda} + (A+\bar{A})^\top \mathcal{P} (A+\bar{A}) \\
& \qquad - (A+\bar{A})^\top \mathcal{P} (B+\bar{B}) [\mathcal{R}_{\bar\lambda} + (B+\bar{B})^\top \mathcal{P} (B+\bar{B})]^{-1} (B+\bar{B})^\top \mathcal{P} (A+\bar{A}).
\end{aligned}
\]
These equations admit a unique positive semidefinite stabilizing solution under standard MF-LQR stabilizability/detectability criteria [2307.07129].

#### Controller Law

The optimal state-feedback is affine in the agent's own state and in the population mean:
\[
u_t^{i,*} = -K x_t^i - (\bar{K} - K)\bar{x}_t + k_0,
\]
with
\[
K = (R + B^\top P B)^{-1} B^\top P A, \qquad
\bar{K} = [\mathcal{R}_{\bar\lambda} + (B+\bar{B})^\top \mathcal{P} (B+\bar{B})]^{-1} (B+\bar{B})^\top \mathcal{P} (A+\bar{A}),
\]
and $k_0$ arises if the noise is nonzero-mean in the dual formulation but is zero otherwise [2307.07129]. The feedback structure matches that of classical mean-field LQR, but the gain matrices internalize the variance penalty via the modified cost matrices.

## 4. Risk Constraint Enforcement and Computational Aspects

The optimal Lagrange multiplier $\lambda^*$ is computed via a primal–dual loop. At each iteration:
1. Update $Q_\lambda$, $\mathcal{Q}_{\bar\lambda}$;
2. Solve the two Riccati equations for $P$, $\mathcal{P}$;
3. Form $K$, $\bar{K}$, $k_0$;
4. Simulate closed-loop dynamics and evaluate $J_c(\lambda)$;
5. Update $\lambda$ by a projected subgradient step:
\[
\lambda_{k+1} = [\lambda_k + \eta (J_c(\lambda_k) - \Gamma)]_+.
\]
Each iteration requires $O(d_x^3)$ operations (Riccati/Lyapunov equations) and is independent of the agent population $n$. The resulting gains do not depend on $n$, which ensures scalability [2307.07129].

## 5. Structural Properties and Solution Comparison

- **Independence from Number of Players**: All Riccati equations and controller gains depend only on local and mean-field coupling ($A, B, \bar{A}, \bar{B}$) and cost weights, not on $n$. This makes the approach viable for large-scale networks [2307.07129].

- **Existence and Uniqueness**: Unique stabilizing Riccati solutions and dual multipliers $\lambda^*\geq 0$ are guaranteed by standard LQR system-theoretic conditions and strong duality [2307.07129].

- **Risk Parameter Influence**: As $\Gamma$ is decreased (tighter variance constraint), $\lambda^*$ increases, which modifies $Q_\lambda$, leading to more conservative $K, \bar{K}$ (higher-magnitude feedback gains). This reduces the amplitude of risky excursions (e.g., overshoots) at the cost of modestly increased average cost $J$ [2307.07129].

- **Comparison to Risk-Neutral MF-LQR**: Setting $\lambda=0$ recovers the standard mean-field LQR controller, which ignores state variance [2307.07129].

## 6. Practical Example and Performance Analysis

A microgrid frequency control scenario is used as a high-dimensional case study. Each area (agent) maintains a $4$-dimensional state including local frequency, generation, tie-line flow, and area control error integral. System matrices $A, \bar{A}, B$ are drawn from standard load frequency control (LFC) parameters and quadratic costs $Q, R$ are assigned.

`Table: Impact of Variance Constraint on Controller Behavior`
| $\Gamma$ (risk tolerance) | $\lambda^*$ (dual) | Gain Magnitude | Overshoot | Average Cost $J$ |
|--------------------------|--------------------|---------------|-----------|------------------|
| High                     | $\approx 0$        | Baseline      | Large     | Baseline         |
| Moderate                 | Moderate           | Increased     | Reduced   | Slightly higher  |
| Low                      | Large              | Largest       | Minimal   | Highest          |

Reducing $\Gamma$ leads to increased $\lambda^*$ and higher feedback gain magnitude, yielding faster disturbance damping and less vulnerability to rare high-variance events [2307.07129].

## 7. Generalizations and Methodological Significance

Risk-constrained MF-SLQR controllers extend classic LQR and mean-field LQR by systematically regulating not only expected cost but also rare-event risk, via variance-type constraints. The affine law and dual-based synthesis parallel but augment the classical mean–fluctuation separation and are compatible with standard Riccati-based implementation. Scalability and algorithmic simplicity are preserved, and the general methodology readily extends to other risk proxies and limit regimes [2307.07129].

These contributions are directly relevant for modern applications requiring explicit robustness to stochastic volatility and population-level coupling, such as large-scale energy systems and networked autonomous agents.

Source: https://www.emergentmind.com/topics/mean-field-stochastic-lqr-mf-slqr-controller