---
title: Ergodic LQ Closed-Loop Optimal Control
url: https://www.emergentmind.com/topics/ergodic-linear-quadratic-closed-loop-optimal-control
type: topic
---

# Ergodic LQ Closed-Loop Optimal Control

Ergodic linear-quadratic (LQ) closed-loop optimal control addresses the synthesis and analysis of control laws that minimize long-run average (ergodic) quadratic performance criteria for linear stochastic systems, possibly with nontrivial mean-field (distribution-dependent) and time-varying or random coefficients. The ergodic regime focuses on steady-state or long-term average behavior, contrasting with discounted or finite-horizon models, and its mathematical underpinnings lie in stochastic control theory, Riccati equations, and invariant measure analysis. This field encompasses advances in theory for mean-field dynamics, systems with uncertainty, periodic/random coefficients, risk-sensitive objectives, as well as robustness under structural uncertainty.

## 1. Fundamental Formulation of the Ergodic LQ Closed-Loop Problem

At the core, the ergodic LQ closed-loop problem seeks an admissible control law that minimizes an ergodic cost functional associated with a controlled stochastic differential equation (SDE):

- **State equation** (general, possibly mean-field):  
  $$ 
  dx_t = [A\,x_t + \bar A\,m_t + B\,u_t + b]\,dt + [C\,x_t + \bar C\,m_t + D\,u_t + \sigma]\,dW_t,\quad m_t = \mathbb{E}[x_t]
  $$
  with $x_t \in \mathbb{R}^n$, $u_t \in \mathbb{R}^m$, $W_t$ a standard Brownian motion.

- **Quadratic cost functional** (possibly mean-field):  
  $$
  J_{\rm erg}(u) = \limsup_{T\to\infty} \frac1{T} \mathbb{E} \int_0^T \bigl[ x_t^\top Q x_t + u_t^\top R u_t + 2x_t^\top S u_t + 2q^\top x_t + 2r^\top u_t + m_t^\top \bar Q m_t \bigr]\,dt
  $$

- **Admissibility** demands uniform moment bounds and the existence of a limiting ergodic law (e.g., a stationary distribution in the $W_2$ metric) [2502.08935], [2004.11088].

The ergodic framework is robust to transients: the cost is determined by the long-term, steady-state interplay between the process dynamics and feedback law, possibly involving invariant or periodic measures.

## 2. Algebraic Riccati Equations and Verification

The closed-loop optimal synthesis hinges on the solution of (generalized) algebraic Riccati equations (AREs) that arise from the stationary Hamilton–Jacobi–Bellman (HJB) equation:

- **Quadratic value function ansatz** yields coupled Riccati equations for the state and mean-field terms:
  $$
  A^\top P + P A + C^\top P C + Q - (P B + C^\top P D + S^\top) (R + D^\top P D)^{-1} (B^\top P + D^\top P C + S) = 0
  $$
  with analogous equations for mean-field (e.g., $\Pi$), affine terms ($P_1$, $p$, $p_1$), and ergodic value $c_0$ [2502.08935], [2004.11088], [2505.05296].

- **Random/periodic coefficients** generalize the Riccati equation to *backward stochastic Riccati equations* (BSREs) or periodic Riccati ODEs. Existence and uniqueness in the random periodic setting are obtained via monotone BSDE iteration and $L^2$-contraction [2601.08672].

- **Finiteness and admissibility** may hold even with indefinite weights, under suitable Riccati inequalities. Existence of a stabilizing solution implies the ergodic problem is well-posed [2004.11088].

## 3. Structure of the Ergodic LQ Optimal Feedback

The optimal closed-loop control is linear (affine if nonhomogeneous) in the state and, for mean-field systems, includes a term dependent on the mean:

- **General form (mean-field LQ):**
  $$
  u_t^* = \Theta^* (x_t - m_t) + \bar\Theta^* m_t + \theta^*
  $$
  with explicit formulas for gains:
  $$
  \Theta^* = - (R + D^\top P D)^{-1}(B^\top P + D^\top P C + S)
  $$
  $$
  \bar\Theta^* = - (R + D^\top P D)^{-1}(B^\top \Pi + D^\top P \bar C + S)
  $$
  and
  $$
  \theta^* = - (R + D^\top P D)^{-1}(B^\top p + D^\top P \sigma + r)
  $$
  [2502.08935], [2505.05296]. In purely homogeneous cases ($C = \bar C = \sigma = b = q = r = 0$), the feedback specializes to classical LQ structure [2502.08935], [2004.11088].

- **For random periodic coefficients**, the optimal law becomes periodic in $t$:  
  $$
  u^*(t) = -R(t)^{-1}[B(t)^\top P(t) + D(t)^\top P(t)C(t)]X(t) - R(t)^{-1}[B(t)^\top \eta(t) + \rho(t)]
  $$
  where $P(t), \eta(t)$ are solutions to periodic BSREs and BSDEs [2601.08672].

- **Stationarity and Invariant Measures:** When the closed-loop is mean-square stabilizing, an invariant Gaussian measure exists, with the cost evaluated via this measure [2004.11088], [2505.05296].

## 4. Asymptotic Analysis: Equivalence, Turnpike, and Invariant Measures

A distinctive trait of ergodic LQ control is the equivalence between the infinite time-averaged cost and the expectation with respect to an invariant or periodic measure:

- **Reduction to one-period average**: For periodic (or random periodic) coefficients, the ergodic cost over $[0,\infty)$ reduces to an expectation over a single period, weighted by the invariant (periodic) measure [2601.08672], [2505.05296].

- **Turnpike property**: The finite-horizon LQ optimal state-control pair converges exponentially fast to the ergodic (infinite-horizon) solution, except near initial and terminal boundary layers. Quantitatively, for $t$ away from boundaries,
  $$
  \mathbb{E}[|X_T(t)-\bar X(t)|^2 + |u_T(t)-\bar u(t)|^2] \leq K' (e^{-\lambda' t} + e^{-\lambda'(T-t)})
  $$
  This demonstrates transient region localization and long-run stability [2502.08935].

- **Convergence of Riccati trajectories:** Time-varying Riccati equations for the finite-horizon problem converge exponentially to their algebraic (ergodic) limits [2502.08935].

- **Invariant and periodic measures for state laws:** Under stabilizing feedback, Markov transition kernels contract in Wasserstein distance to unique invariant or periodic measures, capturing steady-state behavior [2505.05296].

## 5. Extensions: Mean-Field, Periodic, Random, and Adaptive Ergodic LQ Control

Recent work generalizes classical ergodic LQ theory in several directions.

- **Mean-field systems:** Coupling through the empirical mean of the state introduces additional Riccati equations, periodic measure structures, and requires completion of the square in both state and mean [2502.08935], [2505.05296].

- **Random periodic and stochastic coefficients:** Ergodic theory for random periodic stochastic systems relies on random periodic mean-square exponential stability. Backward stochastic Riccati equations with random periodicity yield time-varying, pathwise-periodic feedback gains and cost representations over a single period [2601.08672].

- **Indefinite weights:** Ergodic LQ optimal control remains well-posed under indefinite $Q$ and $R$ if suitable Riccati inequalities admit solutions. Regularization with an $\varepsilon$-penalized cost (for small $\varepsilon > 0$) yields existence and computes the minimal ergodic value as $\varepsilon \to 0$ [2004.11088].

- **Adaptive LQ control:** In the presence of unknown system matrices, certainty-equivalence adaptive control with appropriate weighted least-squares estimation, random regularization, and diminishing excitation achieves ergodic optimality. The key is that parameter estimation in the cost-relevant subspace is consistent, and the diminishing exploration noise has vanishing contribution to the ergodic cost [2409.09250].

## 6. Risk Constraints, Robustness, and Numerical Performance

Recent developments address the robustification of ergodic LQ policies to unpredictable or heavy-tailed noise environments:

- **Ergodic-risk criteria** quantify long-term variance or tail risk by considering limits of martingale increments associated with risk functionals. Asymptotic variance of empirical costs admits closed-form expressions given the invariant distribution [2503.05878].

- **Constrained LQR synthesis** incorporates a bound on the asymptotic risk, formulating a constrained optimization over the space of stabilizing linear feedbacks, with solutions characterized via modified discrete algebraic Riccati equations and resolved through primal-dual algorithms [2503.05878].

- **Stability and functional CLT:** Exponential mean-square stability of the closed-loop system ensures unique invariant measures, bounded second moments, and enables functional central limit theorems (for long-run empirical costs) [2503.05878].

- **Numerical examples** on high-dimensional aerospace models demonstrate that optimal ergodic risk-constrained controllers suppress large noise-induced state excursions at minimal increase in average cost [2503.05878].

---

**References:**
- [2601.08672]: Ergodic LQ control with random periodic coefficients
- [2505.05296]: Ergodic LQ mean-field control with periodic coefficients
- [2502.08935]: Ergodicity and turnpike for mean-field LQ control
- [2409.09250]: Adaptive ergodic LQ control
- [2503.05878]: Ergodic-risk constrained LQR
- [2004.11088]: Classical ergodic LQ with indefinite weights

All detailed formulas, proof structures, and theoretical advances above can be found in the cited references. The ergodic LQ closed-loop control literature continues to expand, addressing increasingly complex dynamics, information structures, and performance objectives.

Source: https://www.emergentmind.com/topics/ergodic-linear-quadratic-closed-loop-optimal-control