Ergodic-Risk Constraints in Control

• Ergodic-risk constraints are long-run risk measures that replace average-cost metrics with specifications based on log-moment growth rates or asymptotic fluctuation variance.
• They transform control problems into multiplicative dynamic programming and nonlinear eigenvalue formulations, thereby addressing variability and tail behavior.
• This framework integrates duality and stability conditions to design robust controllers that perform well even under heavy-tailed noise or extreme deviations.

Ergodic-risk constraints are long-run control constraints in which admissibility is specified by an asymptotic risk quantity rather than by an ordinary average-cost bound. In the classical ergodic risk-sensitive control literature, the relevant quantity is a logarithmic exponential growth rate,

$$J_x(c,\zeta) = \limsup_{T\to\infty}\frac{1}{\gamma T} \log \mathbb{E}_x^\zeta\!\left[\exp\!\left(\int_0^T \gamma\, c(X_t,\zeta_t)\,dt\right)\right],$$

or its discrete-time analogue, so that both the objective and the constraint can be posed as long-run log-moment functionals. More recent work also uses the term for constraints on the asymptotic fluctuation level of a risk functional, typically through a functional central limit theorem and an asymptotic conditional variance. In both cases, the central theme is that the constraint regulates long-run variability, tail behavior, or cumulative uncertainty, rather than only mean performance (Biswas et al., 2022, Talebi et al., 2024, Talebi et al., 10 Feb 2025).

1. Classical ergodic risk-sensitive formulation

Ergodic risk-sensitive control is an infinite-horizon stochastic control framework in which performance is measured not by an ordinary average cost, but by an exponential-of-integral criterion. In the ergodic setting, the horizon $T\to\infty$, and the value is normalized by $T$, leading to a long-run growth rate. For discrete-time controlled Markov chains, the corresponding criterion is

$$J_x(c,\zeta) = \limsup_{T\to\infty}\frac{1}{\gamma T} \log \mathbb{E}_x^\zeta\!\left[\exp\!\left(\sum_{t=0}^{T-1}\gamma\, c(X_t,\zeta_t)\right)\right],$$

while for controlled diffusions the same form appears with the time integral. The sign of $\gamma$ encodes risk attitude: $\gamma>0$ is risk-averse, $\gamma=0$ is the risk-neutral limit recovering classical ergodic control, and $\gamma<0$ is risk-seeking (Biswas et al., 2022).

A major point emphasized in the literature is that the exponential criterion accounts for fluctuations around the mean, unlike a plain average-cost criterion. Two motivations recur. First, a linear average cost ignores fluctuations, whereas exponential criteria weight higher moments and therefore penalize variability and tail behavior. Second, the exponential criterion is multiplicative and therefore supports a multiplicative dynamic programming principle, whereas quadratic mean-variance type objectives often fail to satisfy dynamic programming because they are not time-consistent. This is the structural background from which ergodic-risk constraints emerge (Biswas et al., 2022).

The standard additive ergodic criterion,

$$\limsup_{T\to\infty}\frac{1}{T}\mathbb{E}\!\left[\int_0^T c(X_t,\zeta_t)\,dt\right],$$

is therefore replaced by

$$\limsup_{T\to\infty}\frac{1}{\gamma T} \log \mathbb{E}\!\left[e^{\gamma \int_0^T c\,dt}\right].$$

This changes the optimization from additive to multiplicative, converts the Bellman equation into a nonlinear eigenvalue problem, and naturally penalizes variability and rare bad excursions. In the limit $T\to\infty$0, the risk-sensitive criterion reduces to the classical ergodic cost, at least formally (Biswas et al., 2022).

2. Exponential-growth-rate constraints

The survey literature formulates an explicit constrained problem in which the control minimizes one ergodic risk-sensitive criterion while satisfying another:

$T\to\infty$1

Here the second cost $T\to\infty$2 is constrained through its exponential growth rate. This is an ergodic risk constraint: the constraint is not on the mean of $T\to\infty$3, but on its exponential growth rate. The survey states that this is stronger and more tail-sensitive than an average-cost constraint (Biswas et al., 2022).

The constrained problem is converted into an unconstrained Lagrangian form,

$T\to\infty$4

where $T\to\infty$5 is the Lagrange multiplier and $T\to\infty$6 is the ergodic risk-sensitive game value induced by the Kullback–Leibler variational form. The significance of this transformation is explicit: it turns an ergodic-risk constraint into a standard unconstrained risk-sensitive game or eigenvalue problem. The same treatment yields a corresponding primal linear program with variables $T\to\infty$7, and a dual program over occupation measures, embedding the constrained problem into a convex optimization framework (Biswas et al., 2022).

This constrained formulation is closely tied to the variational representation of exponential moments. In the discrete-time variational formulas, terms of the form

$T\to\infty$8

appear, where the Kullback–Leibler divergence measures the cost of deviating from the nominal transition kernel. The same large-deviation structure underlies the general formula

$T\to\infty$9

Accordingly, ergodic-risk constraints in this sense are inseparable from entropy-penalized robustness: the constraint is imposed on a long-run cumulant generating function, and the resulting optimization admits a robust game interpretation (Biswas et al., 2022).

3. Fluctuation-based ergodic-risk constraints

A second, newer construction defines ergodic risk through the long-run cumulative fluctuation of a measurable risk functional $T$0. The one-step uncertain part relative to past information is

$T$1

or, equivalently,

$T$2

The corresponding ergodic-risk quantity is specified through the normalized cumulative uncertainty

$T$3

together with the asymptotic conditional variance

$T$4

This criterion is not about the mean cost, but about the asymptotic distribution of accumulated fluctuations. It is therefore a long-horizon risk measure aimed at long-term cumulative uncertainty and extreme deviations beyond mean performance (Talebi et al., 2024, Talebi et al., 10 Feb 2025).

In the linear quadratic case, the constrained optimization problem takes the form

$T$5

where $T$6 and

$T$7

For stabilizing linear feedback $T$8, the state covariance satisfies

$T$9

With quadratic risk functionals

$$J_x(c,\zeta) = \limsup_{T\to\infty}\frac{1}{\gamma T} \log \mathbb{E}_x^\zeta\!\left[\exp\!\left(\sum_{t=0}^{T-1}\gamma\, c(X_t,\zeta_t)\right)\right],$$0

the asymptotic conditional variance becomes the risk budget variable. The constraint therefore does not alter the average-cost objective directly; it imposes a robustness specification on the long-run fluctuation level of the risk process (Talebi et al., 10 Feb 2025).

The explicit quadratic formula is central:

$$J_x(c,\zeta) = \limsup_{T\to\infty}\frac{1}{\gamma T} \log \mathbb{E}_x^\zeta\!\left[\exp\!\left(\sum_{t=0}^{T-1}\gamma\, c(X_t,\zeta_t)\right)\right],$$1

with

$$J_x(c,\zeta) = \limsup_{T\to\infty}\frac{1}{\gamma T} \log \mathbb{E}_x^\zeta\!\left[\exp\!\left(\sum_{t=0}^{T-1}\gamma\, c(X_t,\zeta_t)\right)\right],$$2

The corresponding asymptotic normality statement is

$$J_x(c,\zeta) = \limsup_{T\to\infty}\frac{1}{\gamma T} \log \mathbb{E}_x^\zeta\!\left[\exp\!\left(\sum_{t=0}^{T-1}\gamma\, c(X_t,\zeta_t)\right)\right],$$3

when $$J_x(c,\zeta) = \limsup_{T\to\infty}\frac{1}{\gamma T} \log \mathbb{E}_x^\zeta\!\left[\exp\!\left(\sum_{t=0}^{T-1}\gamma\, c(X_t,\zeta_t)\right)\right],$$4, together with

$$J_x(c,\zeta) = \limsup_{T\to\infty}\frac{1}{\gamma T} \log \mathbb{E}_x^\zeta\!\left[\exp\!\left(\sum_{t=0}^{T-1}\gamma\, c(X_t,\zeta_t)\right)\right],$$5

The data explicitly emphasize that this framework handles heavy-tailed process noise as long as the noise has a finite fourth moment, and does not require exponential integrability. In simulations with Student-$$J_x(c,\zeta) = \limsup_{T\to\infty}\frac{1}{\gamma T} \log \mathbb{E}_x^\zeta\!\left[\exp\!\left(\sum_{t=0}^{T-1}\gamma\, c(X_t,\zeta_t)\right)\right],$$6 noise with $$J_x(c,\zeta) = \limsup_{T\to\infty}\frac{1}{\gamma T} \log \mathbb{E}_x^\zeta\!\left[\exp\!\left(\sum_{t=0}^{T-1}\gamma\, c(X_t,\zeta_t)\right)\right],$$7, the risk-constrained controller reduces long-run fluctuation sensitivity by about $$J_x(c,\zeta) = \limsup_{T\to\infty}\frac{1}{\gamma T} \log \mathbb{E}_x^\zeta\!\left[\exp\!\left(\sum_{t=0}^{T-1}\gamma\, c(X_t,\zeta_t)\right)\right],$$8 compared with LQR, while the average cost increases by only about $$J_x(c,\zeta) = \limsup_{T\to\infty}\frac{1}{\gamma T} \log \mathbb{E}_x^\zeta\!\left[\exp\!\left(\sum_{t=0}^{T-1}\gamma\, c(X_t,\zeta_t)\right)\right],$$9 (Talebi et al., 10 Feb 2025).

4. Dynamic programming, eigenvalue structure, and duality

In classical ergodic risk-sensitive control, the optimality equations are nonlinear eigenvalue problems rather than additive Bellman equations. For a controlled discrete-time chain with transition kernel $\gamma$0, the optimality equation is

$\gamma$1

for $\gamma$2. For a controlled diffusion with generator

$\gamma$3

the ergodic risk-sensitive HJB eigen-equation is

$\gamma$4

For continuous-time Markov chains,

$\gamma$5

These are the baseline structures from which constrained formulations inherit their analytical machinery (Biswas et al., 2022).

The fluctuation-based constrained LQR problem is treated through a Lagrangian,

$\gamma$6

or, in the general affine formulation,

$\gamma$7

Under Slater’s condition,

$\gamma$8

the constrained problem admits strong duality and can be solved through the saddle-point problem

$\gamma$9

with complementary slackness

$\gamma>0$0

In the tractable case $\gamma>0$1, the minimizing controller has the Riccati-like form

$\gamma>0$2

where

$\gamma>0$3

This makes the constraint enter the synthesis as an additional quadratic penalty in the Riccati equation (Talebi et al., 10 Feb 2025).

Algorithmically, the same source gives a primal-dual policy optimization method. The inner step uses a Riemannian quasi-Newton or Hewer-style update,

$\gamma>0$4

while the outer step updates the multiplier by projected ascent,

$\gamma>0$5

The stated complexity for obtaining an $\gamma>0$6-accurate solution is

$\gamma>0$7

This preserves the average-performance objective while enforcing the ergodic-risk constraint through the dual variable (Talebi et al., 10 Feb 2025).

5. Stability, existence, and admissibility conditions

The classical literature repeatedly ties ergodic-risk constraints to existence and stability conditions. For discrete-time chains, the survey lists finite state spaces, countable state spaces with Doeblin or near-monotone conditions, and general Borel spaces via discounted approximation. For controlled diffusions it lists local Lipschitz and nondegeneracy assumptions, coercive or near-monotone running cost, blanket stability or Lyapunov conditions, and uniform ellipticity in some results. For continuous-time Markov chains it lists irreducibility under stationary policies, stability or simultaneous Doeblin conditions, near-monotone costs, and finite jump rates or Lyapunov drift conditions. In many of these settings, the risk-sensitive optimality equation has a positive solution $\gamma>0$8, and any minimizing selector is optimal (Biswas et al., 2022).

Near-monotonicity plays a special role because it encourages stabilizing controls. The survey summarizes it as

$\gamma>0$9

for an appropriate threshold $\gamma=0$0, typically the optimal ergodic value. This condition helps ensure existence of an optimal stationary policy even without strong a priori stability assumptions. Later diffusion work sharpens this by combining a two-region structural hypothesis with a Foster–Lyapunov drift condition on one subset and near-monotonicity with inf-compact running cost on the complement, yielding a unique positive solution to the multiplicative HJB equation and a complete characterization of optimal stationary Markov controls (Biswas et al., 2022, Anugu et al., 2 Nov 2025).

The fluctuation-based ergodic-risk constraint literature imposes a different but related admissibility regime. The closed-loop policy is restricted to affine stationary Markov policies

$\gamma=0$1

with $\gamma=0$2 stabilizing and $\gamma=0$3 stabilizable. To obtain irreducibility and a unique invariant measure, the papers further assume that $\gamma=0$4 is controllable, and that the chain is positive Harris recurrent and $\gamma=0$5-uniformly ergodic. For quadratic ergodic-risk, the finite fourth moment condition

$\gamma=0$6

is explicitly required. The papers emphasize that this is what makes the framework applicable to heavy-tailed noise, as long as the fourth moment exists (Talebi et al., 2024, Talebi et al., 10 Feb 2025).

6. Related directions, adjacent formulations, and common confusions

The direct constrained formulations above should be distinguished from nearby uses of “ergodic” and “risk.” In multi-agent ergodic exploration under smoke-based visibility, smoke density defines a visibility coefficient

$\gamma=0$7

that modifies the expected information distribution, for example through

$\gamma=0$8

The paper states explicitly that smoke is not formulated as a hard constraint on the optimization problem and is not a paper about explicit “ergodic-risk constraints” or formal risk-sensitive optimization. It is therefore conceptually adjacent, not mathematically equivalent, to ergodic-risk constrained planning (Wittemyer et al., 6 Mar 2025).

A second nearby line is finite-horizon multistage risk-constrained control with nested conditional risk mappings. There the proposed constraint is

$\gamma=0$9

which accounts for propagation of uncertainty in time on a scenario tree. The same source states that it does not use “ergodic” in the classical infinite-horizon stationary-average sense. This clarifies a common terminological confusion: nested time-consistent risk constraints and ergodic-risk constraints both regulate cumulative uncertainty over time, but they are not the same construction (Sopasakis et al., 2019).

Finance supplies another related but distinct interpretation. Forward entropic risk measures built from exponential forward performance processes are governed by an ergodic BSDE,

$\gamma<0$0

with forward utility

$\gamma<0$1

The risk measure is represented by

$\gamma<0$2

and for long maturities it converges exponentially fast to a constant independent of the initial factor state. This is a long-run risk statement with an ergodic constant $\gamma<0$3, but it is not an explicit control constraint of the Lagrangian type above. A plausible implication is that the expression “ergodic-risk constraint” now covers a family of long-run, stationary, and tail-sensitive restrictions whose exact mathematical form depends on whether the model is built from exponential growth rates, asymptotic fluctuation limits, or ergodic BSDEs (Chong et al., 2016).

Across these strands, the most stable technical picture is the following. Ergodic-risk constraints replace mean-only admissibility by a long-run risk specification; the specification is either a log-moment growth-rate bound or a bound on asymptotic fluctuation variance; and the analysis proceeds through nonlinear eigenvalue problems, ergodic occupation measures, entropy or quadratic penalties, strong duality, and stability conditions that guarantee the existence of optimal stationary controls (Biswas et al., 2022, Talebi et al., 2024, Talebi et al., 10 Feb 2025).