Almost Lyapunov Theory for Practical Stability

• Almost Lyapunov Theory is a framework that relaxes strict Lyapunov conditions by allowing small, controlled 'bad' regions to certify stability.
• The theory extends classical stability results to stochastic and discrete-time systems through finite-step expected decrease conditions.
• It underpins modern data-driven control and reinforcement learning approaches by ensuring practical convergence and robust invariance properties.

Almost Lyapunov Theory provides a rigorous framework to extend classical Lyapunov-based stability arguments to nonlinear, stochastic, and data-driven systems where the strict decrease condition on the Lyapunov function is permitted to fail on small subsets of the state space. This relaxation underpins both measure-theoretic and function-based approaches that certify practical stability, almost everywhere convergence, and robust invariance properties in modern control, dynamical systems, and reinforcement learning.

1. Foundational Definitions and Relaxed Lyapunov Functions

Classical Lyapunov theory relies on a continuously differentiable, positive-definite function $V : \mathbb{R}^n \to [0,\infty)$ such that $\dot V(x) = \nabla V(x)\cdot f(x) < 0$ everywhere except the origin for the system $\dot x = f(x)$, $f(0) = 0$ (Liu et al., 2018). This guarantees asymptotic stability.

Almost Lyapunov theory relaxes the strict negativity condition. Fix $0 < c_1 < c_2$, set $D = \{x : c_1 \le V(x) \le c_2\}$ (compact), and define for decay rate $a > 0$: $\Omega = \{x \in D : \dot V(x) \ge -a V(x)\}$ $V$ is an almost Lyapunov function with rate $a$ on $D$ if:

• $\dot V(x) < -a V(x)$ for all $x \in D\setminus\Omega$
• $\dot V(x) \ge -a V(x)$ for all $x \in \Omega$

Crucially, each connected component $\Omega^*$ of $\Omega$ must satisfy $|\Omega^*| \le \epsilon$ for some $\epsilon > 0$. This allows $V$ to be non-decreasing or even increasing on small, arbitrarily shaped "bad sets" as long as their volume is prescribed to be sufficiently small (Liu et al., 2018, Chang et al., 2021).

In stochastic and discrete-time settings, analogous relaxations apply. For example, the almost Lyapunov criterion allows for only a decrease after a finite number of steps $T$ rather than at every step (Qin et al., 2019): $$\mathbb{E}[V(x_{k+T})|\mathcal{F}_k] \le V(x_k) - \alpha V(x_k)$$ with $T \in \mathbb{N}$, $\alpha > 0$.

2. Main Theorems and Invariance Results

The central conclusion is a practical convergence theorem [(Liu et al., 2018), Theorem 3.1]. Under local Lipschitzness, positivity, and appropriate decay and bad-set-volume conditions, the following holds:

• For any $x(0)=x_0 \in D$ with $V(x_0) < c_2 - h\epsilon^{1/n} - g\epsilon$ (explicit constants $h, g$), the trajectory remains in $D$ for all $t\ge 0$ and enters the set $\{V \le c_1 + h\epsilon^{1/n} + g\epsilon\}$ for all sufficiently large $t$.
• As $\epsilon \to 0$, the result reverts to that of strict Lyapunov conditions.

In stochastic settings, almost Lyapunov (finite-step) theorems yield that if $V$ exhibits an expected decrease only after finite steps, one obtains asymptotic or exponential stability in probability, with explicit rate estimates (Qin et al., 2019). For invariance, the forward-invariant set

$\mathcal{R} = \{ x : V(x) < c_2 - r(\epsilon) \}$

remains invariant for all $t \ge 0$ and trajectories ultimately converge into a smaller inner strip around $\{V\le c_1\}$ (Chang et al., 2021).

These principles apply to both deterministic and stochastic systems, continuous and discrete time, and extend to almost everywhere stability outside small violation sets (Vaidya, 2015, Karabacak et al., 2016).

3. Volume and Measure-Based Methods

A key innovation in almost Lyapunov analysis is the shift from strict pointwise negativity of $\dot V$ to a volumetric or measure-theoretic smallness of $\Omega$, the set where the decrease is non-strict or violated. This is formalized via:

• Explicit volume constraints on each connected component of $\Omega$ (Liu et al., 2018)
• Lyapunov measures, where instead of functions, a measure $\bar\mu$ is constructed such that the induced Markov operator shrinks mass outside a small neighborhood at each step (Vaidya, 2015)

The duality between function and measure perspectives is realized via the relationship between the Perron–Frobenius operator (propagating measures) and the Koopman operator (propagating functions) (Vaidya, 2015, Karabacak et al., 2016): $$\langle \mathbb{P} \mu, f \rangle = \langle \mu, \mathbb{U} f \rangle$$ In almost everywhere stability, Lyapunov density functions $\rho$ satisfy subinvariance properties under the Frobenius–Perron operator, enabling convergence results for almost all initial conditions (outside measure-zero sets) (Karabacak et al., 2016).

4. Computational and Data-Driven Extensions

Modern implementations, notably in control and reinforcement learning, fit the almost Lyapunov framework by:

• Learning parameterized Lyapunov critic functions $V_\theta$ via sample-based losses that approximate the Lie derivative and penalize violations only when their frequency (volume) is above threshold (Chang et al., 2021)
• Certifying sample-based satisfaction of the almost Lyapunov conditions by estimating the violation set's volume on a grid or via Monte Carlo (Chang et al., 2021, Cheng et al., 29 Sep 2025)
• Using diffusion models (e.g., $S^2$Diff) that sample entire control trajectories, biasing the generative process toward regions with negative Lie derivative except on a small set, thus realizing almost Lyapunov conditions in expectation (Cheng et al., 29 Sep 2025)

Tables of empirical metrics in these studies report small violation-set fractions and demonstrate invariance and stability across a variety of high-dimensional tasks.

5. Application Domains and Key Examples

Almost Lyapunov theory provides practical certification in scenarios where strict conditions are unattainable, due to model uncertainty, perturbations, or algorithmic approximations. Key application areas include:

• Nonlinear systems where $\dot V$ is hard to bound globally or only verified on sampled data (Liu et al., 2018, Chang et al., 2021)
• Stochastic systems (e.g., consensus under random communication graphs, distributed solvers under random asynchronous updating) where contraction can only be certified in blocks or in expectation (Qin et al., 2019)
• Safe model-based control via diffusion policies in robotics and aerospace, with empirical evidence of strong safety and stability guarantees despite model-free or approximate learning (Cheng et al., 29 Sep 2025)

A concrete example (Liu et al., 2018): In $\mathbb{R}^2$, for

$$\dot x = \begin{pmatrix} -\lambda(x) & -\mu \ \mu & -\lambda(x) \end{pmatrix} x$$

with $V(x)=\|x\|^2$, inside a small disk $B_\rho(x_c)$ one has $\dot V>0$. By quantifying the disk's volume and applying the almost Lyapunov theorem, practical convergence to a neighborhood of the origin is rigorously certified.

6. Implications, Generalizations, and Open Directions

The almost Lyapunov paradigm fundamentally extends the class of systems for which stability can be certified, relaxing the classical requirement of strict negativity to a setting where only the "bad" set's measure matters. This approach is essential for:

• Robustness to numerical and model-based errors in $L_f V(x)$ calculations, as encountered in learning-based or data-driven methods (Chang et al., 2021)
• Enabling systematic construction of certificate functions when strict Lyapunov candidates do not exist, e.g., via randomized sampling or convex optimization subject to volume constraints (Liu et al., 2018)
• Achieving global results by iterating local practical convergence over level bands $D(c)$, leading to global uniform asymptotic stability under uniform bad-set volume control (Liu et al., 2018)

Open problems highlighted include developing less conservative volume bounds (e.g., through variable tube radii), relaxing the non-vanishing vector field assumption, extending to invariant sets beyond the origin, and exploiting finer geometric characteristics (e.g., “thinness”) rather than pure volume (Liu et al., 2018).

In stochastic and high-dimensional settings, almost Lyapunov theory underpins operator-theoretic certificates of geometric decay and almost sure stability, with explicit algorithms for constructing Lyapunov measures and for finite-dimensional Markov approximations (“coarse stability”) (Vaidya, 2015).

7. Summary Table: Key Variants of Almost Lyapunov Theory

Setting	Relaxed Condition	Main Guarantee
Continuous-time nonlinear (deterministic)	$\dot V < -aV$ outside $\Omega$, $\text{vol}(\Omega) \leq \epsilon$	Invariance and practical convergence (Liu et al., 2018)
Discrete-time stochastic	$\mathbb{E}[V(x_{k+T})] \leq V(x_k) - \alpha V(x_k)$	Almost sure/exponential stability (Qin et al., 2019)
Measure-theoretic (a.e. stability)	Lyapunov density $\rho > 0$, $P\rho < \rho$ a.e.	Convergence for Lebesgue almost all $x$ (Karabacak et al., 2016)
Data-driven/control via neural critics	Lie derivative violated only on small volume sets	Certified robust policy invariance and attraction (Chang et al., 2021, Cheng et al., 29 Sep 2025)

Almost Lyapunov theory, by leveraging volumetric or measure-theoretic constraints, robustly bridges Lyapunov stability with modern computational, stochastic, and data-driven dynamical systems (Liu et al., 2018, Vaidya, 2015, Qin et al., 2019, Chang et al., 2021, Cheng et al., 29 Sep 2025, Karabacak et al., 2016).