Hypocoercivity in Discrete Time Markov Chains

• Hypocoercivity is characterized by uniform exponential convergence to equilibrium in non-reversible, discrete time Markov chains via operator splittings and projections onto mean-zero subspaces.
• The framework leverages spectral properties, similarity transforms, and, when necessary, Jordan block analysis to establish explicit mixing rates and exponential decay bounds.
• Practical implementations include non-reversible random walks and discrete kinetic schemes, where explicit hypocoercivity constants provide sharp non-asymptotic estimates for convergence and χ²-divergence.

The hypocoercivity phenomenon for discrete time Markov chains refers to uniform exponential convergence to equilibrium for a large class of non-reversible, discrete time stochastic processes, even in the absence of classical detailed-balance (i.e., non-selfadjointness of the transition operator). This concept generalizes spectral gap–based “coercivity” to operator splittings where neither part is coercive, originating in kinetic theory and extended here to finite Markov chains, time-discrete kinetic equations, and general linear dynamical systems. The foundational results are operator-theoretic, leveraging spectral properties, similarity transforms, and explicit control of non-reversible dynamics via projections onto mean-zero subspaces in weighted $\ell^2(\pi)$ spaces.

1. Operator-Theoretic Framework and Mean-Zero Subspace

Let $P\in\mathbb{R}^{n\times n}$ be the stochastic, irreducible, aperiodic transition matrix of a Markov chain on finite state space $X$ with stationary law $\pi=(\pi_1, \ldots, \pi_n)>0$. The natural Hilbert space is

$$\ell^2(\pi) = \left\{f:X\to\mathbb{R} \mid \|f\|_{\ell^2(\pi)}^2 = \sum_{i=1}^n \pi_i f(i)^2 < \infty\right\},$$

with inner product $$\langle f,g \rangle_\pi = \sum_{i=1}^n \pi_i f(i)g(i)$$.

The central tool is the orthogonal projection

$U_{\perp 1} := I - 1 \pi^T,$

which projects onto the mean-zero subspace $$V_{\perp 1} = \{f \in \ell^2(\pi): \langle f,1 \rangle_\pi = 0\}$$. This leads to the projected transition operator $PU_{\perp 1}$, acting on $V_{\perp 1}$, which governs convergence to equilibrium: $$P^k f - \pi(f) = (PU_{\perp 1})^k f, \quad \forall f.$$ Hence, the decay $\|P^k f - \pi(f)\|_{\ell^2(\pi)}$ is controlled by the norm $\|(PU_{\perp 1})^k\|$. This operator commutes with $P$ and is the focus for quantitative analysis.

2. Hypocoercivity in the Diagonalizable Case

Assume that $PU_{\perp 1}$ is diagonalizable on $V_{\perp 1}$. There exist $S\in GL(n)$ and a diagonal $$\Lambda = \operatorname{diag}(\lambda_2, \ldots, \lambda_n)$$, with $|\lambda_i|<1$, such that $PU_{\perp 1}=S\Lambda S^{-1}$, where $\lambda_1=1$ is the trivial eigenvalue and $\lambda_2,\ldots, \lambda_n$ are nontrivial. Define the operator condition number $\kappa = \|S\|_{op}\|S^{-1}\|_{op}$.

The discrete hypocoercivity estimate asserts: $$\|P^k f\|_{\ell^2(\pi)} = \|(PU_{\perp 1})^k f\| \leq \kappa\,\alpha^k\,\|f\|_{\ell^2(\pi)}, \quad \forall f:\pi(f)=0,$$ where $\alpha := \max_{2\leq i\leq n}|\lambda_i|<1$ and $C:=\kappa$ is an explicit hypocoercivity constant.

Thus, for diagonalizable $PU_{\perp 1}$, one achieves a uniform, computable exponential bound on the convergence rate—exporting the exact concept of hypocoercivity from the continuous to discrete setting.

3. Non-Diagonalizable and General Hypocoercivity

If $PU_{\perp 1}$ is not diagonalizable, the spectral theory must account for Jordan block structure. The Jordan decomposition (after a suitable similarity via $D_\pi^{1/2}$)

$$D_\pi^{1/2}\, PU_{\perp 1}\, D_\pi^{-1/2} = S J S^{-1}$$

involves $J$ block-diagonal with eigenvalues $\lambda_i$, algebraic multiplicities $m_i$, geometric multiplicities $g_i$, and $D_{\lambda_i} = m_i - g_i$.

Standard estimates yield, for $k\ge 1$: $$\|(PU_{\perp 1})^k\|_{\ell^2(\pi)} \leq \kappa(S) \max_i\ k^{D_{\lambda_i}} |\lambda_i|^k \left(\frac{1-|\lambda_i|}{1-|\lambda_i|^{D_{\lambda_i}+1}}\right).$$ The polynomial prefactors in $k$ reflect algebraic non-diagonalizability. Additionally, in control of $\chi^2$ convergence, the factor $(1-\pi_{min})/\pi_{min}$ (for smallest stationary probability) appears, increasing the worst-case bound.

4. Quantitative Mixing, χ²-Divergence, and Submultiplicativity

Mixing to equilibrium is quantified through the pointwise and global $\chi^2$-divergence: $$\chi^2_i(k) = \sum_j \frac{(p^k(i,j)-\pi_j)^2}{\pi_j},\qquad \chi^2(k) = \max_i \chi^2_i(k).$$ The operator-theoretic approach establishes the submultiplicativity property: $\chi^2_i(k+t) \leq \sigma_2(P^t)^2\ \chi^2_i(k),$ where $\sigma_2(P^t) = \|P^t U_{\perp 1}\|_{\ell^2(\pi)}$ is the second singular value. Globally,

$$\chi^2(k) \leq \|(PU_{\perp 1})^k\|^2\, \frac{1-\pi_{min}}{\pi_{min}}.$$

Explicit checks on powers of $PU_{\perp 1}$—by spectral or numerical means—thus yield sharp, non-asymptotic mixing time estimates for any initial state, including non-reversible Markov chains.

5. Hypocontractivity, Discrete Indices, and Short-Time Plateau Phenomenon

The theory is supplemented and clarified by the concept of hypocontractivity from linear dynamical systems (Achleitner et al., 2022). For a (possibly non-symmetric) $P$, after projecting onto mean-zero, the hypocoercivity/hypocontractivity index $dHC(PU_{\perp 1})=m$ is the minimal $m$ such that the iterated dissipation

$$D_m = \sum_{j=0}^m (A^*)^j D_0 A^j,\quad D_0 = I - A^*A,$$

is positive definite, with $A=PU_{\perp 1}$. This implies that $\|A^k\|=1$ for $k\leq m$, but $\|A^{m+1}\|<1$, revealing a finite “plateau” of norm preservation followed by strict contraction and exponential decay. The drop is quantified by the smallest eigenvalue of $D_m$ as

$\|A^{m+1}\|_2 = \sqrt{1-\lambda_{\min}(D_m)} < 1.$

Efficient computation is via rank tests on controllability matrices. Concrete examples for non-reversible 3-state chains demonstrate this effect; e.g., for certain $P$, $dHC=1$ and first strict contraction occurs at $k=2$.

6. Structure of Hypocoercivity for Discrete Kinetic Schemes

The explicit structure of hypocoercivity is illuminated in the context of discrete Fokker-Planck chains (Dujardin et al., 2018). Decompositions $P=L+X$, with $L$ coercive (“collision”) and $X$ skew-adjoint (“transport”) and appropriate commutator control, enable the construction of modified discrete energies $\mathcal{H}$

$$\mathcal{H}[g] = C\|g\|^2 + D\|D_v g\|^2 + E\langle D_v g, S D_x g\rangle + \|D_x g\|^2,$$

with shift $S$, such that for suitable $C,D,E$ and time step $\Delta t$

$$\mathcal{H}(f^n) \leq (1-k\Delta t)^n \mathcal{H}(f^0)$$

with $k>0$ independent of discretization. The approach is robust: the same algebraic argument applies directly to general discrete Markov chains with analogous splitting and spectral gap properties, so long as a Poincaré-type inequality and relevant commutator identities are available.

7. Assumptions, Generalizations, and Limitations

The hypocoercivity framework for discrete time Markov chains applies to any finite, ergodic chain, regardless of reversibility. In the diagonalizable case, hypocoercivity constants $C=\kappa(S)$ and $α=ρ(PU_{\perp 1})$ are explicit; in the general case, polynomial prefactors arise naturally from Jordan block structure. The theory has been shown to cover a wide class of non-reversible chains (notably, momentum-based samplers and non-reversible random walks), and produces the sharpest known mixing bounds without asymptotic approximations.

Limitations include the computational complexity of determining $\kappa(S)$ (condition number of the similarity matrix), which may be mitigated by bounding via graph structure or pseudospectral estimates. Extending to countable or continuous states requires semigroup analogues and spectral calculus for infinite dimensions. Nevertheless, the foundational operator-theoretic approach, involving mean-zero projections and dissipation control, enables a unified and general methodology for non-asymptotic mixing and exponential convergence for discrete-time, non-reversible Markov chains. The interplay of spectral gap, non-selfadjointness, and cross-effects captured by commutators is now understood as a purely algebraic phenomenon, paralleling and extending Villani’s continuous-time hypocoercivity to the discrete-time, discrete-state setting.