Reversible Markov Matrices

• Reversible Markov matrices are stochastic matrices that satisfy detailed balance, ensuring symmetric flux and a unique positive stationary distribution.
• They exhibit real diagonalizability and embeddability via the principal logarithm, with a rich algebraic structure connected to symmetric matrix theory and information geometry.
• Applications span efficient MCMC estimation, robust parameter inference, and the analysis of spectral gaps and random matrix models in stochastic processes.

A reversible Markov matrix is a stochastic matrix describing a finite-state Markov chain with the property that, for a unique strictly positive stationary distribution, the detailed-balance equations hold: the probability flux from state $i$ to $j$ is exactly balanced by the reverse flux from $j$ to $i$. This symmetry endows the transition matrix with powerful spectral and structural properties, enabling precise characterizations of its embeddability into continuous-time Markov semigroups, algebraic parameterizations, analytic and algorithmic techniques for estimation, and explicit connections to symmetric matrix theory and information geometry. The paper of reversible Markov matrices underpins key developments in probability, statistical mechanics, computational Markov chain Monte Carlo (MCMC), and the algebraic theory of stochastic processes.

1. Formal Definitions and Characterizations

Let $M \in \mathbb{R}^{d \times d}$ be a Markov matrix, i.e., $M_{ij} \ge 0$ and $\sum_j M_{ij} = 1$ for all $i$. The matrix $M$ is reversible if there exists a strictly positive probability vector $\boldsymbol{p} > 0$ such that

$$p_i\, M_{ij} = p_j\, M_{ji} \quad \forall i, j \in \{1, \ldots, d\}.$$

$\boldsymbol{p}$ is the unique stationary distribution: $\boldsymbol{p} M = \boldsymbol{p}$.

Equivalently, in matrix notation: $D M = M^T D$ where $D = \mathrm{diag}(p_1, \ldots, p_d)$, and $M$ is similar to the real symmetric matrix $D^{1/2} M D^{-1/2}$.

Weak reversibility admits $p \ge 0$ (allowing zeros), which holds within each self-contained communicating class.

A Markov matrix is reversibly embeddable if there exists a generator $Q$ (Markov rate matrix, i.e., $Q_{ij} \ge 0$ for $i \neq j$, $\sum_j Q_{ij} = 0$) such that $M = e^Q$ and $Q$ satisfies the same detailed-balance equations: $p_i Q_{ij} = p_j Q_{ji}.$

Kolmogorov’s cycle criterion is equivalent: for all closed paths $(j_0, j_1, \ldots, j_k = j_0)$,

$$M_{j_0,j_1} M_{j_1,j_2} \cdots M_{j_{k-1},j_k} = M_{j_0, j_k} M_{j_k, j_{k-1}} \cdots M_{j_2,j_1} M_{j_1, j_0}.$$

See (Jiang et al., 2018).

2. Spectral Theory and Embedding Criteria

A key property of reversible matrices is real diagonalizability: the spectrum $\sigma(M) \subset \mathbb{R}$, and $M$ is similar to a (real) symmetric matrix. All eigenvalues of a Markov matrix satisfy $|\lambda_j| \le 1$; for reversible $M$, they are real.

Reversible Embedding Problem: Given reversible $M$, does there exist a reversible generator $Q$ (i.e., $M = e^Q$ and $Q$ reversible)? The embedding exists uniquely when all eigenvalues of $M$ are positive and real; the solution is the principal matrix logarithm: $$Q = \log M = P\, \operatorname{diag}(\ln \lambda_0, \ldots, \ln \lambda_{d-1})\, P^{-1},$$ where $M = P D P^{-1}$, $$D = \mathrm{diag}(\lambda_0, \ldots, \lambda_{d-1})$$.

Necessary and sufficient condition: $M$ is reversibly embeddable if the principal logarithm $Q = \log M$ is a Markov generator ($Q_{ij} \ge 0$, $\sum_j Q_{ij} = 0$), and satisfies detailed balance with respect to $p$.

For repeated or multiple eigenvalues, use the minimal polynomial and Vandermonde inversion: $Q = \sum_{k=1}^{m-1} \alpha_k (M - I)^k,$ where the coefficients solve the system: $$\log(\lambda_i) = \sum_{k=1}^{m-1} \alpha_k (\lambda_i - 1)^k.$$ Check positivity of off-diagonal entries of $Q$ (Baake et al., 27 Nov 2025, Jia, 2016).

If $M$ is reducible, $M$ is reversibly embeddable iff each irreducible diagonal block is; weak reversibility is handled analogously blockwise. Negative eigenvalues of even multiplicity may allow embeddability, but not reversible embeddability—nonreversible generators can sometimes embed $M$, but $M^2$ may be reversibly embeddable (Baake et al., 27 Nov 2025).

Illustrative Example

Matrix $M$	Embeddability Condition	Reversible Generator $Q$
$2\times 2$	$a+b<1$, $ab>0$ for reversibility	$Q = -\frac{\ln(1-a-b)}{a+b}(M-I)$
Equal-input rank 1	Always (weakly) reversible	Closed-form embedding
$3\times 3$ w/ neg eig.	Not reversibly embeddable	Two nonreversible commuting $Q_{+}, Q_{-}$

3. Algebraic, Parametric, and Geometric Structure

The set of reversible $d \times d$ Markov matrices forms both an exponential family and a mixture (m-) family. Each reversible transition matrix is uniquely identified (up to normalization) by its edge measure $Q = \mathrm{diag}(\pi) P$, which is symmetric: $Q = Q^T$.

The information-geometric perspective characterizes reversible kernels as a doubly-autoparallel submanifold:

• e-family (exponential): Natural parameters $\theta^{ij}$ for unordered pairs $\{i, j\}$ define

$$\log P(x, x') = \sum_{i \leq j} \theta^{ij} g_{ij}(x, x') + R(x') - R(x) - \psi,$$

where $g_{ij}(x, x')$ generates symmetric edge features (Wolfer et al., 2021).

• m-family (mixture): Expectation parameters $$\eta_{ij} = Q[i, j] = \pi(i) P(i, j) + \pi(j) P(j, i)$$ parameterize the convex set.

From the algebraic statistics viewpoint, the detailed-balance equations and Kolmogorov cycle conditions are binomial polynomial equations. The set of strictly positive reversible Markov matrices, on a fixed undirected graph $\mathcal{G}$, forms a toric variety. The explicit parametrization is: $$P_{v \to w} = \frac{s_{vw} \prod_{B \in \mathcal{B}} t_B^{u_{v \to w}(B)}}{Z_v(s, t)}$$ where $s_{vw}$ are symmetric edge weights and $t_B$ are cut parameters with exponents determined by the cocycle basis $\mathcal{B}$. The invariant law is a monomial in the $t_B$ (Pistone et al., 2010).

Reversible kernels are minimal exponential families generated by the m-family of symmetric kernels, and the smallest mixture family containing i.i.d. (memoryless) kernels (Wolfer et al., 2021).

4. Estimation, Sampling, and Uncertainty Quantification

Estimation of reversible Markov matrices from trajectory or count data is formulated as constrained maximum likelihood (MLE) or Bayesian posterior inference under detailed balance.

• In the reversible MLE, one maximizes the likelihood over $(P, \pi)$ subject to

$$\pi_i p_{ij} = \pi_j p_{ji}, \quad \sum_j p_{ij} = 1, \quad p_{ij} \ge 0.$$

The marginal flux variables $x_{ij} = \pi_i p_{ij}$ are symmetrized ($x_{ij} = x_{ji}$), and efficient iterative solvers (fixed-point, convex–concave programming, interior-point) are available. When the stationary distribution $\pi$ is known, estimation becomes convex (Trendelkamp-Schroer et al., 2015, Trendelkamp-Schroer et al., 2016).

• Symmetric Counting Estimator (SCE): For sequential sample paths in a reversible chain, symmetrized length-2 empirical frequencies yield dimension-free, non-asymptotic concentration bounds for the joint measure $\mu(u)\,p(u, v)$, and preserve reversibility by construction (Huang et al., 12 Aug 2024).
• Bayesian Inference: Priors enforcing reversibility (e.g., sparse reversible priors on edge-measures) and specialized MCMC algorithms allow for posterior credible intervals for kinetic observables in high-dimensional state spaces, matching uncertainty to metastability and sampling graph structure (Trendelkamp-Schroer et al., 2015).
• Algorithmic Detection: Efficient $O(n^2)$ algorithms using row- or column-multiplications can symmetrize $P$ to check reversibility, bypassing factorial Kolmogorov-loop checks (Jiang et al., 2018).

5. Structural, Analytic, and Spectral Properties

Reversibility is equivalent to self-adjointness of $P$ as an operator on $L^2(\pi)$. The spectral theorem ensures an orthonormal basis of real eigenvectors, all eigenvalues are real, and $P$ is diagonalizable by a similarity to its symmetric conjugate. The Dirichlet form

$$\mathcal{E}(f, f) = \frac{1}{2} \sum_{i, j} \pi_i P_{ij} (f(i) - f(j))^2$$

controls variance decay and mixing:

• Spectral gap: $\gamma = 1 - \max_{k>0} |\lambda_k|$ determines geometric ergodicity and the decay of correlation/mixing.

Explicit families of finite reversible chains arise from orthogonal polynomial recursions (Hahn, Jacobi, Meixner, Krawtchouk, Hermite) and their associated Jacobi or Hessenberg matrices, with stationary measure and detailed balance given by inner-product orthogonality or recurrence polynomials (Branquinho et al., 2023). For compact, infinite-state reversible chains (e.g. on $\ell^2(\pi)$), asymptotic convergence rates to stationarity decompose as sums of eigencomponents shrinking with powers of $|\lambda_j|$ (Xu et al., 2023).

6. Extensions and Applications

• Random matrix ensembles: Large, sparse, random reversible Markov matrices with controllable spectral gap and nontrivial limiting empirical spectral distribution can be constructed from random graphs via regularized adjacency matrices (Chi, 2015).
• Imprecise/reversible Markov chains: The reversibility concept extends to convex sets of transition matrices (credal sets); reversibility is equivalently symmetry of the edge-measure set, enabling convex programming for expectation bounds of path functionals (Škulj, 8 Jul 2025).
• Independence-preserving involutions: Bijections $(f, g_f)$ on product spaces characterize reversible kernels, and this framework unifies reversible chain constructions in both discrete and continuous state spaces. The involutive augmentation provides explicit coupling/backward maps, critical for advanced MCMC techniques and integrable probability (Piccioni et al., 16 Aug 2024).

7. Illustrative Summary Table

Property	Description	Reference
Reversibility (detailed balance)	$p_i M_{ij} = p_j M_{ji}$	(Baake et al., 27 Nov 2025, Bradley, 2019)
Spectral structure	Real diagonalizable, spectrum real, nonnegative for embeddability	(Jia, 2016)
Principal logarithm test (embedding)	$Q = \log M$, $Q_{ij} \ge 0$ off-diagonal	(Baake et al., 27 Nov 2025)
Algebraic/statistical model	Toric parameterization by edge/cut generators	(Pistone et al., 2010)
Information geometry	Both e- and m-family, doubly autoparallel manifold	(Wolfer et al., 2021)
Estimation algorithms	Symmetric MLE, convex–concave programming, SCE, Bayesian	(Trendelkamp-Schroer et al., 2015, Trendelkamp-Schroer et al., 2016, Huang et al., 12 Aug 2024)
Random matrix models	ESD, spectral gap, sparse graphs	(Chi, 2015)
Extensions	Weak reversibility, blockwise embedding, involutive augmentation	(Baake et al., 27 Nov 2025, Piccioni et al., 16 Aug 2024)

Each entry in this table encapsulates a central theoretical or methodological aspect of reversible Markov matrices, with direct connections to recent and foundational results. These properties ensure that reversible Markov matrices remain a fundamental tool across probability theory, statistical physics, computational algorithms, and stochastic modeling.