Markov Chains in Random Environments (MCREs)

• MCREs are stochastic systems where a random environment modulates transition kernels, yielding conditional Markov or time-inhomogeneous dynamics.
• They integrate both quenched and annealed viewpoints to analyze invariant laws, mixing properties, and simultaneous jump phenomena.
• MCRE theory underpins diverse applications, from queueing networks and branching processes to metastate selection in spin systems via drift and minorization techniques.

Markov chains in random environments (MCREs) are stochastic systems whose transition mechanism is modulated by a random medium. In the cited literature, this includes discrete-time chains with kernels of the form $Q(Y_n,X_n,\cdot)$, continuous-time generators perturbed by exogenous noise, random walks in dynamic space-time fields, and joint state–environment constructions in which Markovianity is restored only after enlarging the state space (Truquet, 2021, Lovas et al., 2019, Erb, 2023). The common structure is that, conditional on a realized environment, the state process is Markov or time-inhomogeneous Markov, while after averaging over the environment one studies quenched and annealed asymptotics, invariant laws, metastates, mixing, or induced transition structures such as simultaneous jumps (Bretó, 2013).

1. Conceptual scope and formal formulations

A general formulation takes a parametric kernel

$Q:Y\times X\times \mathcal B(X)\to[0,1]$

and an environment process $(Y_n)$, with the state process $(X_n)$ satisfying

$$P\!\left(X_{n+1}\in B \mid (X_k)_{k\le n}, (Y_k)\right)=Q(Y_n,X_n,B).$$

In this form, conditional on the full environment path, $X$ is a time-inhomogeneous Markov chain; unconditionally, $X$ is generally not Markov by itself (Lovas et al., 19 Sep 2025). Closely related formulations appear in stationary exogenous environments, where

$$\mathbb P\!\left(Y_t\in A \mid X, Y_{t-1},Y_{t-2},\ldots\right) = P_{X_{t-1}}(Y_{t-1},A),$$

and in state-augmented models where the pair $(X_t,Q_t)$ or $(X_t,Y_t)$ is the natural Markov object (Truquet, 2021, Greco et al., 2021).

This state augmentation is not a technicality but a structural distinction. In the branching process in a Markovian environment, $Q:Y\times X\times \mathcal B(X)\to[0,1]$0 alone is not Markov in general, while $Q:Y\times X\times \mathcal B(X)\to[0,1]$1 is Markov on $Q:Y\times X\times \mathcal B(X)\to[0,1]$2 (Greco et al., 2021). In the broader drift/minorization theory, the environment is exogenous and stationary, whereas in other constructions the environment dynamics may themselves depend on the current state, so that the joint process, not the marginal state process, carries the Markov property (Das, 2016). This suggests that “MCRE” is best understood as a family of random-kernel or joint-process constructions rather than a single canonical model.

A second basic distinction is between quenched and annealed viewpoints. Quenched statements condition on a realized environment path and study the resulting inhomogeneous Markov dynamics. Annealed statements integrate out the environment and analyze the unconditional process, its invariant law, or its dependence coefficients. Several of the cited works are explicitly organized around this dichotomy, and many of their main theorems transfer information from the environment to either quenched convergence or annealed mixing (Erb, 2023, Lovas et al., 19 Sep 2025).

2. Continuous-time counting systems and induced co-jumps

In continuous time, one prominent MCRE formulation is the Markov counting system (MCS) with environment-dependent transition rates. For a finite set of compartments $Q:Y\times X\times \mathcal B(X)\to[0,1]$3, counts $Q:Y\times X\times \mathcal B(X)\to[0,1]$4 record transitions $Q:Y\times X\times \mathcal B(X)\to[0,1]$5, and the compartment process satisfies

$Q:Y\times X\times \mathcal B(X)\to[0,1]$6

When selected rates are multiplied by external white noises derived from Lévy subordinators, especially gamma processes, the conditional process is a time-changed Markov counting process; after integrating over the environment, the unconditional process becomes a new MCS with modified rates that allow simultaneous jumps, or “co-jumps” (Bretó, 2013).

The conceptual point is sharper than mere rate randomization. In the bivariate death-process prototype, two independent linear death processes are driven by the same gamma white noise, and after averaging over that common environment the unconditional pair has strictly positive rates for simultaneous deaths in both components. The paper identifies the exact mechanism by the infinitesimal covariance identity

$Q:Y\times X\times \mathcal B(X)\to[0,1]$7

Thus, for a pure Markov counting system, nonzero infinitesimal covariance comes exactly from transition rates that simultaneously increment both counting processes (Bretó, 2013).

This result corrects a common simplification. In a standard compartmental CTMC with only single-event transitions, simultaneous increments of distinct counts occur with probability $Q:Y\times X\times \mathcal B(X)\to[0,1]$8. Under correlated environmental perturbations, once the environment is integrated out, such simultaneous increments occur with probability $Q:Y\times X\times \mathcal B(X)\to[0,1]$9. The unconditional process therefore cannot, in general, be represented by the original single-arrow diagram; its generator must include joint-jump terms. In the two-strain epidemic example of the paper, common transmission noise produces positive infinitesimal correlation between primary and secondary infections of the same strain, and the corresponding unconditional CTMC is obtained by introducing explicit pairwise infection rates with co-jumps (Bretó, 2013).

The significance for MCRE theory is methodological as well as interpretive. The environment is exogenous, but the unconditional chain remains Markov only after one enlarges the admissible transition structure. Correlated environments therefore alter not just the values of rates, but the very geometry of the generator.

3. Ergodicity, invariant laws, and mixing under stationary environments

A substantial strand of the literature develops Harris-type ergodic theory for MCREs on general state spaces under environment-dependent drift and minorization. One version assumes a Lyapunov function $(Y_n)$0, environment-size sets $(Y_n)$1, and coefficients $(Y_n)$2 such that

$(Y_n)$3

together with a minorization on

$(Y_n)$4

and a maximal-process bound on the environment. Under summability conditions on the resulting rate terms, the law of $(Y_n)$5 converges to a limiting law in weighted total variation, and ergodic theorems for functionals follow (Gerencser et al., 2018).

A complementary construction uses random invariant probability measures indexed by the past environment. Under a random drift inequality

$(Y_n)$6

and a random small-set condition, together with the long-run multiplicative contraction

$(Y_n)$7

the backward iterates $(Y_n)$8 converge almost surely in total variation to a random measure $(Y_n)$9, independent of $(X_n)$0, satisfying

$(X_n)$1

The same framework yields a quenched geometric bound with random $(X_n)$2 and $(X_n)$3 (Truquet, 2021).

A third variant weakens one-step contraction further by requiring only average long-run contractivity,

$(X_n)$4

together with an environment-dependent small-set minorization. Under these assumptions, $(X_n)$5 converges in total variation to a limiting law $(X_n)$6, and if the environment is ergodic then bounded observables satisfy an $(X_n)$7 law of large numbers (Lovas et al., 2019).

Recent work pushes these results toward dependence properties and rates. One general transfer principle proves annealed strong mixing of $(X_n)$8 under random-environment drift/minorization and environment mixing, with

$(X_n)$9

thereby linking the mixing of the observed process to the stability of random drift products and the dependence of the environment (Lovas et al., 19 Sep 2025). On general Polish state spaces with unbounded stationary environments, a coupling theory based on contractivity up to a bounded perturbation and local pairwise minorization yields annealed convergence and strong-mixing rates for $$P\!\left(X_{n+1}\in B \mid (X_k)_{k\le n}, (Y_k)\right)=Q(Y_n,X_n,B).$$0 that track the environment’s $$P\!\left(X_{n+1}\in B \mid (X_k)_{k\le n}, (Y_k)\right)=Q(Y_n,X_n,B).$$1-mixing rate up to logarithmic losses in the geometric and polynomial regimes (Lovas et al., 17 Dec 2025). At the quenched level, a random non-uniform Doeblin condition together with $$P\!\left(X_{n+1}\in B \mid (X_k)_{k\le n}, (Y_k)\right)=Q(Y_n,X_n,B).$$2-mixing of the environment implies effective geometric ergodicity: there exists a unique random family $$P\!\left(X_{n+1}\in B \mid (X_k)_{k\le n}, (Y_k)\right)=Q(Y_n,X_n,B).$$3 with

$$P\!\left(X_{n+1}\in B \mid (X_k)_{k\le n}, (Y_k)\right)=Q(Y_n,X_n,B).$$4

and for every finite $$P\!\left(X_{n+1}\in B \mid (X_k)_{k\le n}, (Y_k)\right)=Q(Y_n,X_n,B).$$5,

$$P\!\left(X_{n+1}\in B \mid (X_k)_{k\le n}, (Y_k)\right)=Q(Y_n,X_n,B).$$6

with $$P\!\left(X_{n+1}\in B \mid (X_k)_{k\le n}, (Y_k)\right)=Q(Y_n,X_n,B).$$7 (Hafouta, 1 Jan 2026).

A recurring limitation is that verification of the stability quantities can be technically delicate. In particular, the summability of

$$P\!\left(X_{n+1}\in B \mid (X_k)_{k\le n}, (Y_k)\right)=Q(Y_n,X_n,B).$$8

remains the hardest step in many dependent, unbounded environments (Lovas et al., 19 Sep 2025).

4. Dynamic environments, time-inhomogeneous kernels, and regeneration

When the environment itself evolves in time, the natural target of convergence may no longer be a fixed stationary law. For a finite time-inhomogeneous chain with kernels $$P\!\left(X_{n+1}\in B \mid (X_k)_{k\le n}, (Y_k)\right)=Q(Y_n,X_n,B).$$9, one proposal defines

$X$0

when the limit exists and is independent of $X$1, and then introduces the quenched mixing profile

$X$2

An evolving-set method adapted to this nonautonomous setting yields conductance-style upper bounds on mixing, and for a lazy walk on an i.i.d. dynamic Erdős–Rényi graph the quenched mixing time is $X$3 with high probability, with a lower bound of order $X$4 (Erb, 2023).

A simpler finite-state model treats the environment at time $X$5 as a random stochastic matrix $X$6, with

$X$7

Here the environment is i.i.d. in time, and the analysis is mainly annealed. For $X$8, if the columns of $X$9 are Dirichlet with parameter $X$0, the long-time column law of $X$1 is exactly Dirichlet with parameter $X$2; for general $X$3, the paper conjectures asymptotic Dirichlet$X$4 columns and documents exponential decay of nontrivial eigenvalues and singular values (Innocentini et al., 2018). This is a random-matrix representation of a time-random environment rather than a full joint chain–environment theory, but it isolates how temporal kernel randomness renormalizes multi-step transition laws.

For random walks in dynamic random environments, two related approaches center the environment seen from the walker. One proves existence, ergodicity, and mutual absolute continuity of an invariant law of the environment process under weak conditional mixing assumptions that do not require the strong uniform mixing used in earlier work (Bethuelsen et al., 2016). Another introduces an auxiliary field $X$5 satisfying a Random Markov Property,

$X$6

together with polynomial decoupling. Under this criterion the walk trajectory can be decomposed into i.i.d. regeneration increments, yielding an annealed law of large numbers for $X$7 and an annealed central limit theorem for $X$8 (Allasia et al., 2024).

These results clarify a frequent misconception: in dynamic environments, “mixing to stationarity” need not mean convergence to a time-independent measure, and regenerative structure need not come from a globally Markov environment. It may instead arise from a time-dependent pullback law or from random local reset points.

5. Metastates, occupation-time fluctuations, and state selection

Not all MCRE-relevant phenomena are trajectory-level recurrence or mixing problems. In mean-field spin systems with quenched random fields generated by an ergodic finite-state Markov chain, the environment enters through the empirical occupation measure

$X$9

and metastate selection is governed by the occupation-time central limit theorem

$$\mathbb P\!\left(Y_t\in A \mid X, Y_{t-1},Y_{t-2},\ldots\right) = P_{X_{t-1}}(Y_{t-1},A),$$0

For nondegenerate chains, if $$\mathbb P\!\left(Y_t\in A \mid X, Y_{t-1},Y_{t-2},\ldots\right) = P_{X_{t-1}}(Y_{t-1},A),$$1 are the minimizers of the free-energy functional and $$\mathbb P\!\left(Y_t\in A \mid X, Y_{t-1},Y_{t-2},\ldots\right) = P_{X_{t-1}}(Y_{t-1},A),$$2 the corresponding stability regions, then the metastate weights are

$$\mathbb P\!\left(Y_t\in A \mid X, Y_{t-1},Y_{t-2},\ldots\right) = P_{X_{t-1}}(Y_{t-1},A),$$3

so the limiting metastate is a convex combination of visible minimizers, with weights determined by the full occupation-time covariance matrix $$\mathbb P\!\left(Y_t\in A \mid X, Y_{t-1},Y_{t-2},\ldots\right) = P_{X_{t-1}}(Y_{t-1},A),$$4 (Formentin et al., 2011).

The dependence on $$\mathbb P\!\left(Y_t\in A \mid X, Y_{t-1},Y_{t-2},\ldots\right) = P_{X_{t-1}}(Y_{t-1},A),$$5 is structurally important. The set of minimizers depends only on the invariant law $$\mathbb P\!\left(Y_t\in A \mid X, Y_{t-1},Y_{t-2},\ldots\right) = P_{X_{t-1}}(Y_{t-1},A),$$6, but the metastate weights depend on the full transition matrix $$\mathbb P\!\left(Y_t\in A \mid X, Y_{t-1},Y_{t-2},\ldots\right) = P_{X_{t-1}}(Y_{t-1},A),$$7 through $$\mathbb P\!\left(Y_t\in A \mid X, Y_{t-1},Y_{t-2},\ldots\right) = P_{X_{t-1}}(Y_{t-1},A),$$8. Two distinct Markov environments with the same invariant law may therefore yield different metastates (Formentin et al., 2011). This is a specifically random-environment effect: temporal dependence of the environment alters macroscopic state selection even when one-point marginals agree.

The degenerate non-reversible case is more striking. In the three-state random-field Potts example studied in the paper, the limiting Gaussian occupation fluctuations lie on a lower-dimensional support, and the metastate contains mixed states with asymmetric weights under exchange of labels $$\mathbb P\!\left(Y_t\in A \mid X, Y_{t-1},Y_{t-2},\ldots\right) = P_{X_{t-1}}(Y_{t-1},A),$$9 and $(X_t,Q_t)$0, despite the invariant law and covariance displaying that symmetry. The paper’s conclusion is that CLT-scale Gaussian fluctuation information can be insufficient; finer pathwise structure of the environment may survive in the macroscopic limit (Formentin et al., 2011).

This broadens the meaning of MCRE beyond transition-kernel modulation. A Markovian environment may control equilibrium selection through occupation statistics, metastates, and pathwise degeneracies, even when the primary object is a mean-field Gibbs system rather than a conventional chain trajectory.

6. Applications, coupled environments, and structural variants

Several specialized models show how MCRE ideas are adapted to particular stochastic systems. In a branching process in a Markovian environment, an irreducible finite-state chain $(X_t,Q_t)$1 selects offspring laws $(X_t,Q_t)$2, and one individual is replaced at each step according to the next environment state. The survival/extinction threshold is governed by

$(X_t,Q_t)$3

If $(X_t,Q_t)$4, extinction is almost sure; if $(X_t,Q_t)$5, positive survival occurs for sufficiently large initial population. On survival, the process satisfies

$(X_t,Q_t)$6

and a central limit theorem with variance determined by a Poisson equation for the environment chain (Greco et al., 2021). This model illustrates how regeneration at returns of the environment to a fixed state can compress Markovian environmental dependence into an embedded i.i.d. structure.

Queueing theory supplies another class of MCRE constructions. In Jackson networks with environment-dependent service multipliers $(X_t,Q_t)$7, rerouting matrices $(X_t,Q_t)$8, and departure-triggered environment jumps, the environment is not Markov on its own because customers leaving the network may force immediate environment changes. Nevertheless, the joint process of queue lengths and environment is a homogeneous CTMC, and under the randomization algorithms of the paper its stationary law has product form

$(X_t,Q_t)$9

where $(X_t,Y_t)$0 is the Jackson product form and $(X_t,Y_t)$1 solves a reduced environment balance equation (Krenzler et al., 2014). This is an endogenous random environment rather than a purely exogenous one.

An even more explicit joint construction starts from generators $(X_t,Y_t)$2 for the basic process on $(X_t,Y_t)$3 and $(X_t,Y_t)$4 for the environment process on $(X_t,Y_t)$5, and builds a continuous-time Markov process on an augmented space with one or more maintenance states. The target stationary measure is

$(X_t,Y_t)$6

and the auxiliary states compensate for the failure of exact balance on $(X_t,Y_t)$7 alone (Das, 2016). The construction is notable because it relaxes earlier requirements that the environment stationary law be independent of the basic state and that only one coordinate change at a time.

Random switching on manifolds provides a different geometric variant. For Feller chains obtained either by random composition of maps or by random switching between flows, invariant measures can be shown to be absolutely continuous, lower semicontinuous, continuous, or smooth by decomposing the transition kernel as

$(X_t,Y_t)$8

with $(X_t,Y_t)$9 regularizing and $Q:Y\times X\times \mathcal B(X)\to[0,1]$00 preserving regularity (Benaïm et al., 2023). In the random-composition model, the environment is an i.i.d. sequence of maps; in the switched-flow skeleton, the mode process is a finite-state endogenous environment. The invariant-distribution problem is then recast as a regularity problem for a random or switched Markov kernel.

Across these applications, several structural dichotomies recur. The environment may be exogenous or state-coupled, stationary or time-inhomogeneous, Markovian or only ergodic, discrete or continuous in time. The marginal state process may fail to be Markov, while the augmented state–environment process remains Markov. Quenched and annealed statements need not agree in strength or interpretation. The cited literature therefore presents MCREs not as a single theorem class, but as a unifying perspective on random-kernel dynamics, random operator cocycles, state-dependent environments, and induced effective generators.