Markov-Modulated Markov Chain

Updated 8 July 2026

Markov-Modulated Markov Chains are two-component processes where a modulator chain governs the transition dynamics of the secondary process.
Different constructions, such as product-state generators and stationary averaging, enable rigorous stability and asymptotic analysis.
These models have practical applications in phylogenetics, finance, wireless networks, and more, addressing regime-switching challenges.

A Markov-modulated Markov chain is a stochastic process whose parameters or transition mechanisms are governed by a Markov chain. In the cited literature, the term covers a discrete-time two-component chain $(X^t,Y^t)$ in which the $X$ -component is itself a Markov chain, continuous-time compound chains on Cartesian-product state spaces, and special cases such as counting processes, diffusions, affine processes, and reflected Brownian motions whose intensities, generators, coefficients, or boundaries depend on an underlying regime process [(Foss et al., 2011); (Baele et al., 2019); (Mandjes et al., 2016)]. A recurring structural property is that the primary dynamics are Markov once the modulator is included in the state, so the joint process remains Markovian while regime switching is represented explicitly (Mandjes et al., 2016).

1. Core definition and terminology

In one canonical discrete-time formulation, a Markov-modulated Markov chain is a two-component Markov chain $(X^t,Y^t)$ , $t=0,1,2,\ldots$ , where the $X$ -component forms a Markov chain itself. The $X$ -component is the modulator, and the $Y$ -component evolves with transition behavior that can depend on both the current and previous states of $X^t$ and $Y^t$ . The analysis in this setting typically assumes that $(X^t)$ is Harris-ergodic, with a unique stationary distribution $X$ 0 (Foss et al., 2011).

In continuous time, the same idea is expressed by letting a regime process switch the local law of a second process. In phylogenetics, the modulating process is a CTMC over $X$ 1 categories or models, and each regime $X$ 2 has its own substitution model, with rate matrix $X$ 3, stationary distribution $X$ 4, and possibly a rate multiplier $X$ 5. In counting-process models, the intensity is a function of a finite-state Markov chain. In affine-process models, some affine coefficients are a function of an exogenous Markov process. These formulations differ in state space and generator structure, but they share the regime-switching mechanism (Baele et al., 2019, Mandjes et al., 2016, Kurt et al., 2021).

The modulator need not play the same inferential role in every model. In some constructions it is hidden, as in self-exciting Markov-modulated counting processes based on observation of a jump process; in others it is an exogenous or directly modeled state process, as in regime-switching default, diffusion, or market models [(Cohen et al., 2013); (Mandjes et al., 2016); (D'Auria et al., 2022)].

2. Canonical mathematical constructions

A standard continuous-time construction places the process on the Cartesian product of regimes and observable states. If the observable character alphabet has size $X$ 6 and the modulator has $X$ 7 regimes, then the combined process has state space size $X$ 8. Its generator is

$X$ 9

where $(X^t,Y^t)$ 0 is the switching-rate matrix and $(X^t,Y^t)$ 1 is the identity matrix of dimension $(X^t,Y^t)$ 2. Finite-time transition probabilities are then computed by

$(X^t,Y^t)$ 3

The block-diagonal term represents within-regime evolution, while $(X^t,Y^t)$ 4 switches regimes without changing the current character state (Baele et al., 2019).

A second canonical construction uses averaging over the stationary law of the modulator. For the discrete-time chain $(X^t,Y^t)$ 5, the auxiliary chain $(X^t,Y^t)$ 6 is defined by stationary averaging of the $(X^t,Y^t)$ 7-transition probabilities:

$(X^t,Y^t)$ 8

This auxiliary chain is the central object in averaged Lyapunov arguments for recurrence and stability (Foss et al., 2011).

A third construction appears in regime-switching counting and diffusion models. For a Markov-modulated counting process, one writes $(X^t,Y^t)$ 9 or, more generally, $t=0,1,2,\ldots$ 0. For a rapidly switching Markov-modulated diffusion, one writes

$t=0,1,2,\ldots$ 1

In both cases, the local law is selected by the current regime, and the pair consisting of regime and signal is analyzed as a Markov process (Mandjes et al., 2016, Huang et al., 2015).

Formulation	State description	Representative relation
Product CTMC	Regime $t=0,1,2,\ldots$ 2 observable state	$t=0,1,2,\ldots$ 3
Averaged discrete-time chain	Modulator $t=0,1,2,\ldots$ 4 and component $t=0,1,2,\ldots$ 5	$t=0,1,2,\ldots$ 6 uses stationary averaging over $t=0,1,2,\ldots$ 7
Regime-switching counting or diffusion	Signal plus finite-state or fast Markov environment	$t=0,1,2,\ldots$ 8, or $t=0,1,2,\ldots$ 9

These constructions are not mutually exclusive. A plausible implication is that product-state generators and stationary averaging are the two main technical devices by which Markov modulation is turned into tractable Markov analysis.

3. Stability, averaging, and asymptotic regimes

For discrete-time Markov-modulated chains, positive recurrence can be obtained from an averaged Lyapunov drift criterion. If there exists a non-negative test function $X$ 0, a measurable function $X$ 1 bounded by $X$ 2, and a function $X$ 3 such that

$X$ 4

with stationary drift

$X$ 5

then the auxiliary chain $X$ 6 is positive recurrent and the same conditions are sufficient for positive recurrence of the original chain $X$ 7. The multivariate extension applies coordinatewise Lyapunov functions $X$ 8 and yields positive recurrence of sets of the form $X$ 9 (Foss et al., 2011).

Rapid switching produces a different kind of simplification. When the transition matrix of the modulating chain is inflated by a factor tending to infinity, the Markov-modulated counting process converges to an equivalent process with intensity replaced by its average under the invariant distribution of the modulator:

$X$ 0

For a single obligor default process,

$X$ 1

and, for an MM Poisson process, the limit is an ordinary Poisson process with intensity $X$ 2 (Mandjes et al., 2016). Related diffusion approximations for a Markov-modulated binomial counting process are obtained by semimartingale central limit theorems; depending on the order of the limits $X$ 3 and $X$ 4, different approximations arise (Spreij et al., 2018).

In small-noise, rapid-switching diffusion models, the pair consisting of the diffusion path and the occupation measure of the Markov chain satisfies a joint sample-path large deviations principle. The proof structure is based on exponential tightness and a local large deviations principle, and contraction then yields large deviations principles for the separate components (Huang et al., 2015).

A further asymptotic regime appears in Markov-modulated generalized Ornstein-Uhlenbeck processes. There, the stationary distribution of the associated Markov-modulated affine recursion has power-law tail asymptotics with tail index $X$ 5 characterized by the spectral equation $X$ 6, where $X$ 7 is the dominant eigenvalue of the Cramér transform matrix $X$ 8. The analysis uses Markov renewal theory and extends Goldie’s implicit renewal theory to the Markov-modulated setting (Alsmeyer et al., 14 Jan 2026).

4. Filtering, smoothing, and computation

When the modulator is hidden, the central problem is to infer the regime process from the observed modulated process. For a self-exciting counting process whose intensity depends on a hidden finite-state Markov chain, the optimal filter and smoother for the hidden chain are finite dimensional and available in closed form. Under a reference probability $X$ 9, with $Y$ 0 a unit-rate Poisson process independent of $Y$ 1, the likelihood process is

$Y$ 2

and the unnormalized filtered density

$Y$ 3

satisfies

$Y$ 4

The normalized filter is $Y$ 5, while smoothing is obtained from a backward equation for $Y$ 6 with $Y$ 7 and $Y$ 8 (Cohen et al., 2013).

In phylogenetic MMMs, the main computational bottleneck is exponentiation of the compound generator $Y$ 9. For $X^t$ 0 models and state-space size $X^t$ 1, general eigen decomposition is required when the $X^t$ 2 differ, with complexity $X^t$ 3. A general MMM framework was implemented in BEAST through flexible XML specification, and the implementation exploits BEAGLE, a high-performance computational library for phylogenetic inference. Empirical benchmarks reported that GPUs dramatically reduced runtime compared to CPU-only computations (Baele et al., 2019).

Tracking problems also appear when the modulated object is not a hidden regime but a changing empirical distribution. In a Markov-modulated duplication-deletion random graph, a stochastic approximation algorithm is used to track the empirical degree distribution as it evolves over time, and the tracking performance is analyzed in terms of mean square error together with a functional central limit theorem for the asymptotic tracking error (Hamdi et al., 2013).

5. Applications across scientific domains

In phylogenetics, Markov-modulated continuous-time Markov chains were introduced to model heterogeneity in the substitution process over time in a site-specific manner. The substitution process, including relative character exchange rates and the overall substitution rate, is allowed to vary across lineages. Empirical studies on bacterial, viral, and plastid genome evolution showed that MMMs impact phylogenetic tree estimation and can substantially improve model fit compared to standard substitution models. For plant plastid genes $X^t$ 4 and $X^t$ 5, log Bayes factors of up to 356 units over standard models were reported, and for the influenza virus HA gene a heterotachy model with switching between four rate multipliers outperformed standard ASRV by a large margin, with log Bayes factors around 2000 (Baele et al., 2019).

In wireless networks, a two-component Markov-modulated chain was used to analyze systems governed by two randomized protocols. The applications involve multiple stations with separate red and green queues, and the stability criteria are necessary and sufficient in the provided examples. Stability regions for arrival rates are derived explicitly by comparing long-run service rates, averaged over the stationary distribution of environmental conditions, to input rates through Lyapunov drift tests (Foss et al., 2011).

In finance and credit risk, Markov-modulated counting processes are used to model default events of some companies, multiple-obligor default counts, and regime-dependent market activity. For a single obligor default process, the one-jump dynamics are modulated by the current Markov state, while for $X^t$ 6 obligors the counting process $X^t$ 7 records the total defaults. In the Markov-modulated binomial counting process, each obligor’s failure/default intensity depends on an underlying Markov chain, and diffusion approximations are established when the number of obligors increases unboundedly and/or the modulating chain is rapidly switched (Mandjes et al., 2016, Spreij et al., 2018).

High-frequency event data provide another application. In a self-exciting Markov-modulated counting framework fitted to TAQ data for SPY during the flash crash of 6 May 2010, the filter successfully detects the onset and recovery of the flash crash, even before the price decline is fully underway, using only trade counts per second after rescaling by $X^t$ 8 to address overdispersion (Cohen et al., 2013).

In reflected diffusion models, a two-sided reflected Markov-modulated Brownian motion has drift, diffusion coefficient, and two boundaries jointly modulated by a finite-state irreducible continuous-time Markov chain. The state-dependent boundaries can generate jumps at regime-switch times because the current level is projected onto the new admissible interval. The same framework is applied to fluid queues and to dividend payout problems with lower barrier zero and an upper barrier subject to control (D'Auria et al., 2011).

In mean-field control and game theory, finite-state Markov chains perturb the coefficients of mean-field type SDEs through a positive function $X^t$ 9, and the resulting framework supports a sufficient stochastic maximum principle, propagation of chaos for the interacting particle system, and approximate Nash equilibrium for the Markov-modulated mean-field game (Tai, 2014).

In mathematical finance with filtration enlargement, a binary discrete-time Markov chain modulates the interest rate, drift, and volatility of the risky asset, and the modulation may be anticipative with respect to the future of the Brownian motion. On intervals between Markov jumps, the enlarged filtration yields a semimartingale decomposition for the Brownian motion and supports optimal portfolio utility analysis in both complete and incomplete markets (D'Auria et al., 2022).

In dynamic network models, the duplication and deletion probabilities of a random graph evolve according to the realization of a finite-state Markov chain. The resulting Markov-modulated duplication-deletion random graph is used to analyze asymptotic degree distributions, search delay, and adaptive tracking of evolving empirical degree distributions, with social networks as the motivating application (Hamdi et al., 2013).

Several standard models appear as special or limiting cases of Markov modulation. In phylogenetics, if the switching rates between models are set to zero and all $Y^t$ 0 are the same, the Markov-modulated model recovers the classical among-site rate variation model with static heterogeneity. If the global switching rate $Y^t$ 1 approaches zero, the same framework becomes a mixture model with fixed weights. The same literature states that MMMs can encompass ASRV, mixture, and covarion models as special cases (Baele et al., 2019).

A common misconception is that Markov modulation only changes a scalar rate. In the cited work, modulation acts on substitution matrices, base frequencies, exchangeabilities, and rate multipliers in phylogenetics; on intensities in counting processes; on drift, diffusion, and barriers in reflected Brownian motion; on coefficients of mean-field SDEs; and on market coefficients such as $Y^t$ 2, $Y^t$ 3, and $Y^t$ 4 in regime-switching finance [(Baele et al., 2019); (D'Auria et al., 2011); (Tai, 2014); (D'Auria et al., 2022)].

A second misconception is that the modulator must always be hidden. Hidden finite-state chains are central to filtering and smoothing for self-exciting counting processes, but other frameworks treat the modulator as an exogenous process or as part of the modeled state itself. Markov-modulated affine processes explicitly generalize standard affine processes by allowing coefficients to depend on an exogenous Markov process, and standard affine processes are recovered when the coefficients are independent of that process [(Cohen et al., 2013); (Kurt et al., 2021)].

A third misconception is that the more parameter-rich model is necessarily preferred. In phylogenetic simulations under GTR, $Y^t$ 5, and $Y^t$ 6, marginal likelihood estimation using generalized stepping-stone correctly preferred the generative model in each case and did not systematically prefer more complex MMMs; when data were generated under standard models, MMMs with many parameters were penalized (Baele et al., 2019).

Taken together, the cited literature indicates that Markov-modulated Markov chains are best understood not as a single narrow model class, but as a general regime-switching methodology. The modulator may be finite-state or more general, observed or hidden, slow or rapidly switching; the modulated object may be a chain, a counting process, a diffusion, an affine process, a reflected Brownian motion, or a random graph. What remains invariant is the strategy of embedding time-varying local dynamics into a larger Markovian state description.