Imprecise Continuous-Time Markov Chains

• ICTMCs are defined by a convex set of admissible rate matrices, allowing for the modeling of uncertainty in transition dynamics.
• They enable robust, nonparametric inference by computing lower and upper envelopes over behaviors using lower transition rate operators.
• Efficient algorithms, such as adaptive time discretization and state lumping, ensure scalable computations even in high-dimensional and hidden Markov extensions.

Imprecise Continuous-Time Markov Chains (ICTMCs) generalize classical continuous-time Markov chains (CTMCs) by replacing the exact specification of the generator (or rate matrix) with a non-singleton set of admissible generators, thereby capturing model uncertainty and epistemic imprecision. This extension enables rigorous nonparametric inference under incomplete knowledge of transition dynamics, supporting robust predictions, estimation, and decision making in domains where precise parameter information is unavailable or unwarranted.

1. Mathematical Foundations and Formal Definition

An ICTMC is specified by a finite state space $X$, a nonempty, closed, convex set of rate matrices $\mathcal{Q} \subset \mathbb{R}^{|X|\times|X|}$ (with row-sum-zero and non-negative off-diagonal entries), and a set of initial distributions $\mathcal{M}$ on $X$ (Krak et al., 2016, Krak et al., 2018). Three nested classes of ICTMCs can be formalized:

• The set $$\mathbb{P}^{\mathrm{WHM}}_{\mathcal{Q},\mathcal{M}}$$ includes all homogeneous Markov processes with generator in $\mathcal{Q}$ and initial law in $\mathcal{M}$.
• The set $\mathbb{P}^{\mathrm{WM}}_{\mathcal{Q},\mathcal{M}}$ allows time-inhomogeneous but Markovian processes, subject to the requirement that all rate matrices along trajectories at all times belong to $\mathcal{Q}$.
• The broadest set, $\mathbb{P}^{\mathrm{W}}_{\mathcal{Q},\mathcal{M}}$, admits all well-behaved processes whose outer partial derivatives lie in $\mathcal{Q} \subset \mathbb{R}^{|X|\times|X|}$0 (Krak et al., 2016).

The principal object of study is not a single CTMC, but the lower (and upper) envelope over all behaviors admitted by this structure—most commonly represented by the lower expectation operator or, equivalently, the evolution semigroup generated by the lower envelope of the rate set (Krak et al., 2016, Bock, 2016).

2. Lower Transition Rate Operators and Dynamic Semigroups

The central operator-theoretic concept in ICTMCs is the lower transition rate operator $\mathcal{Q} \subset \mathbb{R}^{|X|\times|X|}$1, defined by

$\mathcal{Q} \subset \mathbb{R}^{|X|\times|X|}$2

for every $\mathcal{Q} \subset \mathbb{R}^{|X|\times|X|}$3 in the space of real-valued functions on $\mathcal{Q} \subset \mathbb{R}^{|X|\times|X|}$4 (Krak et al., 2016, Bock, 2016).

ICTMC dynamics are governed by the (nonlinear, superadditive) Kolmogorov-type ordinary differential equation: $\mathcal{Q} \subset \mathbb{R}^{|X|\times|X|}$5 This semigroup $\mathcal{Q} \subset \mathbb{R}^{|X|\times|X|}$6 generalizes the matrix exponential $\mathcal{Q} \subset \mathbb{R}^{|X|\times|X|}$7 of precise theory. In the additive (singleton-$\mathcal{Q} \subset \mathbb{R}^{|X|\times|X|}$8) case, it reduces to the standard semigroup, but in the presence of imprecision the dynamics are non-linear and rich in structure (Bock, 2016, Erreygers et al., 2017).

Properties include:

• Semigroup: $\mathcal{Q} \subset \mathbb{R}^{|X|\times|X|}$9.
• Superadditivity and positive homogeneity: Inherited from the definition of $\mathcal{M}$0.
• Norm-differentiability and generalized Kolmogorov equations: Both forward and backward equations admit consistent operator-norm derivatives (Krak et al., 2016).

3. Parameter Estimation, Imprecise Priors, and the IDM Construction

For homogeneous CTMCs, imprecise probabilistic approaches to estimator construction have been developed that parallel Bayesian treatment, but relax prior assumptions via set-valued models.

A crucial construction is the imprecise Dirichlet model (IDM)-based estimator for the transition rate matrix from a finite trajectory. The estimator is formed by:

• Counting observed transitions $\mathcal{M}$1 and sojourn durations $\mathcal{M}$2.
• Defining a set of conjugate Gamma-type priors over the $\mathcal{M}$3, with a "strength" hyperparameter $\mathcal{M}$4 and location matrix $\mathcal{M}$5, allowed to vary over the full simplex (Krak et al., 2018).
• In the continuous-time limit (via discretized Dirichlet–multinomial analysis), the set of plausible rate matrices is

$\mathcal{M}$6

so that $\mathcal{M}$7, subject to row-sum-zero (Krak et al., 2018).

Key properties:

• The parameter $\mathcal{M}$8 regulates the degree of imprecision; $\mathcal{M}$9 retrieves the maximum likelihood estimate, $X$0 broadens each rate to an interval.
• The resulting rate set $X$1 is convex and closed, enabling efficient inference via a closed-form lower transition rate operator:

$X$2

• This estimator delivers robust envelope estimates honoring observed data, incorporating up to $X$3 “pseudo‐transitions” per row, and remains amenable to numerically stable computation of lower/upper expectations (Krak et al., 2018).

4. Inference, Lower Expectations, and Efficient Algorithms

Robust inference in ICTMCs is performed through lower (and upper) expectation functionals: $X$4 which can be computed exactly as the action of the lower semigroup of transition operators on $X$5, under convex, separately-row-specified $X$6 (Krak et al., 2016). For time-slice or multi-slice functions, explicit iterative schemes (forward-Euler, power-triangle) provide polynomial-time guarantees with controlled error bounds (Krak et al., 2016, Erreygers et al., 2017).

Algorithmic advances include:

• Uniform and adaptive time-discretization methods: Choose step count and step-size to ensure error tolerance, tracking the cumulative error and halting once acceptable precision is reached (Erreygers et al., 2017).
• Error bounds: Explicitly controlled via properties of the operator norm and contraction coefficients. For ergodic ICTMCs, stationary distributions and time-asymptotic expectations can be approximated to arbitrary accuracy (Erreygers et al., 2017).
• Normal-cone-based methods: By exploiting the piecewise-linear structure of the rate polyhedron, the optimal solution to the imprecise Kolmogorov backward equation can be traced within a fixed normal cone for maximal intervals, dramatically reducing the number of LP solves relative to standard grid methods (Škulj, 2020).

5. Structural Properties, Ergodicity, and Limit Results

The asymptotic behavior of ICTMCs, including ergodicity and convergence, is governed by the properties of the lower transition rate operator.

• Ergodicity criterion: An ICTMC is ergodic (i.e., lower expectations converge to a unique state-independent value for any bounded observable) if and only if there exists a top class accessible from every state (upper reachability), and every state can lower-reach this class based on the underlying operator structure (Bock, 2016).
• Comparison with precise theory: In the singleton-$X$7 case, these reachability notions collapse to classical irreducibility.
• Generalized limit theorems: The limiting lower expectation is a unique linear functional in the ergodic case, and efficient reachability-based checks yield ergodicity certification without full ODE solution (Bock, 2016).

Specialized quantities such as expected hitting times can be framed via generalized nonlinear systems, with equivalence among homogeneous, inhomogeneous, and fully imprecise models under broad conditions. Minimal nonnegative solutions to ($X$8)(x) = -1, for $X$9, with boundary condition $$\mathbb{P}^{\mathrm{WHM}}_{\mathcal{Q},\mathcal{M}}$$0, deliver tight lower bounds for expected hitting times (Krak, 2022).

6. Large-Scale Inference, Lumpability, and Model Reduction

State-space explosion in large CTMCs motivates aggregation (lumping) and imprecision to attain computational tractability.

• Lumping with imprecise CTMCs: By aggregating states via a surjective mapping $$\mathbb{P}^{\mathrm{WHM}}_{\mathcal{Q},\mathcal{M}}$$1, one defines a family of lumped rate matrices $$\mathbb{P}^{\mathrm{WHM}}_{\mathcal{Q},\mathcal{M}}$$2, whose lower and upper envelope operators induce an ICTMC on the reduced state space (Erreygers et al., 2018).
• For lumped queries (i.e., observables invariant on fibers of $$\mathbb{P}^{\mathrm{WHM}}_{\mathcal{Q},\mathcal{M}}$$3), tight lower and upper bounds can be computed by evolving on $$\mathbb{P}^{\mathrm{WHM}}_{\mathcal{Q},\mathcal{M}}$$4 using the associated lower transition semigroup, bypassing the full original state space.
• Convergence and correctness are guaranteed under irreducibility and convex/separately-specified-row conditions, yielding tractable bounds even for state spaces infeasible to enumerate (Erreygers et al., 2018).

7. Extensions: Hidden Markov Chains and Output-Observation Models

ICTMCs with output observations (hidden Markov structures) generalize the inference problem to settings with partial or noisy observations.

• Robust inference for imprecise continuous-time hidden Markov chains combines the ICTMC model with output emission distributions, both in discrete and continuous spaces (Krak et al., 2017).
• The update of lower expectations given output observations is reduced to solving a generalized Bayes-type rule, wherein a coherent lower prevision inequality characterizes the updated expectation.
• The inference is performed efficiently via polynomial-time dynamic programming (forward-backward) and bisection methods, leveraging the structure of the lower transition semigroup (Krak et al., 2017).

---

References:

• (Krak et al., 2016) Imprecise Continuous-Time Markov Chains
• (Bock, 2016) Convergence of Imprecise Continuous-Time Markov Chains
• (Erreygers et al., 2017) Imprecise Continuous-Time Markov Chains: Efficient Computational Methods with Guaranteed Error Bounds
• (Krak et al., 2018) An Imprecise Probabilistic Estimator for the Transition Rate Matrix of a Continuous-Time Markov Chain
• (Škulj, 2020) Computing bounds for imprecise continuous-time Markov chains using normal cones
• (Erreygers et al., 2018) Computing Inferences for Large-Scale Continuous-Time Markov Chains by Combining Lumping with Imprecision
• (Krak, 2022) Hitting Times for Continuous-Time Imprecise-Markov Chains
• (Krak et al., 2017) Efficient Computation of Updated Lower Expectations for Imprecise Continuous-Time Hidden Markov Chains