---
title: Imprecise Continuous-Time Markov Chains
url: https://www.emergentmind.com/topics/imprecise-continuous-time-markov-chains-ictmcs
type: topic
---

# Imprecise Continuous-Time Markov Chains

Imprecise Continuous-Time Markov Chains (ICTMCs) generalize classical continuous-time Markov chains (CTMCs) by replacing the exact specification of the generator (or rate matrix) with a non-singleton set of admissible generators, thereby capturing model uncertainty and epistemic imprecision. This extension enables rigorous nonparametric inference under incomplete knowledge of transition dynamics, supporting robust predictions, estimation, and decision making in domains where precise parameter information is unavailable or unwarranted.

## 1. Mathematical Foundations and Formal Definition

An ICTMC is specified by a finite state space \( X \), a nonempty, closed, convex set of rate matrices \( \mathcal{Q} \subset \mathbb{R}^{|X|\times|X|} \) (with row-sum-zero and non-negative off-diagonal entries), and a set of initial distributions \( \mathcal{M} \) on \( X \) [1611.05796], [1804.01330]. Three nested classes of ICTMCs can be formalized:

- The set \( \mathbb{P}^{\mathrm{WHM}}_{\mathcal{Q},\mathcal{M}} \) includes all homogeneous Markov processes with generator in \( \mathcal{Q} \) and initial law in \( \mathcal{M} \).
- The set \( \mathbb{P}^{\mathrm{WM}}_{\mathcal{Q},\mathcal{M}} \) allows time-inhomogeneous but Markovian processes, subject to the requirement that all rate matrices along trajectories at all times belong to \( \mathcal{Q} \).
- The broadest set, \( \mathbb{P}^{\mathrm{W}}_{\mathcal{Q},\mathcal{M}} \), admits all well-behaved processes whose outer partial derivatives lie in \( \mathcal{Q} \) [1611.05796].

The principal object of study is not a single CTMC, but the lower (and upper) envelope over all behaviors admitted by this structure—most commonly represented by the lower expectation operator or, equivalently, the evolution semigroup generated by the lower envelope of the rate set [1611.05796], [1602.05478].

## 2. Lower Transition Rate Operators and Dynamic Semigroups

The central operator-theoretic concept in ICTMCs is the **lower transition rate operator** \( \underline Q \), defined by
\[
[\underline Q f](x) = \inf_{Q \in \mathcal{Q}} \sum_{y \in X} Q(x,y) f(y)
\]
for every \( f \) in the space of real-valued functions on \( X \) [1611.05796], [1602.05478].

ICTMC dynamics are governed by the (nonlinear, superadditive) Kolmogorov-type ordinary differential equation:
\[
\frac{d}{dt} \, \underline{T}_t f = \underline Q(\underline{T}_t f)\,, \qquad \underline{T}_0 f = f
\]
This semigroup \( \{\underline{T}_t\}_{t \ge 0} \) generalizes the matrix exponential \( e^{Qt} \) of precise theory. In the additive (singleton-\( \mathcal{Q} \)) case, it reduces to the standard semigroup, but in the presence of imprecision the dynamics are non-linear and rich in structure [1602.05478], [1702.07150].

Properties include:
- **Semigroup:** \( \underline{T}_{t+s} = \underline{T}_t \circ \underline{T}_s \).
- **Superadditivity and positive homogeneity:** Inherited from the definition of \( \underline Q \).
- **Norm-differentiability and generalized Kolmogorov equations:** Both forward and backward equations admit consistent operator-norm derivatives [1611.05796].

## 3. Parameter Estimation, Imprecise Priors, and the IDM Construction

For homogeneous CTMCs, imprecise probabilistic approaches to estimator construction have been developed that parallel Bayesian treatment, but relax prior assumptions via set-valued models.

A crucial construction is the **imprecise Dirichlet model (IDM)**-based estimator for the transition rate matrix from a finite trajectory. The estimator is formed by:
- Counting observed transitions \( n_{xy} \) and sojourn durations \( d_x \).
- Defining a set of conjugate Gamma-type priors over the \( q_{xy} \), with a "strength" hyperparameter \( s \) and location matrix \( A \), allowed to vary over the full simplex [1804.01330].
- In the continuous-time limit (via discretized Dirichlet–multinomial analysis), the set of plausible rate matrices is
  \[
  \mathcal{Q}_s = \left\{ Q \in \mathcal{Q} : q_{xy} = \frac{s\,A(x,y) + n_{xy}}{d_x} \text{ for some } A \in \mathcal{T} \right\}
  \]
  so that \( q_{xy} \in \left[ \frac{n_{xy}}{d_x}, \frac{n_{xy} + s}{d_x} \right] \), subject to row-sum-zero [1804.01330].

Key properties:
- The parameter \( s \) regulates the **degree of imprecision**; \( s=0 \) retrieves the maximum likelihood estimate, \( s>0 \) broadens each rate to an interval.
- The resulting rate set \( \mathcal{Q}_s \) is convex and closed, enabling efficient inference via a closed-form lower transition rate operator:
  \[
  [\underline Q h](x) = \frac{s}{d_x} \min_y [h(y) - h(x)] + \sum_{y \ne x} \frac{n_{xy}}{d_x} [h(y) - h(x)]
  \]
- This estimator delivers robust envelope estimates honoring observed data, incorporating up to \( s \) “pseudo‐transitions” per row, and remains amenable to numerically stable computation of lower/upper expectations [1804.01330].

## 4. Inference, Lower Expectations, and Efficient Algorithms

Robust inference in ICTMCs is performed through lower (and upper) expectation functionals:
\[
\underline{\mathbb{E}}_{\mathcal{P}}[f(X_{u},X_s)\mid X_u=x_u] = \inf_{P \in \mathcal{P}} \mathbb{E}_P[ f(X_{u},X_s) \mid X_u = x_u ]
\]
which can be computed exactly as the action of the lower semigroup of transition operators on \( f \), under convex, separately-row-specified \( \mathcal{Q} \) [1611.05796]. For time-slice or multi-slice functions, explicit iterative schemes (forward-Euler, power-triangle) provide polynomial-time guarantees with controlled error bounds [1611.05796], [1702.07150].

Algorithmic advances include:
- **Uniform and adaptive time-discretization methods:** Choose step count and step-size to ensure error tolerance, tracking the cumulative error and halting once acceptable precision is reached [1702.07150].
- **Error bounds:** Explicitly controlled via properties of the operator norm and contraction coefficients. For ergodic ICTMCs, stationary distributions and time-asymptotic expectations can be approximated to arbitrary accuracy [1702.07150].
- **Normal-cone-based methods:** By exploiting the piecewise-linear structure of the rate polyhedron, the optimal solution to the imprecise Kolmogorov backward equation can be traced within a fixed normal cone for maximal intervals, dramatically reducing the number of LP solves relative to standard grid methods [2012.01029].

## 5. Structural Properties, Ergodicity, and Limit Results

The asymptotic behavior of ICTMCs, including ergodicity and convergence, is governed by the properties of the lower transition rate operator.

- **Ergodicity criterion:** An ICTMC is ergodic (i.e., lower expectations converge to a unique state-independent value for any bounded observable) if and only if there exists a top class accessible from every state (upper reachability), and every state can lower-reach this class based on the underlying operator structure [1602.05478].
- **Comparison with precise theory:** In the singleton-\( \mathcal{Q} \) case, these reachability notions collapse to classical irreducibility.
- **Generalized limit theorems:** The limiting lower expectation is a unique linear functional in the ergodic case, and efficient reachability-based checks yield ergodicity certification without full ODE solution [1602.05478].

Specialized quantities such as expected hitting times can be framed via generalized nonlinear systems, with equivalence among homogeneous, inhomogeneous, and fully imprecise models under broad conditions. Minimal nonnegative solutions to (\( \underline Q h \))(x) = -1, for \( x \notin A \), with boundary condition \( h(A)=0 \), deliver tight lower bounds for expected hitting times [2202.13111].

## 6. Large-Scale Inference, Lumpability, and Model Reduction

State-space explosion in large CTMCs motivates aggregation (lumping) and imprecision to attain computational tractability.

- **Lumping with imprecise CTMCs:** By aggregating states via a surjective mapping \( \Lambda: X \to S^* \), one defines a family of lumped rate matrices \( \mathcal{Q}^* \), whose lower and upper envelope operators induce an ICTMC on the reduced state space [1804.01020].
- For lumped queries (i.e., observables invariant on fibers of \( \Lambda \)), tight lower and upper bounds can be computed by evolving on \( S^* \) using the associated lower transition semigroup, bypassing the full original state space.
- Convergence and correctness are guaranteed under irreducibility and convex/separately-specified-row conditions, yielding tractable bounds even for state spaces infeasible to enumerate [1804.01020].

## 7. Extensions: Hidden Markov Chains and Output-Observation Models

ICTMCs with output observations (hidden Markov structures) generalize the inference problem to settings with partial or noisy observations.

- Robust inference for imprecise continuous-time hidden Markov chains combines the ICTMC model with output emission distributions, both in discrete and continuous spaces [1702.06791].
- The update of lower expectations given output observations is reduced to solving a generalized Bayes-type rule, wherein a coherent lower prevision inequality characterizes the updated expectation.
- The inference is performed efficiently via polynomial-time dynamic programming (forward-backward) and bisection methods, leveraging the structure of the lower transition semigroup [1702.06791].

---
**References:**
- [1611.05796] Imprecise Continuous-Time Markov Chains
- [1602.05478] Convergence of Imprecise Continuous-Time Markov Chains
- [1702.07150] Imprecise Continuous-Time Markov Chains: Efficient Computational Methods with Guaranteed Error Bounds
- [1804.01330] An Imprecise Probabilistic Estimator for the Transition Rate Matrix of a Continuous-Time Markov Chain
- [2012.01029] Computing bounds for imprecise continuous-time Markov chains using normal cones
- [1804.01020] Computing Inferences for Large-Scale Continuous-Time Markov Chains by Combining Lumping with Imprecision
- [2202.13111] Hitting Times for Continuous-Time Imprecise-Markov Chains
- [1702.06791] Efficient Computation of Updated Lower Expectations for Imprecise Continuous-Time Hidden Markov Chains

Source: https://www.emergentmind.com/topics/imprecise-continuous-time-markov-chains-ictmcs