---
title: Neural and Mixture CTMCs
url: https://www.emergentmind.com/topics/neural-and-mixture-ctmcs
type: topic
---

# Neural and Mixture CTMCs

Neural and Mixture Continuous-Time Markov Chains (CTMCs) refer to a generalization of classical CTMC frameworks in which the transition rate functions are nonparametrically parameterized, allowing nonlinear dependencies on the current discrete state and covariates. Neural CTMCs employ neural networks to learn the propensity or transition-rate functions directly from fully observed sample paths, with the objective of inferring or accurately reconstructing complex stochastic dynamics underlying systems such as chemical reaction networks, population dynamics, and gene regulatory networks [2212.05378].

## 1. Definition and Mathematical Setup

A CTMC is defined over a (potentially infinite) discrete state space $\Omega$ with piecewise constant sample paths $X(t) \in \Omega$ and transitions occurring at random times. The primary object is the transition rate $\lambda_{ij}(x,c)$, representing the instantaneous rate at which the process jumps from state $i$ to $j$ given covariates $c$, where $x$ collects state and covariate information. All possible transitions are assumed to correspond to a finite set of known reactions $ρ=1,\ldots,r$ with prescribed stoichiometric changes $\Delta S_{ρ}$:
$$
\lambda_{i,i+\Delta S_ρ}(x, c) = α_ρ(x, c), \quad \lambda_{ij} = 0 \text{ otherwise}.
$$
The total exit rate from state $i$ is $\lambda_{ii}(x,c) = \sum_{ρ=1}^r α_ρ(x,c)$, and the sojourn time in each state is exponentially distributed with this rate. The process is constructed such that for infinitesimal $dt$,
$$
P[X(t+dt) = j \mid X(t) = i, C(t) = c] = \lambda_{ij}(x, c)\,dt + o(dt).
$$
The state vector may comprise multiple species counts $S(t) \in \mathbb{N}^s$ and continuous covariates $C(t) \in \mathbb{R}^c$, concatenated as $x = [S^T, C^T]^T \in \mathbb{R}^{s+c}$.

## 2. Neural Parameterization of Rate Functions

Unlike mass-action or linear log-linear forms, Neural CTMCs do not impose any fixed parametric dependency. Each $\alpha_ρ(x, c) \geq 0$ is parameterized as an output of a neural network, enabling arbitrary nonlinear dependence on both the discrete state and covariates. Collectively,
$$
\alpha(x, c) = [α_1(x, c), ..., α_r(x, c)]^T,
$$
and each component is modeled as a neural function with nonlinearity and output constraining ($\text{softplus}(z) = \log(1+\exp(z))$) to guarantee nonnegativity:
- **In mass-action kinetics**: $α_ρ(x) = k_ρ \prod_i S_i(t)^{φ_{iρ}}$
- **In Neural CTMCs**: $α_ρ(x, c) = \sigma_2(f_ρ(x, c;θ))$, with $f_ρ$ a neural network, $\sigma_2$ softplus.

This methodological extension enables the model to represent non-mass-action kinetics, encompassing a broader class of stochastic processes observed in empirical domains [2212.05378].

## 3. Neural Network Architectures and Implementation

Two principal network configurations are described for $\alpha(x, c)$:
- **Population Dynamics Example (Lotka–Volterra, $s=3$, no covariates)**:
  - Input: $x \in \mathbb{R}^3$
  - Fully connected neural network, 5 layers of 128 neurons with SELU activation, final output layer of $r=9$ neurons (one per reaction) with softplus.
- **Chemical Reaction Network Example (2 species + covariate $T$, $x \in \mathbb{R}^3$)**:
  - Input passed to a dense layer (3$\to$96), reshaped to (3$\times$32) array
  - Batch-wise 1D convolution across rows (10 filters of size 4), SELU activation
  - Flattened and passed through dense layers (290$\to$32$\to$32), then output layer (32$\to r=4$), softplus.

No additional regularization (e.g., dropout or weight decay) was employed beyond SELU/nonlinearity. Neural weights were set using framework default initialization. These choices ensure that the model has sufficient representational capacity to model complex propensities from observed trajectories.

## 4. Likelihood-Based Training and Loss Function

Given fully observed trajectories, the negative log-likelihood (NLL) is derived exactly from the CTMC process:
- Let $T_i = t_{i+1} - t_i$ denote the dwell time in state $x_i$ before the $i$th reaction $ρ_i$.
- The single-step likelihood is
$$
\left[ \sum_{j=1}^r α_j(x_i;θ) \right] \exp\left[ -T_i \sum_{j=1}^r α_j(x_i;θ) \right] \cdot \frac{α_{ρ_i}(x_i;θ)}{\sum_{j=1}^r α_j(x_i;θ)}
$$
- The trajectory-wise NLL becomes
$$
- \log L(θ) = \sum_{i=0}^{N-1} \left[ T_i \sum_{j=1}^r α_j(x_i;θ) - \log α_{ρ_i}(x_i;θ) \right]
$$
or, regrouped by reaction $ρ$,
$$
- \log L(θ) = \sum_{ρ=1}^r \sum_{ℓ=1}^{L^ρ} \left[ T^ρ_ℓ \sum_{j=1}^r α_j(X^ρ_ℓ;θ) - \log α_ρ(X^ρ_ℓ;θ) \right]
$$
where $L^ρ$ is the number of occurrences of reaction $ρ$, $X^ρ$ the pre-reaction states.

Gradient-based optimization (using, e.g., Adam) is then performed on this NLL, requiring only $\mathcal{O}(r\,\text{batch size})$ cost per iteration, a critical advantage for systems with large or infinite state spaces as no explicit $|\Omega| \times |\Omega|$ matrix is constructed [2212.05378].

## 5. Training Workflow and Computational Aspects

The supervised estimation procedure for N-CTMC can be summarized as:
1. Extract all tuples $(x_i, T_i, ρ_i)$ from observed trajectories.
2. Group by reaction $ρ$ to build state matrices $X^ρ$ and dwell-time vectors $T^ρ$.
3. Initialize neural parameters $θ$ with default framework initialization.
4. Iterate until convergence:
   - Evaluate the networks $α(X^ρ;θ)$ for all states and reactions.
   - Compute the loss per reaction as $\sum_{ℓ=1}^{L^ρ} [T^ρ_ℓ \sum_{j=1}^r α_j(X^ρ_ℓ;θ) - \log α_ρ(X^ρ_ℓ;θ)]$.
   - Aggregate and compute total loss across reactions.
   - Use automatic differentiation to obtain gradients, and update parameters.
   - Monitor loss change or gradient norm for stopping.

Gradients propagate efficiently through all network branches and sum operations. Efficient vectorization by reaction type enables scalability for complex reaction networks.

## 6. Empirical Benchmarks and Comparative Results

N-CTMCs were empirically evaluated against log-linear generalized linear model (GLM) baselines of the form $α_j(x) = \exp(w_j^T x + b_j)$ on multiple synthetic systems:
- **Birth–Death Process with Covariate:** Across $5{,}000$–$500{,}000$ transitions, N-CTMC mean absolute error (N-MAE) matches that of a counting-based MLE baseline, with both outperforming the GLM approach for sufficiently large datasets.
- **Population Dynamics (Lotka–Volterra Type, $r=9$ reactions):** With increasing numbers of trajectories ($100$, $1{,}000$, $10{,}000$), N-CTMC error (MAE, MSE) decreases with data volume and surpasses log-linear GLM baselines by over an order of magnitude at $10{,}000$ trajectories.
- **Chemical Reaction Network with Temperature Covariate:** For $100$–$500$ trajectories, N-CTMC achieves low MAE/W-MAE and MSE/W-MSE, while GLM baselines exhibit large bias, especially on states rarely visited in training.

| Setting                        | N-CTMC MAE   | GLM MAE    | N-CTMC MSE  | GLM MSE   |
|-------------------------------|-------------|------------|------------|-----------|
| Birth–Death (500k transitions) | $9.90\times 10^{-3}$ | —         | $1.50\times 10^{-4}$ | —        |
| Population (10,000 trj., 9 rxn) | $0.099$     | $1.028$    | $0.051$    | $3.208$   |
| ChemNet (500 trj., 4 rxn)       | $0.015$     | $0.250$    | $0.002$    | $1.345$   |

In all cases, N-CTMC converges to ground truth at a rate comparable to counting-based MLE (which is only feasible when the state space is not prohibitively large), and demonstrates substantial improvement over log-linear models, particularly in denser state-regions and in settings with nonlinear effects or covariates [2212.05378].

## 7. Significance and Generalization

By parameterizing CTMC transition rates with neural functions, N-CTMCs replace the explicit $|\Omega| \times |\Omega|$ rate matrix with a shared, state-dependent mapping $\alpha(x, c; θ)$. This parameterization enables the learning of arbitrarily complex, nonlinear, and covariate-dependent transition structures. The framework generalizes both mass-action and generalized linear models and applies naturally in birth–death processes, open-loop CTMC control, chemical kinetics with non-mass-action propensities, and temperature- or context-dependent population models. The ability to fit such models directly from fully observed trajectories, with rigorous likelihood-based gradient learning and empirical performance shown across diverse regimes, positions N-CTMCs as a scalable and expressive alternative to previous parametric approaches [2212.05378].

Source: https://www.emergentmind.com/topics/neural-and-mixture-ctmcs