Space-Time Markov Chain Approximation (STMCA)

• STMCA is a mathematical framework for approximating continuous-time diffusions on networked metric graphs via discrete Markov chains that match transition probabilities and waiting times.
• It employs a systematic discretization by partitioning graphs into cells and embedding a jump process to capture spatial heterogeneity and local timing with explicit convergence bounds.
• Applications of STMCA include modeling networks in electrical circuits, neuroscience, traffic flow, and chemical reactions, with potential extensions for adaptive schemes and rigorous simulation.

The Space-Time Markov Chain Approximation (STMCA) is a mathematically rigorous framework for approximating continuous-time stochastic processes—particularly general diffusions on networked metric graphs—by discrete-time, discrete-space Markov chains. The STMCA is designed to match, in expectation, the transition probabilities and conditional transition times of the target diffusion process. This construction directly quantifies the approximation error in terms of the geometry of the discretization, with explicit convergence results in Wasserstein distance. Applications span stochastic dynamics on networks, including physical, biological, and engineered systems.

1. Framework and Construction of STMCA

STMCA targets general diffusions on finite metric graphs $\Gamma$ where each edge is a one-dimensional interval endowed with edge-dependent scale ($s_e$) and speed measure ($m_e$) and vertices impose specific coupling or lateral conditions. The approximation process comprises the following steps:

1. Subdivision of the Metric Graph: Partition $\Gamma$ into a finite collection of cells (open intervals along edges and vertex neighborhoods).
2. Discrete Markov Chain Model: Define a discrete-state Markov chain on the set of cell centers, with two crucial properties:
   • The transition probability $p_{x, y} = \mathbb{P}_x\{ T_{U_x} = T_y \}$ matches the probability that the original diffusion, started at $x$, first exits cell $U_x$ through cell $U_y$.
   • The conditional expected transition time $$t_{x, y} = \mathbb{E}_x\{ T_{U_x}\mid T_{U_x} = T_y \}$$ matches the expected time for this exit.
3. Embedding: The jump process generated by the STMCA is constructed so that its sequence of states and jump times is stochastically embedded within the original diffusion.

This scheme produces a time-inhomogeneous (doubly asymmetric in space and time) random walk which captures both the spatial heterogeneity and local timing of the underlying continuous process (Anagnostakis, 31 Jul 2025).

2. Analytical Formulation of Transition Probabilities and Waiting Times

The STMCA utilizes explicit analytical formulas for the required transition probabilities and expected times. For a cell $U = \{(e, x): x \in (a, b)\}$ within an edge $e$, crucial ingredients are:

• The scale function $s_e(x)$ for $e$,
• The speed measure $m_e(dx)$ for $e$,
• The Green kernel

$$G^{(e)}_{a,b}(x, y) = \frac{(s_e(x\wedge y) - s_e(a))(s_e(b) - s_e(x\vee y))}{s_e(b) - s_e(a)},$$

where $x\wedge y = \min(x, y)$ and $x\vee y = \max(x, y)$.

Recursive relations for the expected $k$-th moments $$v^{j}_k(e, x) = \mathbb{E}_x\big[ (T_U)^k \cdot \mathbf{1}_{I(T_U)=j} \big]$$ are given as

$$v^{j}_k(e, x) = k\int_{a}^{b} G^{(e)}_{a,b}(x, y)\; v^{j}_{k-1}(e, y) \; m_e(dy).$$

For cells centered at vertices (junctions), the transition probabilities and times are derived by solving systems involving vertex-specific lateral conditions (including gluing and bias parameters $\beta_e$ at the vertex), yielding limiting relations such as

$$\mathbb{P}_v\{ I(T_{U_h}) = e \} \approx \beta_e, \qquad \mathbb{E}_v\{ T_{U_h} \mid I(T_{U_h}) = e \} \approx h \rho_v$$

as $h \to 0$. These formulas are all explicit, allowing practical implementation once the subdivision and the diffusion coefficients are known (Anagnostakis, 31 Jul 2025).

3. Convergence Analysis and Error Metrics

STMCA convergence is rigorously quantified using the $p$-Wasserstein distance $W_p(T)$ between the law of the target diffusion $X$ and the STMCA process $X^\Delta$ up to a fixed time horizon $T$. A thinness quantifier $\#_1\{\Delta\}$—essentially the maximal diameter of the cells in the subdivision—controls the approximation error:

$$W_p(T)(X, X^\Delta) \leq C \cdot \#_1\{\Delta\}^\alpha,$$

for any $\alpha < \frac{1}{4} \wedge \frac{1}{p}$, with $C$ dependent on model parameters. For adaptive subdivisions (where $\#_1\{\Delta\} \leq |\Delta|^2$), the convergence can be sharpened to

$W_p(T)(X, X^\Delta) \leq C |\Delta|^{2\alpha}.$

The proof relies on coupling the STMCA and the diffusion via the embedding property and employs moment bounds for cell exit times and a Kolmogorov-type continuity theorem (Anagnostakis, 31 Jul 2025).

4. Numerical Performance and Illustrative Examples

The STMCA method is demonstrated on metric graphs with various edge conditions:

• Star graph topology (central vertex plus several edges):
  • Edge 1: behaves like the tail of a geometric Brownian motion.
  • Edge 2: akin to a Cox-Ingersoll-Ross (CIR) process.
  • Edge 3: a standard Brownian motion.
  • Vertex: lateral conditions distribute outgoing transitions equally.
• Boundary behaviors:
  • "Sticky" vertices (with positive $\rho$ parameter): delay exit at the vertex; observed in density plots as a concentration near the vertex.
  • Edges with finite endpoints and singular measures: produce reflected or repelling boundaries.

Numerical simulations (trajectory plots, histograms, density estimates) reveal that STMCA precisely captures not only the local pathwise dynamics but also subtleties such as nontrivial tail behavior, oscillations in density near vertices, and the effects of sticky and reflecting boundary conditions (Anagnostakis, 31 Jul 2025).

5. Applications and Extensions

STMCA is applicable wherever general diffusions on networks or graphs play a role, including:

• Electrical circuit modeling (Kirchhoff-type diffusions on graphs),
• Neuroscience (signal or impulse propagation on neuronal networks),
• Traffic flow and transport dynamics,
• Porous media and chemical reaction networks.

The method flexibly handles a variety of local diffusion behaviors, gluing/lateral conditions at vertices, and different types of boundaries (natural, reflecting, sticky). Its error control is explicit, supporting adaptive refinement strategies to target non-uniform regions efficiently. In particular, choosing the subdivision to adapt to the local speed or scale can—according to the analysis—achieve doubled convergence rates.

Beyond numerical approximation, the STMCA provides a framework for establishing invariance principles, thereby supporting theoretical analysis of limiting behaviors for families of diffusions on complex networks.

6. Theoretical and Methodological Context

The STMCA as introduced in (Anagnostakis, 31 Jul 2025) generalizes earlier Markov chain approximations for SDEs by incorporating both spatial and temporal asymmetry and by matching local exit time statistics, not merely transition probabilities. The approach extends previous work on Markov chain discretizations of diffusions (see e.g., (Cosentino et al., 2021) for SDEs in Euclidean space), but uniquely accounts for the singular geometric and analytic features of metric graphs, such as variable speed measures and gluing conditions.

The convergence in Wasserstein distance distinguishes the STMCA from schemes lacking pathwise error guarantees. Explicit representation of cell-wise transition characteristics makes STMCA amenable to simulation, model checking, and further algorithmic development on networked domains.

7. Limitations and Research Directions

While the STMCA achieves explicit error control and is applicable to general diffusions on finite metric graphs, certain limitations and areas for further research are acknowledged:

• The convergence rate is limited by the thinness quantifier and the tightness of the moment conditions; attaining the optimal rates may require further subdivision adaptivity or regularity assumptions.
• Extension to infinite metric graphs, random graphs, or space–time dependent (non-autonomous) coefficients is an open direction.
• Analytical tractability in high-dimensional graphs may challenge the explicit computation of required Green functionals and transition statistics, suggesting the need for efficient numerical integration or approximation methods.

A plausible implication is that the STMCA framework, with its explicit cell embedding structure and strong convergence results, can serve as a foundation for the analysis of more complex networked stochastic systems and as a platform for rigorous simulation-based inference in spatially structured models.