Semi-Markovian Random Walk Models

• Semi-Markovian random walk is a framework that integrates an embedded Markov chain with non-exponential, state-dependent waiting times to capture memory effects.
• The models include Markov-modulated increments, continuous-time random walks, and persistent walks, where state augmentation restores the Markov property.
• Analytical results reveal local limit theorems, variable-order fractional diffusion equations, and time-based network centralities, advancing applications in physics and network science.

Searching arXiv for recent and relevant papers on semi-Markov random walks, CTRWs, and related formulations. Semi‑Markovian random walk denotes a class of random‑walk models in which the effective evolution in physical time is not Markovian, typically because the step law, holding time, direction, or transition mechanism depends on a renewal structure, an age variable, or a modulating environment. Across the literature, the term covers several closely related constructions: Markov‑modulated additive processes whose increments depend on current and next states; continuous‑time random walks (CTRWs) with non‑exponential, state‑dependent waiting times; persistent or telegraph‑type walks whose direction changes only at renewal epochs; and network random walks whose sojourn time and jump probabilities depend on the elapsed time spent at a node (Ye, 2011, Ricciuti et al., 2017, Meerschaert et al., 2012, Basnarkov, 9 Aug 2025). A common feature is that the position process alone is generally non‑Markovian in calendar time, while an augmented process that includes an age, memory, or environment variable is Markov.

1. Formal definitions and principal model classes

The finite‑state Markov‑modulated formulation considers a discrete‑time Markov chain $X=(X_n)$ on a finite state space $E=\{1,\dots,N\}$ with irreducible and aperiodic transition matrix $P=(p_{i,j})$, together with a family of increment laws $F(i,j,dx)$ on $\mathbb{R}$. The associated semi‑Markovian chain $(Y_n,X_n)$ on $\mathbb{R}\times E$ is defined by

$$\widetilde P\big((u,i),A\times\{j\}\big)=p_{i,j}F(i,j,A),$$

and the random walk is

$$S_0=0,\qquad S_n=Y_1+\cdots+Y_n,\qquad m_n=\min(S_0,\dots,S_n).$$

Conditionally on the Markov path $X_0,\dots,X_n$, the increments are independent but not identically distributed; in this sense $E=\{1,\dots,N\}$0 is a Markov additive process, and the paper explicitly identifies the terminology “semi‑Markovian random walk” with the dependence of the step law on $E=\{1,\dots,N\}$1 (Ye, 2011).

A second major class is the semi‑Markov CTRW. Here one starts from an embedded discrete‑time Markov chain $E=\{1,\dots,N\}$2 on a countable state space $E=\{1,\dots,N\}$3, and replaces exponential waiting times by general holding times $E=\{1,\dots,N\}$4 with survival function

$E=\{1,\dots,N\}$5

The continuous‑time process is

$E=\{1,\dots,N\}$6

The marginal $E=\{1,\dots,N\}$7 is not Markov in $E=\{1,\dots,N\}$8, but the pair $E=\{1,\dots,N\}$9, where $P=(p_{i,j})$0 is the sojourn time in the current state, is Markov; the paper identifies this as “exactly the semi‑Markov property” (Ricciuti et al., 2017).

A third class consists of persistent or telegraph‑type walks. In the variable‑length memory model, $P=(p_{i,j})$1 with $P=(p_{i,j})$2, and the sign process $P=(p_{i,j})$3 is generally not Markovian because the probability of switching depends on the run length already spent in the current sign. Introducing the run‑length variable $P=(p_{i,j})$4, the pair $P=(p_{i,j})$5 becomes a Markov chain on $P=(p_{i,j})$6; the scaling limit yields a continuous‑time process $P=(p_{i,j})$7 whose velocity $P=(p_{i,j})$8 is semi‑Markov and whose sample paths are piecewise linear (Cénac et al., 2012). Closely related “squirrel random walk” models define a discrete‑time walk on $P=(p_{i,j})$9 with unit steps, where the direction flips only at renewal times of a discrete‑time renewal process; this is described as a discrete‑time semi‑Markovian generalization of the telegraph process (Michelitsch et al., 2022, Michelitsch et al., 2022).

A broader probabilistic viewpoint comes from CTRW limit theory. A CTRW is described in space–time by a Markov chain $F(i,j,dx)$0, with position $F(i,j,dx)$1 and renewal counter $F(i,j,dx)$2. Its scaling limit is generally non‑Markovian in position alone, but becomes Markov when augmented by a renewal‑time variable such as the age or residual lifetime (Meerschaert et al., 2012, Gill et al., 2016).

2. Markovization, age augmentation, and semi‑Markov structure

The literature repeatedly treats semi‑Markovian random walks as processes that are non‑Markovian in the observed position variable but Markovian in an enlarged state space. In the variable‑length persistent walk, the sign process $F(i,j,dx)$3 alone is not Markov except when the switching probabilities $F(i,j,dx)$4 are constant in $F(i,j,dx)$5, whereas the pair $F(i,j,dx)$6 is Markov by construction. The memory coordinate $F(i,j,dx)$7 records how long the process has remained in the current state, and this discrete memory becomes an age variable in the continuum limit (Cénac et al., 2012).

The same mechanism appears in CTRW limit theory. If $F(i,j,dx)$8 is the increasing clock process and

$F(i,j,dx)$9

then the CTRW limit position is obtained from a time change of a Markov process $\mathbb{R}$0. The position process $\mathbb{R}$1 is non‑Markovian because the future depends on how long it has been since the last renewal, but the augmented processes $\mathbb{R}$2 and $\mathbb{R}$3, where $\mathbb{R}$4 is the age and $\mathbb{R}$5 the residual lifetime, are Markov and even Hunt in the sense stated in the paper (Meerschaert et al., 2012). The algorithmic paper on CTRW limit distributions makes the same point in discrete approximation: $\mathbb{R}$6 is not a Markov process, but $\mathbb{R}$7 is, where $\mathbb{R}$8 is the residence time since the last jump (Gill et al., 2016).

In semi‑Markov CTRWs on countable state spaces, the age process is the elapsed sojourn time

$\mathbb{R}$9

and $(Y_n,X_n)$0 is Markov although $(Y_n,X_n)$1 alone is not (Ricciuti et al., 2017). On complex networks, the same principle is formulated through an “age of state” variable $(Y_n,X_n)$2: given the pair $(Y_n,X_n)$3 consisting of the current node and its age, the next step distribution is determined by $(Y_n,X_n)$4, so the enlarged state process is Markovian even though the node process alone is not (Basnarkov, 9 Aug 2025).

A discrete model with heterogeneous sojourn time provides an especially explicit finite‑difference realization of this principle. There the natural update for the position probabilities depends on both $(Y_n,X_n)$5 and $(Y_n,X_n)$6, so the process is non‑Markovian in the uniform time index. By restricting to an appropriate subgrid of time–space points and reindexing the lattice, the authors obtain a one‑step Markov recurrence for a transformed density $(Y_n,X_n)$7 on a nonuniform lattice, thereby constructing an embedded Markov chain for a semi‑Markov random walk with location‑dependent deterministic holding times (Chung et al., 2023).

3. Representative analytical results

A central analytical result for Markov‑modulated semi‑Markovian random walks concerns the minimum

$(Y_n,X_n)$8

Under the hypotheses denoted $(Y_n,X_n)$9–$\mathbb{R}\times E$0—uniform exponential moments in a strip, a spread‑out/non‑lattice condition, and centering under the stationary distribution—the matrix kernel

$\mathbb{R}\times E$1

has a dominant eigenvalue $\mathbb{R}\times E$2 with $\mathbb{R}\times E$3 and $\mathbb{R}\times E$4. The main local limit theorem states that for every $\mathbb{R}\times E$5 and every $\mathbb{R}\times E$6,

$\mathbb{R}\times E$7

with $\mathbb{R}\times E$8, and

$\mathbb{R}\times E$9

An equivalent distributional form is

$$\widetilde P\big((u,i),A\times\{j\}\big)=p_{i,j}F(i,j,A),$$0

where $$\widetilde P\big((u,i),A\times\{j\}\big)=p_{i,j}F(i,j,A),$$1 is harmonic for the Markov additive process, increasing in $$\widetilde P\big((u,i),A\times\{j\}\big)=p_{i,j}F(i,j,A),$$2, strictly positive for $$\widetilde P\big((u,i),A\times\{j\}\big)=p_{i,j}F(i,j,A),$$3, and satisfies

$$\widetilde P\big((u,i),A\times\{j\}\big)=p_{i,j}F(i,j,A),$$4

The proof combines spectral perturbation theory for $$\widetilde P\big((u,i),A\times\{j\}\big)=p_{i,j}F(i,j,A),$$5, Wiener–Hopf factorization, Presman’s operator method, and singularity analysis of generating functions (Ye, 2011).

Telegraph‑type semi‑Markovian walks admit exact generating‑function formulae for the propagator. In the squirrel random walk, the step direction remains constant between renewal times and flips at renewal times. The characteristic function $$\widetilde P\big((u,i),A\times\{j\}\big)=p_{i,j}F(i,j,A),$$6 is expressed through the waiting‑time generating function $$\widetilde P\big((u,i),A\times\{j\}\big)=p_{i,j}F(i,j,A),$$7, and the expected position has generating function

$$\widetilde P\big((u,i),A\times\{j\}\big)=p_{i,j}F(i,j,A),$$8

For waiting times with finite mean, the walker remains localized in the average close to the departure site, whereas for fat‑tailed waiting‑time densities with infinite mean the expected position escapes by a sublinear power law (Michelitsch et al., 2022). For generalized Sibuya waiting times, the mean squared displacement exhibits three explicit regimes: $$\widetilde P\big((u,i),A\times\{j\}\big)=p_{i,j}F(i,j,A),$$9

$$S_0=0,\qquad S_n=Y_1+\cdots+Y_n,\qquad m_n=\min(S_0,\dots,S_n).$$0

and

$$S_0=0,\qquad S_n=Y_1+\cdots+Y_n,\qquad m_n=\min(S_0,\dots,S_n).$$1

so the order $$S_0=0,\qquad S_n=Y_1+\cdots+Y_n,\qquad m_n=\min(S_0,\dots,S_n).$$2 of the first diverging moment of the waiting time determines whether the process is ballistic, superdiffusive, or diffusive (Michelitsch et al., 2022).

A different but structurally related analytical setting arises on complex networks. There the survival function at node $$S_0=0,\qquad S_n=Y_1+\cdots+Y_n,\qquad m_n=\min(S_0,\dots,S_n).$$3 is

$$S_0=0,\qquad S_n=Y_1+\cdots+Y_n,\qquad m_n=\min(S_0,\dots,S_n).$$4

in discrete time, or

$$S_0=0,\qquad S_n=Y_1+\cdots+Y_n,\qquad m_n=\min(S_0,\dots,S_n).$$5

in continuous time. The effective transition parameters of the embedded semi‑Markov chain are

$$S_0=0,\qquad S_n=Y_1+\cdots+Y_n,\qquad m_n=\min(S_0,\dots,S_n).$$6

in discrete time and

$$S_0=0,\qquad S_n=Y_1+\cdots+Y_n,\qquad m_n=\min(S_0,\dots,S_n).$$7

in continuous time. The stationary arrival probabilities satisfy the eigenvector relation $$S_0=0,\qquad S_n=Y_1+\cdots+Y_n,\qquad m_n=\min(S_0,\dots,S_n).$$8, and the occupation probability is

$$S_0=0,\qquad S_n=Y_1+\cdots+Y_n,\qquad m_n=\min(S_0,\dots,S_n).$$9

which represents the fraction of time an infinite walk spends on node $X_0,\dots,X_n$0 (Basnarkov, 9 Aug 2025).

4. Diffusion limits, variable‑order dynamics, and anomalous transport

Semi‑Markovian random walks frequently arise as microscopic models of anomalous diffusion. In state‑dependent CTRWs with Mittag–Leffler waiting times

$X_0,\dots,X_n$1

the backward equation for transition probabilities is

$X_0,\dots,X_n$2

while the forward equation, under $X_0,\dots,X_n$3, is

$X_0,\dots,X_n$4

The order of the Caputo derivative depends on the starting state in the backward equation, whereas the order of the Riemann–Liouville derivative depends on the intermediate state in the forward equation. For arbitrary holding times generated by Lévy measures $X_0,\dots,X_n$5, these fractional operators are replaced by Volterra‑type kernels depending on the state (Ricciuti et al., 2017).

Under diffusive spatial scaling, these heterogeneous semi‑Markov CTRWs lead to variable‑order fractional diffusion equations. In the symmetric nearest‑neighbor setting with state‑dependent $X_0,\dots,X_n$6, the forward limit is

$X_0,\dots,X_n$7

and with spatially varying diffusivity $X_0,\dots,X_n$8,

$X_0,\dots,X_n$9

The corresponding backward equation is

$E=\{1,\dots,N\}$00

These equations model anomalous diffusion in heterogeneous media, and the paper notes effects such as anomalous aggregation near points where $E=\{1,\dots,N\}$01 is minimal (Ricciuti et al., 2017).

The heterogeneous‑sojourn discrete walk gives a different local limit. If the jump length is fixed and the sojourn time is $E=\{1,\dots,N\}$02, with for example

$E=\{1,\dots,N\}$03

then after constructing an embedded Markov chain on a nonuniform lattice and passing to the parabolic limit, the limiting field $E=\{1,\dots,N\}$04 solves

$E=\{1,\dots,N\}$05

and the physical density $E=\{1,\dots,N\}$06 satisfies

$E=\{1,\dots,N\}$07

The paper also derives the corresponding Green’s function and shows by Monte Carlo simulation that steady states are proportional to $E=\{1,\dots,N\}$08, so more mass accumulates where the sojourn time is longer (Chung et al., 2023).

From the CTRW‑limit perspective, the same phenomenon is expressed through inverse subordinators. If $E=\{1,\dots,N\}$09 is the space–time limit of the jump chain and $E=\{1,\dots,N\}$10, then the physical‑time process is $E=\{1,\dots,N\}$11 or $E=\{1,\dots,N\}$12 in the uncoupled case. Heavy‑tailed waiting times make $E=\{1,\dots,N\}$13 a stable or tempered stable subordinator, and the inverse time change $E=\{1,\dots,N\}$14 induces subdiffusive or tempered subdiffusive behavior. The semi‑Markov algorithm exploits this structure directly at the level of the Markov pair $E=\{1,\dots,N\}$15 (Meerschaert et al., 2012, Gill et al., 2016).

5. Network, web, and citation formulations

On complex networks, semi‑Markovian random walk means that a walker moves on graph nodes, spends a random sojourn time at each node, and chooses the next neighbor according to probabilities that depend on the elapsed time already spent at the node. In the discrete‑time formulation, $E=\{1,\dots,N\}$16 is the conditional probability that, given the walker is at node $E=\{1,\dots,N\}$17 with age $E=\{1,\dots,N\}$18, the next step is to node $E=\{1,\dots,N\}$19, with $E=\{1,\dots,N\}$20 representing the probability to remain at $E=\{1,\dots,N\}$21 and increase the age to $E=\{1,\dots,N\}$22. In continuous time, the corresponding objects are age‑dependent rates $E=\{1,\dots,N\}$23 and survival functions $E=\{1,\dots,N\}$24 (Basnarkov, 9 Aug 2025).

This framework yields two distinct stationary measures. The arrival or jump distribution solves $E=\{1,\dots,N\}$25, while the occupation probability

$E=\{1,\dots,N\}$26

measures the fraction of time spent at node $E=\{1,\dots,N\}$27. The paper distinguishes this from the classical stationary distribution of a Markov random walk, which measures the fraction of visits. Accordingly it proposes “time rank” as an alternative to visit‑based ranking: a node may be visited infrequently but retain a large occupation probability because each visit is long (Basnarkov, 9 Aug 2025).

The discrete web‑surfing example chooses a directed Erdős–Rényi graph with $E=\{1,\dots,N\}$28 and edge probability $E=\{1,\dots,N\}$29, sets the maximal sojourn time equal to the node out‑degree $E=\{1,\dots,N\}$30, and arranges the transition probabilities so that the probability of eventually following each outgoing hyperlink is $E=\{1,\dots,N\}$31, while the age at which the link is chosen depends on its order. Theoretical occupation probabilities obtained from the eigenvector of $E=\{1,\dots,N\}$32 agree with Monte Carlo simulation, and differ from the occupation probabilities of the classical Markov random walk (Basnarkov, 9 Aug 2025).

A larger citation‑network example uses the strongly connected component of the High Energy Physics Theory citation network with $E=\{1,\dots,N\}$33 nodes. Interpreting nodes as papers and directed edges as citations, the semi‑Markov walk assumes that reading time is proportional to the number of references, again taking $E=\{1,\dots,N\}$34. The resulting time‑rank emphasizes papers that are both authoritative and long. The same paper also defines a semi‑Markovian PageRank variant by mixing the age‑dependent random walk with teleportation of probability $E=\{1,\dots,N\}$35, which modifies the survival to

$E=\{1,\dots,N\}$36

This produces a time‑based analogue of PageRank in which age is reset at teleportation (Basnarkov, 9 Aug 2025).

6. Relations to broader theory, applications, and recurring misconceptions

A recurring misconception is to identify semi‑Markovian random walk with a single model. The literature instead uses the term for several formally distinct but structurally analogous constructions: Markov additive processes with Markov‑modulated increments (Ye, 2011), CTRWs with state‑dependent holding times (Ricciuti et al., 2017), persistent walks with variable‑length memory (Cénac et al., 2012), renewal‑driven telegraph walks (Michelitsch et al., 2022), random walks with heterogeneous sojourn time (Chung et al., 2023), and age‑dependent network diffusion (Basnarkov, 9 Aug 2025). The unifying point is not a unique transition rule, but the combination of an embedded Markov mechanism with non‑memoryless holding times or age dependence.

A second misconception is that non‑Markovianity prevents a Markov description. The papers consistently show the opposite: a semi‑Markovian random walk is typically Markov after augmenting the state by age, residual lifetime, run length, node age, or environmental state (Ricciuti et al., 2017, Meerschaert et al., 2012, Gill et al., 2016). This enlarged‑state description is not merely formal; it is the basis for transition kernels, numerical algorithms, spectral analysis, and scaling limits.

Applications span branching processes in random environments, anomalous diffusion in heterogeneous media, web surfing and citation ranking, and persistent transport with trapping or intermittency. The Markov‑modulated minimum asymptotics were motivated in part by branching processes in Markovian random environments (Ye, 2011). Variable‑order CTRWs connect microscopic heterogeneous trapping to macroscopic variable‑order fractional heat equations (Ricciuti et al., 2017). The semi‑Markov approach to CTRW limits provides explicit finite‑dimensional distributions and a practical algorithm that accommodates arbitrary initial age distributions, including equilibrium residence‑time laws when the waiting‑time tail is integrable (Meerschaert et al., 2012, Gill et al., 2016). Renewal‑driven telegraph walks furnish analytically tractable models with ballistic, superdiffusive, and diffusive regimes determined by the tail of the switching‑time law (Michelitsch et al., 2022, Michelitsch et al., 2022).

Taken together, these results suggest a coherent interpretation: semi‑Markovian random walk is best viewed as a general framework for random motion in which the microscopic transition mechanism is governed by an embedded Markov chain together with non‑exponential or age‑dependent temporal structure. Depending on the model class, this temporal structure appears as a modulating Markov environment, a residence‑time variable, a run length, a renewal clock, or a node age; depending on the scaling, it yields local limit theorems, fractional or Volterra equations, telegraph limits, or time‑based network centralities (Ye, 2011, Ricciuti et al., 2017, Cénac et al., 2012, Basnarkov, 9 Aug 2025).