---
title: Multi-Time Markov Renewal Equations
url: https://www.emergentmind.com/topics/multi-time-markov-renewal-equations
type: topic
---

# Multi-Time Markov Renewal Equations

Searching arXiv for the cited papers to ground the article in current sources.
Multi-Time Markov Renewal Equations are renewal identities in which the renewal mechanism is indexed by several time coordinates, several observation times, or an infinite ordered family of elapsed-time variables, while the underlying dynamics retain a Markov-renewal or semi-Markov structure. A central recent formulation is the discrete \(d\)-dimensional equation
\[
L = G + q * L,
\]
with solution
\[
L = u * G,\qquad u := (\mathbb{I}_s - q)^{(-1)} = \sum_{n\ge 0} q^{(n)},
\]
for matrix-valued sequences on \(\mathbb{N}^d\) [2508.01891]. Related arXiv strands treat multi-time joint laws at several calendar times for continuous-time Markov renewal processes [2403.15221], infinite-times renewal equations driven by the sequence of elapsed times since past events [2304.01605], and matrix Markov renewal equations with one time variable but multidimensional state space, which are explicitly distinguished from genuinely multi-parameter multi-time models [2503.05468].

## 1. Terminological scope and basic meanings

Current arXiv usage assigns the expression “multi-time” to several closely related constructions. In the discrete multidimensional setting, time is indexed by \(k_{1:d}\in \mathbb{N}^d\) with the componentwise partial order \(k \le l\) iff \(k_u \le l_u\) for all \(u=1,\dots,d\). The process is a \(d\)-dimensional multi-time Markov renewal chain \((J,S) := (J_n,S_n)_{n\ge 0}\), where \(J_n\) is a finite-state embedded chain and \(S_n\in \mathbb{N}^d\) is a multi-time jump epoch [2508.01891].

In continuous time, the phrase appears in the analysis of joint laws at several observation times \(0<t_1<\cdots<t_m\le T\). There the state is described by the current state \(Z_t\) and backward recurrence time \(V_t\), and multi-time distributions are written through iterated convolutions of the semi-Markov kernel and survival factors [2403.15221].

A third meaning arises in infinite-memory models. The state is then the ordered sequence \(s=(s_1,s_2,\dots)\), where \(s_i\) is the elapsed time since the \(i\)-th most recent event and \(0\le s_1\le s_2\le \cdots\). The corresponding renewal equation becomes an infinite-dimensional transport-plus-renewal PDE or an equivalent hierarchy of marginal equations [2304.01605].

| Setting | Time/state structure | Representative relation |
|---|---|---|
| Discrete multi-time Markov renewal chain | \(k_{1:d}\in\mathbb{N}^d\), finite \(E\) | \(L = G + q * L\) |
| Continuous-time multi-time joint laws | \(0<t_1<\cdots<t_m\le T\), \((V_t,Z_t)\) | Iterated convolutions of \(Q\) with survival factors |
| Infinite-times renewal equation | \(s\in C_\infty\) with \(0\le s_1\le s_2\le \cdots\) | \(\sum_i \partial_{s_i} n_\infty + p_\infty n_\infty = 0\) with renewal via shift |

A recurrent misconception is to identify “multi-time” with “multidimensional” in the sense of vector-valued state alone. The paper on asymptotic expansions of Markov renewal equations states explicitly that it studies a single time variable \(t\) but a multidimensional state space \(p\) types, and that extending to genuinely multi-parameter renewal equations would require analytic tools for several complex variables [2503.05468].

## 2. Discrete multi-time Markov renewal chains and the fundamental equation

The discrete algebraic formulation begins with a finite state space \(E=\{1,\dots,s\}\), the matrix space \(M_s:=\mathbb{R}^{s\times s}\), and the set \(M_s(\mathbb{N}^d)\) of \(M_s\)-valued sequences \(A(k_{1:d})\). For \(A,B\in M_s(\mathbb{N}^d)\), the \(d\)-dimensional convolution product is
\[
[A*B](k_{1:d}) := \sum_{l+l'=k_{1:d}} A(l)B(l').
\]
Entrywise,
\[
[A*B]_{ij}(k_{1:d}) = \sum_{r\in E}\sum_{l+l'=k_{1:d}} \alpha_{ir}(l)\beta_{rj}(l'),
\]
so convolution acts as matrix multiplication over the commutative ring of real \(d\)-dimensional sequences [2508.01891].

The identity sequence \(\mathbb{I}_s\) is defined by \(\mathbb{I}_s(0_d)=I_s\) and \(\mathbb{I}_s(k_{1:d})=O_s\) otherwise. Convolutional powers are
\[
A^{(0)}:=\mathbb{I}_s,\qquad A^{(n)}:=A*A^{(n-1)},\quad n\ge 1,
\]
equivalently
\[
A^{(n)}(k_{1:d})=\sum_{l^1+\cdots+l^n=k_{1:d}} A(l^1)\cdots A(l^n).
\]
The algebra \((M_s(\mathbb{N}^d),+,\cdot,*)\) is a unital associative \(\mathbb{R}\)-algebra with identity \(\mathbb{I}_s\), and \(A\) has a convolutional inverse iff \(A(0_d)\) is nonsingular [2508.01891].

A \(d\)-dimensional multi-time Markov renewal chain is a pair \((J,S)\) with inter-jump times \(X_n:=S_n-S_{n-1}\), \(S_0=0_d\), and counting process
\[
N(k_{1:d}) := \sup\{n\in\mathbb{N}: S_n\le k_{1:d}\}
= \min_{1\le u\le d} N^{[u]}(k_u).
\]
Its time-homogeneous semi-Markov kernel is
\[
q_{ij}(k_{1:d}) := \mathbb{P}(J_{n+1}=j,\ S_{n+1}-S_n=k_{1:d}\mid J_n=i),
\]
with \(q_{ij}(k_{1:d})\ge 0\), \(\sum_j\sum_{k_{1:d}} q_{ij}(k_{1:d})=1\), and optionally \(q_{ij}(0_d)=0\) to exclude instantaneous transitions across all dimensions [2508.01891].

The fundamental multi-time Markov renewal equation is
\[
L = G + q * L.
\]
Its componentwise form is
\[
L_{ij}(k_{1:d}) = G_{ij}(k_{1:d}) + \sum_{m\in E}\sum_{0_d\le l\le k_{1:d}} L_{im}(k_{1:d}-l)\, q_{mj}(l).
\]
If \(q(0_d)=O_s\), then \((\mathbb{I}_s-q)(0_d)=I_s\), so \(\mathbb{I}_s-q\) is invertible in the convolution algebra. Hence the equation has the unique solution
\[
L = u * G,\qquad
u := (\mathbb{I}_s-q)^{(-1)} = \sum_{n\ge 0} q^{(n)},
\]
and \(u\) satisfies
\[
u = \mathbb{I}_s + q * u.
\]
This resolvent form is the direct multi-time analogue of the classical one-dimensional renewal matrix [2508.01891].

## 3. Canonical renewal objects: transition, renewal, and first passage

Once the resolvent \(u\) is available, the principal quantities of the theory are written as explicit multi-time renewal solutions. The associated multi-time semi-Markov chain is
\[
Z_{k_{1:d}} := J_{N(k_{1:d})}.
\]
Its transition function
\[
P_{ij}(k_{1:d}) := \mathbb{P}(Z_{k_{1:d}}=j \mid Z_{0_d}=i)
\]
satisfies
\[
P = u * \widetilde{H},
\]
where \(\widetilde{H}:=dg(\widetilde{H}_E)\) is the diagonal matrix sequence of complementary cdfs of sojourn times by current state. This is precisely of the form \(L=G+q*L\) with \(G:=\widetilde{H}\) [2508.01891].

The multi-time Markov renewal function \(U\) is defined by
\[
U_{ij}(k_{1:d}) := \mathbb{E}_i[\widetilde{N}_j(k_{1:d})],
\]
where \(\widetilde{N}_j(k_{1:d})\) counts visits to state \(j\) up to multi-time \(k_{1:d}\). It satisfies
\[
U = \mathbb{I}_s + q * U,
\]
so, up to the diagonal structure made explicit in the paper,
\[
U = \mathbb{I}_s * u.
\]
This is the multi-time analogue of the classical Markov renewal function [2508.01891].

First-passage laws admit both an algebraic closed form and a recursive multi-time renewal equation. If \(g_{ij}(k_{1:d}) := \mathbb{P}_i(S^{(j)}_0 = k_{1:d})\) is the first-passage pmf to state \(j\), then
\[
g = (u-\mathbb{I}_s) * dg(u)^{(-1)}.
\]
Conditioning on the first step yields the alternative recursion
\[
g_{ij} = q_{ij} + \sum_{r\ne j} q_{ir} * g_{rj}.
\]
The second formula makes the structural analogy with the classical one-dimensional first-passage renewal equation explicit, the difference being that convolution is now multidimensional and the order on indices is componentwise [2508.01891].

The same framework also produces ergodic passage-time identities. For an irreducible ergodic multi-time Markov renewal chain with stationary distribution \(\nu\) of the embedded chain \(J\), the recurrence-time moments satisfy
\[
\mu_{jj}^{(u)} = \frac{\sum_{i=1}^s \nu_i m_i^{[u]}}{\nu_j},
\qquad
\mu_{jj} = \frac{\sum_{i=1}^s \nu_i m_i}{\nu_j},
\]
together with the stated formula for \(\mu_{jj}^{(u,v)}\). These identities generalize one-dimensional ergodic relations to the multi-time setting [2508.01891].

## 4. Continuous-time multi-time laws and renewal-preserving filtering

In continuous time, Markov renewal theory is formulated through an embedded chain \((J_n)_{n\ge 0}\), jump times \((T_n)_{n\ge 0}\), holding times \(S_{n+1}:=T_{n+1}-T_n\), and the semi-Markov kernel
\[
Q_{ij}(t) = \mathbb{P}(J_{n+1}=j,\ S_{n+1}\le t\mid J_n=i),
\]
assumed absolutely continuous with density \(q_{ij}(t)=dQ_{ij}(t)/dt\). The renewal measure is
\[
U(t)=I+Q(t)+Q^{*2}(t)+Q^{*3}(t)+\cdots,
\]
and it satisfies the Volterra equation
\[
U(t)=I+\int_0^t U(t-s)\, dQ(s).
\]
When densities exist, the renewal density matrix satisfies
\[
u(t)=q(t)+\int_0^t u(s)\, q(t-s)\, ds
\]
[2403.15221].

The continuous-time multi-time content enters through the backward recurrence time
\[
V_t := t-T_n \quad \text{for } T_n\le t < T_{n+1},
\]
and the current state \(Z_t:=J_n\). A classical Markov renewal equation for the joint law of \((V_t,Z_t)\) is
\[
\mathbb{P}(Z_t=z,\ V_t\le v)
= \int_{[0,v]} G_z(u)\sum_y \eta_y\, m(y\to z,t-u)\, du,
\]
with density
\[
p(V_t=v,\ Z_t=z)=G_z(v)\sum_y \eta_y\, m(y\to z,t-v).
\]
For \(0<t_1<\cdots<t_m\le T\), the joint law over \((Z_{t_\ell},V_{t_\ell})_{\ell=1}^m\) is expressed via iterated convolutions of \(Q\) and survival factors. Compactly, matrix-Volterra formulations use \(U(t)\) and survival multipliers to express multi-time distributions; practically, the expression is organized as “renewal blocks” \(Q^{(*n)}\) separated by survival factors [2403.15221].

A major structural result is Anderson’s filtering theorem. If the original state space is partitioned as \(E=E_{\mathrm{obs}}\cup E_{\mathrm{hid}}\) and the semi-Markov kernel has block form
\[
Q(t)=
\begin{bmatrix}
A(t) & B(t)\\
C(t) & D(t)
\end{bmatrix},
\]
then the filtered process on \(E_{\mathrm{obs}}\) is again a Markov renewal process with kernel
\[
\widetilde Q(t)
=
A(t)+\int_0^t B(du)\int_0^{t-u}\Big[\sum_{k\ge 0} D^{(k)}(ds)\Big]\, C(t-u-s),
\]
and, in Laplace domain,
\[
\widetilde Q^*(s)=a^*(s)+b^*(s)(I-d^*(s))^{-1}c^*(s).
\]
Two subclasses are singled out: one in which \(Y\) is itself a renewal process, and one in which \((X,Y)\) remains a Markov renewal process after coarse-graining [2403.15221].

This filtered continuous-time theory supports exact information-theoretic calculations for Poisson-type channels. For a counting output \(Y\) with predictable intensity \(\lambda_t\),
\[
I(X_{[0,T]};Y_{[0,T]})
=
\int_0^T
\big[
\mathbb{E}\,\phi(\lambda_t^{XY})-\mathbb{E}\,\phi(\lambda_t^Y)
\big]\, dt,
\qquad
\phi(z)=z\ln z,
\]
and the filtered hazard into state \(z\) is
\[
\Lambda_z^{\mathrm{MrP}}(v,y)=\frac{\widetilde q_{yz}(v)}{S_y(v)}
\]
or, equivalently in the paper’s notation,
\[
\Lambda_z^{\mathrm{MrP}}(v,y)=\frac{\widetilde Q_{yz}(v)}{F_y(v)}.
\]
The framework is applied to bacterial gene expression, where filtering is analytically tractable [2403.15221].

## 5. Algebraic inversion, FFT computation, and asymptotics on large multi-index domains

The multidimensional convolution algebra is designed for computation as well as theory. If \(A(0_d)\) is nonsingular, the convolutional inverse admits the representation
\[
A^{(-1)}
=
[A(0_d)]^{-1}\sum_{n=0}^\infty (\mathbb{I}_s-A_0)^{(n)}
=
\Big[\sum_{n=0}^\infty (\mathbb{I}_s-A_0^*)^{(n)}\Big][A(0_d)]^{-1},
\]
with coefficientwise truncation finite at each fixed \(k_{1:d}\). The same paper gives the classical recurrence for \(A^{(-1)}(k_{1:d})\), Newton’s inversion method for formal power series, and a convolutional Gauss–Jordan algorithm based on elementary row operations implemented by convolution with elementary matrix sequences [2508.01891].

The computational complexity reported for FFT-based inversion is
\[
O\!\left(s^3 \prod_{i=1}^d k_i \log_2 k_i\right),
\]
while direct convolution scales as \(O\!\left(s^3 \prod k_i^2\right)\). The reported experiments are as follows [2508.01891]:

| Experiment | FFT-based method | Direct method |
|---|---|---|
| 1D, \(s=3\), up to \(512\) | convolution \(\sim 0.45\)s | \(36.34\)s |
| 2D, \(s=3\), \(32\times 32\) | convolution \(2.19\)s | \(101.43\)s |
| 2D inverse, \(s=3\), \(32\times 32\) | Gauss–Jordan \(1.71\)s | \(89.68\)s |

The paper states the corresponding speedups as \(\times 80.8\) in the one-dimensional convolution example, \(\times 46.3\) in the two-dimensional convolution example, and Gauss–Jordan \(\times 52.4\) faster than direct in the two-dimensional inversion example. It also reports a larger two-dimensional run, \(128\times 128\), with FFT \(7.78\)s versus direct \(83.09\)s while operating on much larger domains [2508.01891].

A parallel development based on multi-index convolution and multivariate formal power series writes scalar renewal and matrix Markov renewal equations as
\[
u = \delta_0 + f*u,\qquad
U = I\delta_0 + F*U,
\]
with generating functions
\[
U(z)=\frac{1}{1-F(z)}
\quad\text{and}\quad
U(z)=[I-F(z)]^{-1},
\]
respectively. The practical inversion schemes combine FFT-based multidimensional convolution with Newton-type reciprocal iteration, including the matrix update
\[
U_{2m}=U_m[2I-(I-F)U_m]\mod 2m,
\]
and convolution on a padded grid of size \(L_1\times \cdots \times L_d\) is stated to have complexity
\[
O\!\left(N_L\sum_{u=1}^d \log L_u\right),
\qquad N_L:=\prod L_u
\]
[2606.31455].

The same framework provides asymptotics under proportional growth. If \(k/|k|_1\to \lambda\) with \(\lambda^u\in(0,1)\) and \(\sum \lambda^u=1\), and \(\mu_u=\mathbb{E}[X_1^{[u]}]\in (0,\infty)\), then the directional rate is
\[
\mu_\lambda := \min_{1\le u\le d} \frac{\lambda^u}{\mu_u}.
\]
The strong laws are
\[
\frac{N(k)}{|k|_1}\to \mu_\lambda \quad \text{a.s.},
\qquad
\frac{M(k)}{|k|_1}\to \mu_\lambda.
\]
For additive functionals,
\[
\frac{W_h(k)-A_h N(k)}{\sqrt{|k|_1}}
\Rightarrow \mathcal N_s(0,\mu_\lambda \Sigma_h).
\]
If there is a unique rate-determining coordinate \(u_\lambda\), then
\[
\frac{N(k)-k_{u_\lambda}/\mu_{u_\lambda}}{\sqrt{|k|_1}}
\Rightarrow
\mathcal N\!\left(0,\frac{\lambda^{u_\lambda}\sigma_{u_\lambda}^2}{\mu_{u_\lambda}^3}\right).
\]
The paper also states that fixed-horizon observations induce a genuinely multivariate right-censoring mechanism, leading to an exact nonparametric maximum likelihood estimator and its asymptotic normality [2606.31455].

## 6. Infinite-times renewal equations and memory of the whole past

The infinite-times formulation replaces finitely many time coordinates by the whole ordered sequence of elapsed times since previous events. For finite \(N\), with state \([s]_N=(s_1,\dots,s_N)\in C_N:=\{0\le s_1\le \cdots \le s_N\}\), the renewal PDE is
\[
\partial_t n_N + \sum_{i=1}^N \partial_{s_i} n_N + p_N([s]_N)n_N = 0,
\]
together with the renewal boundary condition
\[
n_N(t,s_1=0,s_2,\dots,s_N)
=
\int_0^\infty p_N(s_2,\dots,s_N,u)\, n_N(t,s_2,\dots,s_N,u)\, du.
\]
The shift operator is
\[
T(s_1,\dots,s_N)=(0,s_1,\dots,s_{N-1}),
\]
and in the infinite-dimensional case \(T(s_1,s_2,\dots)=(0,s_1,s_2,\dots)\). The formal generator on test functions is
\[
L\phi([s]) = \sum_i \partial_{s_i}\phi([s]) + p([s])\big(\phi(T[s])-\phi([s])\big)
\]
[2304.01605].

The hazards are assumed to have the finite-window decomposition
\[
p_N([s]_N)=\sum_{i=1}^N \phi_i(s_1,\dots,s_i),
\]
with \(\phi_i\ge 0\), tail norms
\[
E_{K,N}:=\sum_{i=K+1}^N \|\phi_i\|_{L^\infty},
\qquad
E_K:=E_{K,\infty},
\]
and uniform lower and upper bounds
\[
0<a_-\le p_N([s]_N)\le a_+.
\]
Under these conditions, for each finite \(N\) there exists a unique weak solution \(n_N\in C([0,\infty);L^1(C_N))\) that preserves positivity, mass, and support [2304.01605].

Two rigorous notions are given for the infinite-times equation. One is a hierarchy formulation for all finite marginals \(n^{(K)}\), with coupling terms
\[
E^{(K)}(t,d[s]_K)=\sum_{i=K+1}^{\infty}\int n^{(i)}(t,[s]_i)\phi_i([s]_i)\, d[s]_{K+1,i},
\]
satisfying
\[
|E^{(K)}(t,\cdot)|_{L^1(C_K)}\le E_K.
\]
The other is an infinite-dimensional measure formulation on \(C_\infty\) using test functions depending on finitely many coordinates. Their equivalence is proved via the Kolmogorov extension theorem [2304.01605].

Long-time behavior is established in both strong and weak metrics. For finite \(N\), Doeblin bounds yield exponential convergence in \(L^1(C_N)\) to a unique steady state. For the infinite-times model, uniform-in-time strong approximation of marginals and convergence of steady states hold when the hazard tail decays fast enough so that \((t_N/a_N)E_N\to 0\). A second route uses a Monge–Kantorovich distance built from the weighted truncated cost
\[
V_{N,\beta,\alpha}([s]_N,[s']_N)
=
\sum_{i=1}^N (1+\beta)^{-i}\min\{|s_i-s_i'|,\alpha\},
\]
together with the Lipschitz condition
\[
|p_N([s]_N)-p_N([s']_N)|
\le \delta\, V_{N,\beta,\alpha}([s]_N,[s']_N).
\]
If \(y:=\beta a_-/(1+\beta)-\delta>0\), then
\[
T_{V_{N,\beta,\alpha}}(n_N(t),m_N(t))
\le e^{-yt}\, T_{V_{N,\beta,\alpha}}(n_N(0),m_N(0)),
\]
and the paper gives the corresponding infinite-times analogue [2304.01605].

## 7. Related matrix theories, precursors, and application domains

A related but distinct line of work studies one-time vector-valued Markov renewal equations of the form
\[
F(t)=f(t)+\boldsymbol{\mu}*F(t),
\]
where \(\boldsymbol{\mu}\) is a \(p\times p\) matrix of locally finite measures on \([0,\infty)\). The renewal matrix is
\[
\boldsymbol{U}=\sum_{n=0}^\infty \boldsymbol{\mu}^{*n},
\]
and the unique solution is
\[
F(t)=\boldsymbol{U}*f(t).
\]
The asymptotic behavior is governed by the characteristic equation
\[
\det(I-L(\lambda))=0,
\]
the pole orders \(k(\lambda)\) of \((I-L(z))^{-1}\), and the Malthusian parameter \(\alpha\) determined by \(\rho_{L(\alpha)}=1\). In the primitive case, \(\alpha\) is a simple pole; in reducible cases \(k(\alpha)\) can exceed \(1\), which yields polynomial corrections \(t^k e^{\alpha t}\) [2503.05468]. The same paper states explicitly that extending this residue-calculus approach to genuinely multi-parameter renewal equations would require several complex variables, multidimensional Laplace transforms, and spectral analysis on product half-spaces [2503.05468].

An earlier probabilistic precursor is the quasi-stochastic matrix framework. Given a quasi-stochastic matrix \(Q\), a matrix of distribution functions \(F\), and the matrix kernel
\[
(Q\otimes F)(t)=[q_{ij}F_{ij}(t)]_{i,j},
\]
the matrix renewal measure is
\[
V = I\delta_0 + (Q\otimes F) + (Q\otimes F)^{*2} + (Q\otimes F)^{*3} + \cdots,
\]
with renewal equation
\[
V = I\delta_0 + (Q\otimes F)*V.
\]
The harmonic transform
\[
P=D^{-1}QD
\]
converts \(Q\) into a stochastic matrix, and in the nonarithmetic positive-drift case the renewal theorem gives
\[
\lim_{t\to\infty} V_{ij}((t,t+h]) = \frac{v_i u_j h}{\mu}.
\]
The exposition connecting this paper to multi-time Markov renewal equations states that the framework naturally yields multi-time integral equations via iterated matrix convolution [1312.3090].

Applications are spread across several domains. In communication theory and stochastic filtering, the continuous-time framework supports simulation-free or reduced-simulation evaluation of mutual information and mutual information rate for Poisson-type channels, including an application to bacterial gene expression [2403.15221]. In discrete multi-time renewal theory, the stated applications include a binomial--multiset identity, two-attribute warranty evaluation, alternating-renewal availability computation, and discretization-based approximations of continuous-time bivariate renewal and availability models [2606.31455]. The infinite-times PDE was motivated by neuroscience, where prediction of the next discharge can depend on the last two discharge times and possibly more; the same paper also mentions epidemics/contact tracing and connections to Hawkes and Wold processes [2304.01605]. The algebraic multidimensional Markov renewal chain framework identifies reliability, maintenance, and biological growth as natural application areas for heterogeneous clocks and jointly distributed waiting times [2508.01891].

Taken together, these works show that Multi-Time Markov Renewal Equations are not a single formalism but a family of renewal structures built around the same core ingredients: a Markovian state update, renewal kernels or hazards, convolutional resolvents, and explicit control of multi-time dependence. This suggests that the subject is best understood as a common operator-theoretic and probabilistic language spanning discrete multi-index time, continuous multi-time observation, and infinite-memory renewal dynamics.

Source: https://www.emergentmind.com/topics/multi-time-markov-renewal-equations