Multi-Time Markov Renewal Chains

• Multi-Time Markov Renewal Chains are a multidimensional extension of classical renewal theory that incorporates state-dependent kernels and multidimensional sojourn times.
• The framework employs matrix-valued convolution algebra to solve renewal equations, enabling efficient computation and scaling over complex multidimensional grids.
• This approach unifies semi-Markov processes with multidimensional time, providing practical insights into recurrence, ergodicity, and statistical inference.

Searching arXiv for papers on multi-time and related Markov renewal theory. Multi-Time Markov Renewal Chains are a multidimensional extension of classical Markov renewal theory in which jump times evolve on the lattice $\mathbb N^d$ rather than along a single time axis. In the formulation introduced in "Time multidimensional Markov Renewal chains -- An algebraic approach" (Kordalis et al., 3 Aug 2025), the process is a pair $(J,S)=(J_n,S_n)_{n\in\mathbb N}$, where $J_n$ is the visited state and $S_n$ is a $d$-dimensional jump time, with inter-jump vector $X_n=S_n-S_{n-1}$. The defining property is a Markov renewal kernel on multidimensional sojourns: the conditional law of $(J_{n+1},S_{n+1}-S_n)$ given the past depends only on $J_n$. This places multi-time Markov renewal chains at the intersection of semi-Markov modeling, renewal equations, and multidimensional convolution algebra. Closely related work clarifies what these models are not: purely i.i.d. multi-time renewal chains have a similar renewal-equation structure but lack the state-dependent kernel that characterizes genuine Markov renewal dynamics (Kordalis et al., 30 Jun 2026).

1. Formal definition and position within renewal theory

In the finite-state framework of (Kordalis et al., 3 Aug 2025), the state space is

$E=\{1,\dots,s\},$

and a multi-time Markov renewal chain is the pair

$(J,S) := (J_n,S_n)_{n\in\mathbb N},$

with $(J,S)=(J_n,S_n)_{n\in\mathbb N}$0, $(J,S)=(J_n,S_n)_{n\in\mathbb N}$1, and $(J,S)=(J_n,S_n)_{n\in\mathbb N}$2 the $(J,S)=(J_n,S_n)_{n\in\mathbb N}$3-dimensional sojourn time. The Markov renewal property is

$(J,S)=(J_n,S_n)_{n\in\mathbb N}$4

When this law does not depend on $(J,S)=(J_n,S_n)_{n\in\mathbb N}$5, the chain is homogeneous and is governed by the multi-time semi-Markov kernel

$(J,S)=(J_n,S_n)_{n\in\mathbb N}$6

This definition reduces to the ordinary one-time Markov renewal chain when $(J,S)=(J_n,S_n)_{n\in\mathbb N}$7, and a renewal process is recovered as the special case $(J,S)=(J_n,S_n)_{n\in\mathbb N}$8 (Kordalis et al., 3 Aug 2025). The multidimensional extension therefore preserves the usual state-dependent semi-Markov structure while replacing scalar time by several concurrent time coordinates.

A useful contrast is provided by the discrete multi-time renewal model of (Kordalis et al., 30 Jun 2026). There, the process is

$(J,S)=(J_n,S_n)_{n\in\mathbb N}$9

with i.i.d. $J_n$0-valued increments. That framework develops multi-time renewal equations and asymptotics, but it is explicitly described as not truly Markov renewal because there is no auxiliary state process and no state-dependent kernel. This distinction is central: multi-time Markov renewal chains are semi-Markov objects, whereas multi-time renewal chains in the i.i.d. sense are state-free renewal processes.

2. Multidimensional time, counting processes, and kernel structure

The time domain in (Kordalis et al., 3 Aug 2025) is the lattice $J_n$1 with componentwise order

$J_n$2

Jump times satisfy $J_n$3 in this partial-order sense, although some coordinates of $J_n$4 may be zero. The paper explicitly allows instantaneous jumps in some components. This accommodates settings in which one time coordinate advances while another does not.

The number of jumps up to time $J_n$5 is

$J_n$6

and the relation

$J_n$7

shows that the global count is controlled by the slowest coordinate (Kordalis et al., 3 Aug 2025). This provides the basic interpretation of multidimensional time in the model.

The kernel admits several associated distributions. The unconditional sojourn-time law in state $J_n$8 is

$J_n$9

with cumulative and survival forms

$S_n$0

$S_n$1

$S_n$2

Conditioning also on the next state gives

$S_n$3

and the embedded Markov chain transition probabilities are obtained by summing the kernel over all multi-indices: $S_n$4

The model also has one-dimensional marginals. For each coordinate $S_n$5, summing out the remaining time dimensions yields

$S_n$6

so each coordinate induces an ordinary one-time Markov renewal chain, possibly with instantaneous transitions (Kordalis et al., 3 Aug 2025). This shows that the multidimensional theory is not a replacement for classical Markov renewal theory but a superstructure whose marginals retain the classical form.

3. Renewal equations and convolution algebra

A distinctive feature of the multi-time theory is its algebraic treatment through convolution of matrix-valued sequences. The relevant space is

$S_n$7

the set of $S_n$8 matrix-valued sequences on $S_n$9. For $d$0, convolution is defined by

$d$1

The paper states that $d$2 is a unital associative $d$3-algebra with identity $d$4, and that it is commutative iff $d$5 (Kordalis et al., 3 Aug 2025). Invertibility is characterized by the constant term: if $d$6 is nonsingular, then $d$7 has a convolution inverse; if $d$8 is singular, then it does not.

This algebra makes the multi-time Markov renewal equation

$d$9

formally parallel to the one-time case. If $X_n=S_n-S_{n-1}$0, then

$X_n=S_n-S_{n-1}$1

and the unique solution is

$X_n=S_n-S_{n-1}$2

The same resolvent-like object controls the principal derived quantities. The transition function of the associated discrete-time semi-Markov chain

$X_n=S_n-S_{n-1}$3

satisfies

$X_n=S_n-S_{n-1}$4

where

$X_n=S_n-S_{n-1}$5

Likewise, the multidimensional renewal function

$X_n=S_n-S_{n-1}$6

satisfies

$X_n=S_n-S_{n-1}$7

hence

$X_n=S_n-S_{n-1}$8

(Kordalis et al., 3 Aug 2025).

A closely related but non-Markovian algebra appears in (Kordalis et al., 30 Jun 2026), where the basic renewal equation is

$X_n=S_n-S_{n-1}$9

for the renewal function $(J_{n+1},S_{n+1}-S_n)$0 of a pure i.i.d. multi-time renewal chain. The similarity of the convolutional fixed-point form helps explain why multi-time Markov renewal chains admit an algebraic treatment. A plausible implication is that multidimensional convolution is the natural common language for both state-free and state-dependent multi-time renewal models, with matrix-valued kernels distinguishing the Markov renewal case.

4. First-passage, recurrence, and ergodic structure

Beyond renewal equations, (Kordalis et al., 3 Aug 2025) develops first-passage and recurrence formulas in the multidimensional setting. The first passage distributions $(J_{n+1},S_{n+1}-S_n)$1 satisfy

$(J_{n+1},S_{n+1}-S_n)$2

and also the recursion

$(J_{n+1},S_{n+1}-S_n)$3

This is the direct multi-time analogue of the usual decomposition into first-step hit and delayed hit through an intermediate state.

The paper defines recurrence, positive recurrence, $(J_{n+1},S_{n+1}-S_n)$4-periodicity, ergodicity, and irreducibility in terms of the marginal chains and the embedded Markov chain $(J_{n+1},S_{n+1}-S_n)$5 (Kordalis et al., 3 Aug 2025). When $(J_{n+1},S_{n+1}-S_n)$6 is irreducible and ergodic with stationary distribution $(J_{n+1},S_{n+1}-S_n)$7, recurrence moments are expressed through stationary weights and local sojourn moments. For example,

$(J_{n+1},S_{n+1}-S_n)$8

and

$(J_{n+1},S_{n+1}-S_n)$9

These formulas quantify how the embedded chain and multidimensional sojourn statistics jointly determine long-run recurrence behavior.

The transformation result

$J_n$0

for $J_n$1 provides another structural device (Kordalis et al., 3 Aug 2025). The paper applies it to product-time variables such as $J_n$2, whose induced kernel is

$J_n$3

This makes covariance-type quantities accessible through ordinary one-dimensional Markov renewal analysis of transformed sojourns.

A broader conceptual link appears in (Chan et al., 20 May 2026), where a stationary spacing sequence with finite memory is recast as a Markov chain on blocks and then analyzed via a Markov renewal theorem with exponential rate. That paper treats a one-dimensional setting, but its conclusion that renewal theory can be extended from i.i.d. increments to dependent, finite-memory spacings suggests a wider interpretation of “multi-time” or sequential Markov renewal structure: not only multiple clock coordinates, but also renewal mechanisms driven by dependence across successive increments.

5. Statistical inference and asymptotic estimation

Statistical work on Markov renewal models remains largely one-time, but it supplies tools that are immediately relevant to the multi-time setting. In the discrete-time finite-state model of (Ogata et al., 2023), a Markov renewal chain

$J_n$4

with kernel sequence

$J_n$5

is estimated nonparametrically by

$J_n$6

Under irreducibility, aperiodicity, and positive recurrence, the paper proves the joint limit

$J_n$7

in the product space $J_n$8, with covariance

$J_n$9

(Ogata et al., 2023). The same paper derives multi-dimensional asymptotic normality for the convolution inverse $E=\{1,\dots,s\},$0, the semi-Markov distribution matrix sequence $E=\{1,\dots,s\},$1, and the reliability vector sequence $E=\{1,\dots,s\},$2.

These results are not formulated for multidimensional time coordinates, but they are directly aligned with the algebraic program of (Kordalis et al., 3 Aug 2025): both frameworks represent derived quantities as renewal-transform functionals of the kernel. This suggests that asymptotic inference for multi-time Markov renewal chains may plausibly proceed by extending the product-space CLT and convolution-linearization arguments from the one-time kernel array to the multidimensional kernel $E=\{1,\dots,s\},$3. That is an inference-oriented interpretation rather than an explicit theorem of the cited papers.

A second inferential perspective arises in the pure multi-time renewal setting (Kordalis et al., 30 Jun 2026). There, fixed-horizon observation induces a multivariate right-censoring mechanism through the terminal age vector

$E=\{1,\dots,s\},$4

and the paper derives an exact nonparametric maximum likelihood estimator together with asymptotic normality. Since this model is explicitly state-free, it does not yield estimation theory for multi-time Markov renewal chains themselves. It does, however, show that genuinely multivariate censoring phenomena are intrinsic once observation is truncated on $E=\{1,\dots,s\},$5.

6. Computation, applications, and neighboring formulations

The computational contribution of (Kordalis et al., 3 Aug 2025) is a multidimensional adaptation of Gauss-Jordan elimination for matrix-valued sequence inversion. Elementary row operations are encoded as convolutional matrix sequences, and the paper proves that for $E=\{1,\dots,s\},$6 the following are equivalent: $E=\{1,\dots,s\},$7 has a convolutional inverse, $E=\{1,\dots,s\},$8 is row equivalent to $E=\{1,\dots,s\},$9, and $(J,S) := (J_n,S_n)_{n\in\mathbb N},$0 can be written as a finite convolution product of elementary sequences. This yields a constructive inversion procedure for $(J,S) := (J_n,S_n)_{n\in\mathbb N},$1 and therefore for the solution of multi-time Markov renewal equations.

The paper also states that FFT-based convolution is dramatically faster than direct convolution, and that Gauss-Jordan inversion is often faster than Newton’s method and much faster than direct computation (Kordalis et al., 3 Aug 2025). The reported complexity when FFT and Newton-type inversion are used in the background is roughly

$(J,S) := (J_n,S_n)_{n\in\mathbb N},$2

In the related state-free framework (Kordalis et al., 30 Jun 2026), the corresponding computational program combines FFT-based multidimensional convolution with Newton-type reciprocal iteration. The parallel is important: both papers treat multi-index convolution not merely as notation, but as the basis of scalable numerical evaluation on large grids.

Applications in (Kordalis et al., 3 Aug 2025) are framed mainly around finite-state systems with multidimensional sojourn descriptors, and the paper gives a 3-state, 2-time example with bivariate Poisson sojourns shifted by $(J,S) := (J_n,S_n)_{n\in\mathbb N},$3. It computes state-dependent first and second moments, covariance $(J,S) := (J_n,S_n)_{n\in\mathbb N},$4, recurrence moments $(J,S) := (J_n,S_n)_{n\in\mathbb N},$5, and recurrence covariances and correlations. In (Kordalis et al., 30 Jun 2026), applications include two-attribute warranty evaluation, alternating-renewal availability computation, and discretization-based approximations of continuous-time bivariate renewal and availability models. Those examples belong formally to pure renewal theory rather than Markov renewal theory, but they indicate the kinds of multi-attribute time structures for which a state-dependent extension is natural.

Another neighboring line of work studies synchronization rather than multidimensional clock coordinates. For two independent time-inhomogeneous Markov chains, the simultaneous renewal time

$(J,S) := (J_n,S_n)_{n\in\mathbb N},$6

has explicit expectation bounds under dominating-sequence and regularity assumptions (Golomoziy, 2017, Golomoziy, 2020). This is a multi-time problem in the sense of multiple renewal clocks evolving in parallel, not in the sense of a single $(J,S) := (J_n,S_n)_{n\in\mathbb N},$7-valued clock. The distinction matters because the underlying mathematics is coupling and overshoot control rather than multidimensional convolution, yet both settings study common regenerative structure across more than one temporal index.

7. Conceptual scope, misconceptions, and current significance

A recurrent misconception is to identify any multidimensional renewal model with a multi-time Markov renewal chain. The comparison between (Kordalis et al., 3 Aug 2025) and (Kordalis et al., 30 Jun 2026) shows why this is incorrect. A genuine multi-time Markov renewal chain requires a state process $(J,S) := (J_n,S_n)_{n\in\mathbb N},$8 and a state-dependent kernel $(J,S) := (J_n,S_n)_{n\in\mathbb N},$9. A pure multi-time renewal chain has only i.i.d. increment vectors and therefore lacks the semi-Markov dependence mechanism.

A second misconception is to think that multidimensional time merely duplicates several one-dimensional models. In fact, the key relation

$(J,S)=(J_n,S_n)_{n\in\mathbb N}$00

shows that the coordinates interact through the partial order and through the slowest-clock effect (Kordalis et al., 3 Aug 2025). The model is therefore not a collection of independent marginal renewal chains, even though each coordinate admits a marginal kernel.

The present significance of the topic lies in its synthesis of three strands. First, the algebraic strand treats multidimensional semi-Markov kernels as matrix-valued formal power series with explicit inversion theory (Kordalis et al., 3 Aug 2025). Second, the asymptotic strand shows that kernel-based functionals of one-time Markov renewal chains admit joint central limit theorems and renewal-transform delta methods (Ogata et al., 2023). Third, the probabilistic strand demonstrates that Markov-dependent renewal mechanisms can still yield quantitative renewal theorems with exponential remainders when dependence is controlled by geometric ergodicity and regeneration (Chan et al., 20 May 2026). Taken together, these results suggest that multi-time Markov renewal chains are emerging as a framework in which multidimensional clocks, state-dependent sojourn laws, and computationally explicit renewal equations can be studied within a unified semi-Markov formalism.