--- title: Multi-Time Markov Renewal Chains url: https://www.emergentmind.com/topics/multi-time-markov-renewal-chains type: topic --- # Multi-Time Markov Renewal Chains Searching arXiv for recent papers on multi-time and related Markov renewal theory. Multi-Time Markov Renewal Chains are a multidimensional extension of classical Markov renewal theory in which jump times evolve on the lattice $\mathbb N^d$ rather than along a single time axis. In the formulation introduced in "Time multidimensional Markov Renewal chains -- An algebraic approach" [2508.01891], the process is a pair $(J,S)=(J_n,S_n)_{n\in\mathbb N}$, where $J_n$ is the visited state and $S_n$ is a $d$-dimensional jump time, with inter-jump vector $X_n=S_n-S_{n-1}$. The defining property is a Markov renewal kernel on multidimensional sojourns: the conditional law of $(J_{n+1},S_{n+1}-S_n)$ given the past depends only on $J_n$. This places multi-time Markov renewal chains at the intersection of semi-Markov modeling, renewal equations, and multidimensional convolution algebra. Closely related work clarifies what these models are not: purely i.i.d. multi-time renewal chains have a similar renewal-equation structure but lack the state-dependent kernel that characterizes genuine Markov renewal dynamics [2606.31455]. ## 1. Formal definition and position within renewal theory In the finite-state framework of [2508.01891], the state space is \[ E=\{1,\dots,s\}, \] and a multi-time Markov renewal chain is the pair \[ (J,S) := (J_n,S_n)_{n\in\mathbb N}, \] with $J_n\in E$, $S_n\in\mathbb N^d$, and $X_n:=S_n-S_{n-1}$ the $d$-dimensional sojourn time. The Markov renewal property is \[ \mathbb{P}(J_{n+1}=j,\, S_{n+1}-S_n=k_{1:d}\mid J_{0:n},S_{0:n}) = \mathbb{P}(J_{n+1}=j,\, S_{n+1}-S_n=k_{1:d}\mid J_n). \] When this law does not depend on $n$, the chain is homogeneous and is governed by the multi-time semi-Markov kernel \[ q_{ij}(k_{1:d}) := \mathbb{P}(J_{n+1}=j,\; S_{n+1}-S_n=k_{1:d}\mid J_n=i). \] This definition reduces to the ordinary one-time Markov renewal chain when $d=1$, and a renewal process is recovered as the special case $s=1$ [2508.01891]. The multidimensional extension therefore preserves the usual state-dependent semi-Markov structure while replacing scalar time by several concurrent time coordinates. A useful contrast is provided by the discrete multi-time renewal model of [2606.31455]. There, the process is \[ S_0 = 0_d,\qquad S_n=\sum_{i=1}^n X_i, \] with i.i.d. $\mathbb N^d$-valued increments. That framework develops multi-time renewal equations and asymptotics, but it is explicitly described as not truly Markov renewal because there is no auxiliary state process and no state-dependent kernel. This distinction is central: multi-time Markov renewal chains are semi-Markov objects, whereas multi-time renewal chains in the i.i.d. sense are state-free renewal processes. ## 2. Multidimensional time, counting processes, and kernel structure The time domain in [2508.01891] is the lattice $\mathbb N^d$ with componentwise order \[ k\le l \iff k_u\le l_u \text{ for all }u=1,\dots,d. \] Jump times satisfy $S_nk_{1:d}\mid J_n=i), \] \[ \widetilde H_i(k_{1:d}) := 1-\mathbb P(X_{n+1}\le k_{1:d}\mid J_n=i). \] Conditioning also on the next state gives \[ f_{ij}(k_{1:d}) := \mathbb P(X_{n+1}=k_{1:d}\mid J_n=i,J_{n+1}=j), \] and the embedded Markov chain transition probabilities are obtained by summing the kernel over all multi-indices: \[ p_{ij} := \mathbb P(J_{n+1}=j\mid J_n=i) = \sum_{k_{1:d}} q_{ij}(k_{1:d}). \] The model also has one-dimensional marginals. For each coordinate $u$, summing out the remaining time dimensions yields \[ q^{[u]}_{ij}(k_u) = \sum_{k_{1:u-1},k_{u+1:d}} q_{ij}(k_{1:d}), \] so each coordinate induces an ordinary one-time Markov renewal chain, possibly with instantaneous transitions [2508.01891]. This shows that the multidimensional theory is not a replacement for classical Markov renewal theory but a superstructure whose marginals retain the classical form. ## 3. Renewal equations and convolution algebra A distinctive feature of the multi-time theory is its algebraic treatment through convolution of matrix-valued sequences. The relevant space is \[ \mathcal M_s(\mathbb N^d), \] the set of $s\times s$ matrix-valued sequences on $\mathbb N^d$. For $A,B\in\mathcal M_s(\mathbb N^d)$, convolution is defined by \[ [A*B](k_{1:d}) := \sum_{l+l'=k_{1:d}} A(l)B(l'). \] The paper states that $(\mathcal M_s(\mathbb N^d),+,*)$ is a unital associative $\mathbb R$-algebra with identity $\mathbbm{I}_s$, and that it is commutative iff $s=1$ [2508.01891]. Invertibility is characterized by the constant term: if $A(0_d)$ is nonsingular, then $A$ has a convolution inverse; if $A(0_d)$ is singular, then it does not. This algebra makes the multi-time Markov renewal equation \[ L = G + q*L \] formally parallel to the one-time case. If $q(0_d)=O_s$, then \[ u := (\mathbbm{I}_s-q)^{(-1)} = \sum_{n\ge 0} q^{(n)}, \] and the unique solution is \[ L = u*G. \] The same resolvent-like object controls the principal derived quantities. The transition function of the associated discrete-time semi-Markov chain \[ Z_{k_{1:d}} := J_{N(k_{1:d})} \] satisfies \[ P = u*\widetilde H, \] where \[ P_{ij}(k_{1:d}) := \mathbb P(Z_{k_{1:d}}=j\mid Z_{0_d}=i). \] Likewise, the multidimensional renewal function \[ U_{ij}(k_{1:d}) := \mathbb E_i[\widetilde N_j(k_{1:d})] \] satisfies \[ U = \mathbbm{I}_s + q*U, \] hence \[ U = \mathbbm{I}_s*u \] [2508.01891]. A closely related but non-Markovian algebra appears in [2606.31455], where the basic renewal equation is \[ M = F + f*M \] for the renewal function $M(k)=\mathbb E[N(k)]$ of a pure i.i.d. multi-time renewal chain. The similarity of the convolutional fixed-point form helps explain why multi-time Markov renewal chains admit an algebraic treatment. A plausible implication is that multidimensional convolution is the natural common language for both state-free and state-dependent multi-time renewal models, with matrix-valued kernels distinguishing the Markov renewal case. ## 4. First-passage, recurrence, and ergodic structure Beyond renewal equations, [2508.01891] develops first-passage and recurrence formulas in the multidimensional setting. The first passage distributions $g_{ij}$ satisfy \[ g = (u-\mathbb I_s)*\mathrm{dg}(u)^{(-1)}, \] and also the recursion \[ g_{ij} = q_{ij} + \sum_{r\neq j} q_{ir}*g_{rj}. \] This is the direct multi-time analogue of the usual decomposition into first-step hit and delayed hit through an intermediate state. The paper defines recurrence, positive recurrence, $r_{1:d}$-periodicity, ergodicity, and irreducibility in terms of the marginal chains and the embedded Markov chain $J$ [2508.01891]. When $J$ is irreducible and ergodic with stationary distribution $\nu$, recurrence moments are expressed through stationary weights and local sojourn moments. For example, \[ \mu_{jj}^{(u)}=\frac{\sum_{i=1}^s \nu_i m_i^{[u]}}{\nu_j}, \] and \[ \mu_{jj}^{(u,v)} = \frac{ \sum_{i=1}^s \nu_i m_i^{[u,v]} + \sum_{i=1}^s\sum_{r\neq j} \nu_i p_{ir} \big[ m_{ir}^{[u]} \mu_{rj}^{(v)} + m_{ir}^{[v]} \mu_{rj}^{(u)} \big] }{\nu_j}. \] These formulas quantify how the embedded chain and multidimensional sojourn statistics jointly determine long-run recurrence behavior. The transformation result \[ q^{[\phi]}_{ij}(k) = \sum_{\substack{v:\,\phi(v)=k}} q_{ij}(v) \] for $\Phi_n=\phi(X_n)$ provides another structural device [2508.01891]. The paper applies it to product-time variables such as $Y_n=X_n^{[u]}X_n^{[v]}$, whose induced kernel is \[ q^{[y]}_{ij}(k) = \sum_{\substack{k_{1:d}: k_u k_v = k}} q_{ij}(k_{1:d}). \] This makes covariance-type quantities accessible through ordinary one-dimensional Markov renewal analysis of transformed sojourns. A broader conceptual link appears in [2605.21638], where a stationary spacing sequence with finite memory is recast as a Markov chain on blocks and then analyzed via a Markov renewal theorem with exponential rate. That paper treats a one-dimensional setting, but its conclusion that renewal theory can be extended from i.i.d. increments to dependent, finite-memory spacings suggests a wider interpretation of “multi-time” or sequential Markov renewal structure: not only multiple clock coordinates, but also renewal mechanisms driven by dependence across successive increments. ## 5. Statistical inference and asymptotic estimation Statistical work on Markov renewal models remains largely one-time, but it supplies tools that are immediately relevant to the multi-time setting. In the discrete-time finite-state model of [2304.03924], a Markov renewal chain \[ (J,S)=(J_n,S_n)_{n\ge 0} \] with kernel sequence \[ q=(q(k))_{k\in\mathbb N},\qquad q(k)=(q_{ij}(k))_{i,j\in E}, \] is estimated nonparametrically by \[ \widehat q_{ij}(k,M):= \frac{1}{N_i(M)} \sum_{n=1}^{N(M)} \mathbf 1_{\{J_{n-1}=i,\;J_n=j,\;X_n=k\}. \] Under irreducibility, aperiodicity, and positive recurrence, the paper proves the joint limit \[ \sqrt{M}\,(\widehat q(M)-q)\ \Rightarrow\ N(0,V^q) \] in the product space $\mathbb R^{\widetilde E}$, with covariance \[ V^q_{(i,j,k),(i',j',k')} = v_i\,\mathbf 1_{\{i=i'\}}\,q_{ij}(k)\, \bigl(\mathbf 1_{\{j=j',\,k=k'\}}-q_{i'j'}(k')\bigr) \] [2304.03924]. The same paper derives multi-dimensional asymptotic normality for the convolution inverse $\Psi=(\delta I-q)^{-1}$, the semi-Markov distribution matrix sequence $P$, and the reliability vector sequence $R$. These results are not formulated for multidimensional time coordinates, but they are directly aligned with the algebraic program of [2508.01891]: both frameworks represent derived quantities as renewal-transform functionals of the kernel. This suggests that asymptotic inference for multi-time Markov renewal chains may plausibly proceed by extending the product-space CLT and convolution-linearization arguments from the one-time kernel array to the multidimensional kernel $q_{ij}(k_{1:d})$. That is an inference-oriented interpretation rather than an explicit theorem of the cited papers. A second inferential perspective arises in the pure multi-time renewal setting [2606.31455]. There, fixed-horizon observation induces a multivariate right-censoring mechanism through the terminal age vector \[ U_m=m-S_{N(m)}, \] and the paper derives an exact nonparametric maximum likelihood estimator together with asymptotic normality. Since this model is explicitly state-free, it does not yield estimation theory for multi-time Markov renewal chains themselves. It does, however, show that genuinely multivariate censoring phenomena are intrinsic once observation is truncated on $\mathbb N^d$. ## 6. Computation, applications, and neighboring formulations The computational contribution of [2508.01891] is a multidimensional adaptation of Gauss-Jordan elimination for matrix-valued sequence inversion. Elementary row operations are encoded as convolutional matrix sequences, and the paper proves that for $A\in\mathcal M_s(\mathbb N^d)$ the following are equivalent: $A$ has a convolutional inverse, $A$ is row equivalent to $\mathbb I_s$, and $A$ can be written as a finite convolution product of elementary sequences. This yields a constructive inversion procedure for $(\mathbb I_s-q)^{(-1)}$ and therefore for the solution of multi-time Markov renewal equations. The paper also states that FFT-based convolution is dramatically faster than direct convolution, and that Gauss-Jordan inversion is often faster than Newton’s method and much faster than direct computation [2508.01891]. The reported complexity when FFT and Newton-type inversion are used in the background is roughly \[ \mathcal O\!\left(s^3\prod_{i=1}^d k_i\log_2(k_i)\right). \] In the related state-free framework [2606.31455], the corresponding computational program combines FFT-based multidimensional convolution with Newton-type reciprocal iteration. The parallel is important: both papers treat multi-index convolution not merely as notation, but as the basis of scalable numerical evaluation on large grids. Applications in [2508.01891] are framed mainly around finite-state systems with multidimensional sojourn descriptors, and the paper gives a 3-state, 2-time example with bivariate Poisson sojourns shifted by $(1,1)$. It computes state-dependent first and second moments, covariance $c_i^{[1,2]}$, recurrence moments $\mu_{jj}^{(u)}$, and recurrence covariances and correlations. In [2606.31455], applications include two-attribute warranty evaluation, alternating-renewal availability computation, and discretization-based approximations of continuous-time bivariate renewal and availability models. Those examples belong formally to pure renewal theory rather than Markov renewal theory, but they indicate the kinds of multi-attribute time structures for which a state-dependent extension is natural. Another neighboring line of work studies synchronization rather than multidimensional clock coordinates. For two independent time-inhomogeneous Markov chains, the simultaneous renewal time \[ T=\inf\{t>0:\ \exists m,n \text{ such that } t=\tau^{(1)}_m=\tau^{(2)}_n\} \] has explicit expectation bounds under dominating-sequence and regularity assumptions [1701.01246, 2001.02442]. This is a multi-time problem in the sense of multiple renewal clocks evolving in parallel, not in the sense of a single $\mathbb N^d$-valued clock. The distinction matters because the underlying mathematics is coupling and overshoot control rather than multidimensional convolution, yet both settings study common regenerative structure across more than one temporal index. ## 7. Conceptual scope, misconceptions, and current significance A recurrent misconception is to identify any multidimensional renewal model with a multi-time Markov renewal chain. The comparison between [2508.01891] and [2606.31455] shows why this is incorrect. A genuine multi-time Markov renewal chain requires a state process $J_n$ and a state-dependent kernel $q_{ij}(k_{1:d})$. A pure multi-time renewal chain has only i.i.d. increment vectors and therefore lacks the semi-Markov dependence mechanism. A second misconception is to think that multidimensional time merely duplicates several one-dimensional models. In fact, the key relation \[ N(k_{1:d})=\min_{1\le u\le d} N^{[u]}(k_u) \] shows that the coordinates interact through the partial order and through the slowest-clock effect [2508.01891]. The model is therefore not a collection of independent marginal renewal chains, even though each coordinate admits a marginal kernel. The present significance of the topic lies in its synthesis of three strands. First, the algebraic strand treats multidimensional semi-Markov kernels as matrix-valued formal power series with explicit inversion theory [2508.01891]. Second, the asymptotic strand shows that kernel-based functionals of one-time Markov renewal chains admit joint central limit theorems and renewal-transform delta methods [2304.03924]. Third, the probabilistic strand demonstrates that Markov-dependent renewal mechanisms can still yield quantitative renewal theorems with exponential remainders when dependence is controlled by geometric ergodicity and regeneration [2605.21638]. Taken together, these results suggest that multi-time Markov renewal chains are emerging as a framework in which multidimensional clocks, state-dependent sojourn laws, and computationally explicit renewal equations can be studied within a unified semi-Markov formalism. Source: https://www.emergentmind.com/topics/multi-time-markov-renewal-chains