Hierarchical Semi-Markov Models

• Hierarchical semi-Markov frameworks are sequential models that represent persistence over segments arranged in multiple temporal or semantic scales.
• They extend traditional Markov models by incorporating recursive parent-child state hierarchies, Bayesian sharing, partial pooling, and regime modulation for richer dynamics.
• Applications span activity recognition, natural language processing, and financial modeling, where explicit duration modeling and multilevel structure enhance performance.

Hierarchical semi-Markov framework denotes a family of sequential models in which persistence is represented over segments rather than isolated time steps, while those segments are organized across multiple temporal or semantic scales. In the strongest recursive formulations, a state at one level persists over an interval and its internal realization is a sequence of child states at the next lower level, yielding nested sequential structure with explicit parent-child synchronization; in other formulations, hierarchy appears through Bayesian sharing, regime-modulated continuous dynamics, or multilevel pooling rather than through a latent recursive state tree (Tran et al., 2014, Truyen et al., 2010, Dube et al., 22 Sep 2025).

1. Concept and scope

Within the literature, the phrase hierarchical semi-Markov does not refer to a single mathematical template. In the recursive sequential sense, the framework is exemplified by Hierarchical Semi-Markov Conditional Random Fields (HSCRFs), described as “deep generalisations” of HMMs and linear-chain CRFs and as bounded-depth members of the probabilistic context-free grammar family. In that setting, a high-level segment persists for some duration, and within that segment a lower-level Markov process unfolds; each lower-level state may itself expand into another subchain, continuing down a bounded hierarchy (Tran et al., 2014).

A second meaning arises in Bayesian nonparametrics. The Hierarchical Dirichlet Process Hidden Semi-Markov Model (HDP-HSMM) is hierarchical because transition distributions are coupled through a hierarchical Dirichlet process, and semi-Markov because persistence is modeled through explicit, state-specific duration distributions rather than through repeated self-transitions (Johnson et al., 2012). Here the hierarchy is over shared transition structure and prior coupling, not necessarily over recursively nested parent and child segments.

A third meaning is multilevel parameter hierarchy. In the activity-sequence model of daily human behavior, the framework is explicitly called hierarchical, but the paper states that the hierarchy is not a latent-state hierarchy in the HMM sense. Instead, slot-level, group-specific routing and hazard parameters are shrunk toward block-level prototypes across day-part blocks and demographic groups, yielding a time-inhomogeneous observed-state semi-Markov generator with Bayesian partial pooling (Dube et al., 22 Sep 2025).

A fourth, closely related construction appears in continuous time. The hierarchical Markov model for ion-channel modal gating is not presented as a semi-Markov process at the full-state level, yet its aggregated modal dwell times are generally non-exponential phase-type distributions. At the coarse modal level it therefore reproduces behavior often associated with semi-Markov models, while the full latent process remains an ordinary CTMC (Siekmann et al., 2016).

These usages establish that the framework is better understood as a design space than as a single canonical model class. The common thread is the joint treatment of duration and multiscale structure; the main variation lies in where the hierarchy is placed—latent state recursion, prior coupling, parameter pooling, or regime modulation.

2. Recursive latent-state mechanics and formal structure

The most explicit recursive formulation is the HSCRF. The starting contrast is the linear-chain CRF

$$P(x|o)=\frac{1}{Z(o)}\prod_{t=1}^{T-1}\phi_t(x_t,x_{t+1},o),$$

with local Markov property

$P(x_t\mid x_{\neg t})=P(x_t\mid x_{t-1},x_{t+1}).$

Semi-Markov models relax pointwise transitions by allowing a state to persist over a segment, and hierarchical semi-Markov models extend this further so that a parent-state at one level contains, as its internal realization, a sequence of child states at the next lower level. The resulting architecture is described as “fractal”: each level, viewed in isolation, resembles a Markov chain, but each state in that chain is itself refined into another chain below (Tran et al., 2014).

In HSCRFs, the hidden configuration comprises hierarchical state variables $x$ and transition variables

$$e=(e_1,e_2,\ldots,e_T), \qquad e_t=d,\; 1\le d\le D,$$

where $e_t=d$ means that the transition at time $t$ occurs at hierarchical level $d$. The conditional model is written

$P(x,e\mid o)=\frac{1}{Z(o)}\Phi(x,e,o).$

A central structural property is that for any model of depth $D$, there is exactly one transition occurring at a specific time $t$. If the transition occurs at level $P(x_t\mid x_{\neg t})=P(x_t\mid x_{t-1},x_{t+1}).$0, then all states at levels $P(x_t\mid x_{\neg t})=P(x_t\mid x_{t-1},x_{t+1}).$1 remain unchanged, while all states at levels $P(x_t\mid x_{\neg t})=P(x_t\mid x_{t-1},x_{t+1}).$2 must already have ended. Once $P(x_t\mid x_{\neg t})=P(x_t\mid x_{t-1},x_{t+1}).$3 is fixed, the remaining conditional structure $P(x_t\mid x_{\neg t})=P(x_t\mid x_{t-1},x_{t+1}).$4 collapses into a Markov tree (Tran et al., 2014).

The earlier HSCRF formulation for recursive sequential data makes the recursive structure fully explicit through state variables $P(x_t\mid x_{\neg t})=P(x_t\mid x_{t-1},x_{t+1}).$5 and ending indicators $P(x_t\mid x_{\neg t})=P(x_t\mid x_{t-1},x_{t+1}).$6 across levels $P(x_t\mid x_{\neg t})=P(x_t\mid x_{t-1},x_{t+1}).$7. It imposes deterministic nestedness constraints such as: if a state ends, all descendants must end; if a state persists, all ancestors must persist; bottom-level states do not persist; and all levels terminate at the final time. The global conditional distribution is

$P(x_t\mid x_{\neg t})=P(x_t\mid x_{t-1},x_{t+1}).$8

where $P(x_t\mid x_{\neg t})=P(x_t\mid x_{t-1},x_{t+1}).$9. The global potential factorizes into four contextual clique types: state-persistence $x$0, state-transition $x$1, state-initialization $x$2, and state-ending $x$3, each parameterized in log-linear form by observation-dependent features (Truyen et al., 2010).

This recursive semantics yields a strict parent-child synchronization rule: a parent can transition only after its active child chain terminates, and during the lifetime of the child the parent must remain unchanged. The resulting object is neither a flat segment model nor a generic multilayer graph; it is a depth-bounded conditional random field over nested segments, durations, transitions, and hierarchical legality constraints.

3. Inference, learning, and computational trade-offs

Exact inference in recursive hierarchical semi-Markov models is substantially more involved than in flat chains. For HSCRFs, the main exact scheme is the Asymmetric Inside-Outside (AIO) algorithm, which defines symmetric inside/outside masses over parent segments and asymmetric inside/outside masses over parent segments conditioned on the identity of the returning child. From these quantities one obtains the partition function, marginal probabilities, expected sufficient statistics, and a generalized Viterbi decoder by replacing summation with maximization. The resulting learning algorithm supports fully supervised, partially supervised, and constrained settings through dynamic programming over only the legal completions of the observed labels (Truyen et al., 2010).

The conference paper on MCMC for HSCRFs addresses the main computational bottleneck of exact inference. It contrasts two exact alternatives already in the literature: an inside-outside-style dynamic program with complexity $x$4, and a DBN linearization that is linear in sequence length but exponential in model depth. The proposed Rao-Blackwellised Gibbs sampler samples only the transition-level sequence $x$5 and integrates out the state variables $x$6: $x$7 with Gibbs conditionals

$x$8

By exploiting the one-transition-at-a-time property and a walking-chain procedure, a full sweep is claimed to be obtainable in $x$9 time. The empirical evaluation is simulation-based: with $$e=(e_1,e_2,\ldots,e_T), \qquad e_t=d,\; 1\le d\le D,$$0, $$e=(e_1,e_2,\ldots,e_T), \qquad e_t=d,\; 1\le d\le D,$$1, 50 training sequences, and 50 test sequences, KL divergence and absolute difference improve over iterations but slow markedly after roughly 500 iterations, whereas maximal-marginal decoding accuracy rises quickly and reaches a high level after only about 10 iterations (Tran et al., 2014).

Bayesian nonparametric explicit-duration models follow a different inference pattern. In the HDP-HSMM, the latent sequence is represented by segment labels $$e=(e_1,e_2,\ldots,e_T), \qquad e_t=d,\; 1\le d\le D,$$2 and durations $$e=(e_1,e_2,\ldots,e_T), \qquad e_t=d,\; 1\le d\le D,$$3, with backward messages

$$e=(e_1,e_2,\ldots,e_T), \qquad e_t=d,\; 1\le d\le D,$$4

and duration-summing HSMM recursions. The baseline complexity is

$$e=(e_1,e_2,\ldots,e_T), \qquad e_t=d,\; 1\le d\le D,$$5

reduced to

$$e=(e_1,e_2,\ldots,e_T), \qquad e_t=d,\; 1\le d\le D,$$6

when durations are truncated. The paper develops both a direct-assignment sampler and a weak-limit blocked Gibbs sampler; the latter restores conjugacy through auxiliary variables after self-transitions are removed and is reported to mix much faster in practice (Johnson et al., 2012).

Across these formulations, the core trade-off is stable. Exact dynamic programming preserves full posterior structure but can be cubic in length or worse under richer hierarchies. Approximate samplers and weak-limit constructions reduce dimensionality or improve mixing, but introduce dependence on convergence behavior, truncation, or model-family assumptions.

4. Principal families of hierarchical semi-Markov modeling

The main variants differ in what exactly is made hierarchical and how duration is represented.

Family	Hierarchical mechanism	Duration mechanism
HSCRF (Truyen et al., 2010)	Recursive parent-child latent state hierarchy	Segment persistence potentials at each level
HDP-HSMM (Johnson et al., 2012)	Hierarchical Dirichlet-process sharing of transition rows	Explicit state-specific duration distributions
Activity-sequence model (Dube et al., 22 Sep 2025)	Block-level and group-level partial pooling of observed-state parameters	Discrete hazard over run-length bins
Modal gating model (Siekmann et al., 2016)	Top-level mode process controls within-mode CTMC kinetics	Phase-type modal dwell times via Markov embedding
Semi-Markov-modulated diffusion (D'Amico et al., 2012)	Finite-state semi-Markov regime process modulates a lower continuous diffusion	Holding-time law in the upper semi-Markov regime process

The observed-state activity model is especially notable because it separates “whether to exit” from “where to go if exiting.” The one-step kernel is

$$e=(e_1,e_2,\ldots,e_T), \qquad e_t=d,\; 1\le d\le D,$$7

where $$e=(e_1,e_2,\ldots,e_T), \qquad e_t=d,\; 1\le d\le D,$$8 is a duration-aware hazard and $$e=(e_1,e_2,\ldots,e_T), \qquad e_t=d,\; 1\le d\le D,$$9 is the router’s conditional destination distribution. Hierarchy enters through Dirichlet and Beta shrinkage toward day-part prototypes rather than through latent recursion (Dube et al., 22 Sep 2025).

Continuous-time regime-modulated constructions provide another branch of the framework. In the semi-Markov-modulated interest-rate model, a finite-state semi-Markov process $e_t=d$0 controls regime-specific diffusions such as Vasicek, Hull–White, or CIR, and recursive equations are derived for moments of the discount factor by conditioning on whether a regime switch occurs before the horizon (D'Amico et al., 2012). In the ion-channel modal-gating model, a top-level aggregated mode process $e_t=d$1 governs lower-level within-mode generators $e_t=d$2, and the full CTMC generator is assembled by Kronecker sums and products: $e_t=d$3 This is formally a hierarchical CTMC, but at the aggregated modal level it inherits non-exponential phase-type sojourns (Siekmann et al., 2016).

Several related models are adjacent rather than fully semi-Markov. The multiscale animal-behavior model uses an upper-level Markov chain over internal states that selects among lower-level HMMs over production states; the authors explicitly position it as a strong scaffold for later hidden semi-Markov extensions because dwell times remain geometric at both levels (Leos-Barajas et al., 2017). The Infinite Mixture Model of Markov Chains similarly has two levels—super states and component-specific Markov chains—but no explicit duration distribution; segmentation arises implicitly through boundary states and transition structure rather than through HSMM-style duration variables (Reubold et al., 2017). The tracepoint-based runtime model is an absorbing semi-Markov chain built from observable events across software layers and is explicitly not hierarchical, though its event-centric construction suggests a plausible substrate for hierarchical extensions (Bielmeier et al., 30 Jul 2025).

5. Applications and empirical behavior

The HSCRF was demonstrated on activity recognition from surveillance video and noun-phrase chunking. In the activity-recognition experiment, the hierarchy had depth 3 with 3 persistent high-level activities and 12 sub-trajectories. Under full supervision, the reported accuracies were $e_t=d$4 at level $e_t=d$5 and $e_t=d$6 at level $e_t=d$7, compared with $e_t=d$8 and $e_t=d$9 for a DCRF and $t$0 for a flat CRF at the lower level. Under partial supervision, the partially observed HSCRF achieved $t$1 at level $t$2 and $t$3 at level $t$4, while a partially observed flat CRF achieved $t$5 at the bottom level. In noun-phrase chunking, HSCRF with POS tags supplied at test time consistently outperformed the baselines reported in the figure, including SCRF, SemiCRF, and DCRF variants (Truyen et al., 2010).

The Bayesian nonparametric explicit-duration branch shows similar empirical gains when duration is genuinely informative. On synthetic HSMM data with non-geometric durations, the HDP-HMM fails to reproduce duration statistics and tends to infer the wrong number of states, whereas the Poisson-HDP-HSMM recovers transitions, durations, emissions, and state cardinality. On real power-disaggregation data, the reported mean accuracies are $t$6 for an EM-trained factorial HMM, $t$7 for a factorial HDP-HMM, and $t$8 for a factorial HDP-HSMM (Johnson et al., 2012).

Observed-state hierarchical semi-Markov modeling has recently been used to generate realistic daily human-activity traces from American Time Use Survey diaries. The model uses $t$9 ten-minute slots, $d$0 activity states, and survey-weighted estimation over 252,808 respondents. On held-out data, the baseline Markov model $d$1 has NLL $d$2, while the baseline semi-Markov model $d$3 has NLL $d$4; the difference is evaluated by a paired bootstrap with 2,000 respondent-level resamples. Top-1 accuracy remains roughly $d$5 across models, while the demographic factors giving the largest predictive gains are Sex, Household Size, and Day-Type (Dube et al., 22 Sep 2025).

Continuous-time variants show that the framework is not confined to symbolic sequence labeling. In ion-channel modal gating, the hierarchical Markov construction preserves the top-level modal sojourn-time density in the full model and provides a better representation than the earlier flat model of the same dataset; for most datasets, one active-mode state sufficed while the inactive mode required two states (Siekmann et al., 2016). In financial modeling, semi-Markov modulation has been used to drive regime-dependent Vasicek, Hull–White, and CIR short-rate dynamics, with recursive equations derived for higher-order moments of the discount factor and a Monte Carlo algorithm specified for simulation (D'Amico et al., 2012).

6. Limitations, misconceptions, and open directions

A recurring misconception is that all hierarchical semi-Markov models use the same notion of hierarchy. The activity-sequence model is explicit that its hierarchy is a multilevel parameter hierarchy, not a latent-state hierarchy in the HMM sense (Dube et al., 22 Sep 2025). The ion-channel modal-gating model is equally explicit that the integrated process is exactly a CTMC and not a semi-Markov process at the full-state level; its semi-Markov character appears only after aggregation by mode (Siekmann et al., 2016). The term therefore spans materially different mechanisms, and claims about interpretability, tractability, or duration semantics do not transfer automatically across all variants.

The main algorithmic limitation is inference cost. Exact HSCRF inference remains cubic in sequence length, while alternative exact linearizations can be exponential in depth. The Rao-Blackwellised Gibbs sampler was introduced precisely because exact inference is prohibitive for large-scale problems with arbitrary length and depth, but the same paper is explicit that the work is preliminary: the chain may mix slowly, the convergence plots show slow decay of KL divergence after initial gains, the walking-chain derivation is omitted from the conference version, and the experiments are synthetic rather than on large real-world data (Tran et al., 2014).

Statistical and modeling limitations are also prominent in semi-Markov variants outside latent recursive HSCRFs. In the Markov-renewal proportional-hazards setting, future work is stated to require advanced covariate structures and more general non-Markovian dynamics (Cuicizion, 27 Jan 2025). In predictive runtime analysis from tracepoints, the proposed semi-Markov chain is explicitly flat rather than hierarchical and lacks analytical first-passage formulas, uncertainty bounds, hidden-state structure, and long-memory mechanisms (Bielmeier et al., 30 Jul 2025). These gaps indicate that many practically useful semi-Markov models remain only partially hierarchical, or only partially inferentially developed.

From a verification standpoint, semi-Markov models are also difficult to compare compositionally. The “faster-than” relation over semi-Markov processes is undecidable in general; nevertheless, there is an additive approximation algorithm for the time-bounded faster-than problem over slow residence-time functions, and the exact faster-than problem is in $d$6 for unambiguous semi-Markov processes (Pedersen et al., 2017). This suggests that a fully general hierarchical semi-Markov theory will likely require restricted subclasses, compositional abstractions, or approximate semantics to remain analyzable.

Current research directions are correspondingly diverse. Explicitly stated open problems include improving RBGS mixing speed, combining RBGS with contrastive divergence for learning, and testing HSCRF approximations on real large-scale problems with arbitrary length and depth (Tran et al., 2014). In observed-state hierarchy, stated extensions include multi-covariate interaction models, more granular activity taxonomies, and multi-agent household models with coupled individual sequences (Dube et al., 22 Sep 2025). A plausible implication is that future hierarchical semi-Markov frameworks will be increasingly hybrid: recursive where deep nesting is indispensable, partially pooled where sparse heterogeneity dominates, and Markov-embedded where continuous-time tractability matters more than explicit duration kernels.