Hierarchical Hidden Markov Models

• Hierarchical Hidden Markov Models (HHMMs) are probabilistic models where latent states are organized into multiple hierarchical levels. This structure allows for the representation of observations across different temporal, semantic, or structural scales, distinguishing them from conventional hidden Markov models (HMMs) with their single-level structure (Khanna et al., 2009).
• HHMMs are applied across various domains, such as biological data (e.g., animal movement, genomic analysis), human activity recognition, financial market analysis, and sequential data modeling.
• HHMMs have several extensions, including discriminative models like the Hierarchical Semi-Markov Conditional Random Field (HSCRF). Recursive, multi-scale, Bayesian and nonparametric methods offer different trade-offs between expressiveness and computational tractability.

Hierarchical hidden Markov models (HHMMs) extend hidden Markov models (HMMs) by organizing latent states into multiple levels. A state at one level may select, contain, or govern a lower-level Markov process, allowing observations to be represented at multiple temporal, semantic, or structural scales. In a conventional recursive HHMM, a high-level state invokes a subordinate HMM; the subordinate process executes until termination, after which control returns to the parent. Other models described as hierarchical HMMs use related but distinct constructions, including coarse-scale chains selecting fine-scale HMMs, Bayesian hierarchies coupling multiple HMMs across experimental traces, and nonparametric mixtures of Markov chains. These formulations share multi-level latent structure but differ in whether the hierarchy is temporally nested, block-based, statistical, or parameter-sharing.

1. Conceptual foundations and taxonomy

An ordinary HMM contains a latent state sequence and an observation sequence. At time $t$, the latent state determines the observation distribution and the transition distribution for the next state. Its central conditional-independence assumption is that observations are conditionally independent of the rest of the sequence given their corresponding latent states.

An HHMM introduces a hierarchy of latent processes. In the recursive formulation, a state at level $d$, denoted $s^d$, may embed a child HMM at level $d+1$. When the child terminates, control returns to the parent, which either initializes or transitions to another child process or terminates itself. The hierarchy therefore represents abstraction: a high-level activity is decomposed into temporally ordered lower-level activities, which may themselves contain further subactivities. “Eat breakfast,” for example, can contain entering a kitchen, going to a cupboard, taking cereal, washing dishes, and leaving the kitchen (Truyen et al., 2010).

At each level, an ending indicator $e_t^d\in\{0,1\}$ records whether the state persists or ends. Hierarchical consistency requires that persistence and termination propagate through the hierarchy:

$e_t^d=0\implies e_t^{1:d-1}=0,$

and

$e_t^d=1\implies e_t^{d+1:D}=1.$

A lower-level transition may occur while its parent remains unchanged, whereas a state termination forces termination of all descendants. Bottom-level states emit observations, and special ending states indicate completion of a subchain. The child’s internal history is not carried forward after termination; the parent abstracts over the completed child process.

The term HHMM is used for several related architectures:

Formulation	Hierarchical object	Principal use
Recursive HHMM	Nested parent–child temporal processes	Activities, language, command structure
Multi-scale HHMM	Coarse state selects a fine-scale HMM	Animal behavior, finance
Hierarchically coupled HMM	Shared hyperparameters couple trace-specific HMMs	Single-molecule kinetics
Nonparametric approximation	HDP-HMM or mixture of Markov chains	Segmentation and prediction

These formulations should not be treated as probabilistically identical. The hierarchically coupled model, for example, has the hierarchy $$\psi\rightarrow\theta_n\rightarrow z_n\rightarrow x_n$$ across signals or molecules rather than a nested temporal hierarchy (Meent et al., 2013). Similarly, the two-level animal-movement model is an outer HMM whose frame-level emissions are likelihoods from alternative lower-level HMMs (Leos-Barajas et al., 2017).

2. Recursive state semantics and graphical structure

A fixed-depth recursive HHMM contains levels $1,\ldots,D$. At time $t$, $d$0 denotes the state at level $d$1 and $d$2 denotes its ending indicator. At the top level, states transition as in an ordinary HMM. Under a parent state $d$3, child states transition among themselves, with transition behavior conditioned on the direct parent. A child process has an explicit termination mechanism; after termination, the parent may transition to another child or terminate.

The hierarchy is therefore a collection of nested Markov chains. The state hierarchy specifies semantic abstraction, while the ending indicators specify temporal containment. If a child segment occupies the interval $d$4, it is entirely contained within the lifespan of its parent. This differs from a cascaded strategy, in which each level is modeled separately and the output of one level is passed to the next. Cascaded models suffer from propagated errors and prevent information at one level from informing inference at another. HHMMs infer all levels jointly.

The generative HHMM models a joint distribution,

$d$5

where $d$6 denotes hierarchical hidden states and $d$7 observations. Transitions depend on the relevant parent, while observations are generated locally, typically at production or bottom-level states. Durations arise implicitly from the number of time steps for which a state persists. In the dynamic Bayesian-network representation, duration is not represented by an explicit segment-persistence clique in the same manner as in a semi-Markov conditional model.

Inference and learning can be performed using Inside–Outside-style dynamic programming. The reported complexity for recursive HHMM inference is

$d$8

where $d$9 is the largest state-space size, $s^d$0 is hierarchy depth, and $s^d$1 is sequence length. Collapsing the state stack into an HHMM “mega-state” yields

$s^d$2

which is practical only for shallow hierarchies (Truyen et al., 2010).

A related architecture is the two-level block-structured HHMM. In animal movement modeling, an internal state $s^d$3 selects one of $s^d$4 fine-scale HMMs for frame $s^d$5. Within that frame, a production-state chain $s^d$6 generates observations. The internal chain captures coarse behavioral persistence, while the production chain captures fine-scale states. The entire model can be evaluated as an outer HMM whose emission likelihood for internal state $s^d$7 is the forward likelihood of the corresponding inner HMM:

$s^d$8

This construction avoids coarsening fine-resolution observations while permitting inference at broader temporal scales (Leos-Barajas et al., 2017).

3. Discriminative, semi-Markov, and constrained extensions

The Hierarchical Semi-Markov Conditional Random Field (HSCRF) combines HHMM-style recursive structure with CRF-based discriminative modeling. A conventional CRF directly models $s^d$9 rather than separately specifying $d+1$0 and $d+1$1. Its conditional distribution is

$d+1$2

The HSCRF replaces local transitions with recursively contained segments. A state segment $d+1$3 starts at $d+1$4, persists through $d+1$5, and ends at $d+1$6. Its persistence potential can depend on the segment’s duration, start and end times, and observations anywhere in the interval. This permits features involving words over an entire phrase, phrase length, contextual windows, start and end cues, average velocity over an activity segment, and arbitrary combinations of labels and observations (Truyen et al., 2010).

The HSCRF uses four contextual clique types:

1. State persistence: scores an entire state segment.
2. State transition: scores a child transition under an unchanged parent.
3. State initialization: scores entry into a child process.
4. State ending: scores return of control from a child to its parent.

Its global potential is the product of these persistence, transition, initialization, and ending potentials:

$d+1$7

Unlike a conventional fixed-connectivity graphical model, the active contextual cliques depend dynamically on the ending indicators. Each potential can use a log-linear parameterization,

$d+1$8

An HSCRF can be viewed as a nested SemiCRF: each parent segment contains a sequence of child segments. A flat SemiCRF is a special case of an HSCRF, represented as a three-level model with dummy root and bottom states. HSCRFs preserve exact polynomial-time inference by restricting cross-level dependencies to recursive containment rather than permitting arbitrary dynamic-CRF interactions.

Inside–Outside recursions compute partition functions, segment marginals, expected feature counts, and constrained likelihoods. MAP inference replaces sums with maxima in a generalized Viterbi algorithm. Partial supervision is handled by inserting consistency indicators that set dynamic-programming entries to zero whenever a candidate segment conflicts with observed labels, boundaries, or ending indicators. This supports partially observed states, segment labels, start and end times, and partial information at test time.

The HSCRF was evaluated on human activities of daily living and noun-phrase chunking. In indoor-surveillance activity recognition, a fully supervised depth-three HSCRF achieved 100.0% activity-level accuracy and 93.9% sub-trajectory accuracy, compared with 96.5% and 89.7% for a DCRF and 82.6% for a flat CRF at the sub-trajectory level. Under partial supervision, the HSCRF achieved 80.2% activity-level and 90.4% sub-trajectory accuracy. In noun-phrase chunking, hierarchical joint modeling of NP and POS information performed better when POS labels were supplied through constrained inference. The principal limitations were cubic dependence on sequence length, fixed hierarchy depth, recursive containment, segmentation assumptions, and memory consumption of approximately $d+1$9 (Truyen et al., 2010).

4. Multi-scale and domain-specific HHMMs

HHMMs have been applied to domains in which observations exhibit multiple temporal or semantic scales.

Animal movement

The multi-scale animal-movement model uses internal states to select alternative production-level HMMs. In harbor-porpoise data, hourly internal states governed dive-level production processes. Three production states represented short, shallow, smooth dives; moderately long and deep dives; and long, deep, highly wiggly dives. One internal regime was dominated by the first two production states, whereas another was dominated by the third and was interpreted as foraging. The construction allowed occasional fine-scale deviations inside persistent broad-scale behavioral regimes (Leos-Barajas et al., 2017).

For garter snakes, three production states represented motionless behavior, slow exploratory movement, and rapid escape movement. Three internal states represented different levels of behavioral flexibility: relatively persistent movement, frequent switching, and strong behavioral inflexibility. The model therefore separated the movement state occupied by an animal from the broader tendency to switch among movement states (Leos-Barajas et al., 2017).

Human activity and gesture recognition

Online human-activity recognition uses a root HHMM with sub-HMMs for activity beginning, ongoing recognition, and activity ending. The beginning detector uses the first three sensor events, the ongoing recognizer updates the activity posterior with each new event, and the ending detector identifies the termination pattern. The resulting activity pane is then corrected using duration and time-of-day densities. On two smart-home datasets, the HHMM achieved 65.20% and 60% accuracy, respectively, although several fixed-window baselines obtained higher F1 values in some conditions. The architecture supports ongoing recognition, variable-length segmentation, and interrupted activities (Asghari et al., 2019).

A vector autoregressive HHMM models gestures with an upper movement state $e_t^d\in\{0,1\}$0 and a lower segment state $e_t^d\in\{0,1\}$1. Each movement–segment pair emits a vector autoregressive signal,

$e_t^d\in\{0,1\}$2

The fitted model used $e_t^d\in\{0,1\}$3 and up to five segment states per gesture. In a movement-to-sound interface, the model used six-dimensional accelerometer and gyroscope observations sampled at 100 Hz. Relative to online KNN, VAR-HHMM produced higher recall for all but one movement, fewer misclassified blocks, and fewer delayed detections under Android-related latency constraints, while showing somewhat lower precision because of earlier movement activation (Marković et al., 2016).

Financial regime analysis

A two-level financial HHMM separates long-term market regimes from short-term return dynamics. A coarse state represents bullish, intermediate, or bearish conditions, while each coarse state selects a fine-scale HMM for daily returns. The coarse observation is a 30-trading-day average return, and fine-scale observations are daily log-returns. The resulting likelihood is a coarse forward recursion whose emission contribution is the likelihood of the corresponding fine-scale HMM.

For the DAX and S&P 500, three coarse states and two fine states were selected. In the DAX analysis, the bullish state had the highest marginal expected return and lowest marginal volatility, while the bearish state had the lowest expected return and highest volatility. The fine-scale process distinguished local return behavior within a persistent coarse regime, so short-lived negative returns did not necessarily induce an immediate transition to a bearish state (Oelschläger et al., 2020).

Weakly supervised semantic frame induction

A hierarchical HMM for adaptive command interfaces represents semantic slots at an upper layer and slot values at a lower layer. Non-emitting [START](https://www.emergentmind.com/topics/sub-task-aware-robotic-transformer-start) and [END](https://www.emergentmind.com/topics/eccentric-nuclear-disk-end) states factorize transitions between concrete slot values into shared slot-level structure. Filler states model command units that do not directly express a slot value. Expression sharing ties emissions for values that occur in different contextual slots, such as source and target suits.

Training uses Baum–Welch with frame-level constraints but no utterance-to-slot alignment. Transition sharing improved slot-order learning, while expression sharing reduced data requirements when contextual slots genuinely used common expressions. On PATCOR, top HMM/HHMM configurations achieved overall $e_t^d\in\{0,1\}$4 scores of 86.6% for phoneme unigrams, 86.2% for phoneme bigrams, 88.0% for word unigrams, and 73.0% for word bigrams. The degradation for word bigrams reflected sparsity and excessive contextual specificity (Loo et al., 2019).

Genomic multiple testing

In GWAS, an HHMM models both SNP association states and latent chromosome-region types. The lower state $e_t^d\in\{0,1\}$5 indicates whether SNP $e_t^d\in\{0,1\}$6 is associated; the upper state $e_t^d\in\{0,1\}$7 indicates the latent type of the surrounding region. Region types remain fixed within blocks of size $e_t^d\in\{0,1\}$8 and may change at block boundaries according to a transition matrix.

The hierarchical local index of significance is

$e_t^d\in\{0,1\}$9

Forward–backward inference combines local evidence about the SNP with posterior uncertainty about its region type. The oracle procedure orders SNPs by HLIS and rejects the largest initial set whose average HLIS is at most $e_t^d=0\implies e_t^{1:d-1}=0,$0. Under the stated HHMM assumptions, the oracle procedure controls FDR at level $e_t^d=0\implies e_t^{1:d-1}=0,$1 and has optimality properties for marginal false non-discovery rate among threshold procedures. The data-driven method estimates the HHMM by EM and substitutes the estimates into the HLIS calculation (Wang et al., 2022).

5. Bayesian, nonparametric, and ensemble hierarchies

Not every model called hierarchical HMM is a recursively nested HHMM. Several important approaches place the hierarchy in parameters, datasets, or component identities.

Hierarchically coupled HMMs

For single-molecule data, each trace $e_t^d=0\implies e_t^{1:d-1}=0,$2 has its own HMM parameters

$e_t^d=0\implies e_t^{1:d-1}=0,$3

while shared hyperparameters $e_t^d=0\implies e_t^{1:d-1}=0,$4 describe the ensemble distribution. The hierarchy is

$e_t^d=0\implies e_t^{1:d-1}=0,$5

Trace-specific means, precisions, transition matrices, and initial-state distributions vary across molecules but are coupled through Normal-Gamma and Dirichlet priors. This provides soft parameter sharing, probabilistic state correspondence, and regularization for short or noisy traces. Variational empirical Bayes alternates trace-level variational HMM inference with ensemble-level hyperparameter updates. The model estimates transition probabilities and posterior free-energy-like quantities, but it does not supply a general conversion from discrete-time transition matrices to continuous-time kinetic rates (Meent et al., 2013).

Marginalized inference for embedded HMMs

For hierarchical models containing many embedded discrete HMMs, MCMC can become inefficient because explicit latent-state sampling introduces as many as $e_t^d=0\implies e_t^{1:d-1}=0,$6 nuisance variables and creates strong state–parameter correlations. Discrete filtering integrates the latent states out of the likelihood exactly, leaving only top-level parameters in the MCMC state. The likelihood of one sequence is computed by the forward recursion with approximately $e_t^d=0\implies e_t^{1:d-1}=0,$7 operations.

The approach can also compress repeated observation histories. If $e_t^d=0\implies e_t^{1:d-1}=0,$8 sequences contain only $e_t^d=0\implies e_t^{1:d-1}=0,$9 distinct histories with multiplicities $e_t^d=1\implies e_t^{d+1:D}=1.$0, then

$e_t^d=1\implies e_t^{d+1:D}=1.$1

Dynamic block sampling addresses residual posterior correlations among top-level parameters. In capture–recapture examples, combining filtering, repeated-data reduction, and blocking produced reported improvements ranging from approximately 60-fold in the Dipper model to nearly 1,000-fold in the Goose model relative to latent-state MCMC (Turek et al., 2016).

Nonparametric segmentation

The segmented infinite HMM adds a binary segmentation variable $e_t^d=1\implies e_t^{d+1:D}=1.$2 to an iHMM. If $e_t^d=1\implies e_t^{d+1:D}=1.$3, the ordinary state-dependent transition $e_t^d=1\implies e_t^{d+1:D}=1.$4 governs the next state; if $e_t^d=1\implies e_t^{d+1:D}=1.$5, the next state is drawn from a global distribution $e_t^d=1\implies e_t^{d+1:D}=1.$6. Thus $e_t^d=1\implies e_t^{d+1:D}=1.$7 models structured within-segment dynamics while $e_t^d=1\implies e_t^{d+1:D}=1.$8 models transitions between segments.

The siHMM does not represent explicit high-level regime identities, high-level transition matrices, or regime-specific lower-level models. It is therefore a boundary-oriented approximation to an HHMM. Its SVI inference uses an augmented forward–backward algorithm with cost $e_t^d=1\implies e_t^{d+1:D}=1.$9 and memory $$\psi\rightarrow\theta_n\rightarrow z_n\rightarrow x_n$$0 under truncation. On synthetic and real segmentation tasks, it matched or exceeded a more elaborate sub-iHMM baseline in several settings, especially when components shared the same observed state space but differed in transition dynamics (Saeedi et al., 2016).

The Infinite Mixture Model of Markov Chains similarly simplifies HHMM structure by using an HDP-HMM over super-state identities and one Markov chain over categorical sub-states for each super state. A segment continuation variable and a boundary symbol represent entry and exit. The model uses truncated blocked Gibbs sampling. On Facebook navigation data, it achieved 61.41% next-action accuracy on all data, compared with 9.84% for a finite mixture of Markov chains and approximately 1% for global Markov models. Its components are directly interpretable as behavioral routines, but the model is not an arbitrary-depth recursively nested HHMM (Reubold et al., 2017).

6. Inference, identifiability, and limitations

HHMM inference commonly relies on dynamic programming. Forward–backward or Inside–Outside procedures compute marginal likelihoods and posterior state probabilities; Viterbi or max-product recursions compute the most probable hierarchical path. In block-structured models, inner forward recursions calculate fine-scale HMM likelihoods and an outer recursion propagates them through the coarse chain. In generative recursive models, termination and return-of-control variables must be incorporated into the recursion.

Numerical stability is important because hierarchical likelihoods multiply many probabilities. Log-space calculations, scaling procedures, and log-sum-exp operations are used to prevent underflow. Positive potentials in conditional models can grow exponentially with sequence length, requiring analogous stabilization.

Parameter estimation includes maximum likelihood, EM or Baum–Welch, direct numerical optimization, variational inference, variational empirical Bayes, stochastic variational inference, blocked Gibbs sampling, and MCMC. Direct likelihood optimization is often computationally attractive because latent paths can be marginalized by forward recursions. Bayesian methods provide posterior uncertainty and can address multimodality, but hierarchical posteriors may exhibit strong correlation, label switching, and local modes.

Parallel tempering targets tempered posteriors

$$\psi\rightarrow\theta_n\rightarrow z_n\rightarrow x_n$$1

where $$\psi\rightarrow\theta_n\rightarrow z_n\rightarrow x_n$$2. Hot chains flatten posterior barriers and can exchange configurations with the $$\psi\rightarrow\theta_n\rightarrow z_n\rightarrow x_n$$3 chain. For HHMMs, the method can improve exploration of multimodal and highly correlated posteriors, but its effectiveness depends on proposal scales, temperature spacing, swap frequency, and overlap between neighboring tempered distributions. In a harbor-porpoise HHMM, parallel tempering showed promise but only limited improvement over ordinary single-update Metropolis–Hastings, emphasizing that it is not a guarantee of convergence (Sacchi et al., 2020).

Several limitations recur across HHMM formulations:

1. Fixed or selected hierarchy: recursive HHMMs usually require a predefined depth, while multi-scale models require meaningful frame boundaries, state numbers, or block sizes.
2. Recursive containment: arbitrary cross-level dependencies that violate parent–child nesting cannot be represented directly.
3. Segmentation assumptions: processes without clear exit points must be approximated with long segments or analyst-defined frames.
4. Computational cost: state cardinalities, hierarchy depth, segment boundaries, parent–child connectivity, and transition structures can make exact inference expensive.
5. Identifiability: latent labels are arbitrary up to permutation. In multi-level models, internal states may also be weakly distinguished when they induce similar lower-level dynamics.
6. Model misspecification: Gaussian emissions, conditional independence, geometric durations, or finite-state assumptions may not capture long tails, asymmetry, drift, continuous behavior, or heterogeneous observation processes.
7. Optimization and sampling: EM and generalized EM can converge to local optima; MCMC can mix slowly because of posterior correlation and multimodality.
8. Interpretation: latent states acquire semantic labels only after examining their emissions, transitions, durations, or decoded sequences.

The central methodological trade-off is between expressiveness and tractability. Full recursive HHMMs can represent explicit high-level identities, state-dependent lower-level processes, termination, duration, and multiple levels of abstraction, but inference requires substantial bookkeeping. Restricted models such as the siHMM, multi-scale block HHMM, HSCRF, or IMMC sacrifice some hierarchy or dependency structure to retain polynomial-time or comparatively efficient inference. Across applications, HHMMs are most useful when the data contain genuine hierarchical organization—nested activities, coarse behavioral regimes, repeated routines, semantic slot structure, or heterogeneous genomic regions—and when the additional structure can be specified without exceeding the computational and identifiability limits of the model.