HTPGM: Hierarchical Probabilistic Time Series Forecasting

• HTPGM is a probabilistic generative framework for hierarchical time series forecasting that enforces consistency across aggregated and disaggregated data.
• It leverages latent state dynamics, tailored emission distributions, and reconciliation techniques from Bayesian, deep variational, and hybrid models to capture nonstationarity and sparsity.
• Empirical results demonstrate that HTPGM architectures yield improved forecast accuracy and robust uncertainty quantification across diverse application domains.

A Hierarchical Time Series Probabilistic Generative Module (HTPGM) is a formalism and suite of modeling techniques for producing probabilistic forecasts of time series data with inherent hierarchical structure. This structure arises when individual time series can be grouped and related such that predictions at multiple levels of aggregation or disaggregation are needed, and the relationships (often defined as summations or constraints) must be enforced in the joint generative process. Modern HTPGM architectures encompass classical Bayesian models, deep neural and transformer-based variational models, and hybrid approaches, providing exactly or approximately coherent probabilistic forecasts, robust uncertainty quantification, and strong empirical performance across a variety of application domains.

1. Structural and Mathematical Foundations

HTPGMs impose a hierarchy on the collection of time series, most frequently represented via rooted trees or aggregation matrices. If $y_{i,t}$ is the value of node $i$ at time $t$ and $\mathcal{C}_i$ the set of its children, the hierarchy requires for all $i$ with children:

$$y_{i,t} = \sum_{j \in \mathcal{C}_i} \phi_{ij}\,y_{j,t}$$

with known coefficients $\phi_{ij}$ specifying the aggregation weights. The module's generative process typically incorporates:

• Latent State Dynamics: For each series, possibly AR or ARMA/ARIMA or factor model–driven, potentially sharing parameters across the hierarchy or via hierarchical priors.
• Emission Distributions: Distributional assumptions matched to the data regime (e.g., Negative Binomial, Poisson for count/sparse series, Gaussian for dense series).
• Hierarchical Consistency: Enforced either by design (top-down generative structure, conditional priors) or post hoc reconciliation/alignment of marginal and aggregate forecasts.
• Probabilistic Forecasts: The model outputs full predictive distributions or samples for each node and time horizon.

Modules range from classical state space models with explicit dynamic latent variables and hierarchical priors (Chapados, 2014), through deep sequence architectures utilizing conditional VAEs and Transformers (Wang et al., 2024, Tong et al., 2022), to top-down Dirichlet-proportion generative processes (Das et al., 2022) and RNNs with refinement and reconciliation blocks (Kamarthi et al., 2024).

2. Principal Model Variants and Generative Mechanisms

2.1 Hierarchical Bayesian State Space Models

These models, such as the H-NBSS, posit global hyperpriors on emission and transition/innovation parameters (e.g., AR coefficients, mean-reversion levels, overdispersion) and share information across series via the hierarchical prior structure. For count-valued data:

• Each series $\ell$ possesses latent state $\eta_{\ell,t}$, AR(1) evolution, covariate effects $\mathbf{x}_{\ell,t}$, and observation $y_{\ell,t}$ modeled as zero-inflated Negative Binomial:

$$y_{\ell,t} \sim z_\ell\,\delta_0 + (1-z_\ell)\mathrm{NB}\left(\mu = \exp(\eta_{\ell,t}+\mathbf{x}_{\ell,t}^\top\theta_\ell), \alpha_\ell \right)$$

Inference proceeds via joint Laplace approximation over latent states, leveraging sparsity of the GMRF precision structure (Chapados, 2014).

2.2 Deep Hierarchical Variational Models

Recent advances utilize deep stochastic generative processes with hierarchical latent variables, as in the Hierarchical Time Series Variational Transformer (HTV-Trans):

• Observed series $x_{1:T,n}$, multi-layer latent hierarchy $z_{t_l,n}^l$, with top-down dependencies and downsampled resolutions per layer.
• Generative chain: $$p(z^L)\,\prod_{l=L-1}^{1}p(z^l_{t_l}|z^{l+1}_{\lceil t_l/K_l\rceil},h)\,\prod_{t}p(x_{t}|z^1_{t})$$, where $h$ are deterministic features from the transformer encoder.
• The model captures multi-scale nonstationary structure, leveraging variational inference with reparameterization gradients and a structured ELBO objective (Wang et al., 2024).

2.3 Top-Down Dirichlet Proportions Generative Models

For hierarchical coherence by construction, the Dirichlet Proportions Model (DirProp) defines:

• At each forecast time, sample root series from a neural parameterized distribution (e.g., Negative Binomial).
• For each non-leaf parent, sample child proportions from Dirichlet, then disaggregate the parent’s value to the children via these proportions.
• The resulting model guarantees $\forall\,p$, $y_{t}^{(p)} = \sum_{c\in L(p)} y_{t}^{(c)}$, yielding strictly coherent probabilistic forecasts without post hoc reconciliation (Das et al., 2022).

2.4 Reconciliation via Bayesian Posterior Combination

For decoupled forecasting (independent models at different hierarchy levels or nodes), probabilistic reconciliation is performed at the forecast stage:

• Gaussian base- and aggregate-level predictive distributions, linked via the structural constraint $u = Ax$.
• Closed-form reconciling posterior:

$p(x|y) \propto p(x)\,\frac{\eta(Ax)}{p(Ax)}$

with $p(x)$ the product of forecasts, $\eta(u)$ the aggregate model, and reconciliation achieved via a minimum-trace or Bayesian update (Elgavish, 2023).

3. Training, Inference, and Algorithmic Details

3.1 Variational and Bayesian Inference

In deep models, amortized variational inference is leveraged using RNNs, CNNs, or MLP encoders to approximate $q(z|x)$, the latent posterior, matched in structure to the generative hierarchy. Key components include:

• Evidence Lower Bound (ELBO) objectives over all series; ELBO terms for data likelihood, forecast, reconstruction, plus multi-level KL divergences (Wang et al., 2024, Tong et al., 2022).
• Reparameterization gradients for stochastic sampling of latent variables.
• Ablation reveals importance of both hierarchical depth and inclusion of nonstationarity in prior parameterization (Wang et al., 2024).

In classical models and Bayesian reconciliation, inference typically involves:

• Laplace approximation for latent states and parameters in state-space settings, exploiting precision structure (Chapados, 2014).
• Gibbs/blockwise sampling (joint MCMC) when conjugacy or conditional updates are tractable (Hollyman et al., 2022).
• Closed-form updates for linear-Gaussian posteriors, as in modular forecast reconciliation (Elgavish, 2023).

3.2 Hierarchy-Aware Neural Architectures

RNN/GRU base blocks predict per-node parameters, optionally refined by hierarchy-aware mechanisms (e.g., weighted sums, mixing coefficients, or attention), and reinforced through penalties that align each parent’s predicted distribution with the sum-induced child distribution (e.g., via Jensen–Shannon divergence) (Kamarthi et al., 2024).

4. Hierarchical Reconciliation and Probabilistic Coherence

Enforcing hierarchical coherence is achieved via several mechanisms:

• By-construction (generative): DirProp’s top-down disaggregation and latent-variable splitting yield joint distributions guaranteeing aggregation constraints.
• Regularization: HAILS model imposes a differentiable loss (DCRS; sum of nodewise JSDs between parent and induced-child distributions) added to the main likelihood (Kamarthi et al., 2024).
• Post hoc Bayesian Reconciliation: Aggregated via closed-form correction of marginal forecasts to enforce the hierarchical (summation) constraints at every forecast horizon, using minimum-trace (MinT) or Bayesian updates (Elgavish, 2023, Hollyman et al., 2022).

5. Empirical Evaluation and Benchmarking

HTPGM-based architectures achieve or exceed state-of-the-art accuracy and calibration on benchmarks:

• On multivariate time series datasets (e.g., ETTh2, Weather), HTV-Trans attains minimum MAE/MSE for long horizons; for instance, ETTh2 $H$=96, MSE=0.300, compared to 0.346 (Fedformer) and 0.476 (NS-Transformer). Degradation occurs when critical hierarchical or nonstationary components are ablated (Wang et al., 2024).
• In industrial forecasting with massive hierarchies and sparseness (10,000+ nodes), HAILS improves forecast accuracy by 8.5% overall, with a 23% gain for sparse series. CRPS and WRMSSE are reduced by up to 40% and 12%, respectively, at the lower levels (Kamarthi et al., 2024).
• The “Hierarchies Everywhere” Bayesian factor model consistently delivers the lowest Energy Score and highest $R^2$ across aggregation layers, outperforming classical post hoc reconciliation (Hollyman et al., 2022).

6. Extensions, Scalability, and Future Directions

HTPGMs are extensible to:

• Deep architectures with decomposition of latent factors into interpretable trend/seasonality components and non-autoregressive (multi-step) decoding, enabling forecast interpretability (Tong et al., 2022).
• Hybrid regimes, such as mixed discrete/continuous emissions (Poisson for sparse/dense), neural-state augmentation (hierarchical refinement with learnable mixing), and dynamic adjustment of reconciliation penalties.
• Archetypal scalability via minibatch SGD, GPU-parallelized nodewise RNNs, and block-diagonal covariance estimation, yielding $O(N)$–$O(N^2)$ per-epoch costs at massive scale (Kamarthi et al., 2024).
• Nonlinear and non-Gaussian reconciliation via Monte Carlo or variational inference, embedding hierarchy constraints as soft factors in complex models (Elgavish, 2023).

Open challenges remain in optimal hierarchical structure specification, efficient approximate inference in high-dimensional regimes, and jointly modeling cross-series dependence beyond hierarchy-imposed structure.

7. Comparative Summary of Representative Approaches

Model/Variant	Reconciliation Mechanism	Emission Distribution(s)
H-NBSS (Chapados, 2014)	Hierarchical priors	Zero-inflated Negative Binomial
DirProp (Das et al., 2022)	By-construction (top-down)	Neural NB + Dirichlet Proportions
HTV-Trans (Wang et al., 2024)	Latent fusion + transformer	Gaussian (deep latent emission)
HAILS (Kamarthi et al., 2024)	JSD regularization (DCRS)	Poisson (sparse), Gaussian (dense)
Bayesian Reconciliation (Elgavish, 2023)	Post hoc, closed-form	Gaussian (extensions possible)
HE (Bayesian Factor) (Hollyman et al., 2022)	MCMC, in-model reconciliation	DLM + hierarchical factor

These variants collectively demonstrate the spectrum of HTPGM realizations, from fully generative hierarchical Bayesian models to deep, architecture-driven modules, unified by the core mandate: producing coherent, uncertainty-calibrated probabilistic forecasts across the full information hierarchy.