Theoretical Framework for Online Time-Series Forecasting

• TOT is a unified theoretical framework for online time-series forecasting that formalizes sequential prediction under distribution shifts by conditioning on latent causal variables.
• It characterizes fundamental forecasting accuracy bounds and model identifiability while enabling plug-in modular algorithmic instantiations across diverse predictors.
• Empirical evaluations demonstrate significant MSE reductions and high latent estimation correlations, validating TOT's theoretical and practical benefits in dynamic regimes.

A Theoretical Framework for Online Time-Series Forecasting (TOT) formalizes and unifies a broad class of learning procedures for sequential prediction under distribution shift and nonstationarity, centering on the explicit identification and conditioning on latent state variables within causal generative models. The TOT framework characterizes both fundamental bounds for achievable forecasting accuracy and model identifiability, and prescribes modular algorithmic instantiations that can be incorporated as plug-ins across diverse neural and statistical backbones. TOT thus serves as a foundation for algorithm construction, comparison, and analysis for streaming time-series prediction in adversarial, stochastic, and hybrid regimes.

1. Formal Problem Setup and Data-Generating Model

TOT models a multivariate time series $\{x_t\}_{t \ge 1},\ x_t \in \mathbb{R}^n$, as generated by dynamic nonlinear mappings with unobserved latent variables that induce nonstationary and shifting distributions. The structural model is: $$\begin{aligned} x_t &= g(x_{t-1}, z_t, \epsilon_t),\quad \epsilon_t \sim p_\epsilon, \ z_{t,i} &= f_i(\operatorname{Pa}_d(z_{t,i}), \operatorname{Pa}_e(z_{t,i}), \xi_{t,i}),\quad \xi_{t,i}\sim p_{\xi_i}, \end{aligned}$$ where $$g:\mathbb{R}^{2n}\times\mathbb{R}^n \to \mathbb{R}^n$$ is a general, possibly non-invertible mixing function, and each $z_{t,i}$ evolves according to potentially sparse time-delayed and contemporaneous parental dependencies.

At each present time $t$, the forecaster observes a lagged window $\{x_{t-\tau}, \ldots, x_t\}$ and seeks to produce a forecast $\hat{x}_{t+1}$ to minimize mean-squared error, conditioned on all available information summarized as the filtration $\mathcal{F}_t$.

2. Theoretical Results: Bayes Risk Reduction and Identifiability

TOT establishes that explicit conditioning on latent variables provably reduces the Bayes risk of forecasting. Key findings include:

• Risk Tightening: For $\mathcal{F}_t^{(x)}=\sigma(x_{t-\tau:t})$, $\mathcal{F}_t^{(xz)} = \sigma(x_{t-\tau:t}, z_t)$, and $$\begin{aligned} x_t &= g(x_{t-1}, z_t, \epsilon_t),\quad \epsilon_t \sim p_\epsilon, \ z_{t,i} &= f_i(\operatorname{Pa}_d(z_{t,i}), \operatorname{Pa}_e(z_{t,i}), \xi_{t,i}),\quad \xi_{t,i}\sim p_{\xi_i}, \end{aligned}$$0 (with $$\begin{aligned} x_t &= g(x_{t-1}, z_t, \epsilon_t),\quad \epsilon_t \sim p_\epsilon, \ z_{t,i} &= f_i(\operatorname{Pa}_d(z_{t,i}), \operatorname{Pa}_e(z_{t,i}), \xi_{t,i}),\quad \xi_{t,i}\sim p_{\xi_i}, \end{aligned}$$1 an estimator), the risks satisfy $$\begin{aligned} x_t &= g(x_{t-1}, z_t, \epsilon_t),\quad \epsilon_t \sim p_\epsilon, \ z_{t,i} &= f_i(\operatorname{Pa}_d(z_{t,i}), \operatorname{Pa}_e(z_{t,i}), \xi_{t,i}),\quad \xi_{t,i}\sim p_{\xi_i}, \end{aligned}$$2, with equality $$\begin{aligned} x_t &= g(x_{t-1}, z_t, \epsilon_t),\quad \epsilon_t \sim p_\epsilon, \ z_{t,i} &= f_i(\operatorname{Pa}_d(z_{t,i}), \operatorname{Pa}_e(z_{t,i}), \xi_{t,i}),\quad \xi_{t,i}\sim p_{\xi_i}, \end{aligned}$$3 if $$\begin{aligned} x_t &= g(x_{t-1}, z_t, \epsilon_t),\quad \epsilon_t \sim p_\epsilon, \ z_{t,i} &= f_i(\operatorname{Pa}_d(z_{t,i}), \operatorname{Pa}_e(z_{t,i}), \xi_{t,i}),\quad \xi_{t,i}\sim p_{\xi_i}, \end{aligned}$$4 is functionally invertible of $$\begin{aligned} x_t &= g(x_{t-1}, z_t, \epsilon_t),\quad \epsilon_t \sim p_\epsilon, \ z_{t,i} &= f_i(\operatorname{Pa}_d(z_{t,i}), \operatorname{Pa}_e(z_{t,i}), \xi_{t,i}),\quad \xi_{t,i}\sim p_{\xi_i}, \end{aligned}$$5 (Li et al., 21 Oct 2025).
• Blockwise Identifiability: If a model matches the four-fold marginal joint density of $$\begin{aligned} x_t &= g(x_{t-1}, z_t, \epsilon_t),\quad \epsilon_t \sim p_\epsilon, \ z_{t,i} &= f_i(\operatorname{Pa}_d(z_{t,i}), \operatorname{Pa}_e(z_{t,i}), \xi_{t,i}),\quad \xi_{t,i}\sim p_{\xi_i}, \end{aligned}$$6, then, under mild smoothness, blockwise identifiability holds: $$\begin{aligned} x_t &= g(x_{t-1}, z_t, \epsilon_t),\quad \epsilon_t \sim p_\epsilon, \ z_{t,i} &= f_i(\operatorname{Pa}_d(z_{t,i}), \operatorname{Pa}_e(z_{t,i}), \xi_{t,i}),\quad \xi_{t,i}\sim p_{\xi_i}, \end{aligned}$$7 for some diffeomorphism $$\begin{aligned} x_t &= g(x_{t-1}, z_t, \epsilon_t),\quad \epsilon_t \sim p_\epsilon, \ z_{t,i} &= f_i(\operatorname{Pa}_d(z_{t,i}), \operatorname{Pa}_e(z_{t,i}), \xi_{t,i}),\quad \xi_{t,i}\sim p_{\xi_i}, \end{aligned}$$8.
• Componentwise Identifiability under Sparsity: If $$\begin{aligned} x_t &= g(x_{t-1}, z_t, \epsilon_t),\quad \epsilon_t \sim p_\epsilon, \ z_{t,i} &= f_i(\operatorname{Pa}_d(z_{t,i}), \operatorname{Pa}_e(z_{t,i}), \xi_{t,i}),\quad \xi_{t,i}\sim p_{\xi_i}, \end{aligned}$$9 has sparse instantaneous connections, each $$g:\mathbb{R}^{2n}\times\mathbb{R}^n \to \mathbb{R}^n$$0 is identified (up to permutation) from four consecutive observations, enforceable via sparsity-penalized models.

These results provide theoretical underpinnings for plug-in latent estimation methods, ensuring that improved Bayesian lower bounds on error and practical recoverability of latent structure are attainable for a broad class of time-series models (Li et al., 21 Oct 2025).

3. Algorithmic Blueprint and Model Architecture

TOT operationalizes the theoretical guarantees with a modular, model-agnostic pipeline that can be incorporated around any encoder–forecaster backbone:

• Variational Encoder-Decoder: A variational autoencoder (VAE) over the observation history encodes approximate posteriors $$g:\mathbb{R}^{2n}\times\mathbb{R}^n \to \mathbb{R}^n$$1 and decodes with $$g:\mathbb{R}^{2n}\times\mathbb{R}^n \to \mathbb{R}^n$$2, maximizing the ELBO.
• Latent and Observation Noise Estimators: Two independent MLP-based networks estimate (i) latent transition noise $$g:\mathbb{R}^{2n}\times\mathbb{R}^n \to \mathbb{R}^n$$3 and (ii) observation mixing noise $$g:\mathbb{R}^{2n}\times\mathbb{R}^n \to \mathbb{R}^n$$4 to enforce independent-noise structure through change-of-variables log-likelihood regularization.
• Forecasting Module: A residual forecaster $$g:\mathbb{R}^{2n}\times\mathbb{R}^n \to \mathbb{R}^n$$5 maps estimated latent trajectories $$g:\mathbb{R}^{2n}\times\mathbb{R}^n \to \mathbb{R}^n$$6 and a reduced $$g:\mathbb{R}^{2n}\times\mathbb{R}^n \to \mathbb{R}^n$$7-history embedding $$g:\mathbb{R}^{2n}\times\mathbb{R}^n \to \mathbb{R}^n$$8 to multi-step predictions.
• Sparsity Penalty: An $$g:\mathbb{R}^{2n}\times\mathbb{R}^n \to \mathbb{R}^n$$9 loss on the Jacobian of the observation wrt latent mapping enforces componentwise identifiability.
• Total Loss: The sum $z_{t,i}$0 balances reconstruction, regularization, and identifiability (Li et al., 21 Oct 2025).

A table summarizes the principal modules:

Module	Function	Loss/Regularizer
VAE (Encoder/Decoder)	Posterior/likelihood estimation	$z_{t,i}$1
Noise Estimator (MLPs)	Inverse-dynamics for $z_{t,i}$2 and $z_{t,i}$3	$z_{t,i}$4
Sparsity Penalty	Enforce one-to-one mapping of latents	$z_{t,i}$5
Residual Forecaster	Prediction from $z_{t,i}$6, $z_{t,i}$7	Main prediction loss

This architecture can be seamlessly integrated with existing backbone predictors, enabling practical realization of the theoretical latent conditioning and identifiability results.

4. Empirical Evaluation and Performance

TOT has been benchmarked both on synthetic data and on real-world datasets, including ETTh2, ETTm1, WTH, ECL, Traffic, and Exchange (Li et al., 21 Oct 2025). Empirical findings include:

• On synthetic generative models, the mean correlation coefficient (MCC) between true and estimated latents reaches $z_{t,i}$8 for TOT, compared to $z_{t,i}$9 for IDOL and $t$0 for TDRL.
• Forecasting mean squared error (MSE) is significantly reduced using TOT-derived $t$1 as compared to baselines using only observed $t$2.
• On real data, incorporating TOT into five strong online or concept-drift backbones consistently reduces MSE/MAE by $t$3–$t$4\% relative to the backbone alone.
• Ablation studies confirm that omitting the sparsity penalty or noise prior terms degrades both identifiability and forecasting accuracy.

The benefits are robust to backbone choice and cannot be attributed to increased parameter count alone, supporting the theoretical claim that latent-augmented predictors have strictly lower Bayes risk under the modeling assumptions.

5. Comparison with Related Paradigms

TOT offers several distinct advantages in relation to classical and contemporary frameworks:

• Causal Generative Emphasis: In contrast to pure autoregressive or black-box deep models, TOT's reliance on latent-variable causal dynamics enables both risk improvement and identifiability, even under strong nonstationarity.
• Plug-and-Play Design: TOT does not prescribe a fixed network but rather acts as a modular augmentation, compatible with latent forecasting approaches such as LSTD for long-short-term disentanglement (Cai et al., 18 Feb 2025), proactive drift adaptation (Zhao et al., 2024), and explicit tensor subspace tracking (Luan et al., 2024).
• Theoretical Guarantees: Unlike adaptive methods or buffer-based continual learners, the correctness of TOT's risk reductions and identifiability can be established analytically under transparent structural assumptions.

Other frameworks such as SOCO (Wintenberger, 2021) focus primarily on regret under adversarial or stochastic settings, and do not leverage explicit latent identification or causal modeling. Variants such as conformal prediction (Sabashvili, 26 Jan 2026) address post-hoc uncertainty calibration but do not guarantee forecasting optimality via structural latents.

6. Extensions, Open Questions, and Future Directions

The TOT framework invites several lines of further exploration:

• Extension to Nonparametric and Infinite-Dimensional Latents: Current identifiability proofs rely on finite, component-wise latents and injectivity; relaxing these constraints is an active research direction.
• Real-Time and Robustness Enhancements: Online implementation poses challenges for stable training, hyperparameter tuning (e.g., regularization strengths, window lengths), and statistical robustness to rare/abrupt regime shifts.
• Hybridization with Distribution-Free Inference: Combining TOT's risk-optimality with conformal or distribution-free interval prediction may provide both tight point forecasts and valid predictive uncertainty.

Empirical results suggest that the framework's theoretical advantages translate directly to practical improvements in online time-series forecasting tasks, across application domains and backbone model classes.

7. Summary and Significance

TOT establishes a rigorous foundation for model-based online time-series forecasting in nonstationary environments, ensuring that the explicit modeling and estimation of latent variables both enhances predictability (sharpening Bayes risk) and supports learnable, provably identifiable, fast-converging algorithms. The construction is agnostic to backbone architecture, widely empirically validated, and theoretically well-motivated—marking it as a central reference framework for future developments in streaming predictive systems (Li et al., 21 Oct 2025).