Pretrained Time-Series Foundation Models

• Pretrained TSFMs are large Transformer-based models trained on diverse time series data using self-supervised forecasting to capture complex temporal dynamics.
• They incorporate architectural innovations such as mixture-of-experts backbones and dynamic patch tokenizations to efficiently model nonlinear dependencies and regime shifts.
• TSFMs enable robust zero-shot and fine-tuned adaptation for tasks including forecasting, classification, anomaly detection, and reasoning across domains like energy, finance, and healthcare.

Pretrained Time-Series Foundation Models (TSFMs) are a class of large neural network models, typically based on Transformer or closely related architectures, trained on massive and diverse corpora of time series data via self-supervised forecasting objectives. These models are designed to capture broad temporal patterns, nonstationary trends, and domain-generalizable structures, thereby enabling zero-shot or data-efficient adaptation to a wide array of downstream forecasting, classification, reasoning, and anomaly detection tasks.

1. Foundations and Pretraining Paradigms

TSFMs are constructed on the foundation model paradigm, as formalized in (Meyer et al., 15 Oct 2025), wherein a single model is pretrained at scale over numerous domains and temporal regimes. Standard architectures include deep encoder–decoder or decoder-only Transformers, often with hundreds of millions to billions of parameters, enabling them to represent nonlinear dependencies, cross-variate effects, and structural breaks encountered in energy, finance, healthcare, and other time series domains (Li et al., 19 Jan 2026).

Pretraining is dominated by supervised forecasting objectives: given a multivariate look-back window $\mathbf{x}_{1:L,d}$, the TSFM predicts a future horizon $\mathbf{y}_{L+1:L+T,d}$. The primary loss is typically mean squared error (MSE) or a quantile-based pinball loss, applied over all forecast steps and possibly quantile levels, with formulaic uniform averaging:

$$\mathcal{L}_\text{pretrain} = \frac{1}{B\,D\,T} \sum_{b=1}^{B}\sum_{d=1}^{D}\sum_{t=1}^{T} \left(y_{b,L+t,d}-\hat{y}_{b,L+t,d}\right)^2$$

Though masking, contrastive, and frequency-based objectives are sometimes included, nearly all state-of-the-art TSFMs employ this forecasting-centered regime (Li et al., 19 Jan 2026, Simeone, 11 Feb 2026, Marconi, 9 Jul 2025).

2. Architectural Innovations and Model Families

Several prominent variants of TSFMs exist:

• Mixture-of-Experts (MoE) Backbones: Models like TimeMoE and Moirai employ sparse expert routing to enhance modeling capacity for long-horizon and multiscale phenomena (Li et al., 19 Jan 2026, Yu et al., 8 Dec 2025).
• Custom Patch Tokenizations: Kairos (Feng et al., 30 Sep 2025) advances tokenization with dynamic patching (Mixture-of-Size Dynamic Patching, MoS-DP) and instance-adaptive positional embeddings (IARoPE), enabling finer adaptation to varying signal density and structural periodicity.
• MLP-Mixer Alternatives: TinyTimeMixer eschews attention, relying purely on MLP blocks for local and global feature integration (Simeone, 11 Feb 2026).
• Classification-Robust Variants: KairosHope (Balderas et al., 18 May 2026) replaces quadratic attention with a dual-memory HOPE block and hybrid decision head for classification tasks.
• Hybrid and Cross-Modal Extensions: CoRA (Qin et al., 14 Oct 2025) leverages frozen TSFMs with zero-initialized covariate-injection adapters for structured integration of exogenous, multi-modal (time series, text, image) covariates.

A sample summary of benchmarked model families is:

Model	Core Architecture	Backbone Params	Pretraining Objective
Chronos-2	Transformer (Encoder)	120M	Direct quantile forecasting
TimeMoE	MoE Transformer (Decoder)	up to 1B+	MSE/Huber
Kairos	Transformer + MoS-DP	50M	Weighted quantile loss
TinyTimeMixer	MLP-Mixer	<1M	MSE, no native quantiles
KairosHope	HOPE dual-memory/MLP-head	8M+	MTSM + InfoNCE (class.)

3. Zero-Shot, Fine-Tuned, and Adaptation Regimes

TSFMs are deployed primarily in "zero-shot" settings—pretrained weights are frozen, and raw time series context is sufficient for out-of-the-box forecasting without per-series or per-domain retraining. This contrasts with classical local models which require on-the-fly fitting and are sensitive to context window size and event novelty (Simeone, 11 Feb 2026, Marconi, 9 Jul 2025, Yu et al., 8 Dec 2025).

Recent work extends TSFMs to support:

• Parameter-efficient adaptation: LoRA, OFT, HRA adapters allow model performance matching or exceeding full fine-tuning with only $<$2% of weights updated, reducing computational and memory costs (Park et al., 1 Jan 2026).
• Plug-in memory modules: TS-Memory (Lyu et al., 12 Feb 2026) distills nonparametric retrieval signals (e.g., kNN quantile corrections) into offline-learned, parametric adapters, enabling constant-time inference and robust adaptation under distribution shift.
• In-context learning (ICL): In-context Time-series Pre-training (ICTP) (Xu et al., 23 Feb 2026) restructures pretraining data into demonstration–query pairs, enabling dynamic task adaptation (e.g., imputation, backtracing) at inference without fine-tuning, yielding $\sim$11% improvement on unseen tasks.

4. Distillation, Arbitration, and Model Pooling

The large parameter footprint and inference latency of TSFMs pose practical barriers to real-time, edge, or low-resource deployment. Knowledge distillation methods, notably DistilTS (Li et al., 19 Jan 2026), address this by:

• Introducing horizon-weighted objectives to counteract gradient domination by short-range forecasts, exponentially upweighting long-horizon errors.
• Employing factorized temporal alignment (FTA) to resolve mismatches between point-wise teacher states and compact, variate-wise student embeddings.
• Achieving up to $1/150$ parameter reduction and $6000\times$ speedup compared to full-sized TSFMs while maintaining competitive or superior forecasting accuracy.

Model arbitration and expert pooling approaches (TimeRouter (Ning et al., 10 Jun 2026), Synapse (Das et al., 7 Nov 2025)) exploit the heterogeneity across frozen TSFM experts, routing test queries by discriminative meta-features or adaptively weighting expert outputs via context-sensitive calibration (e.g., CRPS-based forward simulation and predictive sampling). This yields performance superior to static ensembling or single-model baselines, especially on high-lumpiness, regime-shifting datasets.

Framework	Routing Principle	Adaptivity	Inference Overhead
TimeRouter	Discriminative + gating	Per-query, selective	$\sim$10ms/series
Synapse	Dynamic arbitration	Per-timestamp, continuous	context-dependent
DistilTS	Knowledge distillation	Static student	$<$ original model

5. Robustness, Redundancy, and Benchmarking

Systematic evaluations highlight both the promise and challenges of scaling TSFMs:

• Adversarial Robustness: TSFMs can be highly vulnerable to minimal, structured input perturbations causing forecast reversals or amplitude shifts even under modest adversarial budgets (e.g., RED$_\text{NMAE}>50\times$ increases in error) (2505.19397). Mixture-of-experts architectures and multi-task pretraining offer increments in resistance.
• Universal Redundancy: Empirical ablation and stable-rank analysis reveal that up to 40% of intermediate layers and $\mathbf{y}_{L+1:L+T,d}$028% of heads in contemporary TSFMs can be removed with negligible penalty in forecast metrics, supporting model compression and deployment efficiency (Bao et al., 2 Feb 2026).
• Benchmark Integrity: The emergence of large pretraining corpora led to widespread test-set leakage and overstated zero-shot generalization claims if benchmark splits overlap with pretraining (Meyer et al., 15 Oct 2025). Robust evaluation protocols now require transparent, non-leaking, rolling, and domain-held-out splits, and a move toward "live" prospective evaluations that accrue performance on future, unseen data.

6. Specialized Applications and Domain Transfer

Application domains now encompass:

• Industry and Energy: TSFMs deliver grid-level load forecasting, process model prediction, and remaining useful life (RUL) estimation, consistently outperforming both classical baselines and from-scratch deep learners—even with limited supervision or context (Simeone, 11 Feb 2026, El-Ghoussani et al., 10 Jun 2026, Yu et al., 8 Dec 2025).
• Finance: While TSFMs, especially in zero-shot and parameter-efficient adaptation modes, exhibit gains in low-signal and low-data regimes, outright dominance over domain-specific econometric models is rare in high-SNR tasks; domain-specific pretraining and architectural tuning remain critical (Marconi, 9 Jul 2025, Alonso et al., 25 Jun 2026, Rahimikia et al., 23 Nov 2025).
• Classification and Reasoning: Extensions to time series classification, process monitoring, and even synergistic LLM–TSFM reasoning (e.g., TS-Reasoner (Yu et al., 3 Oct 2025)) show that sophisticated pretraining and aligned multi-modal training protocols significantly benefit domain-specific and analytic tasks.

7. Practical Implications and Future Directions

TSFMs set a new standard for universal, data-efficient time series modeling across domains, underpinned by large-scale pretraining, compact distillation, and plug-and-play adaptation capabilities. Recommendations for practitioners include:

• Pretrain or select TSFMs on corpora matched to the target domain when possible, employing adaptation modules like CoRA or TS-Memory for covariate-rich or distribution-shifting settings (Qin et al., 14 Oct 2025, Lyu et al., 12 Feb 2026).
• Utilize horizon-weighted and alignment-aware distillation for edge and resource-constrained deployments.
• Monitor for redundancy and adversarial risk, adopting structurally sparse backbones and regularizing multi-task objectives for robustness and efficiency.
• Benchmark under robust, non-leaking protocols, with attention to real-world context, rolling origins, and out-of-domain generalization (Meyer et al., 15 Oct 2025).

Continued progress in self-supervised objectives, adaptive architecture design, efficient adaptation modules, and rigorous evaluation will define the trajectory of TSFMs as the de facto infrastructure for temporal data analysis in both academic and industrial regimes.