Moirai-2.0: Universal TS Forecasting Model

• Moirai-2.0 is a decoder-only time-series foundation model that uses a single-patch input format and quantile regression to achieve universal univariate forecasting.
• It leverages autoregressive multi-token decoding and a diverse pretraining corpus of 36 million series to deliver robust efficiency–accuracy trade-offs across domains.
• Empirical benchmarks show Moirai-2.0 excels in medium-horizon forecasting and transfer learning, though gains diminish in noisy, data-sparse settings.

Moirai-2.0 is a decoder-only time-series foundation model (TSFM) developed for universal univariate forecasting. It implements quantile-based probabilistic output over a broad range of domains and is pretrained on a highly diverse collection of 36 million series amounting to approximately 295 billion observations. Moirai-2.0 adopts a single-patch input format, quantile regression (pinball loss), and an autoregressive multi-quantile decoding algorithm. Empirically, Moirai-2.0 achieves leading efficiency–accuracy trade-offs, best-in-class ranking among TSFMs for medium-horizon forecasting, and robust transfer as a predictive prior in data-sparse regimes. However, its absolute performance exhibits small and sparse gains over naive baselines in noisy settings such as financial return prediction, highlighting limits imposed by domain predictability itself (Liu et al., 12 Nov 2025, Alonso et al., 25 Jun 2026).

1. Model Architecture and Training Protocol

Moirai-2.0 utilizes a pure decoder-only Transformer stack, dispensing with the masked-encoder and multi-patch input of its predecessor. The model treats a univariate input series as a contiguous sequence $x_{1:T}$ subjected to per-instance normalization using statistics computed from the first 30% of each sequence (to avoid label leakage). The normalized sequence is partitioned into non-overlapping patches, each embedding both values and a missing-value mask via a SiLU residual block: $$z_i = \mathrm{SiLU}(W\, [x_i \| m_i] + b) + [x_i \| m_i] \ , \qquad W\in\mathbb{R}^{2p_k \times d},\; m_i\in\{0,1\}^{p_k}$$ where $p_k$ is the patch length and $d$ the embedding dimension.

The Transformer comprises multiple identical layers of causal multi-head self-attention and feedforward networks. All architectures employ residual connections and layer normalization. Typical variants include: 12/24/48 layers, model dimensions $d=768/1024/1536$, and 12/16/24 heads.

Patch tokens progress through the Transformer in causal order, outputting $n_q=9$ quantile forecasts (levels $0.1,0.2,\ldots,0.9$) for each future time step. No mixture-density components are present; all quantiles share a single projection head.

Training employs quantile regression via the pinball loss aggregated over horizon and quantile levels,

$$L_Q = \frac{1}{H|Q|} \sum_{t=1}^H \sum_{\tau\in Q} \max\big(\tau (y_t-\hat y_t^{(\tau)}), (1-\tau)(\hat y_t^{(\tau)}-y_t)\big)$$

accompanied by a patch-wise dropout rate of 50% to promote robustness. Optimization utilizes AdamW with a batch size of 256, learning rate $10^{-3}$, linear warm-up then cosine annealing, and mixed-precision (bf16) (Liu et al., 12 Nov 2025).

2. Pretraining Corpus and Data Handling

Moirai-2.0 is pretrained on approximately 36 million univariate time series totaling around 295 billion points. The corpus aggregates several sources:

• Gift-Eval benchmark and synthetic splits,
• Chronos-Mixup synthetic recombinations,
• KernelSynth Gaussian-process synthetic series,
• Internal Salesforce CloudOps telemetry.

Preprocessing for pretrain includes outlier/anomaly filtering via z-score, patch-level missing-value masking, and per-instance normalization as above. Notably, the pretraining set is not finance-specific and encompasses a broad domain range, giving Moirai-2.0 a highly generic inductive bias (Liu et al., 12 Nov 2025, Alonso et al., 25 Jun 2026).

3. Forecasting Methodology: Quantile and Multi-Token Decoding

Forecasting uses a quantile-based objective: for each horizon step, the model predicts a full vector of quantile levels. Autoregressive multi-quantile decoding constructs full quantile trajectories via a recursive, beam-like expansion:

1. Initial context $y_{1:c}$ is provided.
2. For $$z_i = \mathrm{SiLU}(W\, [x_i \| m_i] + b) + [x_i \| m_i] \ , \qquad W\in\mathbb{R}^{2p_k \times d},\; m_i\in\{0,1\}^{p_k}$$0:
   • Expand: For each context, predict quantiles for the next patch.
   • Pool all candidate forecasts (beam expansion).
   • Collapse: For each quantile, select the corresponding empirical quantile from the expanded set.

This depth-2 beam approach allows the model to produce long-horizon, multi-token forecasts with reduced error accumulation at each stochastic episode (Liu et al., 12 Nov 2025, Alonso et al., 25 Jun 2026).

4. Efficiency, Scaling, and Model Size

Moirai-2.0 exhibits marked efficiency improvements over predecessors. Moirai 2.0 Small (11.4M parameters) is roughly 30× smaller and 2× faster (≈200 ms per task) than Moirai 1.0-Large (360M, ≈400 ms) (Liu et al., 12 Nov 2025). Speed–size–accuracy trade-off plots show that the Small and Base variants provide a sweet spot for most practical tasks:

Model	Params (M)	MASE	CRPS
Moirai 2.0 S	11.4	0.728	0.516
Moirai 2.0 B	87.1	0.732	0.525
Moirai 2.0 L	305	0.743	0.530

Performance plateaus as parameter count increases, and larger models can degrade, suggesting a data–capacity mismatch at fixed corpus scale. Inference can be accelerated with K-V caching (up to 17× for long-horizon rollouts) (Liu et al., 12 Nov 2025).

5. Empirical Performance and Benchmarks

On Gift-Eval, Moirai-2.0 achieves normalized MASE 0.728 and normalized CRPS 0.516 in zero-shot configuration, ranking 5th and 6th among 30 reproducible pretrained models (Liu et al., 12 Nov 2025). Domain-level analysis shows robust ranking in 6 of 7 categories, with visibility in electricity, financial, and retail settings. Performance degrades as forecast horizon lengthens, indicating current architectural limitations for long-range dependencies.

On financial return forecasting (AAPL, AMZN, GOOG, JPM, META), Moirai-2.0 obtains an average rank of 2.9 across 10 rolling-origin forecasting tasks—the best among TSFMs—and wins 3 tasks: both GOOG (linear and log returns) and AMZN (log returns) (Alonso et al., 25 Jun 2026). However, skill over the random-walk baseline is positive only for AMZN and GOOG; e.g., $$z_i = \mathrm{SiLU}(W\, [x_i \| m_i] + b) + [x_i \| m_i] \ , \qquad W\in\mathbb{R}^{2p_k \times d},\; m_i\in\{0,1\}^{p_k}$$1, and only the improvement on GOOG is statistically significant ($$z_i = \mathrm{SiLU}(W\, [x_i \| m_i] + b) + [x_i \| m_i] \ , \qquad W\in\mathbb{R}^{2p_k \times d},\; m_i\in\{0,1\}^{p_k}$$2, Diebold–Mariano test). The iTransformer model, trained from scratch, decisively outperforms on META, illustrating that Moirai-2.0 is not universally optimal for all assets.

6. Design Ablations and Theoretical Positioning

Ablation studies isolate key contributors to Moirai-2.0’s accuracy and efficiency. Introduction of quantile loss yields the largest gain, followed by recursive multi-quantile decoding and the transition to decoder-only architecture. Input patch-masking and multi-token prediction further improve robustness and inference speed.

Theoretically, pretraining on a massive corpus encodes a strong inductive prior, narrowing the parameter posterior for low–sample settings. PAC-Bayes framing suggests that Moirai-2.0’s pretrained weights $$z_i = \mathrm{SiLU}(W\, [x_i \| m_i] + b) + [x_i \| m_i] \ , \qquad W\in\mathbb{R}^{2p_k \times d},\; m_i\in\{0,1\}^{p_k}$$3 offer tight control of generalization error when local adaptation data is scarce. Information-theoretic analysis constrains maximum $$z_i = \mathrm{SiLU}(W\, [x_i \| m_i] + b) + [x_i \| m_i] \ , \qquad W\in\mathbb{R}^{2p_k \times d},\; m_i\in\{0,1\}^{p_k}$$4 and skill achievable in low-predictability domains (e.g., daily equity returns with $$z_i = \mathrm{SiLU}(W\, [x_i \| m_i] + b) + [x_i \| m_i] \ , \qquad W\in\mathbb{R}^{2p_k \times d},\; m_i\in\{0,1\}^{p_k}$$5 nats, so $$z_i = \mathrm{SiLU}(W\, [x_i \| m_i] + b) + [x_i \| m_i] \ , \qquad W\in\mathbb{R}^{2p_k \times d},\; m_i\in\{0,1\}^{p_k}$$6), bounding clinical impact regardless of relative model ranking (Alonso et al., 25 Jun 2026).

7. Limitations and Research Directions

Moirai-2.0’s principal limitations include: degradation at very long horizons ($$z_i = \mathrm{SiLU}(W\, [x_i \| m_i] + b) + [x_i \| m_i] \ , \qquad W\in\mathbb{R}^{2p_k \times d},\; m_i\in\{0,1\}^{p_k}$$7 steps); inability to process multivariate or covariate-enriched series; corpus imbalance (weakness on “Nature” domain); and diminishing returns from additional model parameters. A plausible implication is that future advances may require not just architectural refinements (long-horizon encoding, hierarchical attention) but also corpus expansion and domain-specific data curation.

Priorities for extension involve dataset scaling (especially with environmental data), improved synthetic series generation for broadening long-horizon signal diversity, explicit treatments for covariates and multimodality (text/satellite), and integration with agentic downstream systems for automated risk management and planning (Liu et al., 12 Nov 2025).

---

References:

(Liu et al., 12 Nov 2025) “Moirai 2.0: When Less Is More for Time Series Forecasting” (Alonso et al., 25 Jun 2026) “Pretrained Time-Series Foundation Models for Financial Return Forecasting”