---
title: 'Moirai-2.0: Universal TS Forecasting Model'
url: https://www.emergentmind.com/topics/moirai-2-0
type: topic
---

# Moirai-2.0: Universal TS Forecasting Model

Moirai-2.0 is a decoder-only time-series foundation model (TSFM) developed for universal univariate forecasting. It implements quantile-based probabilistic output over a broad range of domains and is pretrained on a highly diverse collection of 36 million series amounting to approximately 295 billion observations. Moirai-2.0 adopts a single-patch input format, quantile regression (pinball loss), and an autoregressive multi-quantile decoding algorithm. Empirically, Moirai-2.0 achieves leading efficiency–accuracy trade-offs, best-in-class ranking among TSFMs for medium-horizon forecasting, and robust transfer as a predictive prior in data-sparse regimes. However, its absolute performance exhibits small and sparse gains over naive baselines in noisy settings such as financial return prediction, highlighting limits imposed by domain predictability itself [2511.11698][2606.27100].

## 1. Model Architecture and Training Protocol

Moirai-2.0 utilizes a pure decoder-only Transformer stack, dispensing with the masked-encoder and multi-patch input of its predecessor. The model treats a univariate input series as a contiguous sequence $x_{1:T}$ subjected to per-instance normalization using statistics computed from the first 30% of each sequence (to avoid label leakage). The normalized sequence is partitioned into non-overlapping patches, each embedding both values and a missing-value mask via a SiLU residual block:
\[
z_i = \mathrm{SiLU}(W\, [x_i \| m_i] + b) + [x_i \| m_i] \ , \qquad W\in\mathbb{R}^{2p_k \times d},\; m_i\in\{0,1\}^{p_k}
\]
where $p_k$ is the patch length and $d$ the embedding dimension.

The Transformer comprises multiple identical layers of causal multi-head self-attention and feedforward networks. All architectures employ residual connections and layer normalization. Typical variants include: 12/24/48 layers, model dimensions $d=768/1024/1536$, and 12/16/24 heads.

Patch tokens progress through the Transformer in causal order, outputting $n_q=9$ quantile forecasts (levels $0.1,0.2,\ldots,0.9$) for each future time step. No mixture-density components are present; all quantiles share a single projection head.

Training employs quantile regression via the pinball loss aggregated over horizon and quantile levels,
\[
L_Q = \frac{1}{H|Q|} \sum_{t=1}^H \sum_{\tau\in Q} \max\big(\tau (y_t-\hat y_t^{(\tau)}), (1-\tau)(\hat y_t^{(\tau)}-y_t)\big)
\]
accompanied by a patch-wise dropout rate of 50% to promote robustness. Optimization utilizes AdamW with a batch size of 256, learning rate $10^{-3}$, linear warm-up then cosine annealing, and mixed-precision (bf16) [2511.11698].

## 2. Pretraining Corpus and Data Handling

Moirai-2.0 is pretrained on approximately 36 million univariate time series totaling around 295 billion points. The corpus aggregates several sources:
- Gift-Eval benchmark and synthetic splits,
- Chronos-Mixup synthetic recombinations,
- KernelSynth Gaussian-process synthetic series,
- Internal Salesforce CloudOps telemetry.

Preprocessing for pretrain includes outlier/anomaly filtering via z-score, patch-level missing-value masking, and per-instance normalization as above. Notably, the pretraining set is not finance-specific and encompasses a broad domain range, giving Moirai-2.0 a highly generic inductive bias [2511.11698][2606.27100].

## 3. Forecasting Methodology: Quantile and Multi-Token Decoding

Forecasting uses a quantile-based objective: for each horizon step, the model predicts a full vector of quantile levels. Autoregressive multi-quantile decoding constructs full quantile trajectories via a recursive, beam-like expansion:

1. Initial context $y_{1:c}$ is provided.
2. For $t = c+1,\ldots,c+H$:
   - Expand: For each context, predict quantiles for the next patch.
   - Pool all candidate forecasts (beam expansion).
   - Collapse: For each quantile, select the corresponding empirical quantile from the expanded set.

This depth-2 beam approach allows the model to produce long-horizon, multi-token forecasts with reduced error accumulation at each stochastic episode [2511.11698][2606.27100].

## 4. Efficiency, Scaling, and Model Size

Moirai-2.0 exhibits marked efficiency improvements over predecessors. Moirai 2.0 Small (11.4M parameters) is roughly 30× smaller and 2× faster (≈200 ms per task) than Moirai 1.0-Large (360M, ≈400 ms) [2511.11698]. Speed–size–accuracy trade-off plots show that the Small and Base variants provide a sweet spot for most practical tasks:

| Model         | Params (M) | MASE | CRPS |
|:--------------|:----------|:-----|:-----|
| Moirai 2.0 S  | 11.4      | 0.728|0.516 |
| Moirai 2.0 B  | 87.1      | 0.732|0.525 |
| Moirai 2.0 L  | 305       | 0.743|0.530 |

Performance plateaus as parameter count increases, and larger models can degrade, suggesting a data–capacity mismatch at fixed corpus scale. Inference can be accelerated with K-V caching (up to 17× for long-horizon rollouts) [2511.11698].

## 5. Empirical Performance and Benchmarks

On Gift-Eval, Moirai-2.0 achieves normalized MASE 0.728 and normalized CRPS 0.516 in zero-shot configuration, ranking 5th and 6th among 30 reproducible pretrained models [2511.11698]. Domain-level analysis shows robust ranking in 6 of 7 categories, with visibility in electricity, financial, and retail settings. Performance degrades as forecast horizon lengthens, indicating current architectural limitations for long-range dependencies.

On financial return forecasting (AAPL, AMZN, GOOG, JPM, META), Moirai-2.0 obtains an average rank of 2.9 across 10 rolling-origin forecasting tasks—the best among TSFMs—and wins 3 tasks: both GOOG (linear and log returns) and AMZN (log returns) [2606.27100]. However, skill over the random-walk baseline is positive only for AMZN and GOOG; e.g., $\mathrm{Skill}(\mathrm{GOOG}) \approx 0.229$, and only the improvement on GOOG is statistically significant ($p = 0.042$, Diebold–Mariano test). The iTransformer model, trained from scratch, decisively outperforms on META, illustrating that Moirai-2.0 is not universally optimal for all assets.

## 6. Design Ablations and Theoretical Positioning

Ablation studies isolate key contributors to Moirai-2.0’s accuracy and efficiency. Introduction of quantile loss yields the largest gain, followed by recursive multi-quantile decoding and the transition to decoder-only architecture. Input patch-masking and multi-token prediction further improve robustness and inference speed.

Theoretically, pretraining on a massive corpus encodes a strong inductive prior, narrowing the parameter posterior for low–sample settings. PAC-Bayes framing suggests that Moirai-2.0’s pretrained weights $\pi_{\rm pre}(\theta)$ offer tight control of generalization error when local adaptation data is scarce. Information-theoretic analysis constrains maximum $R^2$ and skill achievable in low-predictability domains (e.g., daily equity returns with $I(r_{t+h};\mathcal{F}_t)\approx0.005$ nats, so $\max R^2 \lesssim 1\%$), bounding clinical impact regardless of relative model ranking [2606.27100].

## 7. Limitations and Research Directions

Moirai-2.0’s principal limitations include: degradation at very long horizons ($>1,000$ steps); inability to process multivariate or covariate-enriched series; corpus imbalance (weakness on “Nature” domain); and diminishing returns from additional model parameters. A plausible implication is that future advances may require not just architectural refinements (long-horizon encoding, hierarchical attention) but also corpus expansion and domain-specific data curation.

Priorities for extension involve dataset scaling (especially with environmental data), improved synthetic series generation for broadening long-horizon signal diversity, explicit treatments for covariates and multimodality (text/satellite), and integration with agentic downstream systems for automated risk management and planning [2511.11698].

---

**References:**  
[2511.11698] “Moirai 2.0: When Less Is More for Time Series Forecasting”  
[2606.27100] “Pretrained Time-Series Foundation Models for Financial Return Forecasting”

Source: https://www.emergentmind.com/topics/moirai-2-0