Toto 2.0: Time Series Forecasting Enters the Scaling Era

Abstract: We show that time series foundation models scale: a single training recipe produces reliable forecast-quality improvements from 4M to 2.5B parameters. We release Toto 2.0, a family of five open-weights forecasting models trained under this recipe. The Toto 2.0 family sets a new state of the art on three forecasting benchmarks: BOOM, our observability benchmark; GIFT-Eval, the standard general-purpose benchmark; and the recent contamination-resistant TIME benchmark. This report describes our experimental results and details the design decisions behind Toto 2.0: its architecture and training recipe, training data, and the u-muP hyperparameter transfer pipeline. All five base checkpoints are released under Apache 2.0.

• The paper introduces the first family of time series foundation models that exhibit reliable, monotonic scaling through a novel u-$ hyperparameter transfer mechanism.
• It leverages a redesigned transformer with Contiguous Patch Masking and quantile regression, boosting inference speed and long-horizon forecasting stability.
• The models achieve Pareto-optimal performance on BOOM, GIFT-Eval, and TIME benchmarks, significantly outperforming previous methods with lower latency.

Toto 2.0: Scaling Time Series Foundation Models

Introduction

Toto 2.0 establishes the first family of time series foundation models (TSFMs) that consistently improve in performance as scale increases, advancing TSFM research into the systematic scaling regime previously demonstrated in NLP and computer vision. Comprising a suite of five open-weight models ranging from 4M to 2.5B parameters, Toto 2.0 delivers state-of-the-art results across BOOM, GIFT-Eval, and TIME benchmarks, outpacing both its predecessor (Toto 1.0) and contemporary model families. The suite's training pipeline integrates a novel scaling law-inspired hyperparameter transfer procedure (u-$), large-scale exclusive reliance on internal observability and synthetic data for pretraining, and a redesigned transformer architecture tailored for high-throughput, robust multi-horizon probabilistic forecasting.

Figure 1: CRPS rank versus parameter count on BOOM and GIFT-Eval; Toto 2.0 shows monotonic scaling and Pareto-optimal performance across all model sizes.

Architecture and Training Protocol

Toto 2.0 builds on a patched, decoder-only transformer with alternating causal (time-axis) and full (variate-axis) attention, introducing several pivotal modifications to enhance scaling and forecast efficiency. The model transitions from autoregressive decoding to Contiguous Patch Masking (CPM), enabling parallel multi-step forecasting in a single forward pass and substantially improving inference speed and coherence over long horizons.

A quantile regression head, predicting nine quantiles per target, supplants the numerically fragile Student-T mixture used in Toto 1.0, and training is now governed by the pinball loss, aligning with best practices in recent TSFM literature for stability and calibration at scale. For optimization, Toto 2.0 employs NorMuon, a variant of Muon optimized for sign-valued losses, with per-neuron normalization to adapt to the limited step-size information available under the pinball loss.

Further refinements include a reduced patch size (32), robust input normalization using the $\operatorname{arcsinh}$ transform to accommodate large dynamic ranges in observability metrics, nonlinear patch projections via residual MLPs, and adoption of learned per-dimension attention scaling. No dropout is used, and bias terms are enabled on attention projections only.

Figure 2: The architecture and training/inference protocol of Toto 2.0 featuring CPM, improved normalization, and quantile output head.

Data Pipeline and Composition

The training corpus for Toto 2.0 comprises 5.04T points (for 313M, 1B, 2.5B models) exclusively sourced from Datadog observability data and synthetic series generated using the TempoPFN framework. Public datasets are explicitly excluded from pretraining, in contrast to other leading TSFMs—public data features only at the finetuning stage for specific model variants.

A notable adjustment in data curation reweights sampling intervals, shifting focus from overwhelmingly high-frequency signals (10s) to a more balanced inclusion of longer intervals (5m+), increasing their share in the mix from 5% to 35%. Synthetic data is also increased to cover 57.5% of the total, contributing regime diversity and long-range structure.

Figure 3: Comparison of data composition in Toto 1.0 and Toto 2.0, with near-doubling of internal metrics and a substantial uptick in synthetic data; interval balance is also improved.

Hyperparameter Transfer: Unit-Scaled Parametrization

A distinct advancement in Toto 2.0 is the application of the u-$$(unit-scaled maximal update parametrization) paradigm—previously restricted to NLP and vision—enabling width-invariant transfer of optimizer hyperparameters. Hyperparameters are tuned comprehensively on a 10M-parameter proxy under u-$$ and passed to all five deployed scales without substantial performance loss, eliminating the need for repeated large-scale hyperparameter sweeps.

Figure 4: Illustration of u-$-based hyperparameter transfer yielding width-independence of optima, verified by proxy-to-target transfer.

Empirical Results and Scaling Behavior

BOOM Benchmark

Toto 2.0 demonstrates strong scaling: every model size sits on or near the Pareto frontier for CRPS and MASE on BOOM, with every new scale yielding improved forecasting error. Notably, the 22M parameter model matches or exceeds Toto 1.0's performance while being approximately 7× smaller, and the 4M is competitive despite massive parameter reductions. All Toto 2.0 models outperform competing foundation models at comparable sizes.

Figure 5: Across BOOM, all Toto 2.0 models achieve lower CRPS and MASE than external competitors; even sub-25M models surpass the original Toto 1.0.

GIFT-Eval Benchmark

On GIFT-Eval, the main public cross-domain TSFM benchmark, Toto 2.0's three largest base models are the highest-performing foundation models by CRPS rank. When finetuned and ensembled (FnF), Toto 2.0 further outpaces all submissions—including those of agentic and highly adapted systems—exhibiting robust transfer and adaptability for practical deployment.

Figure 6: Toto 2.0 family ranks top three CRPS and achieves Pareto-optimal results on MASE rank; finetuned and ensembled variants dominate all leaderboard categories.

TIME Benchmark

TIME, designed to mitigate data contamination and assess true generalization, is another domain Toto 2.0 dominates: the 313M, 1B, and 2.5B models sweep the top metrics, again validating monotonic scaling. Although some rank-metric inversions appear between 313M and 1B, every Toto 2.0 size outperforms Toto 1.0 and all external foundation models, regardless of parameter count.

Figure 7: On TIME, the top three models by every major metric are Toto 2.0 sizes, outpacing Chronos-2 and PatchTST-FM.

Latency and Long-Horizon Stability

The CPM-based parallel decoding design yields orders-of-magnitude improvements in inference latency, particularly for long-horizon forecasts. Toto 2.0 models are significantly faster than Toto 1.0 in all configurations, and maintain efficiency well beyond the training horizon—single-pass mode is stable up to ~768 steps, while block decoding provides further reach with limited degradation.

Long-horizon synthetic signal forecasting further demonstrates that model scale correlates with stability and fidelity: only the largest Toto 2.0 variants produce coherent extrapolations at 8,192-step horizons, whereas prior foundation models fail to preserve structure beyond shorter contexts.

(Figure 8)

Figure 8: Inference latency: Toto 2.0 achieves much lower forward pass times both at fixed horizon and when scaling horizon length—single-pass and block decoding settings compared to prior art.

(Figure 9)

Figure 9: Long-horizon forecasts on synthetic signals reveal stability and multi-scale fidelity scaling with Toto 2.0 size, in contrast to baseline collapse in prior models.

Implications and Future Directions

The demonstration of reliable, monotonic scaling in TSFMs with Toto 2.0 repositions the research focus from foundational scaling questions to advanced challenges in model data curation, long-horizon generalization, and practical application in real-world telemetry. The efficacy of unit-scaled hyperparameter transfer for width invariance is likely of broader methodological value, and the quantile-output paradigm positions Toto 2.0 as a reference implementation for probabilistic time series modeling.

Open research areas highlighted include:

• Closing the Gap with Classical Methods: While Toto 2.0 outperforms statistical baselines on benchmark error metrics, explicit long-horizon extrapolation and uncertainty growth remain challenging, suggesting need for further architectural or objective innovations.
• Principled Data Curation: Post-scaling, research emphasis will increasingly shift toward systematic, curriculum-based, or filtering-aware data pipeline design.
• Metrics as a Unique Modality: Further work is anticipated to exploit the specific structure and heterogeneity of high-frequency observability time series beyond generic time series strategies.
• Multimodal World Models: Integration of metrics, logs, traces, and structured/unstructured telemetry towards holistic observability modeling and root-cause analysis represents a future strategic direction for both academia and industry.

Conclusion

Toto 2.0 sets a new technical bar for TSFM research, providing an open-weight, robustly scaling, and production-grade forecasting family. Its architecture and training innovations, systematic scaling protocol, and leading empirical performance across demanding benchmarks reposition the field to focus on data, long-horizon reasoning, and multimodal integration. The paradigm of predictable scaling in time series forecasting, long established in NLP, is now firmly realized.

Citation: "Toto 2.0: Time Series Forecasting Enters the Scaling Era" (2605.20119).