TSGym: Component-Level Forecasting

• TSGym is a framework that decomposes deep forecasting into modular components, enabling precise evaluation of design choices such as normalization and tokenization.
• It organizes forecasting into preprocessing, encoding, architecture, and optimization stages, offering a comprehensive component-level analysis of multivariate time-series models.
• By leveraging meta-learning for zero-shot configuration selection, TSGym achieves competitive performance across diverse datasets including Weather, Traffic, and ILI.

Searching arXiv for TSGym and a few directly related MTSF component papers for accurate citations. TSGym is a framework for benchmarking, analyzing, and automatically constructing deep models for multivariate time-series forecasting (MTSF) at the component level rather than the whole-model level. Introduced in "TSGym: Design Choices for Deep Multivariate Time-Series Forecasting" (Liang et al., 21 Sep 2025), it treats modern forecasting systems as compositions of recurring design choices—such as normalization, decomposition, tokenization, channel handling, backbone type, attention mechanism, and optimization settings—rather than as indivisible named architectures. Its central claim is that forecasting performance depends less on allegiance to a single model family than on selecting an effective combination of components for the characteristics of a given dataset. On that basis, TSGym combines large-scale component-level benchmarking with a meta-learning system that performs zero-shot automated model construction for new datasets.

1. Problem setting and conceptual orientation

TSGym studies the standard MTSF setting in which a multivariate series with $C$ variables is observed over a historical window of length $L$, and the objective is to predict the next $T$ time steps. The work follows prior long-horizon forecasting practice by predicting all future steps directly rather than autoregressively, in order to avoid error accumulation when $T>1$. In the appendix, the future forecast tensors used in the metrics are written as $$\mathbf{X}, \widehat{\mathbf{X}} \in \mathbb{R}^{H \times C}$$, with $H$ future time points.

The paper’s motivation is methodological as much as empirical. Prior MTSF evaluation is described as too holistic: benchmarks typically compare complete end-to-end models, even though those models differ along multiple internal dimensions. This obscures which parts actually matter, when they help, and how they interact. TSGym therefore reframes the field’s central question from “which named model is best?” to “which component combination is best for this dataset?” That reframing is the basis both for its analytical program and for its automated construction procedure.

A major consequence of this position is that TSGym is not presented as conventional model selection, standard hyperparameter tuning, or narrow-family neural architecture search. It is instead a pipeline-level component-composition framework whose searchable space spans preprocessing, encoding, architecture, and optimization simultaneously.

2. Component-level decomposition of the forecasting pipeline

TSGym organizes deep forecasting as a four-stage pipeline:

$$\text{Series Preprocessing} \rightarrow \text{Series Encoding} \rightarrow \text{Network Architecture} \rightarrow \text{Network Optimization}$$

Across that pipeline it studies 16 design dimensions. The decomposition is intended to isolate the contribution of fine-grained choices such as whether a model uses channel independence, series patching, or a particular normalization scheme, instead of attributing performance gains to an entire architecture label.

Stage	Design dimensions	Candidate choices
Preprocessing	normalization; decomposition; series sampling/mixing	None, Stat, RevIN, DishTS; None, MA, MoEMA, DFT; False/True
Encoding	channel independent; sequence length; series embedding/tokenization	False/True; $48, 96, 192, 512$; Inverted Encoding, Positional Encoding, Series Patching
Architecture	network type; series attention; feature attention; $d_{\text{model}}$; $d_{\text{ff}}$; encoder layers	MLP, RNN, Transformer, LLM, TSFM; Null, SelfAttn, AutoCorr, SparseAttn, FrequencyAttn, DestationaryAttn; Null, SelfAttn, SparseAttn, FrequencyAttn; 64 or 256; 256 or 1024; 2 or 3
Optimization	epochs; loss function; learning rate; learning-rate strategy	10, 20, 50; MSE, MAE, HUBER; $L$0; Null, Type1

This search space includes components inspired by influential MTSF methods explicitly named in the paper, including series patching from PatchTST, channel-independent strategy from PatchTST, inverted encoding from iTransformer, multi-scale mixing from TimeMixer, multiple attention variants from the Transformer-for-time-series literature, and LLM and TSFM backbone choices. The appendix compares TSGym to ADGym and reports a design space size of about 7,962,624 combinations (Liang et al., 21 Sep 2025).

The significance of this decomposition is twofold. Scientifically, it enables component-level claims—for example, whether tokenization matters more than attention, or whether Transformers outperform MLPs only under certain preprocessing conditions. Practically, it allows TSGym to construct pipelines that need not coincide with any previously published off-the-shelf model.

3. Formalization and automated model construction

TSGym formalizes the design space through a set of design dimensions,

$L$1

where each $L$2 is a set of candidate design choices. The valid model space is the Cartesian product

$L$3

so that each composed model $L$4 is a tuple of one choice per dimension.

Because the full space is too large for exhaustive evaluation, TSGym samples a subset of valid combinations,

$L$5

Suppose there are $L$6 training datasets and $L$7 sampled pipelines. Evaluating them yields a performance matrix

$L$8

where $L$9 is the forecasting performance, using MSE, of model $T$0 on dataset $T$1. Since raw MSE values are not directly comparable across datasets, TSGym converts each row into a normalized rank,

$T$2

with smaller values indicating better relative performance.

Dataset representation is based on meta-features $T$3 extracted from the training split only. These include statistical, temporal, spectral, fractal, and distribution shift features; the appendix states that TSGym uses 1404-dimensional meta-features. Model configurations are represented by categorical design choices encoded with a LabelEncoder and then mapped through an nn.Embedding, producing learned continuous component embeddings.

The meta-predictor is a two-layer MLP that learns the mapping

$T$4

and more specifically

$T$5

At selection time on a new dataset, TSGym extracts meta-features from the training split, predicts ranks for candidate component combinations, and selects the top-ranked pipeline or pipelines. The paper characterizes this as zero-shot at selection time, because it does not require evaluating all candidates on the target test dataset.

The search process operates at two levels. Candidate configurations are initially sampled randomly from the valid space, and the paper later introduces Optuna to sample more intelligently through Bayesian optimization, using 50 random trials for cold start and then 50 guided trials. The appendix states that the meta-predictor is optimized with Pearson loss, with the explicit goal of learning ranking quality rather than absolute error.

4. Empirical findings from component-level analysis

A substantial part of TSGym’s contribution lies in its component-level empirical analysis rather than only in its final benchmark scores. The most consistent general finding is that series normalization matters a great deal. RevIN and Stat normalization are repeatedly among the best choices and are described as almost essential defaults for handling non-stationary dynamics. The appendix sharpens this into a conditional statement: normalization helps on high-drift datasets and may hurt when drift is negligible (Liang et al., 21 Sep 2025).

By contrast, decomposition is useful but clearly dataset-dependent. MA-based decomposition helps on some datasets such as ETTm1, while on others—notably ETTh1, ECL, ETTh2, Traffic—no decomposition can be better. The paper therefore rejects the view of decomposition as a universal add-on.

The analysis of channel independence is similarly conditional. Channel-independent strategies outperform channel-dependent ones on most datasets, with Traffic and ILI as explicit exceptions. However, the appendix emphasizes that CI versus CD cannot be determined simply from inter-channel correlation; even highly correlated datasets may still prefer CI. This is one of the paper’s clearest arguments against simplistic architecture heuristics.

Tokenization emerges as one of the most consequential design dimensions. Both patch-wise encoding / series patching and inverted encoding outperform traditional point-wise positional encoding across many settings. In the appendix, positional encoding is said to win only rarely, while patching and inverted encoding dominate. This supports the paper’s broader argument that some gains typically attributed to backbone innovations are better understood as gains in representation design.

The results on backbones are notably anti-dogmatic. Transformers are not universally superior: in the main text, Transformer outperforms MLP mainly on Weather, Traffic, and ILI, whereas simpler models remain competitive or better elsewhere. The appendix goes further, stating that Transformers are less robust than MLPs by IQR-based variability in many settings, and that the claim that Transformers have a higher upper bound than MLPs is not reliably supported. In that sense, TSGym directly challenges architecture tribalism in MTSF.

The discussion of attention is similarly restrained. Using attention to model temporal dependencies shows no significant overall advantage, and different attention variants have limited separation in average performance. Yet the appendix adds nuance: de-stationary, frequency-enhanced, and sparse attention outperform vanilla self-attention in more settings, with de-stationary attention especially effective on strongly non-stationary datasets such as Traffic and Weather.

A phase-level ablation further reports that Series Preprocessing and Series Encoding contribute larger average gains than Network Architecture or Network Optimization. That result is central to TSGym’s interpretation of the field: the dominant sources of performance are often upstream of the backbone. The inclusion of LLMs and TSFMs is also treated empirically rather than ideologically. GPT4TS is described as stable and competitive, Time-LLM performs poorly under many configurations, and the benefit of adding LLM or TSFM options is dataset-specific rather than uniform.

5. Benchmarks, datasets, and comparative performance

TSGym is evaluated on 10 datasets. The long-term forecasting benchmark includes ETTh1, ETTh2, ETTm1, ETTm2, Electricity (ECL), Traffic, Weather, Exchange, and ILI. The short-term benchmark is M4. For most long-term datasets, forecast horizons are $T$6; for ILI they are $T$7. On M4, the reported horizons are $T$8. The paper states that the past sequence length is set to 36 for ILI and 96 for others in standard long-term comparison.

The baseline set includes strong deep MTSF models such as DUET, TimeMixer, MICN, SegRNN, TimesNet, PatchTST, Crossformer, Autoformer, and DLinear in the main paper, with a substantially larger appendix catalog spanning RNNs, CNNs, MLPs, Transformers, LLM-based methods, and TSFMs. The appendix also compares against the AutoML methods AutoGluon-TimeSeries and AutoTS.

On long-term forecasting, the main comparison table reports that TSGym achieves the best MSE/MAE on 11 occasions and is the top-count method overall. Representative summaries in the paper state that TSGym is best on ETTm1, ETTm2, ETTh2, Traffic, and Weather, has the best MSE on ECL, is slightly behind SegRNN on one ETTh1 table, is outperformed by DUET on Exchange, and faces strong competition from TimesNet and PatchTST on ILI (Liang et al., 21 Sep 2025).

On M4, TSGym attains the best OWA and MASE, while SMAPE is described as essentially tied and slightly behind TimeMixer in one summarized table. The appendix’s larger M4 table gives TSGym the most first-place counts overall. In comparisons with AutoML, TSGym outperforms AutoGluon-TimeSeries and AutoTS on OWA, SMAPE, and MASE in short-term forecasting, and on most long-term datasets, with especially favorable results on ETTm1, ETTh1, ETTh2, ECL, and Weather.

Transferability and robustness are core interpretive themes of these results. Robustness is supported indirectly by the repeated success of shift-aware components such as RevIN, Stat, and de-stationary attention, and directly by the appendix’s claim-level analysis showing that normalization helps on high-shift datasets and can be unnecessary or harmful on low-shift datasets. Transferability is evidenced by zero-shot selection on unseen datasets and by a case study in which adding datasets similar to ILI in meta-feature space improves ILI performance substantially.

The implementation details reported for the meta-predictor are specific. It is trained for 100 epochs with early stopping, using Adam, learning rate 0.001, and batch size 512. All experiments were run on 4 NVIDIA A100 GPUs with 80GB, and 8 GPUs with 40GB memory, with parallel execution. The paper also states that all code is publicly available at https://github.com/SUFE-AILAB/TSGym.

6. Significance, limitations, and future directions

TSGym’s principal significance lies in its reformulation of MTSF methodology. Rather than treating forecasting architectures as monolithic competitors, it treats them as assemblies of reusable design choices. This has analytical value because it exposes which pipeline stages drive performance, and practical value because it enables automated construction of pipelines that are tailored to dataset characteristics and need not duplicate existing published models.

The framework is best described, in the paper’s own positioning, as a hybrid automated pipeline-construction framework. It performs component selection, includes aspects of hyperparameter tuning, conducts architecture search in a broad sense by choosing among MLP, RNN, Transformer, LLM, and TSFM, and differs from fixed-model AutoML because its unit of automation is the componentized pipeline rather than the named model. That distinction is central to its claim that component-level construction can outperform both state-of-the-art forecasting models and standard AutoML systems.

The limitations are also explicit. TSGym is not exhaustive and does not cover every possible MTSF component. It incurs a large upfront benchmarking cost, since many component combinations must be evaluated to build the performance matrix and train the meta-predictor. Component interactions remain complex even after decomposition. Some datasets still favor specialized baselines, with Exchange identified as a notable harder case and ILI as a difficult regime sometimes better served by specific baselines or large-model variants. The integration of LLMs and TSFMs is presented as promising but immature, and the current system is largely single-objective, emphasizing predictive quality rather than explicit accuracy-versus-cost trade-offs.

The future directions identified in the paper are correspondingly concrete: expand the design space with new forecasting techniques, improve meta-learning, add multi-objective optimization balancing accuracy and computational cost, and extend the approach beyond forecasting to broader time-series tasks. A plausible implication is that TSGym’s lasting contribution may be less a single benchmark score than a methodological template for studying time-series models as compositional systems whose effectiveness depends on dataset-adaptive assembly rather than architectural branding alone.