---
title: 'TSGym: Component-Level Forecasting'
url: https://www.emergentmind.com/topics/tsgym
type: topic
---

# TSGym: Component-Level Forecasting

Searching arXiv for TSGym and a few directly related MTSF component papers for accurate citations.
TSGym is a framework for benchmarking, analyzing, and automatically constructing deep models for multivariate time-series forecasting (MTSF) at the component level rather than the whole-model level. Introduced in "TSGym: Design Choices for Deep Multivariate Time-Series Forecasting" [2509.17063], it treats modern forecasting systems as compositions of recurring design choices—such as normalization, decomposition, tokenization, channel handling, backbone type, attention mechanism, and optimization settings—rather than as indivisible named architectures. Its central claim is that forecasting performance depends less on allegiance to a single model family than on selecting an effective combination of components for the characteristics of a given dataset. On that basis, TSGym combines large-scale component-level benchmarking with a meta-learning system that performs zero-shot automated model construction for new datasets.

## 1. Problem setting and conceptual orientation

TSGym studies the standard MTSF setting in which a multivariate series with \(C\) variables is observed over a historical window of length \(L\), and the objective is to predict the next \(T\) time steps. The work follows prior long-horizon forecasting practice by predicting all future steps directly rather than autoregressively, in order to avoid error accumulation when \(T>1\). In the appendix, the future forecast tensors used in the metrics are written as \(\mathbf{X}, \widehat{\mathbf{X}} \in \mathbb{R}^{H \times C}\), with \(H\) future time points.

The paper’s motivation is methodological as much as empirical. Prior MTSF evaluation is described as too holistic: benchmarks typically compare complete end-to-end models, even though those models differ along multiple internal dimensions. This obscures which parts actually matter, when they help, and how they interact. TSGym therefore reframes the field’s central question from “which named model is best?” to “which component combination is best for this dataset?” That reframing is the basis both for its analytical program and for its automated construction procedure.

A major consequence of this position is that TSGym is not presented as conventional model selection, standard hyperparameter tuning, or narrow-family neural architecture search. It is instead a pipeline-level component-composition framework whose searchable space spans preprocessing, encoding, architecture, and optimization simultaneously.

## 2. Component-level decomposition of the forecasting pipeline

TSGym organizes deep forecasting as a four-stage pipeline:

\[
\text{Series Preprocessing} \rightarrow \text{Series Encoding} \rightarrow \text{Network Architecture} \rightarrow \text{Network Optimization}
\]

Across that pipeline it studies **16 design dimensions**. The decomposition is intended to isolate the contribution of fine-grained choices such as whether a model uses channel independence, series patching, or a particular normalization scheme, instead of attributing performance gains to an entire architecture label.

| Stage | Design dimensions | Candidate choices |
|---|---|---|
| Preprocessing | normalization; decomposition; series sampling/mixing | None, Stat, RevIN, DishTS; None, MA, MoEMA, DFT; False/True |
| Encoding | channel independent; sequence length; series embedding/tokenization | False/True; \(48, 96, 192, 512\); Inverted Encoding, Positional Encoding, Series Patching |
| Architecture | network type; series attention; feature attention; \(d_{\text{model}}\); \(d_{\text{ff}}\); encoder layers | MLP, RNN, Transformer, LLM, TSFM; Null, SelfAttn, AutoCorr, SparseAttn, FrequencyAttn, DestationaryAttn; Null, SelfAttn, SparseAttn, FrequencyAttn; 64 or 256; 256 or 1024; 2 or 3 |
| Optimization | epochs; loss function; learning rate; learning-rate strategy | 10, 20, 50; MSE, MAE, HUBER; \(10^{-3}, 10^{-4}\); Null, Type1 |

This search space includes components inspired by influential MTSF methods explicitly named in the paper, including series patching from PatchTST, channel-independent strategy from PatchTST, inverted encoding from iTransformer, multi-scale mixing from TimeMixer, multiple attention variants from the Transformer-for-time-series literature, and LLM and TSFM backbone choices. The appendix compares TSGym to ADGym and reports a design space size of about **7,962,624** combinations [2509.17063].

The significance of this decomposition is twofold. Scientifically, it enables component-level claims—for example, whether tokenization matters more than attention, or whether Transformers outperform MLPs only under certain preprocessing conditions. Practically, it allows TSGym to construct pipelines that need not coincide with any previously published off-the-shelf model.

## 3. Formalization and automated model construction

TSGym formalizes the design space through a set of design dimensions,
\[
\mathcal{DD}=\{DD_1,\ldots,DD_k\},
\]
where each \(DD_i\) is a set of candidate design choices. The valid model space is the Cartesian product
\[
\mathcal{M}=DD_1 \times DD_2 \times \cdots \times DD_k,
\]
so that each composed model \(M_j\) is a tuple of one choice per dimension.

Because the full space is too large for exhaustive evaluation, TSGym samples a subset of valid combinations,
\[
\mathcal{M}_s \subset \mathcal{M}.
\]
Suppose there are \(n\) training datasets and \(m\) sampled pipelines. Evaluating them yields a performance matrix
\[
\mathbf{P}\in \mathbb{R}^{n \times m},
\]
where \(\mathbf{P}_{i,j}\) is the forecasting performance, using MSE, of model \(M_j\) on dataset \(\mathcal{D}_i\). Since raw MSE values are not directly comparable across datasets, TSGym converts each row into a normalized rank,
\[
\mathbf{R}_{i,j}=rank(P_{i,j})/m \in [0,1],
\]
with smaller values indicating better relative performance.

Dataset representation is based on meta-features \(\mathbf{E}_i^{meta}\) extracted from the training split only. These include statistical, temporal, spectral, fractal, and distribution shift features; the appendix states that TSGym uses **1404-dimensional** meta-features. Model configurations are represented by categorical design choices encoded with a `LabelEncoder` and then mapped through an `nn.Embedding`, producing learned continuous component embeddings.

The meta-predictor is a **two-layer MLP** that learns the mapping
\[
f(\mathcal{D}_i, M_j)=\mathbf{R}_{i,j},
\]
and more specifically
\[
f:\mathbf{E}_{i}^{meta}, \mathbf{E}_{j}^{comp} \mapsto \mathbf{R}_{i,j}.
\]
At selection time on a new dataset, TSGym extracts meta-features from the training split, predicts ranks for candidate component combinations, and selects the top-ranked pipeline or pipelines. The paper characterizes this as **zero-shot at selection time**, because it does not require evaluating all candidates on the target test dataset.

The search process operates at two levels. Candidate configurations are initially sampled randomly from the valid space, and the paper later introduces **Optuna** to sample more intelligently through Bayesian optimization, using **50 random trials for cold start** and then **50 guided trials**. The appendix states that the meta-predictor is optimized with **Pearson loss**, with the explicit goal of learning ranking quality rather than absolute error.

## 4. Empirical findings from component-level analysis

A substantial part of TSGym’s contribution lies in its component-level empirical analysis rather than only in its final benchmark scores. The most consistent general finding is that **series normalization matters a great deal**. **RevIN** and **Stat normalization** are repeatedly among the best choices and are described as almost essential defaults for handling non-stationary dynamics. The appendix sharpens this into a conditional statement: normalization helps on high-drift datasets and may hurt when drift is negligible [2509.17063].

By contrast, **decomposition** is useful but clearly dataset-dependent. MA-based decomposition helps on some datasets such as **ETTm1**, while on others—notably **ETTh1, ECL, ETTh2, Traffic**—no decomposition can be better. The paper therefore rejects the view of decomposition as a universal add-on.

The analysis of **channel independence** is similarly conditional. Channel-independent strategies outperform channel-dependent ones on most datasets, with **Traffic** and **ILI** as explicit exceptions. However, the appendix emphasizes that CI versus CD cannot be determined simply from inter-channel correlation; even highly correlated datasets may still prefer CI. This is one of the paper’s clearest arguments against simplistic architecture heuristics.

**Tokenization** emerges as one of the most consequential design dimensions. Both **patch-wise encoding / series patching** and **inverted encoding** outperform traditional point-wise positional encoding across many settings. In the appendix, positional encoding is said to win only rarely, while patching and inverted encoding dominate. This supports the paper’s broader argument that some gains typically attributed to backbone innovations are better understood as gains in representation design.

The results on **backbones** are notably anti-dogmatic. Transformers are not universally superior: in the main text, Transformer outperforms MLP mainly on **Weather, Traffic,** and **ILI**, whereas simpler models remain competitive or better elsewhere. The appendix goes further, stating that Transformers are less robust than MLPs by IQR-based variability in many settings, and that the claim that Transformers have a higher upper bound than MLPs is **not** reliably supported. In that sense, TSGym directly challenges architecture tribalism in MTSF.

The discussion of **attention** is similarly restrained. Using attention to model temporal dependencies shows **no significant overall advantage**, and different attention variants have limited separation in average performance. Yet the appendix adds nuance: **de-stationary**, **frequency-enhanced**, and **sparse** attention outperform vanilla self-attention in more settings, with de-stationary attention especially effective on strongly non-stationary datasets such as **Traffic** and **Weather**.

A phase-level ablation further reports that **Series Preprocessing** and **Series Encoding** contribute larger average gains than **Network Architecture** or **Network Optimization**. That result is central to TSGym’s interpretation of the field: the dominant sources of performance are often upstream of the backbone. The inclusion of **LLMs** and **TSFMs** is also treated empirically rather than ideologically. **GPT4TS** is described as stable and competitive, **Time-LLM** performs poorly under many configurations, and the benefit of adding LLM or TSFM options is dataset-specific rather than uniform.

## 5. Benchmarks, datasets, and comparative performance

TSGym is evaluated on **10 datasets**. The long-term forecasting benchmark includes **ETTh1, ETTh2, ETTm1, ETTm2, Electricity (ECL), Traffic, Weather, Exchange,** and **ILI**. The short-term benchmark is **M4**. For most long-term datasets, forecast horizons are \(\{96, 192, 336, 720\}\); for ILI they are \(\{24, 36, 48, 60\}\). On M4, the reported horizons are \(\{6, 8, 13, 14, 18, 48\}\). The paper states that the past sequence length is set to **36 for ILI** and **96 for others** in standard long-term comparison.

The baseline set includes strong deep MTSF models such as **DUET, TimeMixer, MICN, SegRNN, TimesNet, PatchTST, Crossformer, Autoformer,** and **DLinear** in the main paper, with a substantially larger appendix catalog spanning RNNs, CNNs, MLPs, Transformers, LLM-based methods, and TSFMs. The appendix also compares against the AutoML methods **AutoGluon-TimeSeries** and **AutoTS**.

On long-term forecasting, the main comparison table reports that TSGym achieves the best MSE/MAE on **11 occasions** and is the top-count method overall. Representative summaries in the paper state that TSGym is best on **ETTm1, ETTm2, ETTh2, Traffic,** and **Weather**, has the best MSE on **ECL**, is slightly behind **SegRNN** on one **ETTh1** table, is outperformed by **DUET** on **Exchange**, and faces strong competition from **TimesNet** and **PatchTST** on **ILI** [2509.17063].

On **M4**, TSGym attains the best **OWA** and **MASE**, while **SMAPE** is described as essentially tied and slightly behind **TimeMixer** in one summarized table. The appendix’s larger M4 table gives TSGym the most first-place counts overall. In comparisons with AutoML, TSGym outperforms **AutoGluon-TimeSeries** and **AutoTS** on **OWA, SMAPE,** and **MASE** in short-term forecasting, and on most long-term datasets, with especially favorable results on **ETTm1, ETTh1, ETTh2, ECL,** and **Weather**.

Transferability and robustness are core interpretive themes of these results. Robustness is supported indirectly by the repeated success of shift-aware components such as **RevIN**, **Stat**, and **de-stationary attention**, and directly by the appendix’s claim-level analysis showing that normalization helps on high-shift datasets and can be unnecessary or harmful on low-shift datasets. Transferability is evidenced by zero-shot selection on unseen datasets and by a case study in which adding datasets similar to **ILI** in meta-feature space improves ILI performance substantially.

The implementation details reported for the meta-predictor are specific. It is trained for **100 epochs** with **early stopping**, using **Adam**, learning rate **0.001**, and batch size **512**. All experiments were run on **4 NVIDIA A100 GPUs with 80GB**, and **8 GPUs with 40GB memory**, with parallel execution. The paper also states that all code is publicly available at **https://github.com/SUFE-AILAB/TSGym**.

## 6. Significance, limitations, and future directions

TSGym’s principal significance lies in its reformulation of MTSF methodology. Rather than treating forecasting architectures as monolithic competitors, it treats them as assemblies of reusable design choices. This has analytical value because it exposes which pipeline stages drive performance, and practical value because it enables automated construction of pipelines that are tailored to dataset characteristics and need not duplicate existing published models.

The framework is best described, in the paper’s own positioning, as a **hybrid automated pipeline-construction framework**. It performs **component selection**, includes aspects of **hyperparameter tuning**, conducts architecture search in a broad sense by choosing among **MLP, RNN, Transformer, LLM,** and **TSFM**, and differs from fixed-model AutoML because its unit of automation is the componentized pipeline rather than the named model. That distinction is central to its claim that component-level construction can outperform both state-of-the-art forecasting models and standard AutoML systems.

The limitations are also explicit. TSGym is **not exhaustive** and does not cover every possible MTSF component. It incurs a **large upfront benchmarking cost**, since many component combinations must be evaluated to build the performance matrix and train the meta-predictor. Component interactions remain complex even after decomposition. Some datasets still favor specialized baselines, with **Exchange** identified as a notable harder case and **ILI** as a difficult regime sometimes better served by specific baselines or large-model variants. The integration of **LLMs** and **TSFMs** is presented as promising but immature, and the current system is largely **single-objective**, emphasizing predictive quality rather than explicit accuracy-versus-cost trade-offs.

The future directions identified in the paper are correspondingly concrete: expand the design space with new forecasting techniques, improve meta-learning, add **multi-objective optimization** balancing accuracy and computational cost, and extend the approach beyond forecasting to broader time-series tasks. A plausible implication is that TSGym’s lasting contribution may be less a single benchmark score than a methodological template for studying time-series models as compositional systems whose effectiveness depends on dataset-adaptive assembly rather than architectural branding alone.

Source: https://www.emergentmind.com/topics/tsgym