TSCOMP: Benchmark for Deep Time-Series Forecasting

• TSCOMP is a benchmark framework for deep multivariate time-series forecasting, decomposing pipelines into preprocessing, encoding, architecture, and optimization stages.
• The framework employs a constrained orthogonal design to evaluate over 20,000 model-dataset configurations, revealing that series preprocessing is the dominant factor in performance variance.
• TSCOMP leverages meta-learning for zero-shot model configuration, recombining fine-grained components to outperform state-of-the-art methods on long-term forecasting benchmarks.

Searching arXiv for the topic and related usages of “TSCOMP” to ground the article in current papers. TSCOMP most specifically denotes a benchmark-driven framework for deep multivariate time-series forecasting that replaces comparison among holistic architectures with systematic component-level decomposition, benchmarking, and recomposition. In this usage, TSCOMP is presented as “the first large-scale benchmark that systematically deconstructs deep forecasting methods into their core, fine-grained components” across series preprocessing, encoding strategies, network architectures, and optimization methods, and it couples that benchmark with a performance corpus of over 20,000 model-dataset evaluations for zero-shot model construction on new datasets (Liang et al., 26 May 2026).

1. Definition and problem setting

In multivariate time-series forecasting, TSCOMP studies the direct multi-step prediction problem with $C$ variables: given historical data

$$\boldsymbol{\chi} = \{\boldsymbol{x}_1^t,\ldots,\boldsymbol{x}_C^t\}_{t=1}^{L},$$

predict the next $T$ steps

$$\hat{\boldsymbol{\chi}} = \{\hat{\boldsymbol{x}}_1^t,\ldots,\hat{\boldsymbol{x}}_C^t\}_{t=L+1}^{L+T}.$$

The framework explicitly avoids recursive rollout in order to avoid error accumulation (Liang et al., 26 May 2026).

The central methodological shift is from monolithic model comparison to compositional analysis. Rather than asking whether a full model such as DLinear, iTransformer, or TimeMixer is superior on a given benchmark, TSCOMP asks which internal components drive performance, which components interact favorably, and how those components can be recombined into new pipelines. The paper states that previous benchmarks such as TSB/TSlib, TFB, or OpenLTM mainly answer “which full model is better on which dataset,” whereas TSCOMP targets a granular, component-level understanding of model behavior (Liang et al., 26 May 2026).

Formally, the forecasting pipeline is represented as a composition of component dimensions. If $\mathcal{DD}=\{DD_1,\dots,DD_k\}$ denotes the set of component dimensions, then the model space is

$$\mathcal{M} = DD_1 \times DD_2 \times \cdots \times DD_k = \{(DC_1,\dots,DC_k)\mid DC_i \in DD_i\}.$$

Each element of $\mathcal{M}$ is a concrete forecasting pipeline instantiation. This design makes TSCOMP simultaneously a benchmark, a structured model-design space, and a corpus for subsequent meta-learning (Liang et al., 26 May 2026).

2. Component decomposition of forecasting pipelines

TSCOMP organizes the pipeline into four stages: Series Preprocessing $\rightarrow$ Series Encoding $\rightarrow$ Network Architecture $\rightarrow$ Network Optimization. The design space spans 4 pipeline stages, 11 dimensions, and 49 concrete components (Liang et al., 26 May 2026).

Stage	Dimension	Components explicitly listed
Series preprocessing	Series normalization	w/o Norm, Stat, RevIN, DishTS
Series preprocessing	Series decomposition	w/o Decomp, Moving Average, MoEMA, DFT
Series preprocessing	Series sampling / mixing	w/o Mixing, w/ Mixing
Series encoding	Channel independence vs. dependence	Channel Dependent, Channel Independent
Series encoding	Series tokenization	Point Encoding, Series Patching, Inverted Encoding, Ortho Encoding
Series encoding	Timestamp embedding	w/o Embedding, w/ Embedding
Network architecture	Backbone type	DNN, NormLin, GRU, xLSTM, Plain Transformer without attention, SelfAttn, AutoCorr, SparseAttn, FrequencyAttn, DestationaryAttn, GPT4TS, TimeLLM, Timer, Moment, TimeMoE, Chronos
Network architecture	Feature attention	w/o Attn, SelfAttn, SparseAttn
Network architecture	Retrieval-Augmented Generation	w/o RAG, w/ RAG
Network optimization	Sequence length	$$\boldsymbol{\chi} = \{\boldsymbol{x}_1^t,\ldots,\boldsymbol{x}_C^t\}_{t=1}^{L},$$0
Network optimization	Loss function	MSE, MAE, HUBER, DBLoss, PSLoss, FreDFLoss

Within this decomposition, some dimensions are presented as especially consequential. Series normalization is reported as the single most influential dimension, with variance explained of approximately $$\boldsymbol{\chi} = \{\boldsymbol{x}_1^t,\ldots,\boldsymbol{x}_C^t\}_{t=1}^{L},$$1. Channel independence explains $$\boldsymbol{\chi} = \{\boldsymbol{x}_1^t,\ldots,\boldsymbol{x}_C^t\}_{t=1}^{L},$$2, tokenization $$\boldsymbol{\chi} = \{\boldsymbol{x}_1^t,\ldots,\boldsymbol{x}_C^t\}_{t=1}^{L},$$3, network architecture $$\boldsymbol{\chi} = \{\boldsymbol{x}_1^t,\ldots,\boldsymbol{x}_C^t\}_{t=1}^{L},$$4, and network optimization $$\boldsymbol{\chi} = \{\boldsymbol{x}_1^t,\ldots,\boldsymbol{x}_C^t\}_{t=1}^{L},$$5 in the ANOVA-based variance decomposition reported for the benchmark (Liang et al., 26 May 2026).

Several component-specific patterns are also stated. RevIN and simple statistical normalization are among the most impactful components overall. Channel independence is consistently beneficial in aggregate, especially for MLPs, RNNs, and TSFMs, although it can be harmful on highly correlated datasets. Inverted Encoding and Ortho Encoding are reported as robust improvements over raw point encoding. By contrast, timestamp embedding explains very little variance overall—approximately $$\boldsymbol{\chi} = \{\boldsymbol{x}_1^t,\ldots,\boldsymbol{x}_C^t\}_{t=1}^{L},$$6 in the reported setup (Liang et al., 26 May 2026).

The decomposition stage is treated cautiously. Moving Average, MoEMA, and DFT are described as often neutral or harmful in aggregate, with architecture-specific exceptions such as DFT with some TSFMs or Moving Average with LLMs. This suggests that TSCOMP treats decomposition not as a default inductive bias but as a conditional design choice whose value depends on the surrounding pipeline (Liang et al., 26 May 2026).

3. Constrained orthogonal design and benchmark corpus

The full Cartesian design space contains over $$\boldsymbol{\chi} = \{\boldsymbol{x}_1^t,\ldots,\boldsymbol{x}_C^t\}_{t=1}^{L},$$7 potential configurations, but many combinations are invalid or computationally infeasible. Examples given include incompatibility between inverted encoding and channel-independent processing, and the requirement that pre-trained TSFMs use specific encodings. TSCOMP therefore adopts a constrained orthogonal experimental design with pairwise coverage rather than exhaustive enumeration (Liang et al., 26 May 2026).

The design objective is that for any valid pair of components, there exists at least one sampled configuration in which both appear. The paper describes a greedy pairwise-coverage algorithm. It initializes the set of valid pairwise interactions $$\boldsymbol{\chi} = \{\boldsymbol{x}_1^t,\ldots,\boldsymbol{x}_C^t\}_{t=1}^{L},$$8, seeds a model pool $$\boldsymbol{\chi} = \{\boldsymbol{x}_1^t,\ldots,\boldsymbol{x}_C^t\}_{t=1}^{L},$$9 with an initial set $T$0, generates random valid candidates using a constraint checker $T$1, and repeatedly selects the candidate that covers the largest number of not-yet-covered interactions. If the candidate adds new coverage, it is inserted into $T$2 and the corresponding interactions are removed from $T$3; otherwise the search stops (Liang et al., 26 May 2026).

In practice, this procedure produces roughly 136 models per horizon. The resulting benchmark evaluates on 13 long-term forecasting datasets plus M4, with long-term horizons $T$4 or shorter horizons for low-sample datasets, and M4 horizons $T$5 depending on frequency. The corpus comprises roughly 20,760 model-dataset-horizon evaluations, consistent with the paper’s more general statement that the benchmark yields over 20,000 model-dataset evaluations (Liang et al., 26 May 2026).

For long-term forecasting, the principal metrics are

$T$6

For M4, the benchmark uses SMAPE, MAPE, MASE, and OWA, with

$T$7

$T$8

and

$T$9

To enable pooled statistical modeling across datasets and horizons, the benchmark standardizes MSE across tasks and constructs a performance matrix $$\hat{\boldsymbol{\chi}} = \{\hat{\boldsymbol{x}}_1^t,\ldots,\hat{\boldsymbol{x}}_C^t\}_{t=L+1}^{L+T}.$$0, subsequently converting entries to normalized ranks

$$\hat{\boldsymbol{\chi}} = \{\hat{\boldsymbol{x}}_1^t,\ldots,\hat{\boldsymbol{x}}_C^t\}_{t=L+1}^{L+T}.$$1

with lower values indicating better relative performance (Liang et al., 26 May 2026).

The paper further reports that higher-order interactions exist, but pairwise coverage is justified pragmatically: many three-way interactions are statistically significant, yet the incremental variance explained beyond main and pairwise effects is modest, at roughly $$\hat{\boldsymbol{\chi}} = \{\hat{\boldsymbol{x}}_1^t,\ldots,\hat{\boldsymbol{x}}_C^t\}_{t=L+1}^{L+T}.$$2 in $$\hat{\boldsymbol{\chi}} = \{\hat{\boldsymbol{x}}_1^t,\ldots,\hat{\boldsymbol{x}}_C^t\}_{t=L+1}^{L+T}.$$3, while main effects alone explain approximately $$\hat{\boldsymbol{\chi}} = \{\hat{\boldsymbol{x}}_1^t,\ldots,\hat{\boldsymbol{x}}_C^t\}_{t=L+1}^{L+T}.$$4 of explainable variance (Liang et al., 26 May 2026).

4. Empirical findings and statistical interpretation

TSCOMP’s principal empirical claim is that series preprocessing dominates performance variance. At the pipeline-stage level, the reported ANOVA decomposition is Preprocessing (66.6\%) $$\hat{\boldsymbol{\chi}} = \{\hat{\boldsymbol{x}}_1^t,\ldots,\hat{\boldsymbol{x}}_C^t\}_{t=L+1}^{L+T}.$$5 Encoding (18.3\%) $$\hat{\boldsymbol{\chi}} = \{\hat{\boldsymbol{x}}_1^t,\ldots,\hat{\boldsymbol{x}}_C^t\}_{t=L+1}^{L+T}.$$6 Architecture (8.0\%) $$\hat{\boldsymbol{\chi}} = \{\hat{\boldsymbol{x}}_1^t,\ldots,\hat{\boldsymbol{x}}_C^t\}_{t=L+1}^{L+T}.$$7 Optimization (7.1\%). Within preprocessing, normalization alone contributes 63.0\% of explained variance (Liang et al., 26 May 2026).

At the component level, generalized linear mixed model analysis yields several recurring effects. RevIN and simple statistical normalization have large negative coefficients, indicating improved performance relative to no normalization. DishTS is also favorable but more variable. By contrast, omitting normalization is consistently worse. Decomposition is described as fragile: on average, Moving Average, MoEMA, and DFT often increase MSE, especially for Transformers, although Moving Average can help LLMs and DFT can benefit some TSFMs and specialized losses (Liang et al., 26 May 2026).

Encoding effects are similarly prominent. Channel independence is a strong global default, with especially favorable effects for MLPs, RNNs, and TSFMs. Inverted Encoding and Ortho Encoding are reported as robust winners across architectures. Patching is more nuanced and can underperform for TSFMs despite being used at pre-training. Timestamp embedding contributes little to the explained variance in the reported setup (Liang et al., 26 May 2026).

Optimization effects are architecture-dependent. MAE and HUBER consistently outperform pure MSE, especially for Transformers and RNNs. Specialized time-series losses—DBLoss, PSLoss, and FreDFLoss—are described as useful under specific architectures. Lightweight feature attention also matters: a simple MLP backbone combined with Sparse Feature Attention is reported as a particularly strong combination (Liang et al., 26 May 2026).

Architecture-specific analyses refine these aggregate results. MLPs rely heavily on normalization and channel independence, with orthogonal encoding especially beneficial. RNNs are highly sensitive to channel independence and sequence length, and decomposition often harms them. Transformers are more sensitive to loss design; FrequencyAttn is described as particularly effective, while decomposition tends to hurt. LLM-based models are unusually sensitive to decomposition and multi-scale mixing. TSFMs are extremely sensitive to encoding, with tokenization alone explaining 24.7\% of variance in that family (Liang et al., 26 May 2026).

Dataset characteristics further modulate these effects. On datasets with high distribution shift, such as Exchange and Covid-19, the paper states that simple instance-wise normalization such as RevIN is not enough, and stationary normalization is more robust. Auto-correlation attention is effective on strongly autocorrelated datasets, while DestationaryAttn is particularly effective on non-stationary datasets. Channel independence becomes detrimental on highly correlated datasets such as Traffic and ECL, where channel-dependent modeling performs better (Liang et al., 26 May 2026).

Taken together, these findings support the paper’s central argument that preprocessing and encoding choices exert more influence than backbone complexity. A plausible implication is that a large part of recent model-to-model performance variance in deep forecasting can be reinterpreted as variance induced by hidden component choices rather than by architecture families alone.

5. Corpus-driven automated component selection

TSCOMP does not stop at descriptive benchmarking. It uses the benchmark corpus to learn zero-shot recommendations for component configurations on unseen datasets. The meta-dataset is defined as

$$\hat{\boldsymbol{\chi}} = \{\hat{\boldsymbol{x}}_1^t,\ldots,\hat{\boldsymbol{x}}_C^t\}_{t=L+1}^{L+T}.$$8

where $$\hat{\boldsymbol{\chi}} = \{\hat{\boldsymbol{x}}_1^t,\ldots,\hat{\boldsymbol{x}}_C^t\}_{t=L+1}^{L+T}.$$9 is a dataset, $\mathcal{DD}=\{DD_1,\dots,DD_k\}$0 a component configuration, and $\mathcal{DD}=\{DD_1,\dots,DD_k\}$1 the normalized rank of that configuration on that dataset (Liang et al., 26 May 2026).

Each configuration is encoded as a vector of component indices $\mathcal{DD}=\{DD_1,\dots,DD_k\}$2, transformed through a learnable codebook $\mathcal{DD}=\{DD_1,\dots,DD_k\}$3, and concatenated into a component embedding

$\mathcal{DD}=\{DD_1,\dots,DD_k\}$4

Dataset descriptors are obtained through a proxy supervised task built with TabPFN. Time windows $\mathcal{DD}=\{DD_1,\dots,DD_k\}$5 and next-step targets $\mathcal{DD}=\{DD_1,\dots,DD_k\}$6 are converted into bucketized labels $\mathcal{DD}=\{DD_1,\dots,DD_k\}$7, passed through TabPFN’s encoder, and aggregated into a meta-feature vector $\mathcal{DD}=\{DD_1,\dots,DD_k\}$8, which serves as $\mathcal{DD}=\{DD_1,\dots,DD_k\}$9 (Liang et al., 26 May 2026).

The resulting predictor is a regression map

$$\mathcal{M} = DD_1 \times DD_2 \times \cdots \times DD_k = \{(DC_1,\dots,DC_k)\mid DC_i \in DD_i\}.$$0

implemented as a two-layer MLP and trained to maximize correlation between predicted and true rankings. At inference time on a new dataset, TSCOMP computes meta-features, samples a candidate configuration set, scores each candidate, ranks them by predicted normalized rank, and selects the top-$$\mathcal{M} = DD_1 \times DD_2 \times \cdots \times DD_k = \{(DC_1,\dots,DC_k)\mid DC_i \in DD_i\}.$$1 pipelines for training and optional ensembling (Liang et al., 26 May 2026).

The reported selection quality is high. Among the top-5 predicted configurations per task, 98\% fall within the top quartile of the true performance distribution, and more than 99\% fall within the top half. The paper interprets this as evidence that the corpus captures transferable regularities in component performance across datasets (Liang et al., 26 May 2026).

The paper further reports that the resulting automatically composed models—mostly simple MLPs with selected components—outperform state-of-the-art deep MTSF baselines, AutoML systems, and large time-series models on standard benchmarks. Representative results include best average OWA $$\mathcal{M} = DD_1 \times DD_2 \times \cdots \times DD_k = \{(DC_1,\dots,DC_k)\mid DC_i \in DD_i\}.$$2 on M4 versus the next best $$\mathcal{M} = DD_1 \times DD_2 \times \cdots \times DD_k = \{(DC_1,\dots,DC_k)\mid DC_i \in DD_i\}.$$3, first place on 10 out of 14 dataset-metric combinations in the long-term setting, and superior MSE/MAE to TimeFuse, AutoGluon, AutoTS, GPT4TS, and other large time-series models across several datasets. The offline cost of building the corpus is compared to pre-training a foundation model, while online application is reported to be fast; on ETTh1, TSCOMP-fast is approximately $$\mathcal{M} = DD_1 \times DD_2 \times \cdots \times DD_k = \{(DC_1,\dots,DC_k)\mid DC_i \in DD_i\}.$$4 faster than AutoGluon and more accurate (Liang et al., 26 May 2026).

6. Limitations, extensions, and methodological significance

The benchmark is explicit about several limitations. Its design space is large but not exhaustive; some architectures and preprocessing strategies are not included explicitly. Evaluation is conducted in a supervised full-shot regime rather than low-data or online settings. Pairwise coverage does not guarantee characterization of all higher-order interactions. The benchmark focuses on deterministic point forecasting metrics rather than probabilistic forecasting criteria such as CRPS. Other time-series tasks, including classification, anomaly detection, and representation learning, are outside its immediate scope (Liang et al., 26 May 2026).

The proposed future directions are correspondingly broad. The paper suggests LLM-agent-powered continuous evolution of the corpus, with automated extraction of new components from the literature and integration into the benchmark. It also proposes extensions to probabilistic forecasting, anomaly detection, imputation, few-shot and zero-shot evaluation for TSFMs, and multi-objective analysis involving accuracy, latency, and memory (Liang et al., 26 May 2026).

Methodologically, TSCOMP is positioned against both holistic benchmarks and model-level AutoML systems. Benchmarks such as TSlib, TFB, and BasicTS compare whole models, and systems such as AutoGluon-TimeSeries, AutoTS, AutoForecast, AutoXPCR, and TimeFuse primarily select or combine existing models. TSCOMP instead moves from model selection to component composition, while retaining an interpretable statistical layer through GLMMs, ANOVA, and data-specific interaction analyses (Liang et al., 26 May 2026).

A common misconception in the forecasting literature is that architectural sophistication is the primary determinant of performance. The benchmark directly contests that view by reporting that preprocessing and encoding dominate explained variance, and by showing that simple MLP backbones, when paired with strong normalization, channel strategy, tokenization, and loss design, can outperform more elaborate architectures. This does not imply that architecture is irrelevant; rather, the reported variance decomposition indicates that architecture is only one stage in a larger compositional system (Liang et al., 26 May 2026).

7. Other research usages of the term

Outside the 2026 forecasting benchmark, the string “TSCOMP” appears in other research contexts. In the timed-automata literature around Timed Concurrent State Machines, the supplied overview uses “TSCOMP-style workflow” to denote a timed composition and verification methodology in which TCSM models are composed, transformed into Region CSM, stored in a ready-to-verification form, and then repeatedly multiplied by testing automata for safety and liveness analysis (Daszczuk, 2017). In that usage, the central advantage is summarized as “compose once, verify many times” (Daszczuk, 2017).

A different supplied overview links the term informally to the time–scale–chirp_rate (TSC-R) recovery transform/operator for non-stationary signal analysis. There, “TSCOMP” is interpreted as fitting “very naturally” with a time–scale–chirp operator for multi-component processing, namely a 3D transform $$\mathcal{M} = DD_1 \times DD_2 \times \cdots \times DD_k = \{(DC_1,\dots,DC_k)\mid DC_i \in DD_i\}.$$5 over time, scale, and chirp rate that separates components with crossover instantaneous frequencies when chirp rates differ (Chui et al., 2020). The operator adds the quadratic phase kernel $$\mathcal{M} = DD_1 \times DD_2 \times \cdots \times DD_k = \{(DC_1,\dots,DC_k)\mid DC_i \in DD_i\}.$$6 to a wavelet-like transform and supports ridge-based recovery of component values, instantaneous frequencies, and chirp rates (Chui et al., 2020).

These additional usages indicate that the acronym is context-dependent across arXiv-adjacent discussions. In the most direct and formal sense, however, TSCOMP designates the component-level benchmark and corpus-driven model-construction framework for deep multivariate time-series forecasting introduced in 2026 (Liang et al., 26 May 2026).