---
title: 'TSR-Suite: Time-Series Reasoning Benchmark'
url: https://www.emergentmind.com/topics/time-series-reasoning-suite-tsr-suite
type: topic
---

# TSR-Suite: Time-Series Reasoning Benchmark

Time Series Reasoning Suite (TSR-Suite) is a benchmark-and-training framework for time-series reasoning introduced in "TimeOmni-1: Incentivizing Complex Reasoning with Time Series in Large Language Models" [2509.24803]. In that formulation, TSR-Suite defines four atomic tasks spanning three fundamental capabilities—perception, extrapolation, and decision-making—and provides more than 23K samples, including 2.3K examples curated through a human-guided hierarchical annotation process, to support both evaluation and training of time series reasoning models (TSRMs) [2509.24803]. This suggests that TSR-Suite is both a specific dataset introduced with TimeOmni-1 and a broader organizing concept for later work on multimodal, cognitively structured, and context-aware time-series reasoning benchmarks [2606.22126][2601.18744][2604.17295].

## 1. Definition and benchmark philosophy

In the original TimeOmni-1 formulation, TSR-Suite is motivated by the claim that many earlier multimodal time-series datasets do not require genuine time-series reasoning, because their questions are too shallow or their context is insufficient for uniquely determined answers [2509.24803]. Two explicit design principles are proposed.

The first is that question answering must reward reasoning. Let \(M_{\text{RM}}\) denote a reasoning model that produces rationales \(R\) and an answer \(y\), and let \(M_{\text{NRM}}\) denote a non-reasoning model that directly outputs \(y\). A task is reasoning-critical only if
\[
\bar{S}(M_{\text{RM}}) \gg \bar{S}(M_{\text{NRM}}),
\]
where \(\bar{S}(\cdot)\) is the mean task score [2509.24803].

The second is that context must be sufficient. If the time series \(X\) or auxiliary context \(C\) is underspecified, then even infinite reasoning capacity should not materially outperform chance:
\[
\lim_{RC \to \infty} P(\text{correct} \mid X, C) \approx \frac{1}{K},
\]
where \(K\) is the number of options. With sufficient context, the probability should be substantially above \(\frac{1}{K}\) [2509.24803].

Within this framework, a TSRM is defined as a model that explicitly samples rationales and answers:
\[
(R, y) \sim p_{\theta}(R, y \mid X, C)
= p_{\theta}(R \mid X, C)\, p_{\theta}(y \mid R, X, C),
\]
with rationales emitted inside `<think>` tags and final answers inside `<answer>` tags [2509.24803]. This formalization distinguishes TSR-Suite from conventional forecasting or classification benchmarks that compress all intermediate reasoning into latent states and evaluate only end-task accuracy.

## 2. Atomic tasks and capability structure

TSR-Suite formalizes four atomic tasks aligned with three capabilities: perception, extrapolation, and decision-making [2509.24803].

| Capability | Atomic task | Typical output |
|---|---|---|
| Perception | Scenario Understanding | Discrete choice |
| Perception | Causality Discovery | Discrete choice |
| Extrapolation | Event-aware Forecasting | Numerical sequence |
| Decision-making | Decision Making | Discrete choice |

Scenario Understanding presents a single time series together with several textual scenario options and asks which real-world scenario most likely generated the series. The prescribed reasoning template checks series length, magnitude, shape, temporal pattern, alignment with external events, and final elimination of alternatives [2509.24803].

Causality Discovery uses two river discharge time series sampled every 12 hours and asks whether station A causes B, B causes A, or the pair is non-causal. The reasoning template first checks trend consistency, then peak and dip alignment within \(1\)–\(2\) time steps, and finally uses the mean discharge magnitude to determine upstream and downstream direction [2509.24803].

Event-aware Forecasting provides a historical time series, past events in the history window, and descriptions of upcoming events, then asks for a future numerical sequence. The intended reasoning decomposes the task into baseline pattern extraction, estimation of event impact, and composition of baseline and event effects. Evaluation uses mean absolute error:
\[
\text{MAE} = \frac{1}{T}\sum_{t=1}^T |\hat{y}_t - y_t|.
\]
For reinforcement learning, the sequence reward is
\[
\mathcal{R}_{\text{sequence}} =
\begin{cases}
0, & \text{if } \text{len}(\hat{y}) \neq \text{len}(y),\\[4pt]
\exp(-\alpha \cdot \text{MAE}) + \mathcal{R}_{\text{count}}, & \text{otherwise},
\end{cases}
\]
with \(\mathcal{R}_{\text{count}} = 0.1\) [2509.24803].

Decision Making instantiates energy storage arbitrage under dynamic prices. Inputs include 48 hours of historical building load, a price schedule, battery specifications, and a small set of candidate strategies. The reasoning template first forecasts next-day load, then evaluates charge/discharge timing under valley and peak prices, and finally checks savings and feasibility. The paper gives the savings expression
\[
\text{Saving} =
\sum_{h \in \text{peak}}
\min\bigl(\hat{L}(h), P_{\max}^{\text{dis}}\bigr)
\cdot (p_{\text{peak}} - p_{\text{valley}}),
\]
subject to state-of-charge constraints [2509.24803].

This task organization makes TSR-Suite a progressive pipeline from temporal perception to extrapolation and then to action selection. Later benchmark proposals echo this layered view. TSCognition defines a five-task ladder—Decoding, Grounding, Inferring, Extrapolating, and Acting—while TSRBench organizes 15 tasks under Perception, Reasoning, Prediction, and Decision-Making, and HiTSR introduces a four-level taxonomy from numerical read-out to predictive inference [2606.22126][2601.18744][2604.17295]. This suggests that the original TSR-Suite capability structure became a reference point for subsequent taxonomies.

## 3. Dataset composition, annotation, and evaluation protocol

TSR-Suite contains 23,605 question-answer pairs across four tasks and ten domains, with 2,339 high-quality chain-of-thought examples in the curated Stage-1 subset [2509.24803]. The per-task counts are: Scenario Understanding \(609\) Stage-1 train, \(5{,}104\) Stage-2 train, \(200\) ID test, \(899\) OOD test; Causality Discovery \(778\), \(6{,}044\), \(800\), \(800\); Event-aware Forecasting \(400\), \(2{,}780\), \(418\), \(476\); and Decision Making \(552\), \(3{,}284\), \(188\), \(273\) [2509.24803].

The Stage-1 subset supplies rationale supervision. The Stage-2 subset is larger and supports reinforcement learning. The data pipeline uses three components: an LLM Analyzer, human reviewers, and an LLM Rewriter. GPT-4.1 first generates structured reasoning and answers using task-specific templates; if the answer is correct, the rationale is retained. If the answer is incorrect, human reviewers determine whether the problem is sufficiently specified and, if so, write a reasoning trace, which the Rewriter standardizes to the required format [2509.24803].

For Event-aware Forecasting, exact matching is not possible because real trajectories are noisy. The authors therefore retain examples where GPT-4.1 produces relatively low-MAE forecasts and report that ground-truth-guided chains—where the model is told the future values—hurt training compared with self-consistent but slightly imperfect model-generated forecasts [2509.24803]. This is a distinctive feature of TSR-Suite: it treats imperfect but structured forecasting rationales as useful supervision.

TSR-Suite also distinguishes instruction following from reasoning correctness. It reports Success Rate (SR), defined as the fraction of outputs that can be parsed into valid answers using the required `<think>...</think><answer>...</answer>` schema; ACC for discrete tasks; and MAE for Event-aware Forecasting, all computed on valid outputs [2509.24803]. For discrete tasks, correctness reward is
\[
\mathcal{R}_{\text{discrete}} =
\begin{cases}
1, & \hat{y} = y,\\
0, & \text{otherwise},
\end{cases}
\]
and the combined reward is
\[
\mathcal{R}_i
= \lambda \mathcal{R}_{\text{format}}
+ (1-\lambda)\mathcal{R}_{\text{task}},
\]
with \(\lambda = 0.1\) in the reported experiments [2509.24803].

## 4. Training role, TimeOmni-1, and empirical results

TSR-Suite was designed not only as a benchmark but also as a training curriculum for TSRMs. TimeOmni-1 uses Qwen2.5-Instruct-7B as its base model and adopts a two-stage training paradigm: supervised fine-tuning on the curated rationale subset, followed by reinforcement learning with Group Relative Policy Optimization (GRPO) on the larger Stage-2 data [2509.24803].

The supervised stage optimizes the negative log-likelihood of rationale-answer pairs over \(\mathcal{D}=\{(X_i, C_i, R_i, y_i)\}\):
\[
\mathcal{L}(\theta; \mathcal{D})
= -\frac{1}{N} \sum_{i=1}^N \log \pi_\theta\!\left((R'_i, y'_i) \mid X_i, C_i\right).
\]
The reinforcement stage samples multiple rationale-answer trajectories for each input, computes task-grounded rewards, and uses GRPO with group-relative advantages and a KL penalty to the supervised reference model [2509.24803].

The paper reports that reinforcement learning without the Stage-1 rationale injection is ineffective or harmful, whereas CoT-SFT followed by GRPO yields robust gains, especially on causality discovery and decision-making [2509.24803]. Multi-task training over all four TSR-Suite tasks also outperforms single-task training, and perception-plus-extrapolation supervision transfers to decision-making even when direct task-4 supervision is absent [2509.24803].

On the main benchmark, TimeOmni-1 achieves 69.3% ID accuracy and 64.0% OOD accuracy on Causality Discovery, compared with 28.7% and 35.9% for GPT-4.1; 90.7% ID and 87.7% OOD on Scenario Understanding; 47.9% ID and 58.9% OOD on Decision Making, compared with 25.5% and 27.8% for GPT-4.1; and 145.53 OOD MAE on Event-aware Forecasting, compared with 170.78 for GPT-4.1, while also improving valid response rate from 76.1% to 82.3% on that task [2509.24803]. These results position TSR-Suite as both an evaluation protocol and a task mixture for cultivating explicit time-series reasoning rather than pure numerical regression.

## 5. Expansion into a broader benchmark and modeling ecosystem

Later work extends the underlying idea of TSR-Suite in several directions. TSCognition defines cognitive time-series reasoning as a shift from trajectory prediction to context-grounded reasoning over multivariate time series and text, and introduces a multimodal benchmark with 41,086 multiple-choice QA pairs from 15 public sources and five tasks: Decoding, Grounding, Inferring, Extrapolating, and Acting [2606.22126]. TSRBench broadens the scope to 4,125 problems from 14 domains across 15 tasks and four dimensions—Perception, Reasoning, Prediction, and Decision-Making—and reports that scaling laws hold for perception and reasoning but break down for prediction [2601.18744]. HiTSR contributes a four-level taxonomy from numerical read-out to predictive inference and 83k samples, together with LLaTiSA, a dual-view VLM that combines plots and precision-calibrated numerical tables [2604.17295]. TFRBench, in turn, targets forecasting systems and evaluates reasoning traces over cross-channel dependencies, trends, seasonality, and external events across ten datasets and five domains [2604.05364]. TSVer adapts time-series reasoning to fact verification with 287 real-world claims, 400 curated time series, structured evidence ranges, and verdict justifications [2511.01101]. This suggests that the phrase "TSR-Suite" increasingly denotes a family of benchmark designs rather than a single fixed corpus.

A parallel literature addresses the modeling side. TSAlign aligns patch-level time-series representations with semantic directions in an LLM embedding space through attention alignment, gated residual injection, and multivariate fusion, and reports strong gains on TSCognition and TimerBed [2606.22126]. TS-Reasoner instead freezes a pretrained TS foundation model, inserts TS-to-text adapter outputs at `<ts></ts>` placeholders inside a Qwen2.5-7B-Instruct backbone, and uses a two-stage alignment-then-instruction recipe built from 120K synthetic caption pairs and 30K instruction examples [2510.03519]. RationaleTS treats retrieved rationales as guiding reasoning units rather than post-hoc explanations, using label-conditioned rationales, hybrid retrieval over temporal and semantic embeddings, and in-context inference on Finance, Traffic, and Power tasks [2601.02968]. SenTSR-Bench separates TSLM perception from GRLM reasoning and injects RLVR-trained TSLM traces directly into a GRLM’s reasoning buffer for industrial diagnostics [2602.19455]. ReasonTSC shows that, in time-series classification, tailored multi-turn reasoning plus plug-in classifier logits can outperform both vanilla CoT and standalone plug-in models [2506.00807]. Together, these systems indicate that later TSR-style research combines benchmark design with increasingly explicit interfaces between temporal encoders, rationales, retrieval, and general-purpose reasoning models.

## 6. Limitations, misconceptions, and future directions

Several recurrent limitations appear across the TSR-Suite literature. The original TSR-Suite covers ten domains and four tasks, but this is not exhaustive, and its chain-of-thought annotations partly inherit the biases of GPT-4.1 and a limited volume of human review [2509.24803]. Event-aware Forecasting is evaluated by MAE on a single realized future, even though multiple plausible futures may exist [2509.24803]. TSCognition identifies multiple-choice format as a limitation for explanation quality and interactive decision-making, and notes open problems involving noise, missing data, irregular sampling, and more severe domain shifts [2606.22126]. TSRBench shows that strong semantic reasoning does not guarantee accurate context-aware forecasting and that multimodal models still fail to fuse textual and visual representations effectively for reciprocal gains [2601.18744]. HiTSR argues that high-level reasoning benchmarks can remain fragile if lower-level numerical grounding is not explicitly evaluated [2604.17295].

A common misconception addressed by several papers is that LLMs should primarily be used as regressors over tokenized numbers. "Toward Reasoning-Centric Time-Series Analysis" argues instead for reframing time-series analysis from “extracting trends and correlations from observed data” to “a reasoning process,” and warns that many current LLM-based methods exploit numerical regression ability while neglecting deeper reasoning potential [2510.13029]. TSCognition similarly argues that textualization leads to extremely long token sequences and poor discretization of continuous signals, whereas chart-based visual encodings can distort temporal dependencies and fine-grained numeric patterns [2606.22126]. The BlueSky vision paper goes further, defining time-series reasoning systems as models that produce both outputs and reasoning paths,
\[
(y, \mathcal{R}) = f_\theta(\mathcal{X}, \mathcal{C}; \mathcal{T}),
\]
and frames interpretability and trustworthiness as first-class requirements for future systems [2510.16980].

Future work therefore converges on several directions. One is broader task coverage: open-ended QA, explanation generation, counterfactual reasoning, long-form reporting, root-cause analysis, and multi-turn decision dialogue are repeatedly proposed as extensions [2606.22126][2510.13029][2511.01101]. Another is richer multimodality and retrieval, including text, images, tables, logs, and retrieved historical episodes [2510.16980][2604.05364]. A third is more faithful reasoning evaluation, including rationale faithfulness, evidence alignment, attribution consistency, and calibrated uncertainty rather than only end-task accuracy [2510.16980][2511.01101]. This suggests that TSR-Suite, in its mature form, is best understood not as a closed benchmark but as a research program for moving from surface-level time-series recognition to explicit, context-aware, and decision-relevant temporal reasoning.

Source: https://www.emergentmind.com/topics/time-series-reasoning-suite-tsr-suite