RATs40K Anomaly Reasoning Dataset

• RATs40K is a comprehensive multimodal dataset for time series anomaly reasoning that integrates numeric, textual, and visual data across 10 distinct domains.
• The dataset comprises 39,574 samples featuring both univariate and multivariate time series with 14 univariate and 6 multivariate anomaly types, supported by detailed, ensemble-generated explanations.
• It enables rigorous benchmarking of LLMs and multimodal foundation models through structured anomaly categorization, fine-grained reasoning, and robust supervised fine-tuning protocols.

RATs40K is a large-scale, real-world multimodal dataset for Time Series Anomaly Reasoning, explicitly designed to enable generative, fine-grained, explanation-intensive benchmarking of both LLMs and multimodal foundation models in time series anomaly detection and reasoning (Yang et al., 20 Jul 2025).

1. Dataset Scope and Structure

RATs40K (“Reasoning for Anomaly in Time series 40K”) constitutes the first real-world multimodal benchmark dataset explicitly annotated for anomaly reasoning in time series. It contains 39,574 samples spanning 10 distinct domains:

• AIOps systems
• Environment
• Finance
• Healthcare
• IoT
• Industrial sensors
• Server data
• Traffic
• Network records
• Synthetic datasets

Modalities present for each sample include numeric time series ($T$), contextual text ($D$), and visual representation ($V$), supporting both univariate and multivariate inputs. Segment lengths are distributed between 16 and 128 time steps, with an overall anomaly ratio of 83.7%. Annotations include detailed structured explanations generated via ensemble LLM methods and refined by rigorous AI (GPT-4) feedback. The average explanation (“thought”) length is 101.378 tokens.

There are 36,311 univariate and 3,274 multivariate annotated instructions. The dataset's multimodal nature, combined with real and synthetic sources, supports benchmarking models on diverse and heterogeneous anomaly scenarios.

2. Multimodal Data Representation

Each sample in RATs40K is represented as a triplet: $\{T, D, V\}$ where:

• $T$: Numeric time series (either univariate or multivariate)
• $D$: Contextual text (domain/application description and, for multivariate, feature/variable definitions)
• $V$: Visual plot (e.g., a rendered chart or image of the sequence)

This structure is central to evaluating both text-only LLMs and multimodal/vision-LLMs (MLLMs), enabling cross-modal reasoning and facilitating model interpretation.

3. Fine-Grained Anomaly Taxonomy

RATs40K employs an explicit, literature-informed anomaly categorization:

• Univariate anomaly taxonomy: 14 anomaly types (plus “Normal Sequence”), including Point Anomaly, Periodic Change, Trend Change, Change Point, Distributional Change, Amplitude Anomaly, Pattern Change, Sparse Anomaly, Repeated Value, Sudden Flatline, Drift, Sudden Spike, Continuous Segment, and Nonlinear Pattern anomalies. Each is defined by formal behavior in the temporal domain.
• Multivariate anomaly taxonomy: 6 anomaly types (plus “Normal Sequence”), including Covariance Structure, Temporal Dependency, Trend Divergence, Joint Space, Principal Component Space, and Collinearity Shift anomalies. These definitions incorporate inter-channel relations, joint distributions, and latent-space effects.

This taxonomic framework enables not just binary detection, but classification into specific anomaly categories, as well as demand for structured causal or statistical explanation.

4. Annotation, Feedback, and Quality Control Pipeline

RATs40K's labeling process consists of a sophisticated multi-stage pipeline:

1. Data Collection: Source segmentation, context attachment, and initial normal/anomaly label inheritance from base datasets.
2. Ensemble-LLM Label Sampling: A pool of four LLMs (gpt-4o_2024-11-20, gemini-2.5-flash, DeepSeek-R1, Llama-3.3-70B-Instruct) generates candidate “thought” explanations and “action” category selections for each segment conditioned on modality and known anomaly presence.
3. Feedback Ranking and Critique: GPT-4 operates as both an evaluator (Likert 1–5 quantitative scoring and ranking) and critique generator. Its functions include:
   • Scalar and preference-based output ranking
   • Justification and detailed textual feedback
   • Rewriting of explanations and categories as appropriate The highest-ranked and optionally refined output constitutes the dataset label.

Validation and QA mechanisms include:

• Expert alignment studies: Subset of samples evaluated for consistency between GPT-4 labeling and human expert judgments.
• Five-dimension Likert evaluation of reasoning: Language Quality, Factual Soundness, Specificity, Interpretability, Usefulness (mean univariate scores: 4.04–4.58, multivariate: 4.08–4.28).

5. Structured Reasoning and Task Definition

RATs40K is purpose-built for the Time-RA (Time-series Reasoning for Anomaly) task, which moves beyond standard binary or classification-based timeseries anomaly detection to a generative, interpretable framework. For each input sample $(T, D, V)$, the model is expected to yield a triplet:

• Binary anomaly label ($y_l \in \{0,1\}$)
• Fine-grained anomaly category ($a$, univariate or multivariate taxonomy)
• Structured explanation ($D$0, the “thought”)

The full target is $D$1, with the supervised fine-tuning (SFT) objective:

$D$2

The prompt template enforces a three-step workflow: Observation ($D$3), Thought ($D$4, boxed as $D$5), and Action ($D$6, boxed as $D$7). Detailed schema and examples are provided, including ECG case studies for both univariate and multivariate settings.

6. Benchmark Results and Baseline Comparisons

RATs40K enables systematic multi-task benchmarking for LLMs and MLLMs. The principal findings are:

• Binary detection: Models such as Qwen2.5-3B (F1 = 0.9000 for univariate; after SFT, Qwen2.5-7B reaches F1 = 0.8544 for multivariate) match or approach the performance of classic discriminative baselines (e.g., XGBoost, LightGBM, LSTM).
• Fine-grained class (ActionID) matching: All models exhibit much lower F1 (univariate ≈ 0.09; multivariate ≈ 0.166), indicating that detailed anomaly categorization is substantially harder than detection.
• Reasoning (Thought) similarity: Cosine, TF-IDF, Levenshtein, and token-sequence similarity metrics indicate partial semantic alignment but lower structural/literal score averages (univariate best ≈ 0.4122; multivariate ≈ 0.4632).
• Modality effects: MLLMs (e.g., Llava-v1.5, Qwen2.5-VL) show utility for tasks involving visual input, with optimal settings being task- and model-dependent.
• SFT (Supervised Fine-Tuning): SFT consistently enhances ranking, classification, and reasoning quality for all evaluated architectures.
• Transferability: SFT models Fine-tuned on RATs40K evidence strong transfer to held-out and out-of-domain datasets in both univariate and high-dimensional multivariate settings.
• Comparison to unsupervised TSAD: LLMs outperform traditional unsupervised baselines, but do not always surpass strongest discriminative supervised models in binary F1.

7. Experimental Protocols and Usage Considerations

Evaluation in RATs40K employs regular-expression-based extraction for predicted “thought” and “action,” with separate metrics for binary label, class (ID), and reasoning string. Supervised fine-tuning is performed using LoRA for parameter efficiency, and all data is formatted per a fixed, reproducible instruction/prompt template. While the dataset offers extensive benchmarking support and in- and out-of-domain transfer assessment, detailed experimental hyperparameters (such as splits, seeds, or hardware details) are not fully specified in the main text, which may impact exact reproducibility.

Key Metrical Dimensions

Task component	Best Model(s)	Top Metric	Interpretation
Binary detection	Qwen2.5-3B, 7B	F1 ≈ 0.90, 0.85	Near supervised baseline
ActionID	All	F1 ≈ 0.09–0.17	Remains challenging
Thought similarity	All	0.4–0.46 (avg)	Partial alignment

Multimodal variant benchmarks show that the optimal use of tabular, textual, or visual channels is architecture- and setting-dependent.

8. Significance and Distinctive Features

RATs40K introduces a paradigm shift in time series anomaly detection by:

• Enabling fine-grained, explanation-aware anomaly categorization
• Supporting multimodal benchmarking (time series, text, images) across highly heterogeneous real-world domains
• Providing high-quality, interpretable annotations via ensemble LLM generation, GPT-4 ranking/critique, and human expert validation
• Demonstrating the present capabilities and limits of LLMs and MLLMs in structured diagnostic reasoning, category discovery, and generalization

The combination of diverse anomaly taxonomies, formal explanation requirements, multimodality, and rigorous annotation quality controls makes RATs40K a de facto gold standard for future research on interpretable, generative, and foundation-model-based time series anomaly analysis (Yang et al., 20 Jul 2025).