Unified Single-Series Sequence (S3) Overview

• Unified Single-Series Sequence (S3) is a framework that converts heterogeneous, multivariate time series data into a homogeneous sequence of fixed-length tokens.
• It employs segment-wise normalization, time-feature embeddings, and a decoder-only Transformer architecture to enable next-token prediction for tasks like forecasting and anomaly detection.
• Additionally, S3 extends to combinatorial and arithmetic contexts, unifying progression-free sequences and subsequence divergence properties in conditionally convergent series.

The term "Unified Single-Series Sequence" (S3) refers to several foundational constructs united by the theme of sequence transformation and constraint. In contemporary time series modeling, S3 signifies a format that converts arbitrarily heterogeneous, multivariate, and irregular time-series collections into a homogeneous sequence of fixed-length real-valued tokens, facilitating unified generative learning via large-scale decoder-only Transformers (Liu et al., 2024). In parallel, S3 denotes critical combinatorial and arithmetic structures: it is the canonical greedy sequence avoiding three-term arithmetic progressions (the Szekeres sequence) (Tseng, 2011), and it underpins the major theorem in analysis guaranteeing that for any three conditionally convergent real series, a single subsequence can force all partial sums to diverge one-sidedly (Brian, 2018). These instances of S3 are cornerstones for unification in time series machine learning, extremal combinatorics, and additive number theory.

1. Formal Definition and Construction in Time Series Modeling

In the context of large time series models (LTSMs) such as Timer, an S3 is rigorously defined as follows. Given a diverse set of time series—varying in length, sampling frequency, modality, and number of variates—Timer extracts all univariate subsequences, normalizes each via the train split’s mean μ and standard deviation σ, and aggregates these into a pooled collection (Liu et al., 2024). Each S3 instance is created by uniformly sampling a window of length $N·S$ from this pool, where $N$ is the number of tokens and $S$ the segment length in time points: $$X = \{x_1, x_2, \ldots, x_{N·S}\} \in \mathbb{R}^{N·S}$$ This window is then split into $N$ segments (tokens) of length $S$: $$s_i = \{ x_{(i-1)·S+1}, \ldots, x_{i·S} \} \in \mathbb{R}^S,\quad i = 1,\ldots,N$$ The result is a single-series sequence encoding $N·S$ points as an ordered sentence of $N$ tokens, each summarizing $S$ consecutive, real-valued samples. S3 construction dispenses with any requirement for temporal or channel alignment, enabling joint modeling across highly heterogeneous sets.

2. Data Preprocessing, Tokenization, and Embedding

Data preprocessing for S3 involves segment-wise normalization, reinforcement of time context, and randomization to counteract dataset-specific biases (Liu et al., 2024). Specifically:

• Normalization: Each univariate series is standardized using its own train split.
• Time-Feature Embeddings: Each segment index $i$ is linked to a D-dimensional "time embedding" $TE_i$, encoding periodic cues such as hour, day, or timestamp.
• Segment Embedding and Initialization: Each segment $s_i$ is mapped into a D-dimensional space via a learned matrix $W_e \in \mathbb{R}^{D \times S}$, yielding initial hidden states $h^0_i=W_e s_i + TE_i$.
• Causal Batch Mixing: Sampling segments across unrelated series and timestamps ensures the model learns solely from temporal variation and precludes alignment artifacts.

This pipeline transforms aggregate, irregular time series data into a homogeneously embedded sequence, harmonizing the input structure for efficient Transformer-based learning.

3. Unified Generative Objective: Next-Token Prediction

S3 enables all downstream time series analysis tasks—forecasting, segment-level imputation, and anomaly detection—to be recast as autoregressive next-token prediction (Liu et al., 2024). The probabilistic model is defined over the S3 token sequence via a decoder-only Transformer: $$P(s_1, \ldots, s_N) = \prod_{i=1}^N p(s_i \mid s_{<i})$$ The Transformer, after $L$ layers, generates predictions of the next real-valued segment as $\hat{s}_{i+1} = h^L_i W_d$ with $W_d \in \mathbb{R}^{D \times S}$. The universal training objective is the mean-squared error (MSE) over segment reconstructions: $$L_{pretrain} = \frac{1}{NS} \sum_{i=2}^{N+1} \| s_i - \hat{s}_i \|_2^2$$

• Forecasting: The model recursively predicts future segments conditioned on prior context, achieving multi-step prediction via repeated generation.
• Imputation: Random masking and denoising (by autoregressive prediction of masked segments) reduces missing-data errors.
• Anomaly Detection: The per-segment MSE between prediction and observation supplies an unsupervised anomaly score, with lower quantiles on UCR datasets closely tracking ground-truth anomalies.

All tasks thus become instances of conditional next-token regression under S3, with identical model structure and loss.

4. Architectural Implications and Scalability

Timer’s GPT-style decoder-only architecture is optimized for S3, adhering to the principles of causal inference, flexible context handling, and domain generalization (Liu et al., 2024):

• Maintains token-level supervision and gradient flow.
• Accepts arbitrary context lengths, obviating the need for encoder flattening or fixed input size.
• Employs causal self-attention, strictly restricting output dependency to strictly prior segments.
• Incorporates timestamp embeddings for calendar-aware modeling.
• Demonstrates consistent performance gains as model size scales: increasing depth or dimensionality reduces forecasting MSE by 20–40% in few-shot settings. This design ensures that the S3-Transformer pipeline robustly leverages billion-point pre-training across dozens of domains—without bespoke reengineering of task- or data-specific pipelines.

5. Empirical Results: Fine-Tuning and Generalization

S3’s abstraction of sampling irregularities and channel diversity catalyzes unified pre-training on vast and diverse corpora. Notable empirical gains (Liu et al., 2024):

• On the ETTh1 forecasting benchmark, Timer pretrained on UTSD-12G and fine-tuned with only 1% of downstream data achieves MSE=0.362, outperforming both from-scratch models (MSE=0.426) and previous SOTA PatchTST (MSE=0.370).
• For segment-level imputation (5% samples, 25% segments masked), Timer using S3 drops imputation error by up to 17.7% versus from-scratch TimesNet.
• Predictive anomaly detection using S3 locates anomalies in notably lower quantiles than models trained from scratch, sometimes halving the quantile.
• Predictable scaling: doubling decoder layers or hidden size under S3 pre-training systematically lowers few-shot forecast MSE, confirming that large-model pre-training synergizes with the S3 encoding.

These results underline S3’s centrality as a data-format and learning primitive, unifying a wide spectrum of time series tasks and domains.

6. S3 in Additive Number Theory and Series Analysis

The notation $S_3$ also signifies fundamental entities in additive combinatorics and analysis:

• Three-Term Nonaveraging Sequences: The Szekeres $S_3$ is the sequence of nonnegative integers whose base-3 expansion lacks the digit “2.” This sequence is constructed greedily to avoid any three-term arithmetic progression; equivalently, no $x_1,x_2,x_3 \in S_3$ satisfy $x_1 + x_2 = 2x_3$ (Tseng, 2011). Explicitly,

$$n \in S_3 \iff n = \sum_{i=0}^\infty t_i 3^i, \quad t_i \in \{0,1\}$$

This greedy avoidance sets $S_3$ apart for maximal density among progression-free sets with tractable closed-form and asymptotic growth $|S_3 \cap [0,N)| = \Theta(N^{\log_3 2})$.

• Subseries Extraction in Conditionally Convergent Series: For any three conditionally convergent real sequences, there always exists a single subsequence along which all three partial sums diverge one-sidedly (to $+\infty$ or $-\infty$), but this fails for four series (Brian, 2018). The set $A$ of indices achieves a simultaneous "S3" extraction, representing a sharp threshold for multidimensional divergence control in analysis. The construction uses sign-pattern partitions, tame set classification, and combinatorial lemmas to ensure this simultaneous divergence property.

7. S3 as a Unification Principle

Across these domains, S3 serves as a unifying framework:

• In time series modeling, it provides an architecture- and task-invariant carrier of temporal information, enabling GPT-style Transformers to handle forecasting, imputation, and anomaly detection by the same generative learning protocol.
• In extremal combinatorics and series analysis, S3 codifies maximal avoidance and control principles—whether in density-optimal sequences with forbidden patterns or in the simultaneous divergence of conditionally convergent series.

The S3 paradigm thus exemplifies the power of reformatting and subsequence selection for unifying heterogeneous problems, supporting both large-scale empirical modeling (Liu et al., 2024), and deep theoretical results in arithmetic combinatorics (Tseng, 2011, Brian, 2018).