---
title: SDForger Synthetic Time Series Framework
url: https://www.emergentmind.com/topics/sdforger
type: topic
---

# SDForger Synthetic Time Series Framework

SDForger is a framework for generating synthetic multivariate time series utilizing large language models (LLMs) with a compact tabular embedding that enables efficient, high-fidelity data synthesis. The central innovation is transforming both univariate and multivariate time series into textualized embeddings, making it possible to fine-tune autoregressive LLMs—even with limited computational resources and few real instances—and conditionally generate new samples that preserve the target data’s statistical and temporal structure [2505.17103].

## 1. Problem Formulation and Evaluation Metrics

The objective scenario assumes observed multivariate time series 
\[
X = \{ X_i \}_{i=1}^I, \quad X_i \in \mathbb{R}^{C \times L}
\]
with $C$ channels and sequences of length $L$. The generative task is to create $\tilde{I}$ new instances
\[
\tilde X = \{ \tilde X_j \}_{j=1}^{\tilde I}, \quad \tilde X_j \in \mathbb{R}^{C \times L}
\]
matching the joint distribution of $X$.

**Distributional similarity** is quantified primarily via:
- **Marginal Distribution Difference (MDD):**
  \[
  \text{MDD} = \frac{1}{CL} \sum_{c=1}^C \sum_{t=1}^L \left| F_{X^c}(x_t) - F_{\tilde X^c}(\tilde x_t) \right|
  \]
  where $F$ denotes empirical CDFs.
- **Autocorrelation Difference (ACD):**
  \[
  \text{ACD} = \frac{1}{C} \sum_{c=1}^C \sum_{\tau=1}^{\tau_{\max}} \left| \rho_{X^c}(\tau) - \rho_{\tilde X^c}(\tau) \right|
  \]
  with $\rho(\tau)$ the autocorrelation.

Additional **distance-based metrics** employed include Euclidean Distance (ED), Dynamic Time Warping (DTW), and shapelet-based reconstruction error (SHR).

**Forecasting accuracy** is evaluated by training predictive models—such as Tiny Time Mixer (TTM)—on generated data, real data, or combinations. Performance is measured with standard error metrics:
\[
\mathrm{RMSE} = \sqrt{ \frac{1}{N} \sum_{i=1}^N (\hat y_i - y_i)^2 }, \quad
\mathrm{MAPE} = \frac{100}{N} \sum_{i=1}^N \left| \frac{\hat y_i - y_i}{y_i} \right|
\]
This multi-faceted evaluation benchmarks both statistical similarity and utility for downstream forecasting [2505.17103].

## 2. Embedding Construction and Data Representation

Each channel $c$ is modeled as a real-valued function $X_i^c(t)$ over $[0, L]$. Key steps in constructing embeddings are:

- **Functional Basis Projection**: Basis functions $\{b_j^c(t)\}_{j=1}^{k_c}$ (usually via Functional PCA or FastICA) are selected per channel. Embedding coefficients
  \[
  e_{ij}^c = \int_0^L X_i^c(t) b_j^c(t) \, dt, \quad j = 1, \dots, k_c
  \]
  are concatenated over channels to form an embedding vector
  \[
  E_i = (e_{i1}^1, \ldots, e_{ik_1}^1; e_{i1}^2, \ldots, e_{ik_2}^2; \ldots) \in \mathbb{R}^K
  \]
  with $K = \sum_{c=1}^{C} k_c$.

- **Normalization**: Each sequence is mean-standardized at each timestamp ($X_i^c(t) \gets (X_i^c(t) - \mu_t)/\sigma_t$), where $\mu_t, \sigma_t$ are computed across all instances.

- **Tokenization**: Each real-valued coefficient is mapped to a decimal string in $[a, b]$ (e.g., “0.1234”) and tokenized using the LLM’s vocabulary, relying solely on the model's native BPE or byte-level encoding.

No further quantization is introduced beyond the parameterization defined by the basis decomposition and LLM tokenizer.

## 3. LLM Fine-Tuning and Prompt Engineering

- **Prompt Structure**: Each embedding vector $E_i$ is permuted via a random permutation $\pi$ and converted into a fill-in-the-middle text prompt:
  \[
  \text{Input: } \bigcirc_{k=1}^K [\text{value}_{\pi(k)} \text{ is [blank],}] [\text{sep}] \, \text{Target: } \bigcirc_{k=1}^K [ e_{i,\pi(k)} \text{ [answer]} ]
  \]
  where $\bigcirc$ denotes string concatenation. Randomization over $\pi$ mitigates position bias.

- **Optimization Objective**: LLM parameters $\theta$ are optimized to minimize cross-entropy over the combined prompt+target sequence $(\tau_1, \dots, \tau_T)$:
  \[
  \mathcal{L}(\theta) = -\sum_{t=1}^T \log p_{\theta} ( \tau_t | \tau_{<t} )
  \]
  
- **Low-Rank Adapters (LoRA)**: For efficient adaptation, LoRA adapters are inserted within each Transformer block:
  \[
  W \leftarrow W_0 + \Delta W, \quad \Delta W = AB, \quad A \in \mathbb{R}^{d \times r}, B \in \mathbb{R}^{r \times d},\ r \ll d
  \]
  Only $A$ and $B$ are tuned, with $W_0$ frozen.

This workflow enables SDForger to adapt general-purpose LLMs with a minimal number of embedding instances, often as low as $I = 15$–$30$ rows, for high-quality time series synthesis [2505.17103].

## 4. Synthetic Sequence Sampling and Inverse Mapping

- **Autoregressive Sampling**: At inference, SDForger populates a template with $K$ blanks and prompts the LLM to sample, stepwise, in token space by top-$k$ (fixed $k$ highest probability tokens) or nucleus (top-$p$ cumulative probability) selection.

- **Inverse Decoding**: Generated text $\mathcal{G}$ is parsed to extract feature–value pairs $\{ (\pi(k), \tilde a_{\pi(k)}) \}$, which are mapped back to the real domain by reconstructing the original channels using the corresponding basis expansion:
  \[
  \tilde X^c(t) = \sum_{j=1}^{k_c} \tilde e^c_j b^c_j(t)
  \]
  Concatenation across all channels yields synthetic $ \tilde X \in \mathbb{R}^{C \times L} $.

- **Post-processing Filters**: Outputs are filtered to remove NaNs, duplicates, or $\ell_2$-norm outliers before acceptance.

This process yields synthetic sequences with statistics and dynamics aligned to the original distribution.

## 5. Statistical Fidelity and Downstream Use

SDForger’s fidelity is established by:
- **Feature-based metrics**: MDD (marginal), skewness, kurtosis, and ACD (autocorrelation differences).
- **Distance-based metrics**: Euclidean distance, DTW, and SHR via shift-invariant dictionary learning.
- **Multivariate Structure**: Cross-covariances between channels and time-lagged dependencies are computed:
  \[
  \operatorname{Cov}( \tilde X^c(t), \tilde X^{c'}(t+\tau) ) \approx \operatorname{Cov}( X^c(t), X^{c'}(t+\tau) )
  \]
  
For downstream forecasting, TTM is trained on (1) zero-shot, (2) real only, (3) synthetic only, or (4) real+synthetic regimes, with accuracy quantified on held-out test sets (RMSE, MAPE, MASE). Results show synthetic data from SDForger alone often matches utility of real data, with mixed training yielding further gains.

## 6. Comparative Perspective and Algorithmic Features

Compared to GANs and VAEs, SDForger:
- Leverages LLM adaptation via textual prompts, not from-scratch training.
- Handles long sequences in $O(K)$ time, agnostic to sequence length $L$.
- Supports few-shot adaptation with minimal data.
- Enables textual conditioning for semantic or channel-specific control (e.g., “Condition: data is temperature”).

Algorithmic summaries:
- **Fine-tuning**:
  ```python
  # Input: Real time series X, basis {b^c_j}, LLM θ₀.
  # 1. Segment into windows X_i.
  # 2. Compute embeddings e_{ij}^c = ∫ X_i^c(t) b_j^c(t) dt.
  # 3. Form embedding table E∈ℝ^{I×K}.
  # 4. Construct prompts P_i^{FT} (fill-in-the-middle).
  # 5. Fine-tune θ on {P_i^{FT}}, minimizing cross-entropy.
  # Output: Fine-tuned θ*.
  ```

- **Generation**:
  ```python
  # Input: θ*, desired sample count Ũ.
  # while |generated| < Ũ:
  #     a. Sample K-blanks template P^{INF}.
  #     b. Autoregressively sample tokens via top-k/top-p.
  #     c. Parse text → \tilde e ∈ ℝ^K.
  #     d. Filter outputs (NaNs, duplicates, l2 outliers).
  #     e. Decode \tilde e → \tilde X via basis sum.
  #     f. Append \tilde X to outputs.
  # Output: { \tilde X_j }_{j=1}^{Ũ}.
  ```

## 7. Constraints, Extensions, and Outlook

**Limitations** include the need for moderate embedding dimension $K$ (e.g., $K \leq 25$) relative to sample size to prevent overfitting and unstable synthesis. Excessive $K$ slows LLM convergence and degrades sample quality.

**Potential extensions**:
- Encoder-only LLMs (e.g., BERT) with masking for imputation.
- Adaptive data-driven basis selection ($k_c$ per channel).
- Joint time series–text pretraining for richer multimodal generation (e.g., incorporating event annotations).
- Feeding synthetic data back into LLM pretraining to enhance in-context forecasting ability [2505.17103].

SDForger establishes a workflow for high-fidelity, few-shot, multimodally conditioned synthetic time series generation, leveraging the infrastructure and text reasoning of modern LLMs.

Source: https://www.emergentmind.com/topics/sdforger