CovSynth: Covariate Synthesis for TSFMs

Updated 4 July 2026

CovSynth is a covariate synthesis method that transforms target-only series into covariate-enriched training samples using STL decomposition.
It generates event, long-term, and periodic covariates from residuals, trends, and seasonality to mimic realistic exogenous inputs.
Integrated with CITRAS-FM, CovSynth improves forecasting accuracy and real-time efficiency by enabling zero-shot covariate-informed predictions.

CovSynth is a covariate synthesis procedure designed to make covariate-aware pretraining feasible and effective for time series foundation models (TSFMs) despite the scarcity of covariate-rich corpora. In the formulation introduced with CITRAS-FM, it builds realistic, diverse, and target-aligned pseudo-covariates directly from decomposed components of the target series, and it is the key ingredient that exposes the model to covariate-informed forecasting during pretraining, so that at inference time it can exploit real observed and known covariates zero-shot (Yamaguchi et al., 9 Jun 2026).

1. Scope, motivation, and problem setting

CovSynth is motivated by the observation that many real-world forecasting tasks are covariate-informed. Exogenous inputs such as calendar variables, planned events, weather forecasts, and grid-load forecasts materially affect the target’s evolution, and a TSFM must learn to condition on such signals to produce accurate forecasts, especially when known covariates are available throughout the forecast horizon (Yamaguchi et al., 9 Jun 2026).

The central difficulty is that the largest publicly available pretraining corpora are predominantly target-only or have limited covariate diversity. Without covariates during pretraining, TSFMs learn to forecast as if targets are isolated, and often underutilize covariates at test time. CovSynth addresses this by turning any target-only series into a covariate-enriched training instance through decomposition of the target into interpretable components and generation of pseudo-covariates that are described as realistic, diverse, and aligned (Yamaguchi et al., 9 Jun 2026).

The formal forecasting setup distinguishes three variable groups. The target consists of $C_\tau$ variables, with target $\mathbf{X}_{1:T}^{\tau,:} \in \mathbb{R}^{T\times C_\tau}$ , observed covariates $\mathbf{X}_{1:T}^{o,:} \in \mathbb{R}^{T\times C_o}$ available up to time $T$ , and known covariates $\mathbf{X}_{1:T+S}^{\kappa,:} \in \mathbb{R}^{(T+S)\times C_\kappa}$ available through the forecast horizon. The objective is to estimate

$P(\mathbf{X}_{T+1:T+S}^{\tau,:} \mid \mathbf{X}_{1:T}^{\tau,:}, \mathbf{X}_{1:T}^{o,:}, \mathbf{X}_{1:T+S}^{\kappa,:}).$

Within this setup, CovSynth is specifically a pretraining mechanism rather than an inference-time imputation scheme. It enriches pretraining with covariate-informed examples even when only targets are available natively, and its role is therefore architectural and distributional rather than post hoc (Yamaguchi et al., 9 Jun 2026).

2. Decomposition-based formulation

CovSynth begins from seasonal-trend decomposition via LOESS (STL). For each univariate target $y_t$ , $t=1,\dots,T$ , the series is decomposed as

$y_t = T_t + S_t + R_t,$

where $T_t$ is the smooth trend, $\mathbf{X}_{1:T}^{\tau,:} \in \mathbb{R}^{T\times C_\tau}$ 0 the seasonal component at the dataset’s intrinsic frequency, and $\mathbf{X}_{1:T}^{\tau,:} \in \mathbb{R}^{T\times C_\tau}$ 1 the residual variability unexplained by $\mathbf{X}_{1:T}^{\tau,:} \in \mathbb{R}^{T\times C_\tau}$ 2 and $\mathbf{X}_{1:T}^{\tau,:} \in \mathbb{R}^{T\times C_\tau}$ 3 (Yamaguchi et al., 9 Jun 2026).

The decomposition plays two roles. First, it separates nonstationary trend from seasonal dynamics, which is useful because CITRAS-FM applies patch-wise causal scaling to inputs to mitigate nonstationarity without leakage. Second, it localizes the exogenous proxy construction primarily in the residual $\mathbf{X}_{1:T}^{\tau,:} \in \mathbb{R}^{T\times C_\tau}$ 4, so that the synthesized covariates carry information complementary to the target’s own trend and seasonality rather than redundantly reproducing them (Yamaguchi et al., 9 Jun 2026).

This design enforces three statistical properties. Cross-correlations arise because the synthesized covariates are derived from $\mathbf{X}_{1:T}^{\tau,:} \in \mathbb{R}^{T\times C_\tau}$ 5 and mixed with $\mathbf{X}_{1:T}^{\tau,:} \in \mathbb{R}^{T\times C_\tau}$ 6 or $\mathbf{X}_{1:T}^{\tau,:} \in \mathbb{R}^{T\times C_\tau}$ 7. Seasonality alignment is explicit because periodic covariates include $\mathbf{X}_{1:T}^{\tau,:} \in \mathbb{R}^{T\times C_\tau}$ 8, ensuring phase-aligned periodicity with the target. Trend coupling is explicit because long-term covariates include $\mathbf{X}_{1:T}^{\tau,:} \in \mathbb{R}^{T\times C_\tau}$ 9, capturing slow co-movement. The mechanism therefore does not treat covariates as arbitrary perturbations; it constructs them from structured target components under sampled thresholds, mixing weights, and noise scales (Yamaguchi et al., 9 Jun 2026).

The paper states that STL hyperparameters follow standard practice and are chosen to be robust, but it does not enumerate specific parameter values in order to keep CovSynth broadly applicable. It also notes an important edge case: highly irregular series or regime shifts may prevent STL from cleanly separating components, in which case residuals may be dominated by structural breaks and the synthesized covariates may become less representative (Yamaguchi et al., 9 Jun 2026).

3. Covariate classes and synthesis mechanism

The core principle of CovSynth is to use residual variability $\mathbf{X}_{1:T}^{o,:} \in \mathbb{R}^{T\times C_o}$ 0 to emulate external signals that influence the target but are not explained by its own trend or seasonality. It constructs three covariate classes (Yamaguchi et al., 9 Jun 2026).

Covariate class	Construction	Intended effect
Event covariate	$\mathbf{X}_{1:T}^{o,:} \in \mathbb{R}^{T\times C_o}$ 1	Mimics short, irregular exogenous shocks
Long-term covariate	$\mathbf{X}_{1:T}^{o,:} \in \mathbb{R}^{T\times C_o}$ 2	Captures long-run exogenous influences that couple to trend
Periodic covariate	$\mathbf{X}_{1:T}^{o,:} \in \mathbb{R}^{T\times C_o}$ 3	Emulates exogenous seasonally structured drivers aligned with the target’s seasonality

The event covariate is binary and sparse because the threshold is a residual quantile. The long-term and periodic covariates are linear mixtures with additive noise, with mixing weights and noise sampled at training time. CovSynth can also produce multiple covariates at once. For $\mathbf{X}_{1:T}^{o,:} \in \mathbb{R}^{T\times C_o}$ 4, a synthetic covariate is

$\mathbf{X}_{1:T}^{o,:} \in \mathbb{R}^{T\times C_o}$ 5

where $\mathbf{X}_{1:T}^{o,:} \in \mathbb{R}^{T\times C_o}$ 6 instantiates one of the constructions above, with parameters $\mathbf{X}_{1:T}^{o,:} \in \mathbb{R}^{T\times C_o}$ 7 sampled at training time (Yamaguchi et al., 9 Jun 2026).

Observed and known covariates are synthesized separately. At each pretraining step, CovSynth synthesizes up to five observed covariates and five known covariates by independently sampling parameters $\mathbf{X}_{1:T}^{o,:} \in \mathbb{R}^{T\times C_o}$ 8. Observed covariates end at $\mathbf{X}_{1:T}^{o,:} \in \mathbb{R}^{T\times C_o}$ 9, whereas known covariates continue to $T$ 0. Practically, CovSynth generates the full covariate sequences first and the model handles alignment via attention shifting (Yamaguchi et al., 9 Jun 2026).

The paper explicitly contrasts this mechanism with proxy or noise augmentation. Pure noise or random transforms rarely capture realistic cross-correlations or seasonality-phase alignment with targets. CovSynth instead couples covariates to residual variation and to trend and seasonal components, yielding structured signals the model can learn to exploit. It is also described as complementary to structural-causal-model-based synthesis such as Cauker: structural causal synthesis builds multivariate ecosystems, whereas CovSynth upgrades any target-only sequence into covariate-rich training samples with minimal assumptions and computational overhead (Yamaguchi et al., 9 Jun 2026).

The algorithmic steps are direct. Given a raw univariate target $T$ 1, CovSynth computes $T$ 2 by STL, samples thresholds, mixing weights, and noise scales, decides covariate types and whether each is observed or known, constructs event, long-term, and periodic covariates, optionally applies sanity checks such as sparsity or variance bounds, patchifies target and covariates with causal scaling, and then trains with next-token prediction over all patches (Yamaguchi et al., 9 Jun 2026).

4. Integration with CITRAS-FM

CovSynth is tightly integrated into CITRAS-FM, a tiny 7.2M-parameter, patch-based, decoder-only Transformer with four modules: Input Projection, Cross-Time Attention, Cross-Variate Attention, and Output Projection. The model supports univariate, multivariate, and covariate-informed forecasting with real-time CPU inference (Yamaguchi et al., 9 Jun 2026).

Input processing is patch-based with patch length $T$ 3. For target variable $T$ 4, patches are causal-scaled and embedded, and the same embedding is applied to observed and known covariates. Cross-Time Attention is applied per variable with causal masking. The architectural novelty relevant to CovSynth appears in Cross-Variate Attention:

$T$ 5

so that known covariate tokens are left-shifted one patch and $T$ 6 is directly attendable by the target token $T$ 7 (Yamaguchi et al., 9 Jun 2026).

This “Shifted Attention” is crucial because known covariates must be accessible one patch ahead. CovSynth generates full known-covariate sequences through the forecast horizon, and Shifted Attention operationalizes the semantics that horizon-known exogenous inputs should be directly available to the target at the next prediction step. The system therefore combines an overview mechanism at the data level with an alignment mechanism at the attention level (Yamaguchi et al., 9 Jun 2026).

Pretraining uses TSMixup, the Cauker-generated multivariate dataset, and the Gift-Eval pretraining dataset, sampled with equal probability. The reported training setup is: patch size $T$ 8, model dimension $T$ 9, 8 heads, cross-time/cross-variate stacks $\mathbf{X}_{1:T+S}^{\kappa,:} \in \mathbb{R}^{(T+S)\times C_\kappa}$ 0, $\mathbf{X}_{1:T+S}^{\kappa,:} \in \mathbb{R}^{(T+S)\times C_\kappa}$ 1, repeated $\mathbf{X}_{1:T+S}^{\kappa,:} \in \mathbb{R}^{(T+S)\times C_\kappa}$ 2, max input length 1032, batch size 256, 500k steps, AdamW with learning rate $\mathbf{X}_{1:T+S}^{\kappa,:} \in \mathbb{R}^{(T+S)\times C_\kappa}$ 3, weight decay $\mathbf{X}_{1:T+S}^{\kappa,:} \in \mathbb{R}^{(T+S)\times C_\kappa}$ 4, and warmup 10k steps then cosine, using a single V100 (32GB) GPU (Yamaguchi et al., 9 Jun 2026).

The output head is probabilistic: for each target patch it outputs 9 quantiles $\mathbf{X}_{1:T+S}^{\kappa,:} \in \mathbb{R}^{(T+S)\times C_\kappa}$ 5 per time step, trained with the quantile pinball loss

$\mathbf{X}_{1:T+S}^{\kappa,:} \in \mathbb{R}^{(T+S)\times C_\kappa}$ 6

Pretraining uses causal masks and autoregressive next-patch prediction; when $\mathbf{X}_{1:T+S}^{\kappa,:} \in \mathbb{R}^{(T+S)\times C_\kappa}$ 7, forecasts roll out autoregressively (Yamaguchi et al., 9 Jun 2026).

5. Empirical behavior, practical guidance, and limitations

On fev-bench, spanning 100 tasks across various settings, CITRAS-FM achieves fev-all 41.2%, fev-cov 39.0%, fev-multi 54.2%, and fev-uni 31.3%. It is reported as achieving top skill scores among sub-10M models, competitive on covariate-informed tasks, and outperforming models with more than 20 times as many parameters on fev-cov (Yamaguchi et al., 9 Jun 2026).

The most direct empirical evidence for CovSynth itself comes from ablation. Removing CovSynth drops fev-cov skill score from 39.0% to 37.2%, a reduction of 1.8 points. The paper interprets this as confirming that covariate-aware pretraining via CovSynth meaningfully boosts covariate-informed zero-shot forecasting (Yamaguchi et al., 9 Jun 2026).

The same study links this to deployment efficiency. CITRAS-FM is reported to deliver sub-0.1-second CPU inference per window, with 0.05 sec in the Application dataset. The broader claim is that CovSynth enables a tiny model to leverage covariates effectively without requiring large covariate-rich corpora, thereby contributing to a balance between forecasting accuracy and real-time deployability (Yamaguchi et al., 9 Jun 2026).

Several practical recommendations are explicit. STL is recommended for robustness, although other decompositions such as moving averages or classical seasonal filters can be substituted if STL is unavailable, provided the separation $\mathbf{X}_{1:T+S}^{\kappa,:} \in \mathbb{R}^{(T+S)\times C_\kappa}$ 8 is obtained. For parameterization, event covariates should use a mid-to-high quantile range for sparse spikes, mixing weights should be sampled from moderate ranges to avoid trivial collinearity, and small additive noise should be included to ensure diversity. Up to 5 observed and 5 known covariates per series worked well in CITRAS-FM; for other TSFMs, the paper recommends beginning with 1–3 of each and scaling up (Yamaguchi et al., 9 Jun 2026).

The limitations are also explicit. If events are truly exogenous and not reflected in residuals, event covariates synthesized from $\mathbf{X}_{1:T+S}^{\kappa,:} \in \mathbb{R}^{(T+S)\times C_\kappa}$ 9 may miss them. In domains where exogenous covariates have different periodicities or phases than the target, periodic covariates tied to $P(\mathbf{X}_{T+1:T+S}^{\tau,:} \mid \mathbf{X}_{1:T}^{\tau,:}, \mathbf{X}_{1:T}^{o,:}, \mathbf{X}_{1:T+S}^{\kappa,:}).$ 0 may over-align. The paper therefore points to adaptive decomposition, dynamic parameter sampling conditioned on target statistics, and hybrid synthesis blending structural causal models with CovSynth as future directions (Yamaguchi et al., 9 Jun 2026).

The supplied literature also uses “CovSynth” as a broader framing for several adjacent synthesis paradigms, which suggests that the term is not yet standardized across domains.

In synthetic population generation, the framing is directed toward generating individual-level covariates that faithfully preserve the joint dependency structure while meeting known marginal constraints. “SynC” uses a Gaussian copula, predictive model-based merging, and marginal scaling to reconstruct individual-level data from coarse observations, with explicit emphasis on dependency preservation, marginal fidelity, and conditional structure (Wan et al., 2019).

In synthetic tabular data generation, “CovSynth” is also presented as a copula-flow engine. “Copula Flows” separate univariate marginals from multivariate dependence, learn marginals with monotone rational-quadratic spline flows, and learn the copula with an autoregressive flow under a two-stage maximum-likelihood objective. In that usage, the emphasis is high-fidelity synthetic data, mixed-type support, exact likelihood, and interpretable dependence modeling (Kamthe et al., 2021).

A different usage appears in privacy-preserving data release, where the label is attached to a covariance-driven synthetic data generator based on covariance loss under conditional expectation. There the method is formulated through microaggregation, tensorization of low-order moments, differentially private projection and noise addition, and guarantees for most low-dimensional marginals on average rather than all marginals uniformly (Boedihardjo et al., 2021).

In causal transportability, the same label is used for covariate-synthesis methods addressing positivity violations driven by a continuous covariate. That formulation combines an AIPW-based statistical transport model in overlap regions with a mathematical or biophysical model in nonpositive regions, yielding the CovSynth-MSM and CovSynth-CACE estimators for transported treatment effects (Zivich et al., 2023).

These adjacent usages do not define a single unified method. Rather, they locate a recurring idea: synthesis of covariate structure under constraints imposed by forecasting, dependence preservation, privacy, or transportability. Within this broader landscape, the CITRAS-FM formulation is the one that gives CovSynth its most explicit time-series identity: decomposition-based construction of pseudo-covariates for covariate-aware zero-shot forecasting (Yamaguchi et al., 9 Jun 2026).