TFMAdapter for Forecasting with Covariates

• TFMAdapter is a lightweight, instance-level adapter that augments frozen time-series models to integrate future covariates without needing fine-tuning.
• It employs a two-stage approach that first uses a Bayesian Ridge pseudo-forecast generator and then a Gaussian Process regressor to refine forecasts.
• Empirical evaluations report roughly 24%-27% MAE improvements on benchmarks, highlighting its effective adaptation for covariate-based forecasting.

TFMAdapter is a lightweight, instance-level adapter for forecasting with covariates that augments frozen time series foundation models (TSFMs) without fine-tuning. It is designed for the setting in which a pretrained TSFM is strong at univariate forecasting from a brief target history but does not reliably exploit future-known exogenous variables. Rather than retraining the backbone, TFMAdapter learns, within a single forecasting instance, a non-parametric cascade that combines TSFM forecasts with covariates using two classical models: a Bayesian Ridge pseudo-forecast generator and a Gaussian Process regressor. The method is explicitly formulated as a frozen-model adaptation scheme with minimal data and computational overhead, and the reported experiments show average MAE improvements of roughly $24\%$–$27\%$ over base foundation models on forecasting-with-covariates benchmarks (Dange et al., 17 Sep 2025).

1. Problem setting and motivation

TFMAdapter targets a specific failure mode of contemporary TSFMs: strong zero-shot or few-shot univariate forecasting paired with weak use of future-known exogenous variables. In the paper’s formulation, a TSFM can produce a forecast from a target history

$\hat Y_{H+1:H+F} = \mathcal M(Y_{1:H}),$

but many practical tasks require forecasts of the form

$\hat{Y}_{H+1:H+F} = f(Y_{1:H}, \vX_{1:H+F}),$

where $\vX_{1:H+F}$ contains covariates over both the observed history and the forecast horizon. The motivating examples include settings such as electricity price forecasting, where future load and renewable generation forecasts are available in advance (Dange et al., 17 Sep 2025).

A central empirical motivation is the paper’s analysis of Moirai. Although Moirai nominally supports covariates, the authors report that it often does not use them effectively. They test this directly by supplying “oracle” covariates equal to the future target itself and show that Moirai still does not approach perfect forecasting. This is used as evidence that architectural support for covariates is not equivalent to effective covariate utilization (Dange et al., 17 Sep 2025).

TFMAdapter therefore addresses a gap between generic pretrained temporal priors and domain-specific forecasting needs. Its core premise is that a frozen TSFM already encodes useful temporal structure, but that the relationship between those univariate forecasts and exogenous drivers can be learned locally, at inference time, from the current instance’s own history.

2. Formal formulation and adaptation objective

The method assumes a target history

$Y_{1:H} = \{y_1,\dots,y_H\},$

a forecast horizon of length $F$, and covariates

$\vX_{1:H+F} = \{\vx_1,\dots,\vx_{H+F}\},$

with $\vx_t \in \mathbb{R}^d$. The base TSFM is treated primarily as a univariate forecaster: $\hat{Y}_{H+1:H+F} = \mathcal M(Y_{1:H}).$

The adapter objective is instance-level rather than dataset-level. TFMAdapter is learned for a single forecasting problem using only the current series history and the covariates available for that same instance. The paper also introduces $27\%$0, the minimum history length required for $27\%$1 to produce meaningful forecasts. This matters because historical training examples for the adapter can only be constructed from prefixes $27\%$2 with $27\%$3 (Dange et al., 17 Sep 2025).

This design distinguishes TFMAdapter from parameter-efficient fine-tuning in the usual sense. No TSFM parameters are updated. The foundation model is instead queried a small number of times, and the resulting outputs are used to train auxiliary local models around it. The reported method is therefore an adapter in the functional sense—augmenting a frozen foundation model—rather than an internal weight-space or hidden-state adapter.

3. Stage I: pseudo-forecast generation

The first stage exists to avoid repeatedly calling the TSFM across the entire history. Direct training of a historical combiner would require TSFM forecasts for many prefixes $27\%$4, which the paper identifies as the main computational bottleneck. TFMAdapter replaces this with a constant and small number of TSFM calls, using a pseudo-generator to imitate TSFM behavior over the full context (Dange et al., 17 Sep 2025).

The forecastable suffix of the history, from $27\%$5 to $27\%$6, is partitioned into non-overlapping windows of size $27\%$7: $27\%$8 Each window is normalized using the mean and standard deviation of the full history,

$27\%$9

and assigned an average z-score

$\hat Y_{H+1:H+F} = \mathcal M(Y_{1:H}),$0

The algorithm then selects $\hat Y_{H+1:H+F} = \mathcal M(Y_{1:H}),$1 representative windows: those with the lowest, median, and highest $\hat Y_{H+1:H+F} = \mathcal M(Y_{1:H}),$2. For each selected window, the TSFM is called on the preceding prefix: $\hat Y_{H+1:H+F} = \mathcal M(Y_{1:H}),$3

These sparse TSFM forecasts supervise a Bayesian Ridge pseudo-forecast generator $\hat Y_{H+1:H+F} = \mathcal M(Y_{1:H}),$4. The training features include the historical series values, lag features, and sinusoidal positional encodings: $\hat Y_{H+1:H+F} = \mathcal M(Y_{1:H}),$5 Algorithm 1 writes the training design matrix as

$\hat Y_{H+1:H+F} = \mathcal M(Y_{1:H}),$6

with targets

$\hat Y_{H+1:H+F} = \mathcal M(Y_{1:H}),$7

Once fitted, $\hat Y_{H+1:H+F} = \mathcal M(Y_{1:H}),$8 produces pseudo-forecasts across the history,

$\hat Y_{H+1:H+F} = \mathcal M(Y_{1:H}),$9

thereby creating TSFM-like signals at many historical positions without many TSFM invocations. The paper’s algorithm also states

$\hat{Y}_{H+1:H+F} = f(Y_{1:H}, \vX_{1:H+F}),$0

which specifies the future-side pseudo-feature generation used by the subsequent adapter stage (Dange et al., 17 Sep 2025).

4. Stage II: Gaussian Process adaptation and inference

The second stage is a Gaussian Process (GP) regressor that maps pseudo-forecasts and covariates to corrected target values. The observation model is

$\hat{Y}_{H+1:H+F} = f(Y_{1:H}, \vX_{1:H+F}),$1

where $\hat{Y}_{H+1:H+F} = f(Y_{1:H}, \vX_{1:H+F}),$2 is the pseudo-forecast, $\hat{Y}_{H+1:H+F} = f(Y_{1:H}, \vX_{1:H+F}),$3 is the covariate vector, and $\hat{Y}_{H+1:H+F} = f(Y_{1:H}, \vX_{1:H+F}),$4 denotes auxiliary features such as lags and positional encodings (Dange et al., 17 Sep 2025).

The GP uses a composite kernel

$\hat{Y}_{H+1:H+F} = f(Y_{1:H}, \vX_{1:H+F}),$5

where $\hat{Y}_{H+1:H+F} = f(Y_{1:H}, \vX_{1:H+F}),$6 concatenates pseudo-forecasts, lag features, and positional encodings, and $\hat{Y}_{H+1:H+F} = f(Y_{1:H}, \vX_{1:H+F}),$7 contains covariates. The reported kernel families are Matern, RBF, and Linear, including combinations thereof. This decomposition separates TSFM-like temporal signals from direct exogenous effects.

Posterior prediction follows the standard closed-form GP equations printed in the paper: $\hat{Y}_{H+1:H+F} = f(Y_{1:H}, \vX_{1:H+F}),$8 The adapted forecast is the posterior mean

$\hat{Y}_{H+1:H+F} = f(Y_{1:H}, \vX_{1:H+F}),$9

A validation procedure is built into the method. The latest $\vX_{1:H+F}$0 historical points, from $\vX_{1:H+F}$1 to $\vX_{1:H+F}$2, are reserved for validation and used to select the kernel, optimize kernel parameters, and tune an uncertainty threshold. At inference time, if the GP posterior variance at a future step exceeds that threshold, the method falls back to the base TSFM forecast for that step. This fallback mechanism is integral to the reported system and makes the adapter conservative under high uncertainty (Dange et al., 17 Sep 2025).

5. Empirical evaluation

The experiments include both a covariate-usage diagnostic and a broader forecasting benchmark. The diagnostic uses ETT datasets—ETTh1, ETTh2, ETTm1, and ETTm2—to compare Moirai in univariate, multivariate, and forecasting-with-covariates settings, including the oracle-covariate condition $\vX_{1:H+F}$3. The benchmark evaluation uses two groups of datasets: five electricity price forecasting datasets (EPF-BE, EPF-DE, EPF-FR, EPF-NP, EPF-PJM) and eight forecasting-with-covariates datasets derived from ImputeBench (Air Quality, Bafu, Chlorine, Climate, Drift, Electricity, Meteo, Temp) (Dange et al., 17 Sep 2025).

Benchmark group	Datasets	Typical setup
EPF	EPF-BE, EPF-DE, EPF-FR, EPF-NP, EPF-PJM	$\vX_{1:H+F}$4, $\vX_{1:H+F}$5, 2 covariates
ImputeBench-derived	Air Quality, Bafu, Chlorine, Climate, Drift, Electricity, Meteo, Temp	$\vX_{1:H+F}$6 from 288 to 960, $\vX_{1:H+F}$7 from 24 to 96, 4 or 10 covariates

The backbones are Chronos-Bolt-Base (205M), Moirai v1.1 base (91.4M), and TimesFM 2.0 (500M). Baselines include supervised models trained only on the available history—Kernel Regression, TiDE, NBEATSx-I, and TimeXer. Metrics are MAE, SMAPE, and RMSE (Dange et al., 17 Sep 2025).

The headline result is the average MAE improvement over the corresponding base foundation model: 27.267% for Chronos + TFMAdapter, 27.891% for Moirai + TFMAdapter, and 24.961% for TimesFM + TFMAdapter. The paper also reports large instance-level gains on specific datasets. For example, Chronos on Air Quality improves from MAE 0.756 to 0.217, and on Drift from 0.601 to 0.043. The horizon-scaling study further reports that gains increase with forecast length; for Chronos at $\vX_{1:H+F}$8, the gain rises from 5.67% at $\vX_{1:H+F}$9 to 11.84% at $Y_{1:H} = \{y_1,\dots,y_H\},$0 (Dange et al., 17 Sep 2025).

The most important ablation concerns the pseudo-generator. Training the adapter with pseudo-forecasts generated from only $Y_{1:H} = \{y_1,\dots,y_H\},$1 TSFM calls is reported to outperform direct training on true TSFM forecasts obtained from $Y_{1:H} = \{y_1,\dots,y_H\},$2 or $Y_{1:H} = \{y_1,\dots,y_H\},$3 invocations, and to be comparable to direct training with $Y_{1:H} = \{y_1,\dots,y_H\},$4 true-forecast data. This is the paper’s strongest evidence that Stage I is not merely a computational convenience but an effective approximation mechanism (Dange et al., 17 Sep 2025).

6. Position within the adapter literature and interpretive boundaries

TFMAdapter occupies a specific place in the broader adapter landscape. It is not a parameter-efficient internal adapter of the sort used inside transformer blocks, nor an input-side dimensionality reduction layer. Instead, it is a frozen-model, instance-level forecasting adapter that learns a local correction model around a TSFM forecast. This distinguishes it from output-level online wrappers such as AdapTS, which combine frozen FM forecasts with a lightweight online forecaster but do not explicitly target covariate integration within a single inference instance (Lee et al., 18 Feb 2025). It also differs from front-end channel-reduction adapters for multivariate time series classification, which compress input dimensions before a pretrained backbone rather than learning a per-instance forecasting correction with future-known exogenous variables (Feofanov et al., 2024).

In the TSFM ecosystem, TFMAdapter is compatible with a modular view of adapters as external components attached to frozen backbones, a view also reflected in pipeline toolkits that treat adapters as first-class modules around TSFMs (Shastri et al., 30 Nov 2025). Its distinctive contribution is the combination of three properties in one method: no backbone fine-tuning, instance-level fitting, and explicit use of future-known covariates.

Several limitations are implicit in the reported design. The method depends on informative covariates; if exogenous inputs are weak or unreliable, the benefit of the GP refinement may diminish. The pseudo-generator also assumes that TSFM behavior can be approximated from a few representative windows using Bayesian Ridge regression. A plausible implication is that this approximation may degrade when the TSFM’s forecast behavior is highly nonlocal or structurally complex. Likewise, the GP is appropriate for the reported history lengths, but standard GP regression remains sensitive to the size of the local training set. The paper’s own framing suggests that TFMAdapter is most useful precisely because current TSFMs often fail to use covariates well; if future TSFMs were to exploit covariates reliably, the incremental role of this adapter might narrow (Dange et al., 17 Sep 2025).

TFMAdapter is therefore best understood as a practical bridge between generic pretrained temporal inductive bias and domain-specific forecasting requirements. Its central result is that a frozen univariate TSFM, queried only a few times and wrapped by a local Bayesian Ridge–plus–Gaussian Process cascade, can substantially improve forecasting with covariates without fine-tuning the foundation model (Dange et al., 17 Sep 2025).