TabPFN-TS: In-Context Forecasting

• TabPFN-TS is a time-series forecasting method that reformulates the problem as in-context tabular regression using a pretrained TabPFN-v2 model.
• It leverages lightweight feature engineering by incorporating covariates and time embeddings to construct a contextual table fed through a frozen Transformer.
• Empirical results on benchmarks like GIFT-Eval highlight its efficiency and top rank performance on both point and probabilistic forecasts despite its modest 11M parameters.

Searching arXiv for papers on TabPFN-TS and closely related usages. TabPFN-TS is a time-series forecasting method that adapts TabPFN-v2, a general tabular foundation model, to forecasting via lightweight feature engineering rather than task-specific retraining. It was introduced as a simple procedure for both point and probabilistic forecasting, and the original report emphasized that, despite TabPFN-v2 being pretrained solely on synthetic tabular data with no exposure to time series, the resulting system has only 11M parameters and achieves top rank on the public GIFT-Eval leaderboard in both forecasting tasks (Hoo et al., 6 Jan 2025).

1. Definition and lineage

TabPFN-TS is best understood as an in-context reformulation of forecasting into tabular regression. In the surrounding literature, TabPFN-v2 is described as a Transformer-based tabular foundation model pretrained on millions of synthetic regression tasks, and TabPFN-TS inherits the core PFN paradigm: inference proceeds by conditioning on a context table and producing predictions in a single forward pass, without gradient updates (Berthelier et al., 12 May 2026).

This places TabPFN-TS in a distinct position relative to specialized Time Series Foundation Models. Rather than introducing a dedicated temporal architecture, it repurposes a pretrained tabular model and relies on the match between forecasting instances and the model’s learned tabular priors. The original time-series paper presented this as evidence that tabular foundation models can constitute a valuable new approach for forecasting, while the later covariate-focused study treated TabPFN-TS as one of the recent architectures that aim to integrate covariates in zero-shot-style forecasting settings (Hoo et al., 6 Jan 2025).

A recurring misconception is that TabPFN-TS is a time-series-native model in the same sense as sequence-pretrained TSFMs. The supplied descriptions indicate the opposite: the method’s notable property is precisely that it works despite the base model having no pretraining exposure to time series (Hoo et al., 6 Jan 2025).

2. Forecasting as in-context tabular regression

For a univariate target $y$ with covariates $c$, TabPFN-TS constructs an in-context table from a look-back window of length $L$ and horizon covariates over a forecast horizon $H$. The reported input construction is

$$\bigl\{(t-i,\,y_{t-i},\,c_{t-i})\bigr\}_{i=0}^{L-1} \quad\text{and horizon covariates}\quad \bigl\{(t+j,\,c_{t+j})\bigr\}_{j=1}^{H}.$$

Each row is treated as a tabular data point for the Transformer. In this formulation, past rows contain observed targets, while future rows contain covariates and a masked target entry to be predicted (Berthelier et al., 12 May 2026).

The same study formalizes the forecasting problem by denoting

• $$x = [\,y_{t-L+1},\,y_{t-L+2},\,\dots,\,y_t\,]\in\mathbb R^L$$,
• $$c = [\,c_{t-L+1},\dots,c_{t+H}\,]\in\mathbb R^{(L+H)\times d_c}$$,
• $$\hat y = [\,\hat y_{t+1},\dots,\hat y_{t+H}\,]\in\mathbb R^H$$,

and writing the model as

$f_{\mathrm{TabPFN}}:(x,c)\mapsto \hat y.$

Evaluation is often reported with the normalized mean-squared error

$$L_f(x,y) \;=\; \bigl\lVert\,\hat y - \tilde y\bigr\rVert_2^2, \quad \tilde x = \tfrac{x - \mu_x}{\sigma_x},\; \tilde y = \tfrac{y - \mu_x}{\sigma_x}.$$

Here $c$0 are the mean and standard deviation of the look-back values, used to normalize both input and output (Berthelier et al., 12 May 2026).

This tabularization is central to the method. The pretrained TabPFN-v2 is described as acting as a Bayesian posterior predictor over the next $c$1 targets, conditioned on the tabular context without any gradient updates. A plausible implication is that TabPFN-TS succeeds when the induced prompt resembles the synthetic regression tasks from TabPFN-v2 pretraining.

3. Covariate conditioning and temporal feature engineering

The core extension beyond vanilla TabPFN is the explicit representation of time and covariates within each row. The reported row structure is

$c$2

where $c$3 denotes concatenation and “?” is a mask indicating that the model must predict that entry (Berthelier et al., 12 May 2026).

Time embeddings $c$4 are concatenated to each row’s covariate vector to capture periodicities when real-world covariates are used. The supplied formulation uses periodic encodings such as

$c$5

with chosen periods $c$6, $c$7 for hourly data (Berthelier et al., 12 May 2026).

The same source also reports an ablation called TabPFN(cov), in which time embeddings are removed when synthetic covariates alone suffice. This is technically significant because it separates temporal positional information from direct covariate-target mapping. In the reported experiments, removing time embeddings had negligible impact in purely regression-style tasks, which was interpreted as underscoring the advantage conferred by tabular pretraining for direct covariate-conditioned prediction (Berthelier et al., 12 May 2026).

4. Inference pipeline and forecasting behavior

The operational pipeline described for TabPFN-TS is minimal. The key steps are:

1. Normalize $c$8 and, by extension, $c$9 and covariates.
2. Build an in-context table of $L$0.
3. Perform a single forward pass through the frozen Transformer.
4. Extract the $L$1 predicted values (Berthelier et al., 12 May 2026).

The corresponding pseudocode specifies that rows for the look-back interval contain $L$2, whereas horizon rows contain $L$3. The entire table is then passed into the pretrained TabPFN Transformer, which outputs a posterior predictive distribution over the masked entries (Berthelier et al., 12 May 2026).

This single-pass posterior prediction is the mechanism through which the method supports both point and probabilistic forecasting in the original time-series report (Hoo et al., 6 Jan 2025). The exact probabilistic forecasting procedure is not specified in the supplied material, but the covariate-aware formulation explicitly describes the Transformer output as a joint posterior over the masked target entries (Berthelier et al., 12 May 2026).

A second misconception is that effective use of TabPFN-TS necessarily entails task-specific fine-tuning. In the supplied accounts, the defining usage pattern is frozen-model inference with prompt construction and lightweight feature engineering rather than gradient-based adaptation (Berthelier et al., 12 May 2026).

5. Empirical evidence and controlled evaluation

The original introduction of TabPFN-TS reported top rank on the public GIFT-Eval leaderboard in both forecasting tasks, despite the model’s simplicity and 11M-parameter scale (Hoo et al., 6 Jan 2025). A later controlled study then examined a narrower question: how well TabPFN-TS and Chronos-2 integrate simple target-covariate relationships (Berthelier et al., 12 May 2026).

That study defined four tasks:

Task	Relationship
Identity (real data)	$L$4
Sum (synthetic)	$L$5
Aggregate (real + real)	$L$6
Quadratic (synthetic)	$L$7

The experiments varied look-back $L$8 and forecasting horizon $L$9, used the normalized MSE objective above, and employed non-overlapping windows with stride 512 (Berthelier et al., 12 May 2026).

Several findings are explicit. In the Identity experiment, providing the true past target as covariate (“+id”) drastically reduces error for both models, but TabPFN-TS(+id) beats Chronos-2(+id) in all short-horizon settings. In the Aggregate experiment, TabPFN-TS(+cov) achieves normalized MSE $H$0 for $H$1, whereas Chronos-2(+cov) is $H$2; the TabPFN(cov) ablation without time embeddings still maintains top performance. In the Sum and Quadratic tasks, relative error heatmaps $H$3 show large blue regions for small $H$4, indicating that TabPFN-TS outperforms Chronos-2 on short horizons across random-walk and kernel-synth datasets (Berthelier et al., 12 May 2026).

The overall interpretation given in that work is narrow but important: strong benchmark performance of Chronos-2 does not automatically translate into optimal modeling of simple covariate-target dependencies, particularly at short horizons (Berthelier et al., 12 May 2026).

6. Inductive bias, spectral interpretation, and limitations

The controlled covariate study attributes TabPFN-TS’s behavior to its pretraining on i.i.d. regression tasks, which gives it a strong inductive bias for learning direct mappings $H$5. Short-horizon forecasting is described as more closely resembling these tasks because the input-output distributions align more closely with the pretraining distribution (Berthelier et al., 12 May 2026).

By contrast, Chronos-2 is characterized there as being stronger at modeling long-range temporal dependencies via cross-series attention and autoregressive pretraining on multivariate synthetic data. When time-to-target relationships dominate, exemplified by large $H$6 in an AR–nonlinear mix, Chronos-2 regains the lead (Berthelier et al., 12 May 2026). This is a useful boundary condition: the reported advantage of TabPFN-TS is not universal across all temporal regimes.

A related line of work analyzed TabPFN through signal reconstruction rather than forecasting benchmarks and reported that TabPFN has a broader effective frequency capacity than standard ReLU-MLPs, with spectral capacity adapting directly to the number of samples provided in-context, a phenomenon termed Spectral Adaptivity (Zheng et al., 23 Nov 2025). That paper defines a context-kernel

$H$7

and an effective frequency capacity $H$8 derived from the spectral gain $H$9. It further reports an approximate power law $$\bigl\{(t-i,\,y_{t-i},\,c_{t-i})\bigr\}_{i=0}^{L-1} \quad\text{and horizon covariates}\quad \bigl\{(t+j,\,c_{t+j})\bigr\}_{j=1}^{H}.$$0, with $$\bigl\{(t-i,\,y_{t-i},\,c_{t-i})\bigr\}_{i=0}^{L-1} \quad\text{and horizon covariates}\quad \bigl\{(t+j,\,c_{t+j})\bigr\}_{j=1}^{H}.$$1, and describes positional encoding as flattening the eigen-spectrum and raising recoverable frequency (Zheng et al., 23 Nov 2025).

This suggests a mechanistic interpretation for why a tabular PFN can function in time-series settings without temporal pretraining: increasing context size and carefully chosen time encodings may alter the model’s effective reconstruction bandwidth even when parameters remain frozen. The same work explicitly presents TabPFN-based prompting for time-series reconstruction and forecasting as a training-free, hyperparameter-free procedure, optionally augmented with Random Fourier Feature encodings when sampling is sparse or signals contain high-frequency components (Zheng et al., 23 Nov 2025).

7. Terminology and scope

Within the supplied literature, TabPFN-TS most directly denotes the time-series adaptation of TabPFN-v2 introduced in 2025 and then examined in 2026 for covariate-aware forecasting behavior (Hoo et al., 6 Jan 2025). The name is, however, not entirely stable across adjacent work.

A separate summary associated with the TabPFN Text Adapter states that the label “TabPFN-TS” was used there for a text-to-TFM token projection design in which a frozen sentence encoder and frozen TabPFN are connected by a lightweight adapter that maps text embeddings into TabPFN’s embedding space (Tajjar et al., 3 Jun 2026). Another nearby acronym, “PFN-TS,” denotes a Thompson sampling algorithm for contextual bandits that uses PFN posterior predictives together with a subsampled predictive central limit theorem; it is not a forecasting model for time series (Tan et al., 11 May 2026).

For that reason, the term is best interpreted contextually. In forecasting contexts, TabPFN-TS refers to the method that converts look-back windows, horizon covariates, and optional time embeddings into an in-context tabular prompt for TabPFN-v2, thereby enabling zero- or few-shot forecasting without gradient fine-tuning (Berthelier et al., 12 May 2026).