ApolloPFN: Zero-Shot Time Series Forecasting

• ApolloPFN is a zero-shot forecasting model that integrates historical target data and known future exogenous covariates to predict time series outcomes.
• It introduces a temporal synthetic-data prior with SRNGN and root-node excitation to generate structured, ordered synthetic datasets that capture trends and abrupt changes.
• The model employs a time-aware transformer with RoPE embeddings and modified attention mechanisms to effectively capture recency effects and horizon-dependent dependencies.

Searching arXiv for the specified paper and closely related PFN background. ApolloPFN is a prior-data fitted network (PFN) for zero-shot time series forecasting with exogenous variables. It extends PFN methodology from i.i.d. tabular settings to temporally ordered forecasting problems by introducing a temporal synthetic-data prior and a time-aware transformer architecture that can use past target history, past covariates, and future covariates when available (Potapczynski et al., 16 Mar 2026). In the formulation given for ApolloPFN, the historical target series is $(y_t)_{t=1}^T$, the accompanying covariates are $(\mathbf{x}_t)_{t=1}^{T+H}$ with $\mathbf{x}_t \in \mathbb{R}^F$, and the model forecasts

$$(y_{T+1}, \dots, y_{T+H}) = f_\theta(y_1,\dots,y_T,\mathbf{x}_1,\dots,\mathbf{x}_{T+H}).$$

The model is motivated by forecasting domains in which exogenous signals such as promotions, prices, temperature, calendar variables, load, or generation materially affect the target trajectory, especially through spikes, discontinuities, or regime and phase changes.

1. Problem setting and scope

ApolloPFN is defined for forecasting settings in which the target series is accompanied by exogenous covariates extending into the forecast horizon (Potapczynski et al., 16 Mar 2026). The explicit inclusion of $\mathbf{x}_{T+1},\dots,\mathbf{x}_{T+H}$ distinguishes its intended use from univariate forecasters that operate solely on the numerical history of the target. The paper identifies retail demand forecasting, electricity price forecasting, and traffic or sales forecasting as representative applications, with covariates including price, promotion, holidays, system load, zonal load, wind or solar generation, and calendar or event indicators.

The central claim of the problem formulation is that ignoring exogenous variables often causes large forecasting errors when those variables drive abrupt structural behavior in the target. In this setting, future covariates are not treated as auxiliary metadata but as forecast-relevant inputs available at inference time when the underlying application provides them. This permits joint conditioning on historical observations and known future regressors within a single in-context forecasting pass.

ApolloPFN is also explicitly a zero-shot forecaster. In the usage described in the paper, the model does not rely on task-specific fine-tuning for each benchmark dataset; instead, it performs forecasting by applying a PFN trained entirely on synthetic data to unseen forecasting tasks (Potapczynski et al., 16 Mar 2026). This zero-shot framing is central to the paper’s positioning of ApolloPFN relative to both PFNs and time series foundation models.

2. PFN foundations and the move from tabular to temporal data

PFNs are described as models that approximate Bayesian posterior predictive inference directly with a neural network (Potapczynski et al., 16 Mar 2026). Given a synthetic data generator $p(\mathcal{D})$, the PFN objective is to minimize the negative log-likelihood of the test target conditioned on the training data,

$$\mathcal{L}(\theta) = -\mathbb{E}_{p(\mathbf{x},y)} \log q_\theta\!\left(y_{\text{test}} \mid \mathbf{x}_{\text{test}}, \mathcal{D}_{\text{train}}\right),$$

so that the network learns to approximate the posterior predictive distribution

$$p(y_{\text{test}} \mid \mathbf{x}_{\text{test}}, \mathcal{D}_{\text{train}}) = \int p(y_{\text{test}} \mid \mathbf{x}_{\text{test}}, \xi)\, p(\xi \mid \mathcal{D}_{\text{train}})\, d\xi.$$

In this formulation, the network amortizes posterior prediction without explicit latent-variable inference or a closed-form likelihood.

ApolloPFN is presented as a direct response to the mismatch between earlier PFNs and time series data. Earlier PFNs such as TabPFN were built for i.i.d. tabular data, using synthetic tabular datasets sampled from DAGs and structural causal models and a sample-feature separable transformer architecture with attention over features and samples. That design is suitable when the sample axis is exchangeable. Time series, by contrast, are ordered, non-exchangeable, and often contain recency effects, evolving trends, and horizon-dependent covariate structure.

The paper identifies two main modifications required to transport the PFN paradigm into forecasting: synthetic data generation must become temporal, and the architecture must become time-aware (Potapczynski et al., 16 Mar 2026). These two changes are presented as ApolloPFN’s main contributions. A plausible implication is that the paper treats the quality of the synthetic prior and the inductive biases of the architecture as inseparable components of PFN performance in forecasting.

3. Failure modes of tabular PFNs on time series

The paper argues that tabular PFNs adapted to time series by hand-engineered temporal features, referred to as TabPFN-TS, retain intrinsic i.i.d. or order-invariant inductive biases and therefore exhibit several failure modes on forecasting tasks (Potapczynski et al., 16 Mar 2026).

The first failure mode is inability to learn ordered temporal patterns. The paper gives examples such as ramp-up before a promotion, a spike during the promotion, ramp-down after, and then a demand drop. Because the model is not inherently sensitive to temporal order, it cannot reliably represent these ordered motifs. The issue is not merely absence of additional features; it is a structural mismatch between permutation-invariant sample processing and temporally ordered data.

The second failure mode is dependence on manually engineered frequency features, such as day-of-week, month-of-year, or FFT-derived periodic terms,

$$x_{t,j} = \sin\!\left(2\pi \frac{\tau(t)}{P_j}\right) \quad\text{or}\quad x_{t,j} = \cos\!\left(2\pi \frac{\tau(t)}{P_j}\right).$$

If relevant periodicities are not explicitly included, the model may regress toward a mean-like prediction and miss unseen frequencies. This is described as a limitation of feature engineering rather than a robust temporal inductive bias.

The third and fourth failure modes are weak trend extrapolation and lack of recency bias. The paper states that poor modeling of order yields poor extrapolation of trends, especially in short-context settings, and that treating past time points too uniformly is problematic in nonstationary environments where recent observations should matter more. The fifth failure mode is poorly calibrated uncertainty, with confidence intervals reflecting the full historical support rather than uncertainty around the current trend.

Taken together, these arguments formalize the paper’s criticism of simply repurposing tabular PFNs for forecasting. The stated problem is not that PFNs are intrinsically unsuitable for time series, but that order-invariant PFN priors and architectures encode the wrong inductive bias for temporally structured data (Potapczynski et al., 16 Mar 2026).

4. Temporal synthetic prior: SRNGN and root-node excitation

ApolloPFN modifies the synthetic prior so that the training distribution contains explicit temporal structure (Potapczynski et al., 16 Mar 2026). The baseline tabular PFN synthetic generator samples the number of observations $N$, the number of features $(\mathbf{x}_t)_{t=1}^{T+H}$0, a DAG $(\mathbf{x}_t)_{t=1}^{T+H}$1, and an SCM on that graph of the form

$(\mathbf{x}_t)_{t=1}^{T+H}$2

Selected nodes are then assigned as input features and target. In the tabular case, this yields i.i.d. rows; ApolloPFN changes this so that the resulting synthetic datasets are time-indexed sequences.

A principal change is the introduction of the Single Root Node Growing Network (SRNGN). The paper motivates SRNGN by contrasting it with standard random growing network generation, which tends to produce many root nodes, short paths, and disconnected or weakly related feature structures. These properties are characterized as undesirable for time series, where temporal dependencies often unfold along longer chains. SRNGN reverses the logic of the standard random growing network to generate graphs with a single root node, longer causal paths, and connected structure. The graph is initialized with nodes $(\mathbf{x}_t)_{t=1}^{T+H}$3 and $(\mathbf{x}_t)_{t=1}^{T+H}$4 and edge $(\mathbf{x}_t)_{t=1}^{T+H}$5, then grows so that each new node has at least one prior node pointing to it and, with probability $(\mathbf{x}_t)_{t=1}^{T+H}$6, also connects to a popular node. The attachment probabilities are

$(\mathbf{x}_t)_{t=1}^{T+H}$7

where $(\mathbf{x}_t)_{t=1}^{T+H}$8 is the in-degree. The paper reports that this graph prior empirically accelerates training.

The second change is Time Series Root Node Excitation, identified as the key step that makes the synthetic data genuinely temporal. Rather than sampling root-node values independently across observations, ApolloPFN samples temporal root dynamics as a stochastic periodic process:

$(\mathbf{x}_t)_{t=1}^{T+H}$9

The random amplitudes and frequencies are then propagated through the graph to create correlated features and targets, producing synthetic datasets of the form

$\mathbf{x}_t \in \mathbb{R}^F$0

This suggests that ApolloPFN’s prior is designed not merely to encode generic dependence, but to expose the model during training to periodic and trend-like temporal dynamics that are absent from i.i.d. synthetic rows.

The preprocessing follows TabPFN in z-scoring targets using training statistics before they are fed to the model, then inverting predictions afterward (Potapczynski et al., 16 Mar 2026). This continuity with prior PFN practice indicates that ApolloPFN alters the synthetic data regime and temporal inductive bias while preserving the PFN training philosophy.

5. Time-aware architecture and exogenous conditioning

ApolloPFN adapts the TabPFN architecture with temporal inductive biases intended to respect observation order and exploit future covariates at inference time (Potapczynski et al., 16 Mar 2026). The paper emphasizes two architectural mechanisms: positional information and a modified attention mask.

To reflect time order, ApolloPFN adds both relative and absolute positional information. Relative positional bias is introduced via RoPE-style rotational embeddings in sample attention, with query-key interaction of the form

$\mathbf{x}_t \in \mathbb{R}^F$1

so that nearby observations are weighted more heavily. In addition, the model uses explicit sinusoidal absolute positional encodings added to the sample embeddings. The paper’s interpretation is that the combination provides both relative timing information and absolute position information. In the ablations, RoPE is reported as the largest single gain.

The attention mask is also changed. In tabular PFNs, test points do not attend to other test points, which is appropriate for independent test examples. ApolloPFN instead allows full attention among all points in sample attention. The reason given is that, for forecasting, future steps are not independent because future exogenous covariates can inform future targets jointly. Consequently, forecasts are not made independently as $\mathbf{x}_t \in \mathbb{R}^F$2, but in a time-series-aware manner where future context matters:

$\mathbf{x}_t \in \mathbb{R}^F$3

Exogenous covariates are native inputs to ApolloPFN at both train and forecast time. The paper contrasts this with time series foundation models such as Chronos, TimesFM, Sundial, TimeMoE, TimeLLM, and LagLlama, which generally forecast from the univariate history alone (Potapczynski et al., 16 Mar 2026). ApolloPFN’s input signature

$\mathbf{x}_t \in \mathbb{R}^F$4

therefore supports promotions and prices in retail, load and generation in power markets, calendar and event variables, and other known future regressors. A common misunderstanding would be to view full attention over future points as leakage of future targets; the paper’s actual construction allows interaction through future covariates that are assumed known in the application setting, not through future target observations.

6. Training regime, benchmarks, and empirical performance

ApolloPFN is trained entirely on synthetic SCM data with the following hyperparameters (Potapczynski et al., 16 Mar 2026). The number of nodes is sampled uniformly from 20 to 150; the state dimension per node is 6; MLP edge activations are sampled from tanh, sine, abs, identity, log, sigmoid, smooth relu, modulo, and step/indicator; MLP weights are sampled from $\mathbf{x}_t \in \mathbb{R}^F$5; frequencies satisfy $\mathbf{x}_t \in \mathbb{R}^F$6; amplitudes satisfy $\mathbf{x}_t \in \mathbb{R}^F$7. Optimization uses 400K training steps, batch size 64, learning rate $\mathbf{x}_t \in \mathbb{R}^F$8, no weight decay, 20K warmup steps, and cosine annealing to $\mathbf{x}_t \in \mathbb{R}^F$9. During training, the model sees variable context sizes: 34 to 512 samples, 2 to 64 features, and forecast horizon up to 128. The stated purpose is to support generalization across forecasting regimes.

Evaluation covers both exogenous and univariate forecasting benchmarks. For exogenous-variable forecasting, the paper evaluates on electricity price forecasting and the M5 competition. Electricity price forecasting includes Nord Pool, PJM, France, Belgium, and Germany, with exogenous inputs such as system load, zonal load, wind or solar generation, and power generation. M5 includes daily SKU demand with price and promotion covariates, evaluations at state and store aggregation levels, daily, weekly, and monthly versions, and over 30K time series at SKU/store level. For univariate forecasting, the paper evaluates on M1, M3, M4, and Tourism.

The reported metrics are sCRPS for probabilistic forecasting and RMSSE for M5. The paper defines sCRPS as

$$(y_{T+1}, \dots, y_{T+H}) = f_\theta(y_1,\dots,y_T,\mathbf{x}_1,\dots,\mathbf{x}_{T+H}).$$0

where $$(y_{T+1}, \dots, y_{T+H}) = f_\theta(y_1,\dots,y_T,\mathbf{x}_1,\dots,\mathbf{x}_{T+H}).$$1 is the $$(y_{T+1}, \dots, y_{T+H}) = f_\theta(y_1,\dots,y_T,\mathbf{x}_1,\dots,\mathbf{x}_{T+H}).$$2-quantile prediction. RMSSE is defined as

$$(y_{T+1}, \dots, y_{T+H}) = f_\theta(y_1,\dots,y_T,\mathbf{x}_1,\dots,\mathbf{x}_{T+H}).$$3

The paper notes that RMSSE normalizes squared error by the series’ naive-scale variability and penalizes missing spikes from promotions.

The empirical claims are stated in three parts. On electricity price forecasting, ApolloPFN achieves the best or near-best sCRPS across most markets and horizons, with about 12% average improvement over the next best model, TabPFN-TS, and state-of-the-art performance on most datasets. It is reported to outperform TabPFN-TS, Moirai-Large, Chronos-Large, and Sundial-Base. On M5, ApolloPFN is described as highly competitive and often best at state and store aggregation levels across daily, weekly, and monthly horizons, with state-of-the-art performance on most aggregation levels, especially where exogenous variables matter for demand spikes. On classical univariate benchmarks, ApolloPFN is reported to be about 10% better than TabPFN-TS on average, indicating that the time-aware prior and architecture also improve univariate forecasting (Potapczynski et al., 16 Mar 2026).

7. Ablations, limitations, and interpretive significance

The ablation studies focus on the synthetic prior and the temporal architectural modifications (Potapczynski et al., 16 Mar 2026). For synthetic graph generation, training with SRNGN converges faster and achieves better performance earlier than training with the standard random growing network. The paper interprets this as evidence that graph structure in the synthetic prior affects the model’s ability to learn temporal dependencies.

The architectural ablations compare vanilla TabPFN-TS on ApolloPFN’s time-dependent data, adding RoPE, and adding RoPE plus full attention. Performance improves progressively across these variants; RoPE is identified as the largest single gain, and full attention is described as especially important when future covariates need to influence future predictions jointly. The best results arise from the full combination. This suggests that the temporal prior alone is insufficient; the architecture must also encode order and cross-horizon dependence.

The paper is explicit about several limitations. Quadratic attention limits scalability to very long sequences. As an in-context model, ApolloPFN may struggle when the history is very short. Its capabilities are bounded by the synthetic training distribution, so patterns not represented during synthetic training may not generalize. The paper also states that more theoretical work is needed on how synthetic prior design influences generalization, and it identifies hierarchical mechanisms for pooling across related series and compression or quantization for deployment as future directions.

ApolloPFN’s broader significance within the PFN literature lies in demonstrating that prior fitting can be extended from exchangeable tabular data to temporal forecasting by redesigning both the synthetic data generator and the transformer’s inductive biases (Potapczynski et al., 16 Mar 2026). The paper’s bottom-line interpretation is that these changes address the principal failure modes of tabular PFNs on time series and yield strong zero-shot forecasting performance, especially on covariate-rich benchmarks such as M5 and electricity price forecasting, where exogenous variables are integral to capturing spikes and structural shifts.