Reliable generalization to novel covariate futures

Develop forecasting models that generalize reliably from historical intervention events to Novel futures in which covariates are assigned values beyond the range observed in the history, thereby enabling accurate intervention-aware scenario analysis.

Background

WhatIf-Bench evaluates intervention-aware time-series forecasting by presenting models with historical events that alter one or more covariates and then asking them to forecast the target under a future event. Replay futures reuse covariate values seen in the history, whereas Novel futures assign values beyond the historical range and therefore test extrapolation to unseen intervention settings.

Across the benchmark, the evaluated models perform substantially better on Replay futures than on Novel futures. The authors identify this persistent degradation as the central unresolved challenge: models must learn covariate–target relationships from in-context evidence and transfer those relationships to unseen covariate values, including settings with multiple jointly intervened covariates. Solving this problem is necessary for reliable arbitrary scenario analysis.

References

These results point to generalizing to Novel futures as the central open problem, and open new directions for building models that forecast reliably under novel scenarios.

— WhatIf-Bench: Benchmarking Intervention-Aware Time Series Forecasting  (2610.26582 - Ashok et al., 19 Oct 2026) in Section 6, Discussion and Future Work