Robustness of Context-Aware Forecasters to Significant Failures

Develop techniques that increase the robustness of time series forecasting models that integrate natural language context, in order to prevent or reduce significant failures in predictions where forecasts deviate by at least five times the range of the ground truth, while maintaining strong performance on context integration.

Background

The authors’ error analysis reveals that some models produce extreme mispredictions, even when leveraging context, necessitating a cap on the evaluation metric to mitigate outlier effects. These failures occur across multiple tasks and models, including LLM-based forecasters.

The paper identifies robustness against such significant failures as an unresolved issue and calls for methods that can reduce or eliminate these extreme errors without sacrificing the benefits of context-aware forecasting.