Bellman Conformal Inference: Calibrating Prediction Intervals For Time Series (2402.05203v2)

Abstract: We introduce BeLLMan Conformal Inference (BCI), a framework that wraps around any time series forecasting models and provides approximately calibrated prediction intervals. Unlike existing methods, BCI is able to leverage multi-step ahead forecasts and explicitly optimize the average interval lengths by solving a one-dimensional stochastic control problem (SCP) at each time step. In particular, we use the dynamic programming algorithm to find the optimal policy for the SCP. We prove that BCI achieves long-term coverage under arbitrary distribution shifts and temporal dependence, even with poor multi-step ahead forecasts. We find empirically that BCI avoids uninformative intervals that have infinite lengths and generates substantially shorter prediction intervals in multiple applications when compared with existing methods.

• The paper introduces Bellman Conformal Inference, a novel framework that calibrates prediction intervals through multi-step forecasts and dynamic programming.
• It frames calibration as a one-dimensional stochastic control problem to optimize the balance between interval length and coverage.
• Empirical evaluations show that BCI consistently produces shorter, robust intervals compared to Adaptive Conformal Inference, even under distribution shifts.

BeLLMan Conformal Inference: Calibrating Prediction Intervals for Time Series

The paper introduces BeLLMan Conformal Inference (BCI), a novel framework designed to provide calibrated prediction intervals for time series data. This method is applicable to any time series forecasting model, offering an improvement over existing prediction interval methods by leveraging multi-step ahead forecasts and optimizing average interval lengths through a one-dimensional stochastic control problem. The motivation for BCI arises from the inherent challenges in time series forecasting, where uncertain environments and distribution shifts render traditional model-based calibration approaches less effective.

Key Features of BCI

1. Multi-Step Forecasting Integration: Unlike other methods, BCI incorporates multi-step ahead forecasts to adjust the prediction intervals, thus exploiting the information available in longer-term dependencies.
2. Dynamic Programming for Optimization: The core novelty is formulating the calibration problem as a stochastic control problem (SCP) where a dynamic programming (DP) algorithm is employed to find the optimal policy at every time step, enabling the explicit trade-off between interval length and future coverage rates.
3. Robustness Against Distribution Shifts: BCI is proven to maintain long-term coverage regardless of distribution shifts and temporal dependencies in the data, marking a significant theoretical advancement over existing methods that typically require stronger distributional assumptions.

Comparison with Adaptive Conformal Inference (ACI)

The paper distinguishes BCI from Adaptive Conformal Inference (ACI), which achieves calibration without requiring specific assumptions on the time series. However, ACI fails to optimize interval lengths explicitly. BCI enhances this by positioning itself as a model predictive control framework, utilizing multi-step forecasts to strategically balance interval length and coverage.

Empirical Evaluation

The empirical results presented involve various applications including stock price volatility, absolute return forecasting, and Google search trends. The experiments demonstrate that BCI consistently produces shorter prediction intervals compared to ACI, especially when the nominal multi-step ahead prediction intervals are not well-calibrated. In scenarios where the nominal intervals are well-calibrated, BCI's performance aligns with ACI but with the advantage of preventing the occurrence of unbounded intervals.

Implications and Future Directions

The BCI framework has significant implications for a variety of fields relying heavily on time series forecasting, such as finance, epidemiology, and climate science. Practically, its application could lead to more accurate and reliable forecasting systems, which could better inform decision-making processes under uncertainty.

Theoretically, BCI opens new avenues for integrating control theory concepts with statistical learning, particularly in the field of uncertainty quantification. There is potential for future developments in exploring further extensions of the SCP formulation to incorporate more sophisticated machine learning models and broader classes of distribution shifts.

Overall, this work makes a substantial contribution to the literature on prediction interval calibration, offering a robust approach to managing future uncertainties in time series forecasting. The use of stochastic control and dynamic programming within this context is poised to stimulate additional research focused on enhancing predictive accuracy and reliability across diverse application domains.