Conformal PID Control for Time Series Prediction (2307.16895v1)

Abstract: We study the problem of uncertainty quantification for time series prediction, with the goal of providing easy-to-use algorithms with formal guarantees. The algorithms we present build upon ideas from conformal prediction and control theory, are able to prospectively model conformal scores in an online setting, and adapt to the presence of systematic errors due to seasonality, trends, and general distribution shifts. Our theory both simplifies and strengthens existing analyses in online conformal prediction. Experiments on 4-week-ahead forecasting of statewide COVID-19 death counts in the U.S. show an improvement in coverage over the ensemble forecaster used in official CDC communications. We also run experiments on predicting electricity demand, market returns, and temperature using autoregressive, Theta, Prophet, and Transformer models. We provide an extendable codebase for testing our methods and for the integration of new algorithms, data sets, and forecasting rules.

• The paper extends conformal prediction to an online setting by formulating the process as a PID control problem for dynamically adapting prediction intervals.
• It integrates proportional, integral, and derivative components to stabilize coverage and correct systematic prediction errors.
• Experimental results on COVID-19 death and electricity demand forecasting demonstrate improved accuracy and robustness over traditional models.

Overview of "Conformal PID Control for Time Series Prediction"

The paper "Conformal PID Control for Time Series Prediction" addresses the pressing issue of uncertainty quantification in time series prediction under non-ideal conditions characterized by distribution shifts. This document introduces an innovative approach that merges conformal prediction with principles from control theory, specifically adopting a Proportional-Integral-Derivative (PID) controller framework to refine prediction intervals under adversarial and changing conditions. The proposed method is distinguished by its adaptability in environments beset with challenges like seasonality, trends, and distribution shifts, which traditionally hinder prediction accuracy.

Key Contributions

A significant contribution of this research is the extension and enhancement of conformal prediction techniques in an online setting, improving upon existing methods for uncertainty quantification in time series. By conceptualizing the conformal prediction process as a PID control problem, the method allows for prospective modeling of conformal scores, adapting dynamically to systematic errors.

Three core principles underlie the framework:

1. Quantile Tracking (P Control): This involves applying online gradient descent on the quantile loss, ensuring long-run coverage. Unlike existing adaptive conformal inference methods, this approach avoids outputting infinite sets after sequences of miscoverage events.
2. Error Integration (I Control): This principle incorporates coverage error integration into the online quantile update, stabilizing coverage across unbounded score ranges and adjusting for cumulative errors.
3. Scorecasting (D Control): The forward-looking component, wherein an additional model forecasts the quantile of the next score. This step aims to anticipate changes and trends in errors, enhancing both the efficiency of coverage and set size by possibly residualizing systematic trends.

These components collectively form the conformal PID controller, which is mathematically defined and shown through theoretical rigor to achieve long-run coverage under minimal assumptions.

Experimental Evaluation

Extensive experiments validate the framework, demonstrating superior performance in practical applications:

• COVID-19 Death Forecasting: Applying the method to forecast COVID-19 death counts shows improved coverage accuracy and adaptability compared to the baseline ensemble model used by the CDC, demonstrating the PID controller's superior handling of systematic errors linked to geographic and temporal data dependencies.
• Electricity Demand Forecasting: The proposed method demonstrates enhanced coverage and efficient prediction sets by better anticipating intraday variations, showcasing its robustness in a high-frequency, computationally intensive setting.

Implications and Future Directions

The conformal PID control framework potentially shifts practical approaches to time series forecasting by integrating control theory principles, enabling more reliable and adaptable prediction systems in dynamic environments. This approach could be particularly relevant for critical systems where coverage stability and predictive accuracy under changing conditions are paramount.

Future research could explore adaptive learning rates and more refined tuning strategies for online integrators, as well as broader applications across various domains that demand robust time series analysis. Additionally, the modular nature of scorecasting, which operates alongside existing forecasters, invites further investigation into machine learning methods that can effectively predict systematic patterns in residual errors.

By combining the strengths of conformal prediction and PID control, this research offers a novel pathway to enhance forecast reliability amidst the complex, shifting landscapes often encountered in real-world data streams.