Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
173 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A general framework for multi-step ahead adaptive conformal heteroscedastic time series forecasting (2207.14219v9)

Published 28 Jul 2022 in stat.ML and cs.LG

Abstract: This paper introduces a novel model-agnostic algorithm called adaptive ensemble batch multi-input multi-output conformalized quantile regression (AEnbMIMOCQR} that enables forecasters to generate multi-step ahead prediction intervals for a fixed pre-specified miscoverage rate in a distribution-free manner. Our method is grounded on conformal prediction principles, however, it does not require data splitting and provides close to exact coverage even when the data is not exchangeable. Moreover, the resulting prediction intervals, besides being empirically valid along the forecast horizon, do not neglect heteroscedasticity. AEnbMIMOCQR is designed to be robust to distribution shifts, which means that its prediction intervals remain reliable over an unlimited period of time, without entailing retraining or imposing unrealistic strict assumptions on the data-generating process. Through methodically experimentation, we demonstrate that our approach outperforms other competitive methods on both real-world and synthetic datasets. The code used in the experimental part and a tutorial on how to use AEnbMIMOCQR can be found at the following GitHub repository: https://github.com/Quilograma/AEnbMIMOCQR.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (74)
  1. (). Nn5 forecasting competion dataset. http://www.neural-forecasting-competition.com/NN5/. Accessed: April 7, 2023.
  2. TensorFlow: Large-scale machine learning on heterogeneous systems. URL: https://www.tensorflow.org/ software available from tensorflow.org.
  3. Comparison of strategies for multi-step-ahead prediction of time series using neural network. In 2015 International Conference on Advanced Computing and Applications (ACOMP) (pp. 142–149). IEEE.
  4. Conformal prediction: A gentle introduction. Foundations and Trends® in Machine Learning, 16, 494–591. URL: http://dx.doi.org/10.1561/2200000101. doi:10.1561/2200000101.
  5. Predictive inference with the jackknife+, .
  6. Breiman, L. (1996). Bagging predictors. Machine learning, 24, 123–140.
  7. Introduction to time series and forecasting. Springer.
  8. Chatfield, C. (2001). Prediction intervals for time-series forecasting. Principles of forecasting: A handbook for researchers and practitioners, (pp. 475–494).
  9. Exact and robust conformal inference methods for predictive machine learning with dependent data. In Conference On learning theory (pp. 732–749). PMLR.
  10. Cochrane, J. H. (2005). Time series for macroeconomics and finance. Manuscript, University of Chicago, 15, 16.
  11. Multivariate probabilistic forecasting of intraday electricity prices using normalizing flows. Applied Energy, 346, 121370. URL: https://www.sciencedirect.com/science/article/pii/S0306261923007341. doi:https://doi.org/10.1016/j.apenergy.2023.121370.
  12. Long-term forecasting with tide: Time-series dense encoder. arXiv preprint arXiv:2304.08424, .
  13. Aleatory or epistemic? does it matter? Structural safety, 31, 105–112.
  14. Heteroskedastic conformal regression. arXiv, .
  15. Valid prediction intervals for regression problems. Artificial Intelligence Review, 56, 577–613.
  16. Distribution of the estimators for autoregressive time series with a unit root. Journal of the American Statistical Association, 74, 427–431.
  17. Likelihood ratio statistics for autoregressive time series with a unit root. Econometrica, 49, 1057–1072.
  18. Interval forecasting for urban water demand using pso optimized kde distribution and lstm neural networks. Applied Soft Computing, 122, 108875. URL: https://www.sciencedirect.com/science/article/pii/S1568494622002563. doi:https://doi.org/10.1016/j.asoc.2022.108875.
  19. Deepvar: a framework for portfolio risk assessment leveraging probabilistic deep neural networks. Digital finance, 5, 29–56.
  20. Bootstrap estimation of uncertainty in prediction for generalized linear mixed models. Computational Statistics & Data Analysis, 130, 1–17. URL: https://www.sciencedirect.com/science/article/pii/S0167947318301890. doi:https://doi.org/10.1016/j.csda.2018.08.006.
  21. Conformal prediction: a unified review of theory and new challenges. Bernoulli, 29, 1–23.
  22. The limits of distribution-free conditional predictive inference. Information and Inference: A Journal of the IMA, 10, 455–482.
  23. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning (pp. 1050–1059). PMLR.
  24. Bayesian data analysis. Chapman and Hall/CRC.
  25. Adaptive conformal inference under distribution shift. Advances in Neural Information Processing Systems, 34, 1660–1672.
  26. Forecasting conditional extreme quantiles for wind energy. Electric Power Systems Research, 190, 106636.
  27. Bayesian forecasting. Journal of the Royal Statistical Society: Series B (Methodological), 38, 205–228.
  28. Heskes, T. (1996). Practical confidence and prediction intervals. Advances in neural information processing systems, 9.
  29. Long short-term memory. Neural computation, 9, 1735–80. doi:10.1162/neco.1997.9.8.1735.
  30. Bootstrapping time series models. Econometric reviews, 15, 115–158.
  31. Forecasting: principles and practice. OTexts.
  32. Ensemble conformalized quantile regression for probabilistic time series forecasting. IEEE Transactions on Neural Networks and Learning Systems, .
  33. Regression conformal prediction with random forests. Machine learning, 97, 155–176.
  34. Kavousi-Fard, A. (2016). Modeling uncertainty in tidal current forecast using prediction interval-based svr. IEEE Transactions on Sustainable Energy, 8, 708–715.
  35. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30.
  36. Predictive inference is free with the jackknife+-after-bootstrap. Advances in Neural Information Processing Systems, 33, 4138–4149.
  37. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, .
  38. Adaptive, distribution-free prediction intervals for deep networks. In International Conference on Artificial Intelligence and Statistics (pp. 4346–4356). PMLR.
  39. Koenker, R. (2017). Quantile regression: 40 years on. Annual Review of Economics, 9, 155–176.
  40. Quantile regression. Journal of economic perspectives, 15, 143–156.
  41. Kuchibhotla, A. K. (2020). Exchangeability, conformal prediction, and rank tests. arXiv preprint arXiv:2005.06095, .
  42. Uncertainty quantification using bayesian neural networks in classification: Application to biomedical image segmentation. Computational Statistics & Data Analysis, 142, 106816. URL: https://www.sciencedirect.com/science/article/pii/S016794731930163X. doi:https://doi.org/10.1016/j.csda.2019.106816.
  43. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30.
  44. Empirical prediction intervals revisited. International Journal of Forecasting, 30, 217–234.
  45. The accuracy of extrapolation (time series) methods: Results of a forecasting competition. Journal of forecasting, 1, 111–153.
  46. The m4 competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting, 36, 54–74. URL: https://www.sciencedirect.com/science/article/pii/S0169207019301128. doi:https://doi.org/10.1016/j.ijforecast.2019.04.014. M4 Competition.
  47. M5 accuracy competition: Results, findings, and conclusions. International Journal of Forecasting, 38, 1346–1364. URL: https://www.sciencedirect.com/science/article/pii/S0169207021001874. doi:https://doi.org/10.1016/j.ijforecast.2021.11.013. Special Issue: M5 competition.
  48. The m5 uncertainty competition: Results, findings and conclusions. International Journal of Forecasting, 38, 1365–1385.
  49. Machine learning advances for time series forecasting. Journal of economic surveys, 37, 76–111.
  50. Quantile regression forests. Journal of machine learning research, 7.
  51. Noriega, L. (2005). Multilayer perceptron tutorial. School of Computing. Staffordshire University, 4, 5.
  52. Papadopoulos, H. (2008). Inductive conformal prediction: Theory and application to neural networks. In Tools in artificial intelligence. Citeseer.
  53. Reliable prediction intervals with regression neural networks. Neural Networks, 24, 842–851. URL: https://www.sciencedirect.com/science/article/pii/S089360801100150X. doi:https://doi.org/10.1016/j.neunet.2011.05.008. Artificial Neural Networks: Selected Papers from ICANN 2010.
  54. ‘horses for courses’ in demand forecasting. European Journal of Operational Research, 237, 152–163.
  55. Uncertainty quantification for traffic forecasting: A unified approach. In 2023 IEEE 39th International Conference on Data Engineering (ICDE) (pp. 992–1004). IEEE.
  56. Conformalized quantile regression. Advances in neural information processing systems, 32.
  57. Probabilistic forecasting of hourly emergency department arrivals. Health Systems, (pp. 1–17).
  58. A tutorial on conformal prediction. Journal of Machine Learning Research, 9.
  59. Residential probabilistic load forecasting: A method using gaussian process designed for electric load data. Applied Energy, 218, 159–172.
  60. Smith, T. G. et al. (2017). pmdarima: Arima estimators for Python. URL: http://www.alkaline-ml.com/pmdarima [Online; accessed <today>].
  61. Improved conformalized quantile regression. arXiv:2207.02808.
  62. Conformal time-series forecasting. Advances in Neural Information Processing Systems, 34, 6216--6228.
  63. Estimating conditional quantiles with the help of the pinball loss, .
  64. A review and comparison of strategies for multi-step ahead time series forecasting based on the nn5 forecasting competition. Expert systems with applications, 39, 7067--7083.
  65. Combining probabilistic forecasts of covid-19 mortality in the united states. European Journal of Operational Research, 304, 25--41. URL: https://www.sciencedirect.com/science/article/pii/S0377221721005609. doi:https://doi.org/10.1016/j.ejor.2021.06.044. The role of Operational Research in future epidemics/ pandemics.
  66. A review of probabilistic forecasting and prediction with machine learning. arXiv preprint arXiv:2209.08307, .
  67. Vovk, V. (2013). Transductive conformal predictors. In Artificial Intelligence Applications and Innovations: 9th IFIP WG 12.5 International Conference, AIAI 2013, Paphos, Cyprus, September 30–October 2, 2013, Proceedings 9 (pp. 348--360). Springer.
  68. Vovk, V. (2015). Cross-conformal predictors. Annals of Mathematics and Artificial Intelligence, 74, 9--28.
  69. A novel ensemble probabilistic forecasting system for uncertainty in wind speed. Applied Energy, 313, 118796. URL: https://www.sciencedirect.com/science/article/pii/S0306261922002434. doi:https://doi.org/10.1016/j.apenergy.2022.118796.
  70. Gaussian processes for regression. Advances in neural information processing systems, 8.
  71. Winkler, R. L. (1972). A decision-theoretic approach to interval estimation. Journal of the American Statistical Association, 67, 187--191.
  72. Beyond one-step-ahead forecasting: evaluation of alternative multi-step-ahead forecasting models for crude oil prices. Energy Economics, 40, 405--415.
  73. Conformal prediction interval for dynamic time-series. In International Conference on Machine Learning (pp. 11559--11569). PMLR.
  74. Conformal prediction for time series. arXiv:2010.09107.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now