Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Conformal Prediction Regions for Time Series using Linear Complementarity Programming (2304.01075v5)

Published 3 Apr 2023 in eess.SY, cs.LG, and cs.SY

Abstract: Conformal prediction is a statistical tool for producing prediction regions of machine learning models that are valid with high probability. However, applying conformal prediction to time series data leads to conservative prediction regions. In fact, to obtain prediction regions over $T$ time steps with confidence $1-\delta$, {previous works require that each individual prediction region is valid} with confidence $1-\delta/T$. We propose an optimization-based method for reducing this conservatism to enable long horizon planning and verification when using learning-enabled time series predictors. Instead of considering prediction errors individually at each time step, we consider a parameterized prediction error over multiple time steps. By optimizing the parameters over an additional dataset, we find prediction regions that are not conservative. We show that this problem can be cast as a mixed integer linear complementarity program (MILCP), which we then relax into a linear complementarity program (LCP). Additionally, we prove that the relaxed LP has the same optimal cost as the original MILCP. Finally, we demonstrate the efficacy of our method on case studies using pedestrian trajectory predictors and F16 fighter jet altitude predictors.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Social lstm: Human trajectory prediction in crowded spaces. In Proceedings of the IEEE conference on computer vision and pattern recognition, 961–971.
  2. Conformal Prediction: A Gentle Introduction. Found. Trends Mach. Learn., 16(4): 494–591.
  3. Conformal risk control. arXiv preprint arXiv:2208.02814.
  4. Control of systems integrating logic, dynamics, and constraints. Automatica, 35(3): 407–427.
  5. Improved Online Conformal Prediction via Strongly Adaptive Online Learning. In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org.
  6. Applied Mathematical Programming. Addison-Wesley Publishing Company. ISBN 9780201004649.
  7. Guaranteed Dynamic Scheduling of Ultra-Reliable Low-Latency Traffic via Conformal Prediction. IEEE Signal Processing Letters.
  8. The linear complementarity problem. SIAM.
  9. Adaptive Conformal Prediction for Motion Planning among Dynamic Agents.
  10. Conformal prediction under feedback covariate shift for biomolecular design. Proceedings of the National Academy of Sciences, 119(43).
  11. Adaptive conformal inference under distribution shift. Advances in Neural Information Processing Systems, 34: 1660–1672.
  12. Conformal inference for online prediction with arbitrary distribution shifts. arXiv preprint arXiv:2208.08401.
  13. Gurobi Optimization, LLC. 2023. Gurobi Optimizer Reference Manual.
  14. Reliable Prediction Intervals with Directly Optimized Inductive Conformal Regression for Deep Learning. 5th International Workshop on Applied Verification for Continuous and Hybrid Systems (ARCH 2018).
  15. Long short-term memory. Neural computation, 9(8): 1735–1780.
  16. iDECODe: In-Distribution Equivariance for Conformal Out-of-Distribution Detection. Proceedings of the AAAI Conference on Artificial Intelligence, 36(7): 7104–7114.
  17. CODiT: Conformal Out-of-Distribution Detection in Time-Series Data for Cyber-Physical Systems. In Proceedings of the ACM/IEEE 14th International Conference on Cyber-Physical Systems (with CPS-IoT Week 2023), 120–131.
  18. Regression Quantiles. Econometrica, 46(1): 33–50.
  19. Human trajectory forecasting in crowds: A deep learning perspective. IEEE Transactions on Intelligent Transportation Systems.
  20. Safe Planning in Dynamic Environments Using Conformal Prediction. IEEE Robotics and Automation Letters, 8(8): 5116–5123.
  21. Conformal prediction for stl runtime verification. In Proceedings of the ACM/IEEE 14th International Conference on Cyber-Physical Systems (with CPS-IoT Week 2023), 142–153.
  22. Sample-efficient safety assurances using conformal prediction. In Algorithmic Foundations of Robotics XV: Proceedings of the Fifteenth Workshop on the Algorithmic Foundations of Robotics, 149–169. Springer.
  23. Papadopoulos, H. 2008. Inductive conformal prediction: Theory and application to neural networks. In Tools in artificial intelligence. Citeseer.
  24. PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction. In International Conference on Learning Representations.
  25. Model predictive control with signal temporal logic specifications. In 53rd IEEE Conference on Decision and Control, 81–87. IEEE.
  26. Conformalized quantile regression. Advances in neural information processing systems, 32.
  27. A Tutorial on Conformal Prediction. Journal of Machine Learning Research, 9(3).
  28. Conformal time-series forecasting. Advances in Neural Information Processing Systems, 34: 6216–6228.
  29. Learning Optimal Conformal Classifiers. In International Conference on Learning Representations.
  30. Copula Conformal Prediction for Multi-step Time Series Forecasting. arXiv:2212.03281.
  31. Conformal prediction under covariate shift. Advances in neural information processing systems, 32.
  32. Scalable Safe Long-Horizon Planning in Dynamic Environments Leveraging Conformal Prediction and Temporal Correlations. In Long-Term Human Motion Prediction Workshop, International Conference on Robotics and Automation.
  33. Physics Constrained Motion Prediction with Uncertainty Quantification. arXiv preprint arXiv:2302.01060.
  34. Reciprocal velocity obstacles for real-time multi-agent navigation. In 2008 IEEE international conference on robotics and automation, 1928–1935. Ieee.
  35. Vovk, V. 2012. Conditional validity of inductive conformal predictors. In Asian conference on machine learning, 475–490. PMLR.
  36. Algorithmic learning in a random world. Springer Science & Business Media.
  37. Moving Obstacle Avoidance: A Data-Driven Risk-Aware Approach. IEEE Control Systems Letters, 7: 289–294.
  38. Application of conformal prediction interval estimations to market makers’ net positions. In International Symposium on Conformal and Probabilistic Prediction with Applications.
  39. Conformal prediction interval for dynamic time-series. In International Conference on Machine Learning, 11559–11569. PMLR.
  40. Solving linear programs with complementarity constraints using branch-and-cut. Mathematical Programming Computation, 11(2): 267–310.
  41. Adaptive conformal predictions for time series. In International Conference on Machine Learning, 25834–25866. PMLR.
Citations (22)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now