Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Distribution-free Conformal Prediction for Ordinal Classification (2404.16610v2)

Published 25 Apr 2024 in stat.ME and stat.ML

Abstract: Conformal prediction is a general distribution-free approach for constructing prediction sets combined with any machine learning algorithm that achieve valid marginal or conditional coverage in finite samples. Ordinal classification is common in real applications where the target variable has natural ordering among the class labels. In this paper, we discuss constructing distribution-free prediction sets for such ordinal classification problems by leveraging the ideas of conformal prediction and multiple testing with FWER control. Newer conformal prediction methods are developed for constructing contiguous and non-contiguous prediction sets based on marginal and conditional (class-specific) conformal $p$-values, respectively. Theoretically, we prove that the proposed methods respectively achieve satisfactory levels of marginal and class-specific conditional coverages. Through simulation study and real data analysis, these proposed methods show promising performance compared to the existing conformal method.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (32)
  1. Alan Agresti. Analysis of ordinal categorical data. John Wiley & Sons, 2010.
  2. A.N. Angelopoulos and S. Bates. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511, 2021.
  3. Uncertainty sets for image classifiers using conformal prediction. arXiv preprint arXiv:2009.14193, 2020.
  4. The limits of distribution-free conditional predictive inference. Information and Inference: A Journal of the IMA, 10(2):455–482, 2021.
  5. Learning to classify ordinal data: The data replication method. Journal of Machine Learning Research, 8:1393–1429, 2007.
  6. Knowing what you know: valid and validated confidence sets in multiclass and multilabel prediction. The Journal of Machine Learning Research, 22(1):3681–3722, 2021.
  7. A neural network approach to ordinal regression. In 2008 IEEE International Joint Conference on Neural Networks, pages 1279–1284. IEEE, 2008.
  8. An all-at-once unimodal svm approach for ordinal classification. In 2010 Ninth International Conference on Machine Learning and Applications, pages 59–64. IEEE, 2010.
  9. Multiple testing problems in pharmaceutical statistics. CRC press, 2009.
  10. Conformal prediction: a unified review of theory and new challenges. Bernoulli, 29(1):1–23, 2023.
  11. A simple approach to ordinal classification. In 12th European Conference on Machine Learning, Proceedings 12, pages 145–156. Springer, 2001.
  12. Adaptive conformal inference under distribution shift. Advances in Neural Information Processing Systems, 34:1660–1672, 2021.
  13. Ordinal regression methods: survey and experimental study. IEEE Transactions on Knowledge and Data Engineering, 28(1):127–146, 2015.
  14. Frank E Harrell. Ordinal logistic regression. In Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis, pages 311–325. Springer, 2015.
  15. Cautious deep learning. arXiv preprint arXiv:1805.09460, 2018.
  16. Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Machine Learning, 110:457–506, 2021.
  17. Random forest for ordinal responses: prediction and variable selection. Computational Statistics & Data Analysis, 96:57–73, 2016.
  18. Prediction of ordinal classes using regression trees. Fundamenta Informaticae, 47(1-2):1–13, 2001.
  19. J. Lei. Classification with confidence. Biometrika, 101(4):755–769, 2014.
  20. Improving trustworthiness of ai disease rating in medical imaging with ordinal conformal prediction sets. arXiv preprint arXiv:2207.02238, 2022.
  21. Peter McCullagh. Regression models for ordinal data. Journal of the Royal Statistical Society B, 42:109–142, 1980.
  22. Inductive confidence machines for regression. In 13th European Conference on Machine Learning, Proceedings 13, pages 345–356. Springer, 2002.
  23. In defense of one-vs-all classification. The Journal of Machine Learning Research, 5:101–141, 2004.
  24. Classification with valid and adaptive coverage. In Advances in Neural Information Processing Systems, volume 33, pages 3581–3591, 2020.
  25. Least ambiguous set-valued classifiers with bounded error levels. Journal of American Statistical Association, 114:223–234, 2019.
  26. G. Shafer and V. Vovk. A tutorial on conformal prediction. Journal of Machine Learning Research, 9:371–421, 2008.
  27. Conformal prediction under covariate shift. Advances in Neural Information Processing Systems, 32, 2019.
  28. Multi-label classification under uncertainty: a tree-based conformal prediction approach. In Conformal and Probabilistic Prediction with Applications, pages 488–512. PMLR, 2023.
  29. Algorithmic learning in a random world. Springer, 2005.
  30. Vladimir Vovk. Conditional validity of inductive conformal predictors. In Asian Conference on Machine Learning, pages 475–490. PMLR, 2012.
  31. Machine-learning applications of algorithmic randomness. In Sixteenth International Conference on Machine Learning, pages 444–453, 1999.
  32. Conformal risk control for ordinal classification. In Uncertainty in Artificial Intelligence, pages 2346–2355. PMLR, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Subhrasish Chakraborty (1 paper)
  2. Chhavi Tyagi (2 papers)
  3. Haiyan Qiao (1 paper)
  4. Wenge Guo (19 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets