Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash 90 tok/s
Gemini 2.5 Pro 49 tok/s Pro
GPT-5 Medium 22 tok/s
GPT-5 High 36 tok/s Pro
GPT-4o 91 tok/s
GPT OSS 120B 463 tok/s Pro
Kimi K2 213 tok/s Pro
2000 character limit reached

CreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Tasks (2401.05043v3)

Published 10 Jan 2024 in cs.LG and cs.AI

Abstract: Effective uncertainty estimation is becoming increasingly attractive for enhancing the reliability of neural networks. This work presents a novel approach, termed Credal-Set Interval Neural Networks (CreINNs), for classification. CreINNs retain the fundamental structure of traditional Interval Neural Networks, capturing weight uncertainty through deterministic intervals. CreINNs are designed to predict an upper and a lower probability bound for each class, rather than a single probability value. The probability intervals can define a credal set, facilitating estimating different types of uncertainties associated with predictions. Experiments on standard multiclass and binary classification tasks demonstrate that the proposed CreINNs can achieve superior or comparable quality of uncertainty estimation compared to variational Bayesian Neural Networks (BNNs) and Deep Ensembles. Furthermore, CreINNs significantly reduce the computational complexity of variational BNNs during inference. Moreover, the effective uncertainty quantification of CreINNs is also verified when the input data are intervals.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (56)
  1. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information Fusion, 76:243–297, 2021.
  2. Deep ensembles work, but are they necessary? Advances in Neural Information Processing Systems, 35:33646–33660, 2022.
  3. Disaggregated total uncertainty measure for credal sets. International Journal of General Systems, 35(1):29–44, 2006.
  4. Benchmarking bayesian deep learning on diabetic retinopathy detection tasks. In Proceedings of Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
  5. Weight uncertainty in neural network. In Proceedings of the International Conference on Machine Learning, pages 1613–1622. PMLR, 2015.
  6. Imprecise Bayesian neural networks. arXiv preprint arXiv:2302.09656, 2023.
  7. Likelihood-based imprecise regression. International Journal of Approximate Reasoning, 53(8):1137–1154, 2012.
  8. Conservative uncertainty estimation by fitting prior networks. In International Conference on Learning Representations, 2019.
  9. Learning reliable classifiers from small or incomplete data sets: The naive credal classifier 2. Journal of Machine Learning Research, 9(4), 2008.
  10. Bayesian networks with imprecise probabilities: Theory and application to classification. Data Mining: Foundations and Intelligent Paradigms: Volume 1: Clustering, Association and Classification, pages 49–93, 2012.
  11. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 113–123, 2019.
  12. Probability intervals: A tool for uncertain reasoning. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 02(02):167–196, June 1994.
  13. Decomposition of uncertainty in bayesian deep learning for efficient and risk-sensitive learning. In International Conference on Machine Learning, pages 1184–1193. PMLR, 2018.
  14. Large scale structure of neural network loss landscapes. Advances in Neural Information Processing Systems, 32, 2019.
  15. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the International Conference on Machine Learning, pages 1050–1059. PMLR, 2016.
  16. Z.A. Garczarczyk. Interval neural networks. In Proceedings of the IEEE International Symposium on Circuits and Systems, volume 3, pages 567–570 vol.3, 2000.
  17. A survey of uncertainty in deep neural networks. arXiv preprint arXiv:2107.03342, 2021.
  18. Evaluating scalable bayesian deep learning methods for robust computer vision. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 318–319, 2020.
  19. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  20. Bayesian deep ensembles via the neural tangent kernel. Advances in neural information processing systems, 33:1010–1022, 2020.
  21. Interval arithmetic: From principles to implementation. Journal of the ACM (JACM), 48(5):1038–1068, 2001.
  22. The No-U-Turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res., 15(1):1593–1623, 2014.
  23. The two dimensions of worst-case training and their integrated effect for out-of-domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9631–9641, 2022.
  24. Fr3: A fuzzy rule learner for inducing reliable classifiers. IEEE Transactions on Fuzzy Systems, 17(1):138–149, 2008.
  25. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning, 110(3):457–506, 2021.
  26. Quantification of credal uncertainty in machine learning: A critical analysis and empirical comparison. In Proceedings of the Uncertainty in Artificial Intelligence, pages 548–557. PMLR, 2022.
  27. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the International Conference on Machine Learning, pages 448–456. PMLR, 2015.
  28. Hands-on Bayesian neural networks—A tutorial for deep learning users. IEEE Computational Intelligence Magazine, 17(2):29–48, 2022.
  29. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017.
  30. Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114, 2013.
  31. Interval probabilistic neural network. Neural Computing and Applications, 28(4):817–834, 2017.
  32. Cifar-10 (canadian institute for advanced research). 2009.
  33. Multi-layer perceptrons. In Computational intelligence: a methodological introduction, pages 53–124. Springer, 2022.
  34. Fairness without demographics through adversarially reweighted learning. Advances in Neural Information Processing Systems, 33:728–740, 2020.
  35. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in Neural Information Processing Systems, 30, 2017.
  36. Reliable confidence measures for medical diagnosis with evolutionary algorithms. IEEE Transactions on Information Technology in Biomedicine, 15(1):93–99, 2010.
  37. Isaac Levi. The enterprise of knowledge: An essay on knowledge, credal probability, and chance. MIT press, 1980.
  38. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. Advances in Neural Information Processing Systems, 33:7498–7512, 2020.
  39. Shifts: A dataset of real distributional shift across multiple large-scale tasks. In Proceedings of Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
  40. Epistemic deep learning. arXiv preprint arXiv:2206.07609, 2022.
  41. Variational dropout sparsifies deep neural networks. In Proceedings of the International Conference on Machine Learning, pages 2498–2507. PMLR, 2017.
  42. Credal sets representable by reachable probability intervals and belief functions. International Journal of Approximate Reasoning, 129:84–102, 2021.
  43. Learning from failure: De-biasing classifier from biased classifier. Advances in Neural Information Processing Systems, 33:20673–20684, 2020.
  44. Radford M Neal et al. MCMC using Hamiltonian dynamics. Handbook of Markov Chain Monte Carlo, 2(11):2, 2011.
  45. Reading digits in natural images with unsupervised feature learning. 2011.
  46. Detecting failure modes in image reconstructions with interval neural network uncertainty. International Journal of Computer Assisted Radiology and Surgery, 16(12):2089–2097, 2021.
  47. Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems, 32, 2019.
  48. AD Pearman. A weighted maximin and maximax approach to multiple criteria decision making. Journal of the Operational Research Society, 28(3):584–587, 1977.
  49. Distributionally robust neural networks. In Proceedings of the International Conference on Learning Representations, 2020.
  50. Is the volume of a credal set a good measure for epistemic uncertainty? In Uncertainty in Artificial Intelligence, pages 1795–1804. PMLR, 2023.
  51. Reliable classification: Learning classifiers that distinguish aleatoric and epistemic uncertainty. Information Sciences, 255:16–29, 2014.
  52. Ensemble-based uncertainty quantification: Bayesian versus credal inference. In PROCEEDINGS 31. WORKSHOP COMPUTATIONAL INTELLIGENCE, volume 25, page 63, 2021.
  53. Claude Elwood Shannon. A mathematical theory of communication. The Bell system technical journal, 27(3):379–423, 1948.
  54. Neural network model for imprecise regression with interval dependent variables. Neural Networks, 161:550–564, 2023.
  55. Flipout: Efficient pseudo-independent weight perturbations on mini-batches. In Proceedings of the International Conference on Learning Representations, 2018.
  56. Marco Zaffalon. The naive credal classifier. Journal of statistical planning and inference, 105(1):5–21, 2002.
Citations (7)
View on Semantic Scholar
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Don't miss out on important new AI/ML research

See which papers are being discussed right now on X, Reddit, and more:

Explore Trending Papers

“Emergent Mind helps me see which AI papers have caught fire online.”

Philip

Philip

Creator, AI Explained on YouTube