Papers
Topics
Authors
Recent
Search
2000 character limit reached

Morse Neural Networks for Uncertainty Quantification

Published 2 Jul 2023 in stat.ML, cs.AI, and cs.LG | (2307.00667v1)

Abstract: We introduce a new deep generative model useful for uncertainty quantification: the Morse neural network, which generalizes the unnormalized Gaussian densities to have modes of high-dimensional submanifolds instead of just discrete points. Fitting the Morse neural network via a KL-divergence loss yields 1) a (unnormalized) generative density, 2) an OOD detector, 3) a calibration temperature, 4) a generative sampler, along with in the supervised case 5) a distance aware-classifier. The Morse network can be used on top of a pre-trained network to bring distance-aware calibration w.r.t the training data. Because of its versatility, the Morse neural networks unifies many techniques: e.g., the Entropic Out-of-Distribution Detector of (Mac^edo et al., 2021) in OOD detection, the one class Deep Support Vector Description method of (Ruff et al., 2018) in anomaly detection, or the Contrastive One Class classifier in continuous learning (Sun et al., 2021). The Morse neural network has connections to support vector machines, kernel methods, and Morse theory in topology.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Amari, S.-i. Information Geometry and Its Applications. Applied mathematical sciences. Springer, 2016.
  2. Concrete problems in AI safety. https://arxiv.org/abs/1606.06565, 2016.
  3. Generalized energy based models. In International Conference on Learning Representations, 2021.
  4. Morse-Bott theory and equivariant cohomology. 1995.
  5. A proof of the Morse-Bott lemma. Expositiones Mathematicae, 2004.
  6. A connection between cut locus, Thom space and Morse-Bott functions. to appear in Algebraic & Geometric Topology., 2021.
  7. Anomalous example detection in deep learning: A survey. IEEE Access, 8, 2020.
  8. Your GAN is secretly an energy-based model and you should use discriminator driven latent sampling. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), NeurIPS, 2020.
  9. Learning energy-based models by diffusion recovery likelihood. In International Conference on Learning Representations, 2021.
  10. Explaining and harnessing adversarial examples. In ICLR, 2015.
  11. Deep Learning. MIT Press, 2016.
  12. On calibration of modern neural networks. In ICML, 2017.
  13. Reliable graph neural networks for drug discovery under distributional shift. In NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications, 2021.
  14. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations, 2017. URL https://openreview.net/forum?id=Hkg4TI9xl.
  15. Hirsch, M. W. Differential topology. Springer New York, 1997. ISBN 0387901485 3540901485.
  16. Continual learning by using information of each class holistically. AAAI, 2021.
  17. Highly accurate protein structure prediction with alphafold. Nature, 596:583 – 589, 2021.
  18. Measuring and Improving Model-Moderator Collaboration using Uncertainty Estimation, 2021.
  19. Simple and scalable predictive uncertainty estimation using deep ensembles. In NeurIPS, 2017.
  20. An evaluation dataset for intent classification and out-of-scope prediction. In EMNLP-IJCNLP 2019.
  21. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurIPS, volume 31, 2018.
  22. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. In NeurIPS, 2020.
  23. A simple approach to improve single-model deep uncertainty via distance-awareness. JMLR, 2022.
  24. Entropic out-of-distribution detection. In International Joint Conference on Neural Networks (IJCNN), pp.  1–8, 2021.
  25. Density of states estimation for out of distribution detection. In AISTATS, 2021.
  26. Morse, A. P. The behavior of a function on its critical set. Annals of Mathematics, 40:62–70, 1939.
  27. Murphy, K. P. Probabilistic Machine Learning: Advanced Topics. MIT Press, 2023. URL http://probml.github.io/book2.
  28. Understanding the failure modes of out-of-distribution generalization. In ICLR, 2021.
  29. Do deep generative models know what they don’t know? In ICLR, 2019.
  30. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In CVPR, 2015.
  31. Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. In NeurIPS, 2019.
  32. Using mixup as a regularizer can surprisingly improve accuracy & out-of-distribution robustness. In NeurIPS, 2022.
  33. Prasad, S. Cut locus of submanifolds: A geometric and topological viewpoint. 2023.
  34. Likelihood ratios for out-of-distribution detection. In NeurIPS, volume 32, 2019.
  35. A simple fix to Mahalanobis distance for improving near-ood detection. arXiv preprint arXiv:2106.09022, 2021.
  36. Does your dermatology classifier know what it doesn’t know? detecting the long-tail of unseen conditions. Medical Imaging Analysis, 2021.
  37. Deep one-class classification. In ICML, 2018.
  38. A unified survey on anomaly, novelty, open-set, and out of-distribution detection: Solutions and future challenges. TMLR, 2022.
  39. Sard, A. The measure of the critical values of differentiable maps. Bulletin of the American Mathematical Society, 48:883–890, 1942.
  40. Deep learning-based object detection and scene perception under bad weather conditions. Electronics, 11(4), 2022.
  41. ILCOC: An incremental learning framework based on contrastive one-class classifiers. In CVPR, 2021.
  42. Energy-based open-world uncertainty modeling for confidence calibration. In ICCV 2021, 2021.
  43. Conjugate energy-based models. In Third Symposium on Advances in Approximate Bayesian Inference, 2021.
  44. Poisson flow generative models. In NeurIPS, 2022.
  45. Energy-based generative adversarial networks. In International Conference on Learning Representations, 2017.
Citations (4)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up