Out-of-Distribution Detection with a Single Unconditional Diffusion Model (2405.11881v3)
Abstract: Out-of-distribution (OOD) detection is a critical task in machine learning that seeks to identify abnormal samples. Traditionally, unsupervised methods utilize a deep generative model for OOD detection. However, such approaches require a new model to be trained for each inlier dataset. This paper explores whether a single model can perform OOD detection across diverse tasks. To that end, we introduce Diffusion Paths (DiffPath), which uses a single diffusion model originally trained to perform unconditional generation for OOD detection. We introduce a novel technique of measuring the rate-of-change and curvature of the diffusion paths connecting samples to the standard normal. Extensive experiments show that with a single model, DiffPath is competitive with prior work using individual models on a variety of OOD tasks involving different distributions. Our code is publicly available at https://github.com/clear-nus/diffpath.
- Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 427–436, 2015.
- Do deep generative models know what they don’t know? In International Conference on Learning Representations, 2018.
- Generalized out-of-distribution detection: A survey. arXiv preprint arXiv:2110.11334, 2021.
- Likelihood ratios for out-of-distribution detection. Advances in neural information processing systems, 32, 2019.
- Waic, but why? generative ensembles for robust anomaly detection. arXiv preprint arXiv:1810.01392, 2018.
- Density of states estimation for out of distribution detection. In International Conference on Artificial Intelligence and Statistics, pages 3232–3240. PMLR, 2021.
- Denoising diffusion models for out-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2947–2956, 2023.
- Unsupervised out-of-distribution detection with diffusion inpainting. In International Conference on Machine Learning, pages 22528–22538. PMLR, 2023.
- Projection regret: Reducing background bias for novelty detection via diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
- Do we really need to learn representations from in-domain data for outlier detection? arXiv preprint arXiv:2105.09270, 2021.
- Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
- On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.
- Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020a.
- Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
- Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.
- Input complexity and out-of-distribution detection with likelihood-based generative models. In International Conference on Learning Representations, 2019.
- Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
- Genie: Higher-order denoising diffusion solvers. Advances in Neural Information Processing Systems, 35:30150–30166, 2022.
- Denoising diffusion implicit models. In International Conference on Learning Representations, 2020b.
- Numerical Solution of Stochastic Differential Equations. Springer, Berlin, 1992.
- Understanding ddpm latent codes through optimal transport. In The Eleventh International Conference on Learning Representations, 2022.
- Dual diffusion implicit bridges for image-to-image translation. In The Eleventh International Conference on Learning Representations, 2022.
- Applied stochastic differential equations, volume 10. Cambridge University Press, 2019.
- Improving reconstruction autoencoder out-of-distribution detection with mahalanobis distance. arXiv preprint arXiv:1812.02765, 2018.
- Csi: Novelty detection via contrastive learning on distributionally shifted instances. Advances in neural information processing systems, 33:11839–11852, 2020.
- Ssd: A unified framework for self-supervised outlier detection. arXiv preprint arXiv:2103.12051, 2021.
- Detecting out-of-distribution inputs to deep generative models using typicality. arXiv preprint arXiv:1906.02994, 2019.
- Implicit generation and modeling with energy based models. Advances in Neural Information Processing Systems, 32, 2019.
- Multiscale score matching for out-of-distribution detection. In International Conference on Learning Representations, 2020.
- Anomaly detection with robust deep autoencoders. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 665–674, 2017.
- Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In International conference on information processing in medical imaging, pages 146–157. Springer, 2017.
- The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018.
- Consistency models. In International Conference on Machine Learning, pages 32211–32252. PMLR, 2023.
- Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162–8171. PMLR, 2021.
- Vaebm: A symbiosis between variational autoencoders and energy-based models. In International Conference on Learning Representations, 2020.
- Improved contrastive divergence training of energy-based models. In International Conference on Machine Learning, pages 2837–2848. PMLR, 2021.
- Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 31, 2018.
- Maximum likelihood training of score-based diffusion models. Advances in neural information processing systems, 34:1415–1428, 2021.
- Computational optimal transport. Foundations and Trends in Machine Learning, 11(5-6):355–607, 2019.