Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

How to Trust Your Diffusion Model: A Convex Optimization Approach to Conformal Risk Control (2302.03791v3)

Published 7 Feb 2023 in stat.ML, cs.CV, and cs.LG

Abstract: Score-based generative modeling, informally referred to as diffusion models, continue to grow in popularity across several important domains and tasks. While they provide high-quality and diverse samples from empirical distributions, important questions remain on the reliability and trustworthiness of these sampling procedures for their responsible use in critical scenarios. Conformal prediction is a modern tool to construct finite-sample, distribution-free uncertainty guarantees for any black-box predictor. In this work, we focus on image-to-image regression tasks and we present a generalization of the Risk-Controlling Prediction Sets (RCPS) procedure, that we term $K$-RCPS, which allows to $(i)$ provide entrywise calibrated intervals for future samples of any diffusion model, and $(ii)$ control a certain notion of risk with respect to a ground truth image with minimal mean interval length. Differently from existing conformal risk control procedures, ours relies on a novel convex optimization approach that allows for multidimensional risk control while provably minimizing the mean interval length. We illustrate our approach on two real-world image denoising problems: on natural images of faces as well as on computed tomography (CT) scans of the abdomen, demonstrating state of the art performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. A rewriting system for convex optimization problems. Journal of Control and Decision, 5(1):42–60, 2018.
  2. Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313–326, 1982.
  3. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511, 2021.
  4. Learn then test: Calibrating predictive algorithms to achieve risk control. arXiv preprint arXiv:2110.01052, 2021.
  5. Conformal risk control. arXiv preprint arXiv:2208.02814, 2022a.
  6. Image-to-image regression with distribution-free uncertainty quantification and applications in imaging. In International Conference on Machine Learning, pages 717–730. PMLR, 2022b.
  7. MOSEK ApS. The MOSEK optimization toolbox for MATLAB manual. Version 9.0., 2019. URL http://docs.mosek.com/9.0/toolbox/index.html.
  8. Predictive inference with the jackknife+. The Annals of Statistics, 49(1):486–507, 2021.
  9. Conformal prediction beyond exchangeability. arXiv preprint arXiv:2202.13415, 2022.
  10. Distribution-free, risk-controlling prediction sets. Journal of the ACM (JACM), 68(6):1–34, 2021.
  11. Vidmantas Bentkus. On hoeffding’s inequalities. The Annals of Probability, 32(2):1650–1673, 2004.
  12. Pattern recognition and machine learning, volume 4. Springer, 2006.
  13. Vector quantile regression: an optimal transport approach. The Annals of Statistics, 44(3):1165–1192, 2016.
  14. Vector quantile regression and optimal transport, from theory to numerics. Empirical Economics, pages 1–28, 2020.
  15. Monge–kantorovich depth, quantiles, ranks and signs. The Annals of Statistics, 45(1):223–256, 2017.
  16. Diffdock: Diffusion steps, twists, and turns for molecular docking. arXiv preprint arXiv:2210.01776, 2022.
  17. Diffusion models in vision: A survey. arXiv preprint arXiv:2209.04747, 2022.
  18. Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
  19. CVXPY: A Python-embedded modeling language for convex optimization. Journal of Machine Learning Research, 17(83):1–5, 2016.
  20. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050–1059. PMLR, 2016.
  21. Stochastic optimization for large-scale optimal transport. Advances in neural information processing systems, 29, 2016.
  22. Nested conformal prediction and quantile out-of-bag ensemble methods. Pattern Recognition, 127:108496, 2022.
  23. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
  24. Wassily Hoeffding. Probability inequalities for sums of bounded random variables. In The collected works of Wassily Hoeffding, pages 409–426. Springer, 1994.
  25. Equivariant diffusion for molecule generation in 3d. In International Conference on Machine Learning, pages 8867–8887. PMLR, 2022.
  26. Conffusion: Confidence intervals for diffusion models. arXiv preprint arXiv:2211.09795, 2022.
  27. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
  28. Illuminating protein space with a programmable generative model. bioRxiv, 2022.
  29. Stochastic solutions for linear inverse problems using the prior implicit in a denoiser. Advances in Neural Information Processing Systems, 34:13242–13254, 2021.
  30. Brownian motion and stochastic calculus, volume 113. Springer Science & Business Media, 1991.
  31. Snips: Solving noisy inverse problems stochastically. Advances in Neural Information Processing Systems, 34:21757–21769, 2021a.
  32. Stochastic image denoising by sampling from the posterior distribution. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1866–1875, 2021b.
  33. Diffusion models for medical image analysis: A comprehensive survey. arXiv preprint arXiv:2211.07804, 2022.
  34. Regression quantiles. Econometrica: journal of the Econometric Society, pages 33–50, 1978.
  35. What’s behind the mask: Estimating uncertainty in image-to-image problems. arXiv preprint arXiv:2211.15211, 2022.
  36. Efficiently controlling multiple risks with pareto testing. arXiv preprint arXiv:2210.07913, 2022.
  37. Convergence for score-based generative modeling with polynomial complexity. arXiv preprint arXiv:2206.06227, 2022.
  38. Distribution-free prediction bands for non-parametric regression. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76(1):71–96, 2014.
  39. A kernelized stein discrepancy for goodness-of-fit tests. In International conference on machine learning, pages 276–284. PMLR, 2016.
  40. Large-scale celebfaces attributes (celeba) dataset. Retrieved August, 15(2018):11, 2018.
  41. Abdomenct-1k: Is abdominal organ segmentation a solved problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
  42. Latent-nerf for shape-guided generation of 3d shapes and textures. arXiv preprint arXiv:2211.07600, 2022.
  43. Efficient learning of generative models via finite-difference score matching. Advances in Neural Information Processing Systems, 33:19175–19188, 2020.
  44. Inductive confidence machines for regression. In European Conference on Machine Learning, pages 345–356. Springer, 2002.
  45. Conformalized quantile regression. Advances in neural information processing systems, 32, 2019.
  46. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015.
  47. Fast nonlinear vector quantile regression. arXiv preprint arXiv:2205.14977, 2022.
  48. A tutorial on conformal prediction. Journal of Machine Learning Research, 9(3), 2008.
  49. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages 2256–2265. PMLR, 2015.
  50. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
  51. Improved techniques for training score-based generative models. Advances in neural information processing systems, 33:12438–12448, 2020.
  52. Sliced score matching: A scalable approach to density and score estimation. In Uncertainty in Artificial Intelligence, pages 574–584. PMLR, 2020a.
  53. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020b.
  54. Solving inverse problems in medical imaging with score-based generative models. arXiv preprint arXiv:2111.08005, 2021.
  55. Towards a most probable recovery in optical imaging. arXiv preprint arXiv:2212.03235, 2022.
  56. Vladimir Vovk. Cross-conformal predictors. Annals of Mathematics and Artificial Intelligence, 74(1):9–28, 2015.
  57. Algorithmic learning in a random world. Springer Science & Business Media, 2005.
  58. Broadly applicable and accurate protein design by integrating structure prediction networks and diffusion generative models. bioRxiv, 2022.
  59. Measurement-conditioned denoising diffusion probabilistic model for under-sampled medical image reconstruction. arXiv preprint arXiv:2203.03623, 2022.
  60. Dream3d: Zero-shot text-to-3d synthesis using 3d shape prior and text-to-image diffusion models. arXiv preprint arXiv:2212.14704, 2022.
  61. Diffusion models: A comprehensive survey of methods and applications. arXiv preprint arXiv:2209.00796, 2022.
  62. Lion: Latent point diffusion models for 3d shape generation. arXiv preprint arXiv:2210.06978, 2022.
Citations (20)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets