Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

How Good Are Deep Generative Models for Solving Inverse Problems? (2312.12691v1)

Published 20 Dec 2023 in cs.LG and cs.CV

Abstract: Deep generative models, such as diffusion models, GANs, and IMLE, have shown impressive capability in tackling inverse problems. However, the validity of model-generated solutions w.r.t. the forward problem and the reliability of associated uncertainty estimates remain understudied. This study evaluates recent diffusion-based, GAN-based, and IMLE-based methods on three inverse problems, i.e., $16\times$ super-resolution, colourization, and image decompression. We assess the validity of these models' outputs as solutions to the inverse problems and conduct a thorough analysis of the reliability of the models' estimates of uncertainty over the solution. Overall, we find that the IMLE-based CHIMLE method outperforms other methods in terms of producing valid solutions and reliable uncertainty estimates.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (28)
  1. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. ArXiv, abs/2107.07511, 2021.
  2. Image-to-image regression with distribution-free uncertainty quantification and applications in imaging. ArXiv, abs/2202.05265, 2022.
  3. Image colorization: A survey and dataset. ArXiv, abs/2008.10774, 2020.
  4. Distribution-free, risk-controlling prediction sets. J. ACM, 68:43:1–43:34, 2021.
  5. Non-uniform diffusion models. arXiv preprint arXiv:2207.09786, 2022.
  6. Raise: A raw images dataset for digital image forensics. In Proceedings of the 6th ACM Multimedia Systems Conference, pages 219–224, 2015.
  7. Conformal prediction is robust to label noise. ArXiv, abs/2209.14295, 2022.
  8. Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680, 2014.
  9. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
  10. Conffusion: Confidence intervals for diffusion models. ArXiv, abs/2211.09795, 2022.
  11. Qs-attn: Query-selected attention for contrastive learning in i2i translation. arXiv preprint arXiv:2203.08483, 2022.
  12. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1125–1134, 2017.
  13. Denoising diffusion restoration models. In Advances in Neural Information Processing Systems, 2022.
  14. Ke Li and Jitendra Malik. Fast k-nearest neighbour search via Dynamic Continuous Indexing. In International Conference on Machine Learning, pages 671–679, 2016.
  15. Ke Li and Jitendra Malik. Fast k-nearest neighbour search via Prioritized DCI. In International Conference on Machine Learning, pages 2081–2090, 2017.
  16. Ke Li and Jitendra Malik. Implicit maximum likelihood estimation. arXiv preprint arXiv:1809.09087, 2018.
  17. Multimodal image synthesis with conditional implicit maximum likelihood estimation. International Journal of Computer Vision, May 2020.
  18. DivCo : Diverse conditional image synthesis via contrastive generative adversarial network. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16372–16381, 2021.
  19. Mode seeking generative adversarial networks for diverse image synthesis. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1429–1437, 2019.
  20. Chimle: Conditional hierarchical imle for multimodal conditional image synthesis. Advances in Neural Information Processing Systems, 35:280–296, 2022.
  21. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115:211–252, 2015.
  22. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages 2256–2265. PMLR, 2015.
  23. How to trust your diffusion model: A convex optimization approach to conformal risk control. In International Conference on Machine Learning, 2023.
  24. Predictive inference with feature conformal prediction. ArXiv, abs/2210.00173, 2022.
  25. Modulated contrast for versatile image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
  26. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018.
  27. Toward multimodal image-to-image translation. In NIPS, 2017.
  28. Toward multimodal image-to-image translation. In Advances in Neural Information Processing Systems, pages 465–476, 2017.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now