Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Convergence of Nonconvex PnP-ADMM with MMSE Denoisers (2311.18810v1)

Published 30 Nov 2023 in cs.CV

Abstract: Plug-and-Play Alternating Direction Method of Multipliers (PnP-ADMM) is a widely-used algorithm for solving inverse problems by integrating physical measurement models and convolutional neural network (CNN) priors. PnP-ADMM has been theoretically proven to converge for convex data-fidelity terms and nonexpansive CNNs. It has however been observed that PnP-ADMM often empirically converges even for expansive CNNs. This paper presents a theoretical explanation for the observed stability of PnP-ADMM based on the interpretation of the CNN prior as a minimum mean-squared error (MMSE) denoiser. Our explanation parallels a similar argument recently made for the iterative shrinkage/thresholding algorithm variant of PnP (PnP-ISTA) and relies on the connection between MMSE denoisers and proximal operators. We also numerically evaluate the performance gap between PnP-ADMM using a nonexpansive DnCNN denoiser and expansive DRUNet denoiser, thus motivating the use of expansive CNNs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (24)
  1. “An EM algorithm for wavelet-based image restoration,” IEEE Trans. Image Process., vol. 12, no. 8, pp. 906–916, 2003.
  2. A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM Journal on Imag. Sciences, vol. 2, no. 1, pp. 183–202, 2009.
  3. “Fast image recovery using variable splitting and constrained optimization,” IEEE Trans. Image Process., vol. 19, no. 9, pp. 2345–2356, September 2010.
  4. “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Foundations and Trends in Machine Learning, vol. 3, no. 1, pp. 1–122, 2011.
  5. “Plug-and-play priors for model based reconstruction,” IEEE Glob. Conf. Signal Inf. Process., pp. 945–948, 2013.
  6. “Learning deep CNN denoiser prior for image restoration,” in Proc. IEEE Conf. Computer Vision and Pattern Recog. (CVPR), Honolulu, HI, USA, July 2017, pp. 2808–2817.
  7. “Plug-and-play methods for magnetic resonance imaging: Using denoisers for image recovery,” IEEE Signal Process. Mag., vol. 37, no. 1, pp. 105–116, 2020.
  8. “Plug-and-play image restoration with deep denoiser prior,” IEEE Trans. Pattern Anal. and Machine Intell., 2022.
  9. “Plug-and-play priors for bright field electron tomography and sparse interpolation,” IEEE Trans. Comput. Imag., vol. 2, no. 4, pp. 408–423, December 2016.
  10. “Plug-and-play admm for image restoration: Fixed-point convergence and applications,” IEEE Trans. on Comput. Imag., vol. 3, no. 1, pp. 84–98, 2016.
  11. “Plug-and-play methods provably converge with properly trained denoisers,” in Proc. 36th Int. Conf. Machine Learning (ICML), June 2019, pp. 5546–5557.
  12. E. T. Reehorst and P. Schniter, “Regularization by denoising: Clarifications and new interpretations,” IEEE Trans. Comput. Imag., vol. 5, no. 1, pp. 52–67, Mar. 2019.
  13. Y. Sun and et al., “Scalable plug-and-play admm with convergence guarantees,” IEEE Trans. on Comput. Imag., vol. 7, pp. 849–863, 2021.
  14. “Proximal denoiser for convergent plug-and-play optimization with nonconvex regularization,” in Proc. 39th Int. Conf. Machine Learning (ICML), Baltimore, MD, July 17-23, 2022, pp. 9483–9505.
  15. “Plug-and-play methods for integrating physical and learned models in computational imaging,” IEEE Signal Process. Mag., vol. 40, no. 1, pp. 85–97, Jan. 2023.
  16. “Provable convergence of plug-and-play priors with MMSE denoisers,” IEEE Signal Process. Letters, vol. 27, pp. 1280–1284, 2020.
  17. R. Gribonval, “Should penalized least squares regression be interpreted as maximum a posteriori estimation?,” IEEE Trans. Signal Process., vol. 59, no. 5, pp. 2405–2410, May 2011.
  18. Z. Li and J. Li, “A simple proximal stochastic gradient method for nonsmooth nonconvex optimization,” Adv. in Neural Inf. Process. Syst., vol. 31, 2018.
  19. “Convergence of multi-block Bregman ADMM for nonconvex composite problems,” Sci. China Inf. Sciences, vol. 61, 2018.
  20. “Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints,” Int. Journal of Computer Math., vol. 94, no. 8, 2017.
  21. “Structured nonconvex and nonsmooth optimization: algorithms and iteration complexity analysis,” Comput. Optim. and Appl., vol. 72, no. 1, pp. 115–157, 2019.
  22. M. Yashtini, “Multi-block nonconvex nonsmooth proximal admm: Convergence and rates under kurdyka–łojasiewicz property,” Journal of Optim. Theory and Appl., vol. 190, no. 3, pp. 966–998, 2021.
  23. A. Levin and et al., “Understanding and evaluating blind deconvolution algorithms,” in Proc. IEEE Conf. Computer Vision and Pattern Recog (CVPR), 2009.
  24. “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” in Proc. 8th Int. Conf. Computer Vision, July 2001, vol. 2, pp. 416–423.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now