Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Stochastic super-resolution for Gaussian microtextures (2405.15399v2)

Published 24 May 2024 in eess.IV

Abstract: Super-Resolution (SR) is the problem that consists in reconstructing images that have been degraded by a zoom-out operator. This is an ill-posed problem that does not have a unique solution, and numerical approaches rely on a prior on high-resolution images. While optimization-based methods are generally deterministic, with the rise of image generative models more and more interest has been given to stochastic SR, that is, sampling among all possible SR images associated with a given low-resolution input. In this paper, we construct an efficient, stable and provably exact sampler for the stochastic SR of Gaussian microtextures. Even though our approach is limited regarding the scope of images it encompasses, our algorithm is competitive with deep learning state-of-the-art methods both in terms of perceptual metric and execution time when applied to microtextures. The framework of Gaussian microtextures also allows us to rigorously discuss the limitations of various reconstruction metrics to evaluate the efficiency of SR routines.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. Ntire 2017 challenge on single image super-resolution: Dataset and study. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017.
  2. WPPNets and WPPFlows: The power of Wasserstein patch priors for superresolution. SIAM Journal on Imaging Sciences, 16(3):1033–1067, 2023.
  3. Explorable super resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2716–2725, 2020.
  4. Super-resolution with deep convolutional sufficient statistics. In Yoshua Bengio and Yann LeCun, editors, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016.
  5. Super-resolution through neighbor embedding. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004., volume 1, pages I–I, 2004.
  6. A statistically constrained internal method for single image super-resolution. In 26th International Conference on Pattern Recognition, ICPR 2022, Montreal, QC, Canada, August 21-25, 2022, pages 1322–1328. IEEE, 2022.
  7. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations, 2023.
  8. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  9. A compact representation of random phase and Gaussian textures. In ICASSP’12, pages 1381–1384, 2012.
  10. Diffusion models beat gans on image synthesis. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 8780–8794. Curran Associates, Inc., 2021.
  11. Learning a deep convolutional network for image super-resolution. In European Conference on Computer Vision, 2014.
  12. J.L. Doob. Stochastic Processes. Probability and Statistics Series. Wiley, 1953.
  13. Image and video upscaling from local self-examples. ACM Trans. Graph., 28(3):1–10, 2010.
  14. Example-based super-resolution. IEEE Computer Graphics and Applications, 22(2):56–65, 2002.
  15. Gabor noise by example. ACM Trans. Graph., 31(4):73:1–73:9, jul 2012.
  16. A texton for fast and flexible Gaussian texture synthesis. In Proceedings of the 22nd European Signal Processing Conference (EUSIPCO), pages 1686–1690, 2014.
  17. Texton noise. Computer Graphics Forum, 2017.
  18. A texture synthesis model based on semi-discrete optimal transport in patch space. SIAM Journal on Imaging Sciences, 11(4):2456–2493, 2018.
  19. Random Phase Textures: Theory and Synthesis. IEEE Transactions on Image Processing, 20(1):257–267, 2011.
  20. An Algorithm for Gaussian Texture Inpainting. Image Processing On Line, 7:262–277, 2017. https://doi.org/10.5201/ipol.2017.198.
  21. Texture Inpainting Using Efficient Gaussian Conditional Simulation. SIAM Journal on Imaging Sciences, 10(3):1446–1474, 2017.
  22. Microtexture inpainting through Gaussian conditional simulation. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1204–1208, 2016.
  23. Generative Adversarial Nets. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014.
  24. Optimal patch assignment for statistically constrained texture synthesis. In F. Lauze, Y. Dong, and A. B. Dahl, editors, Scale Space and Variational Methods in Computer Vision: 6th International Conference, SSVM 2017, Kolding, Denmark, June 4-8, 2017, Proceedings, pages 172–183, Cham, 2017. Springer International Publishing.
  25. Wasserstein Patch Prior for Image Superresolution. IEEE Transactions on Computational Imaging, 8:693–704, 2022.
  26. Denoising diffusion probabilistic models. In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
  27. Image quality metrics: Psnr vs. ssim. In 2010 20th International Conference on Pattern Recognition, pages 2366–2369, 2010.
  28. Wasserstein generative models for patch-based texture synthesis. In Abderrahim Elmoataz, Jalal Fadili, Yvain Quéau, Julien Rabin, and Loïc Simon, editors, Scale Space and Variational Methods in Computer Vision, pages 269–280, Cham, 2021. Springer International Publishing.
  29. A generative model for texture synthesis based on optimal transport between feature distributions. J. Math. Imaging Vis., 65(1):4–28, 2023.
  30. Sampling network guided cross-entropy method for unsupervised point cloud registration. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 6108–6117, Los Alamitos, CA, USA, oct 2021. IEEE Computer Society.
  31. Perceptual losses for real-time style transfer and super-resolution. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision – ECCV 2016, pages 694–711, Cham, 2016. Springer International Publishing.
  32. Plug-and-play methods for integrating physical and learned models in computational imaging: Theory, algorithms, and applications. IEEE Signal Processing Magazine, 40(1):85–97, 2023.
  33. On the convergence of the conjugate gradient method for singular linear operator equations. SIAM Journal on Numerical Analysis, 9(1):165–181, 1972.
  34. Denoising diffusion restoration models. In ICLR Workshop on Deep Generative Models for Highly Structured Data, 2022.
  35. Accurate image super-resolution using very deep convolutional networks. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1646–1654, 2016.
  36. Glow: Generative Flow with Invertible 1x1 Convolutions. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
  37. Deep laplacian pyramid networks for fast and accurate super-resolution. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5835–5843, 2017.
  38. C. Lantuejoul. Geostatistical Simulation: Models and Algorithms. Collezione legale Pirola. Springer Berlin Heidelberg, 2001.
  39. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 105–114, 2017.
  40. Enhanced deep residual networks for single image super-resolution. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017.
  41. Variational autoencoder for reference based image super-resolution. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 516–525, Los Alamitos, CA, USA, jun 2021. IEEE Computer Society.
  42. SRFlow: Learning the Super-Resolution Space with Normalizing Flow. In Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V, pages 715–732, Berlin, Heidelberg, 2020. Springer-Verlag. event-place: Glasgow, United Kingdom.
  43. Cyclostationary gaussian noise: theory and synthesis. Computer Graphics Forum, 40:xx–yy, 2021.
  44. Semi-unbalanced regularized optimal transport for image restoration. In 31st European Signal Processing Conference, EUSIPCO 2023, Helsinki, Finland, September 4-8, 2023, pages 466–470. IEEE, 2023.
  45. Semi-unbalanced optimal transport for image restoration and synthesis. working paper or preprint, 2024.
  46. Lionel Moisan. Periodic Plus Smooth Image Decomposition. Journal of Mathematical Imaging and Vision, 39(2):161–179, 2011.
  47. Deep learning techniques for inverse problems in imaging. IEEE Journal on Selected Areas in Information Theory, 1(1):39–56, 2020.
  48. R. Penrose. A generalized inverse for matrices. Mathematical Proceedings of the Cambridge Philosophical Society, 51(3):406–413, 1955.
  49. Stochastic super-resolution for gaussian textures. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, 2023.
  50. Solving linear inverse problems provably via posterior sampling with latent diffusion models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
  51. Image Super-Resolution Via Iterative Refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1–14, 2022.
  52. EnhanceNet: Single Image Super-Resolution through Automated Texture Synthesis. In Computer Vision (ICCV), 2017 IEEE International Conference on, pages 4501–4510. IEEE, 2017.
  53. Very deep convolutional networks for large-scale image recognition. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
  54. Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations (ICLR), May 2023.
  55. Score-Based Generative Modeling through Stochastic Differential Equations. 2023.
  56. Amortised map inference for image super-resolution. In International Conference on Learning Representations, 2017.
  57. Deep image prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
  58. Deep image prior. Int. J. Comput. Vis., 128(7):1867–1888, 2020.
  59. Recovering realistic texture in image super-resolution by deep spatial feature transform. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
  60. ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks. In Laura Leal-Taixé and Stefan Roth, editors, Computer Vision – ECCV 2018 Workshops, pages 63–79, Cham, 2019. Springer International Publishing.
  61. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004.
  62. Synthesizing and mixing stationary Gaussian texture models. SIAM J. on Imaging Science, 8(1):476–508, 2014.
  63. Solving inverse problems with piecewise linear estimators: From Gaussian mixture models to structured sparsity. Image Processing, IEEE Transactions on, 21(5):2481–2499, May 2012.
  64. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 586–595, Los Alamitos, CA, USA, June 2018. IEEE Computer Society.
  65. Crossnet: An end-to-end reference-based super resolution network using cross-scale warping. In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision – ECCV 2018, pages 87–104, Cham, 2018. Springer International Publishing.
  66. From learning models of natural image patches to whole image restoration. In 2011 International Conference on Computer Vision, pages 479–486, 2011.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now