Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Burst Super-Resolution with Diffusion Models for Improving Perceptual Quality (2403.19428v3)

Published 28 Mar 2024 in cs.CV

Abstract: While burst LR images are useful for improving the SR image quality compared with a single LR image, prior SR networks accepting the burst LR images are trained in a deterministic manner, which is known to produce a blurry SR image. In addition, it is difficult to perfectly align the burst LR images, making the SR image more blurry. Since such blurry images are perceptually degraded, we aim to reconstruct the sharp high-fidelity boundaries. Such high-fidelity images can be reconstructed by diffusion models. However, prior SR methods using the diffusion model are not properly optimized for the burst SR task. Specifically, the reverse process starting from a random sample is not optimized for image enhancement and restoration methods, including burst SR. In our proposed method, on the other hand, burst LR features are used to reconstruct the initial burst SR image that is fed into an intermediate step in the diffusion model. This reverse process from the intermediate step 1) skips diffusion steps for reconstructing the global structure of the image and 2) focuses on steps for refining detailed textures. Our experimental results demonstrate that our method can improve the scores of the perceptual quality metrics. Code: https://github.com/placerkyo/BSRD

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell., 38(2):295–307, 2016.
  2. Radu Timofte et al. Ntire 2018 challenge on single image super-resolution: Methods and results. In CVPRW, 2018.
  3. Deep back-projection networks for super-resolution. In CVPR, 2018.
  4. Shuhang Gu et al. AIM 2019 challenge on image extreme super-resolution: Methods and results. In ICCVW, 2019.
  5. Deep back-projectinetworks for single image super-resolution. IEEE Trans. Pattern Anal. Mach. Intell., 43(12):4323–4337, 2021.
  6. Deep burst super-resolution. In CVPR, 2021.
  7. Goutam Bhat et al. NTIRE 2021 challenge on burst super-resolution: Methods and results. In CVPRW, 2021.
  8. Goutam Bhat et al. NTIRE 2022 burst super-resolution challenge. In CVPRW, 2022.
  9. Deepsum: Deep neural network for super-resolution of unregistered multitemporal images. IEEE Trans. Geosci. Remote. Sens., 58(5):3644–3656, 2020.
  10. Photo-realistic single image super-resolution using a generative adversarial network. In CVPR, 2017.
  11. Denoising diffusion probabilistic models. In NeurIPS, 2020.
  12. Deep unsupervised learning using nonequilibrium thermodynamics. In ICML, 2015.
  13. Image super-resolution via iterative refinement. IEEE Trans. Pattern Anal. Mach. Intell., 45(4):4713–4726, 2023.
  14. Exploiting diffusion prior for real-world image super-resolution. CoRR, abs/2305.07015, 2023.
  15. Unprocessing images for learned raw denoising. In CVPR, 2019.
  16. Replacing mobile camera ISP with a single deep learning model. In CVPR Workshops, 2020.
  17. Seungjun Nah et al. NTIRE 2019 challenge on video super-resolution: Methods and results. In CVPR Workshop, 2019.
  18. Recurrent back-projection network for video super-resolution. In CVPR, 2019.
  19. Dario Fuoli et al. AIM 2020 challenge on video extreme super-resolution: Methods and results. In ECCV Workshop, 2020.
  20. Space-time-aware multi-resolution video enhancement. In CVPR, 2020.
  21. EBSR: feature enhanced burst super-resolution with deformable alignment. In CVPR Workshops, 2021.
  22. A differentiable two-stage alignment scheme for burst image reconstruction with large shift. In CVPR, 2022.
  23. Deformable convolutional networks. In ICCV, 2017.
  24. BSRT: improving burst super-resolution with swin transformer and flow-guided deformable alignment. In CVPR Workshops, 2022.
  25. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015.
  26. Parallel diffusion models of operator and image for blind inverse problems. In CVPR, 2023.
  27. Gibbsddrm: A partially collapsed gibbs sampler for solving blind inverse problems with denoising diffusion restoration. In ICML, 2023.
  28. Diffusion posterior sampling for general noisy inverse problems. In ICLR, 2023.
  29. Improving diffusion models for inverse problems using manifold constraints. In NeurIPS, 2022.
  30. Denoising diffusion restoration models. In NeurIPS, 2022.
  31. Diffusion model based posterior sampling for noisy linear inverse problems. CoRR, abs/2211.12343, 2022.
  32. Sdedit: Guided image synthesis and editing with stochastic differential equations. In ICLR, 2022.
  33. High-resolution image synthesis with latent diffusion models. In CVPR, 2022.
  34. Burstormer: Burst image restoration and enhancement transformer. In CVPR, 2023.
  35. Recovering realistic texture in image super-resolution by deep spatial feature transform. In CVPR, 2018.
  36. Restormer: Efficient transformer for high-resolution image restoration. In CVPR, 2022.
  37. Perception prioritized training of diffusion models. In CVPR, 2022.
  38. Denoising diffusion implicit models. In ICLR, 2021.
  39. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018.
  40. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NIPS, 2017.
  41. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 13(4):600–612, 2004.
  42. Decoupled weight decay regularization. In ICLR, 2019.
  43. Burst image restoration and enhancement. In CVPR, 2022.
  44. The perception-distortion tradeoff. In CVPR, 2018.
  45. Read: Retrieval-enhanced asymmetric diffusion for motion planning. In CVPR, 2024.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now