Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Frequency Compensated Diffusion Model for Real-scene Dehazing (2308.10510v2)

Published 21 Aug 2023 in cs.CV

Abstract: Due to distribution shift, deep learning based methods for image dehazing suffer from performance degradation when applied to real-world hazy images. In this paper, we consider a dehazing framework based on conditional diffusion models for improved generalization to real haze. First, we find that optimizing the training objective of diffusion models, i.e., Gaussian noise vectors, is non-trivial. The spectral bias of deep networks hinders the higher frequency modes in Gaussian vectors from being learned and hence impairs the reconstruction of image details. To tackle this issue, we design a network unit, named Frequency Compensation block (FCB), with a bank of filters that jointly emphasize the mid-to-high frequencies of an input signal. We demonstrate that diffusion models with FCB achieve significant gains in both perceptual and distortion metrics. Second, to further boost the generalization performance, we propose a novel data synthesis pipeline, HazeAug, to augment haze in terms of degree and diversity. Within the framework, a solid baseline for blind dehazing is set up where models are trained on synthetic hazy-clean pairs, and directly generalize to real data. Extensive evaluations show that the proposed dehazing diffusion model significantly outperforms state-of-the-art methods on real-world images. Our code is at https://github.com/W-Jilly/frequency-compensated-diffusion-model-pytorch.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. I-haze: a dehazing benchmark with real hazy and haze-free indoor images. In International Conference on Advanced Concepts for Intelligent Vision Systems, pages 620–631. Springer, 2018.
  2. Dense-haze: A benchmark for image dehazing with dense-haze and haze-free images. In 2019 IEEE international conference on image processing (ICIP), pages 1014–1018. IEEE, 2019.
  3. Nh-haze: An image dehazing benchmark with non-homogeneous hazy and haze-free images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 444–445, 2020.
  4. O-haze: a dehazing benchmark with real hazy and haze-free outdoor images. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 754–762, 2018.
  5. Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18208–18218, 2022.
  6. Self-guided image dehazing using progressive feature fusion. IEEE Transactions on Image Processing, 31:1217–1229, 2022.
  7. Non-local image dehazing. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1674–1682, 2016.
  8. Glenn D Boreman. Modulation transfer function in optical and electro-optical systems, volume 4. SPIE press Bellingham, Washington, 2001.
  9. Single image dehazing using color ellipsoid prior. IEEE Transactions on Image Processing, 27(2):999–1009, 2017.
  10. Difference of gaussians. Catalogue of Artificial Intelligence Tools, pages 30–30, 1984.
  11. Dehazenet: An end-to-end system for single image haze removal. IEEE Transactions on Image Processing, 25(11):5187–5198, 2016.
  12. Gated context aggregation network for image dehazing and deraining. In 2019 IEEE winter conference on applications of computer vision (WACV), pages 1375–1383. IEEE, 2019.
  13. Amplitude-phase recombination: Rethinking robustness of convolutional neural networks in frequency domain. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 458–467, 2021.
  14. Psd: Principled synthetic-to-real dehazing guided by physical priors. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7180–7189, 2021.
  15. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34:8780–8794, 2021.
  16. Multi-scale boosted dehazing network with dense feature fusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2157–2167, 2020.
  17. Fd-gan: Generative adversarial networks with fusion-discriminator for single image dehazing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 10729–10736, 2020.
  18. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  19. Cycle-dehaze: Enhanced cyclegan for single image dehazing. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 825–833, 2018.
  20. Raanan Fattal. Dehazing using color-lines. ACM transactions on graphics (TOG), 34(1):1–14, 2014.
  21. Unsupervised monocular depth estimation with left-right consistency. In CVPR, 2017.
  22. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020.
  23. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16000–16009, 2022.
  24. Single image haze removal using dark channel prior. IEEE transactions on pattern analysis and machine intelligence, 33(12):2341–2353, 2010.
  25. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
  26. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022.
  27. Image quality metrics: Psnr vs. ssim. In 2010 20th international conference on pattern recognition, pages 2366–2369. IEEE, 2010.
  28. Distribution augmentation for generative modeling. In International Conference on Machine Learning, pages 5006–5019. PMLR, 2020.
  29. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5148–5157, 2021.
  30. Aod-net: All-in-one dehazing network. In Proceedings of the IEEE international conference on computer vision, pages 4770–4778, 2017.
  31. Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing, 28(1):492–505, 2018.
  32. Pdr-net: Perception-inspired single image dehazing network with refinement. IEEE Transactions on Multimedia, 22(3):704–716, 2019.
  33. Dr-net: transmission steered single image dehazing network with weakly supervised refinement. arXiv preprint arXiv:1712.00621, 2017.
  34. End-to-end single image fog removal using enhanced cycle consistent adversarial networks. IEEE Transactions on Image Processing, 29:7819–7833, 2020.
  35. Learning deep priors for image dehazing. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2492–2500, 2019.
  36. From synthetic to real: Image dehazing collaborating with unlabeled real data. In Proceedings of the 29th ACM International Conference on Multimedia, pages 50–58, 2021.
  37. Sdedit: Image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.01073, 2021.
  38. Kevin P Murphy. Machine learning: a probabilistic perspective. MIT press, 2012.
  39. Vision in bad weather. In Proceedings of the seventh IEEE international conference on computer vision, volume 2, pages 820–827. IEEE, 1999.
  40. Bayesian defogging. International journal of computer vision, 98:263–278, 2012.
  41. Ffa-net: Feature fusion attention network for single image dehazing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 11908–11915, 2020.
  42. On the spectral bias of neural networks. In International Conference on Machine Learning, pages 5301–5310. PMLR, 2019.
  43. Multi-scale retinex for color image enhancement. In Proceedings of 3rd IEEE international conference on image processing, volume 3, pages 1003–1006. IEEE, 1996.
  44. AJ Rainal. Sampling technique for generating gaussian noise. Review of Scientific Instruments, 32(3):327–331, 1961.
  45. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015.
  46. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference Proceedings, pages 1–10, 2022.
  47. Image super-resolution via iterative refinement. arXiv preprint arXiv:2104.07636, 2021.
  48. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  49. Domain adaptation for image dehazing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2808–2817, 2020.
  50. The ciede2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations. Color Research & Application: Endorsed by Inter-Society Color Council, The Colour Group (Great Britain), Canadian Society for Color, Color Science Association of Japan, Dutch Society for the Study of Color, The Swedish Colour Centre Foundation, Colour Society of Australia, Centre Français de la Couleur, 30(1):21–30, 2005.
  51. No-reference quality assessment using natural scene statistics: Jpeg2000. IEEE Transactions on image processing, 14(11):1918–1927, 2005.
  52. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages 2256–2265. PMLR, 2015.
  53. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.
  54. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
  55. Vision transformers for single image dehazing. arXiv preprint arXiv:2204.03883, 2022.
  56. Solving inverse problems in medical imaging with score-based generative models. arXiv preprint arXiv:2111.08005, 2021.
  57. Image processing, analysis, and machine vision. Cengage Learning, 2014.
  58. Alexander Tanchenko. Visual-psnr measure of image quality. Journal of Visual Communication and Image Representation, 25(5):874–878, 2014.
  59. Investigating haze-relevant features in a learning framework for image dehazing. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2995–3000, 2014.
  60. van A Van der Schaaf and JH van van Hateren. Modelling the power spectra of natural images: statistics and information. Vision research, 36(17):2759–2770, 1996.
  61. Fully non-homogeneous atmospheric scattering modeling with convolutional neural networks for single image dehazing. arXiv preprint arXiv:2108.11292, 2021.
  62. Fwb-net: front white balance network for color shift correction in single image dehazing via atmospheric light estimation. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2040–2044. IEEE, 2021.
  63. High-frequency component helps explain the generalization of convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8684–8694, 2020.
  64. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004.
  65. Multiscale structural similarity for image quality assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, volume 2, pages 1398–1402. Ieee, 2003.
  66. Computational colour science using MATLAB. John Wiley & Sons, 2012.
  67. Deblurring via stochastic refinement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16293–16303, June 2022.
  68. Fast image dehazing using improved dark channel prior. In 2012 IEEE international conference on information science and technology, pages 663–667. IEEE, 2012.
  69. Fda: Fourier domain adaptation for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4085–4095, 2020.
  70. Densely connected pyramid dehazing network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3194–3203, 2018.
  71. Deep fully convolutional regression networks for single image haze removal. In 2017 IEEE Visual Communications and Image Processing (VCIP), pages 1–4. IEEE, 2017.
  72. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2223–2232, 2017.
  73. A fast single image haze removal algorithm using color attenuation prior. IEEE transactions on image processing, 24(11):3522–3533, 2015.
Citations (15)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - W-Jilly/frequency-compensated-diffusion-model-pytorch (39 stars)