Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
117 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

VIFNet: An End-to-end Visible-Infrared Fusion Network for Image Dehazing (2404.07790v1)

Published 11 Apr 2024 in cs.CV

Abstract: Image dehazing poses significant challenges in environmental perception. Recent research mainly focus on deep learning-based methods with single modality, while they may result in severe information loss especially in dense-haze scenarios. The infrared image exhibits robustness to the haze, however, existing methods have primarily treated the infrared modality as auxiliary information, failing to fully explore its rich information in dehazing. To address this challenge, the key insight of this study is to design a visible-infrared fusion network for image dehazing. In particular, we propose a multi-scale Deep Structure Feature Extraction (DSFE) module, which incorporates the Channel-Pixel Attention Block (CPAB) to restore more spatial and marginal information within the deep structural features. Additionally, we introduce an inconsistency weighted fusion strategy to merge the two modalities by leveraging the more reliable information. To validate this, we construct a visible-infrared multimodal dataset called AirSim-VID based on the AirSim simulation platform. Extensive experiments performed on challenging real and simulated image datasets demonstrate that VIFNet can outperform many state-of-the-art competing methods. The code and dataset are available at https://github.com/mengyu212/VIFNet_dehazing.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Dense-haze: A benchmark for image dehazing with dense-haze and haze-free images, in: Proc. IEEE Int. Conf. Image Process., p. 1014–1018.
  2. Nh-haze: An image dehazing benchmark with non-homogeneous hazy and haze-free images, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., p. 444–445.
  3. Non-local image dehazing, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1674–1682.
  4. Air-light estimation using haze-lines, in: Proc. IEEE Int. Conf. Comput. Photog., pp. 1–9.
  5. Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., p. 11682–11692.
  6. Dehazenet: An end-to-end system for single image haze removal. IEEE Trans. Image. Process. 25, 5187–5198.
  7. Libre: The multiple 3d lidar dataset. IEEE Intell. Veh. Symp. , 1094–1101.
  8. Focal network for image restoration, in: Proc. IEEE Int. Conf. Comput. Vis., pp. 13001–13011.
  9. Learning to predict crisp boundaries, in: Proc. Eur. Conf. Comput. Vis., pp. 562–578.
  10. Non-aligned supervision for real image dehazing. arXiv preprint arXiv:2303.04940 .
  11. Near-infrared guided color image dehazing, in: Proc. IEEE Int. Conf. Image Process., pp. 2363–2367.
  12. Image dehazing transformer with transmission-aware 3d position embedding, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., p. 5812–5820.
  13. Rsdehazenet: Dehazing network with channel refinement for multispectral remote sensing images. IEEE Trans. Geosci. Remote. Sens. 59, 2535–2549.
  14. Single image haze removal using dark channel prior 33, 2341–2353.
  15. Transer: Hybrid model and ensemble-based sequential learning for non-homogenous dehazing, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops, pp. 1670–1679.
  16. Distilling image dehazing with heterogeneous task imitation, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., p. 3462–3471.
  17. Colour image dehazing using near-infrared fusion. IET Image. Process. 11, 587–594.
  18. Darkvisionnet: Low-light imaging via rgb-nir fusion with deep inconsistency prior, in: Proc. AAAI Conf. Artif. Intell., pp. 1104–1112.
  19. Edge-guided multi-domain rgb-to-tir image translation for training vision tasks with challenging labels, in: IEEE Int. Conf. Robot., pp. 8291–8298.
  20. Aod-net: All-in-one dehazing network, in: Proc. IEEE Int. Conf. Comput. Vis., pp. 4780–4788.
  21. Benchmarking single image dehazing and beyond. IEEE Trans. Image. Process. 28, 492–505.
  22. Usid-net: Unsupervised single image dehazing network via disentangled representations. IEEE Trans. Multimedia. 25, 3587–3601.
  23. Lap-net: Level-aware progressive network for image dehazing, in: Proc. IEEE Int. Conf. Comput. Vis., p. 3275–3284.
  24. Dual-scale single image dehazing via neural augmentation. IEEE Trans. Image. Process. 31, 6213–6223.
  25. Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 5802–5811.
  26. Visual transformer with stable prior and patch-level attention for single image dehazing. Neurocomputing 551, 126535.
  27. Learning deep priors for image dehazing, in: Proc. IEEE Int. Conf. Comput. Vis., p. 2492–2500.
  28. A spectral grouping-based deep learning model for haze removal of hyperspectral images. ISPRS J. Photogramm. Remote. Sens. 188, 177–189.
  29. Optics of the atmosphere: scattering by molecules and particles. New York .
  30. Dehazing for multispectral remote sensing images based on a convolutional neural network with the residual architecture. IEEE J. Sel. Top. Appl. Earth. Obs. Remote. Sens. 11, 1645–1655.
  31. Ffa-net: Feature fusion attention network for single image dehazing, in: Proc. AAAI Conf. Artif. Intell., p. 11908–11915.
  32. Single image dehazing via multi-scale convolutional neural networks, in: Proc. Eur. Conf. Comput. Vis., p. 154–169.
  33. Semantic foggy scene understanding with synthetic data, in: Int. J. Comput. Vis., pp. 973–992.
  34. Airsim: High-fidelity visual and physical simulation for autonomous vehicles, in: Field and Service Robotics: Results of the 11th International Conference, pp. 621–635.
  35. Near-infrared fusion via color regularization for haze and color distortion removals. IEEE Transactions on Circuits and Systems for Video Technology 28, 3111–3126.
  36. Vision transformers for single image dehazing. IEEE Trans. Image. Process. 32, 1927–1941.
  37. Visibility in bad weather from a single image, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1–8.
  38. Visible and nir image fusion using weight-map-guided laplacian–gaussian pyramid for improving scene visibility. Sādhanā. 42, 1063–1082.
  39. Multimodal deep fusion network for visibility assessment with a small training dataset. IEEE Access. 8, 217057–217067.
  40. Cycle-snspgan: Towards real-world image dehazing via cycle spectral normalized soft likelihood estimation patch gan. IEEE Trans. Intell. Transp. Syst. 23, 20368–20382.
  41. Multiscale structural similarity for image quality assessment, in: 37th Asilomar. Conf. Signals. Syst. Comput, pp. 1398–1402.
  42. Contrastive learning for compact single image dehazing, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 10551–10560.
  43. Accurate transmission estimation for removing haze and noise from a single image. IEEE Trans. Image. Process. 29, 2583–2597.
  44. Ridcp: Revitalizing real image dehazing via high-quality codebook priors, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 22282–22291.
  45. Thermal infrared guided color image dehazing, in: Proc. IEEE Int. Conf. Image Process., pp. 2465–2469.
  46. Two-step image dehazing with intra-domain and inter-domain adaptation. Neurocomputing 485, 1–11.
  47. High-quality image dehazing with diffusion model. arXiv preprint arXiv:2308.11949 .
  48. Frequency and spatial dual guidance for image dehazing, in: Proc. Eur. Conf. Comput. Vis., pp. 181–198.
  49. A fast single image haze removal algorithm using color attenuation prior. IEEE Trans. Image. Process. 24, 3522–3533.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now