Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Beyond Night Visibility: Adaptive Multi-Scale Fusion of Infrared and Visible Images (2403.01083v1)

Published 2 Mar 2024 in cs.CV

Abstract: In addition to low light, night images suffer degradation from light effects (e.g., glare, floodlight, etc). However, existing nighttime visibility enhancement methods generally focus on low-light regions, which neglects, or even amplifies the light effects. To address this issue, we propose an Adaptive Multi-scale Fusion network (AMFusion) with infrared and visible images, which designs fusion rules according to different illumination regions. First, we separately fuse spatial and semantic features from infrared and visible images, where the former are used for the adjustment of light distribution and the latter are used for the improvement of detection accuracy. Thereby, we obtain an image free of low light and light effects, which improves the performance of nighttime object detection. Second, we utilize detection features extracted by a pre-trained backbone that guide the fusion of semantic features. Hereby, we design a Detection-guided Semantic Fusion Module (DSFM) to bridge the domain gap between detection and semantic features. Third, we propose a new illumination loss to constrain fusion image with normal light intensity. Experimental results demonstrate the superiority of AMFusion with better visual quality and detection accuracy. The source code will be released after the peer review process.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. V Aslantas and Emre Bendes. 2015. A new image quality metric for image fusion: The sum of the correlations of differences. Aeu-international Journal of electronics and communications 69, 12 (2015), 1890–1896.
  2. End-to-end object detection with transformers. In ECCV. Springer, 213–229.
  3. Diffusiondet: Diffusion model for object detection. In CVPR. 19830–19843.
  4. Sanjoy Das and Yunlong Zhang. 2000. Color night vision for navigation and surveillance. Transportation research record 1708, 1 (2000), 40–46.
  5. Object classification using CNN-based fusion of vision and LIDAR in autonomous vehicle environment. IEEE TII 14, 9 (2018), 4224–4231.
  6. Ross Girshick. 2015. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision. 1440–1448.
  7. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 580–587.
  8. LIME: Low-light image enhancement via illumination map estimation. IEEE TIP 26, 2 (2016), 982–993.
  9. Liqiang He and Sinisa Todorovic. 2022. DESTR: Object detection with split transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 9377–9386.
  10. LLVIP: A visible-infrared paired dataset for low-light vision. In CVPR. 3496–3504.
  11. Low-Light Image Enhancement via Stage-Transformer-Guided Network. IEEE TCSVT (2023).
  12. Enlightengan: Deep light enhancement without paired supervision. IEEE TIP 30 (2021), 2340–2349.
  13. Unsupervised night image enhancement: When layer decomposition meets light-effects suppression. In ECCV. Springer, 404–421.
  14. Learning to enhance low-light image via zero-reference deep curve estimation. IEEE TPAMI 44, 8 (2021), 4225–4238.
  15. Different input resolutions and arbitrary output resolution: A meta learning-based deep framework for infrared and visible image fusion. IEEE TIP 30 (2021), 4070–4083.
  16. Hui Li and Xiao-Jun Wu. 2018. DenseFuse: A fusion approach to infrared and visible images. IEEE TIP 28, 5 (2018), 2614–2623.
  17. NestFuse: An infrared and visible image fusion architecture based on nest connection and spatial/channel attention models. IEEE TIM 69, 12 (2020), 9645–9656.
  18. LDRM: Degradation Rectify Model for Low-light Imaging via Color-Monochrome Cameras. In ACM MM. 8406–8414.
  19. Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection. In CVPR. 5802–5811.
  20. Ssd: Single shot multibox detector. In ECCV. Springer, 21–37.
  21. Infrared and visible image fusion methods and applications: A survey. Information fusion 45 (2019), 153–178.
  22. SwinFusion: Cross-domain long-range learning for general image fusion via swin transformer. IJCAI 9, 7 (2022), 1200–1217.
  23. STDFusionNet: An infrared and visible image fusion network based on salient target detection. IEEE TIM 70 (2021), 1–13.
  24. DDcGAN: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion. IEEE TIP 29 (2020), 4980–4995.
  25. FusionGAN: A generative adversarial network for infrared and visible image fusion. Information fusion 48 (2019), 11–26.
  26. Information measure for performance of image fusion. Electronics letters 38, 7 (2002), 1.
  27. You only look once: Unified, real-time object detection. In CVPR. 779–788.
  28. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE TPAMI 39, 6 (2017), 1137.
  29. Assessment of image fusion procedures using entropy, image quality, and multispectral classification. Journal of Applied Remote Sensing 2, 1 (2008), 023522.
  30. Aashish Sharma and Robby T Tan. 2021. Nighttime visibility enhancement by increasing the dynamic range and suppression of light effects. In CVPR. 11977–11986.
  31. Detfusion: A detection-driven infrared and visible image fusion network. In ACM MM. 4003–4011.
  32. DIVFusion: Darkness-free infrared and visible image fusion. Information Fusion 91 (2023), 477–493.
  33. Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network. Information Fusion 82 (2022), 28–42.
  34. PIAFusion: A progressive infrared and visible image fusion network based on illumination aware. Information Fusion 83 (2022), 79–92.
  35. Rethinking the necessity of image fusion in high-level vision tasks: A practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity. Information Fusion (2023), 101870.
  36. YDTR: Infrared and visible image fusion via Y-shape dynamic transformer. IEEE TMM (2022).
  37. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In CVPR. 7464–7475.
  38. Cbam: Convolutional block attention module. In ECCV. 3–19.
  39. Revisiting ap loss for dense object detection: Adaptive ranking pair selection. In CVPR. 14187–14196.
  40. U2Fusion: A unified unsupervised image fusion network. IEEE TPAMI 44, 1 (2020), 502–518.
  41. Group R-CNN for weakly semi-supervised object detection with points. In CVPR. 9417–9426.
  42. Beyond brightening low-light images. IJCV 129 (2021), 1013–1037.
  43. MetaFusion: Infrared and Visible Image Fusion via Meta-Feature Embedding From Object Detection. In CVPR. 13955–13965.
  44. Object detection in 20 years: A survey. Proc. IEEE (2023).

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now