FogGuard: guarding YOLO against fog using perceptual loss (2403.08939v2)
Abstract: In this paper, we present FogGuard, a novel fog-aware object detection network designed to address the challenges posed by foggy weather conditions. Autonomous driving systems heavily rely on accurate object detection algorithms, but adverse weather conditions can significantly impact the reliability of deep neural networks (DNNs). Existing approaches include image enhancement techniques like IA-YOLO and domain adaptation methods. While image enhancement aims to generate clear images from foggy ones, which is more challenging than object detection in foggy images, domain adaptation does not require labeled data in the target domain. Our approach involves fine-tuning on a specific dataset to address these challenges efficiently. FogGuard compensates for foggy conditions in the scene, ensuring robust performance by incorporating YOLOv3 as the baseline algorithm and introducing a unique Teacher-Student Perceptual loss for accurate object detection in foggy environments. Through comprehensive evaluations on standard datasets like PASCAL VOC and RTTS, our network significantly improves performance, achieving a 69.43\% mAP compared to YOLOv3's 57.78\% on the RTTS dataset. Additionally, we demonstrate that while our training method slightly increases time complexity, it doesn't add overhead during inference compared to the regular YOLO network.
- A. J. Hawkins, “Tesla’s ‘phantom braking’ problem is getting worse, and the us government has questions,” https://www.theverge.com/2022/6/3/23153241/tesla-phantom-braking-nhtsa-complaints-investigation, June 2022.
- R. Bellan, “Tesla backs vision-only approach to autonomy using powerful supercomputer,” https://techcrunch.com/2021/06/21/tesla-backs-vision-only-approach-to-autonomy-using-powerful-supercomputer, June 2021.
- ——, “Tesla appears to be turning back to radar for its vehicles,” https://techcrunch.com/2022/12/07/tesla-appears-to-be-turning-back-to-radar-for-its-vehicles/, December 2022.
- M. Bijelic, T. Gruber, F. Mannan, F. Kraus, W. Ritter, K. Dietmayer, and F. Heide, “Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather,” CoRR, 2019. [Online]. Available: http://arxiv.org/abs/1902.08913v3
- J. Guan, S. Madani, S. Jog, S. Gupta, and H. Hassanieh, “Through fog high-resolution imaging using millimeter wave radar,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
- D.-H. Paek, S.-H. KONG, and K. T. Wijaya, “K-radar: 4d radar object detection for autonomous driving in various weather conditions,” in Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc., 2022, pp. 3819–3829. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2022/file/185fdf627eaae2abab36205dcd19b817-Paper-Datasets_and_Benchmarks.pdf
- N. Anjana, J. J. Priestley, V. Nandhini, and V. Elamaran, “Color image enhancement using edge based histogram equalization,” Indian Journal of Science and Technology, vol. 8, no. 29, pp. 1–6, 2015.
- K. K. Lavania, R. Kumar et al., “Image enhancement using filtering techniques,” International Journal on Computer Science and Engineering, vol. 4, no. 1, p. 14, 2012.
- L. Li, R. Wang, W. Wang, and W. Gao, “A low-light image enhancement method for both denoising and contrast enlarging,” in 2015 IEEE international conference on image processing (ICIP). IEEE, 2015, pp. 3730–3734.
- Y. Hu, H. He, C. Xu, B. Wang, and S. Lin, “Exposure: A white-box photo post-processing framework,” ACM Transactions on Graphics (TOG), vol. 37, no. 2, pp. 1–17, 2018.
- W. Liu, G. Ren, R. Yu, S. Guo, J. Zhu, and L. Zhang, “Image-adaptive yolo for object detection in adverse weather conditions,” CoRR, 2021. [Online]. Available: http://arxiv.org/abs/2112.08088v3
- J. Redmon and A. Farhadi, “Yolov3: an incremental improvement,” CoRR, 2018. [Online]. Available: http://arxiv.org/abs/1804.02767v1
- J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution,” CoRR, 2016. [Online]. Available: http://arxiv.org/abs/1603.08155v1
- B. Li, W. Ren, D. Fu, D. Tao, D. Feng, W. Zeng, and Z. Wang, “Benchmarking single-image dehazing and beyond,” IEEE Transactions on Image Processing, vol. 28, no. 1, pp. 492–505, 2019.
- M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, “The PASCAL visual object classes (VOC) challenge,” International journal of computer vision, vol. 88, pp. 303–338, 2010.
- T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13. Springer, 2014, pp. 740–755.
- Q. Qin, K. Chang, M. Huang, and G. Li, “Denet: Detection-driven enhancement network for object detection under adverse weather conditions,” in Proceedings of the Asian Conference on Computer Vision (ACCV), December 2022, pp. 2813–2829.
- A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, “Yolov4: Optimal speed and accuracy of object detection,” 2020.
- Z. Yu and C. Bajaj, “A fast and adaptive method for image contrast enhancement,” in 2004 International Conference on Image Processing, 2004. ICIP ’04., vol. 2, 2004, pp. 1001–1004 Vol.2.
- B. Zhang and J. P. Allebach, “Adaptive bilateral filter for sharpness enhancement and noise removal,” IEEE Transactions on Image Processing, vol. 17, no. 5, pp. 664–678, 2008.
- K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel prior,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 12, p. 2341–2353, Dec 2011. [Online]. Available: http://dx.doi.org/10.1109/TPAMI.2010.168
- C. Chen, M. N. Do, and J. Wang, “Robust image and video dehazing with visual artifact suppression via gradient residual minimization,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14. Springer, 2016, pp. 576–591.
- D. Yang and J. Sun, “Proximal dehaze-net: A prior learning-based deep network for single image dehazing,” in Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
- R. Yu, W. Liu, Y. Zhang, Z. Qu, D. Zhao, and B. Zhang, “Deepexposure: Learning to expose photos with asynchronously reinforced adversarial learning,” Advances in Neural Information Processing Systems, vol. 31, 2018.
- Z. Zhang, H. Zheng, R. Hong, M. Xu, S. Yan, and M. Wang, “Deep color consistent network for low-light image enhancement,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 1899–1908.
- W. Ren, S. Liu, H. Zhang, J. Pan, X. Cao, and M.-H. Yang, “Single image dehazing via multi-scale convolutional neural networks,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14. Springer, 2016, pp. 154–169.
- S. Zhang and F. He, “Drcdn: learning deep residual convolutional dehazing networks,” The Visual Computer, vol. 36, no. 9, pp. 1797–1808, 2020.
- H. Dong, J. Pan, L. Xiang, Z. Hu, X. Zhang, F. Wang, and M.-H. Yang, “Multi-scale boosted dehazing network with dense feature fusion,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 2154–2164.
- Y. Ganin and V. Lempitsky, “Unsupervised domain adaptation by backpropagation,” in International conference on machine learning. PMLR, 2015, pp. 1180–1189.
- M. Hnewa and H. Radha, “Multiscale domain adaptive yolo for cross-domain object detection,” in 2021 IEEE International Conference on Image Processing (ICIP), 2021, pp. 3323–3327.
- H. Zhou, F. Jiang, and H. Lu, “SSDA-YOLO: Semi-supervised domain adaptive YOLO for cross-domain object detection,” Computer Vision and Image Understanding, vol. 229, p. 103649, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1077314223000292
- R. Ranftl, K. Lasinger, D. Hafner, K. Schindler, and V. Koltun, “Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 3, pp. 1623–1637, 2020.
- O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein et al., “Imagenet large scale visual recognition challenge,” International journal of computer vision, vol. 115, pp. 211–252, 2015.
- M. Bijelic, T. Gruber, F. Mannan, F. Kraus, W. Ritter, K. Dietmayer, and F. Heide, “Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
- W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “Ssd: Single shot multibox detector,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14. Springer, 2016, pp. 21–37.
- Soheil Gharatappeh (3 papers)
- Sepideh Neshatfar (3 papers)
- Salimeh Yasaei Sekeh (34 papers)
- Vikas Dhiman (18 papers)