Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
158 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

FlameFinder: Illuminating Obscured Fire through Smoke with Attentive Deep Metric Learning (2404.06653v1)

Published 9 Apr 2024 in cs.CV

Abstract: FlameFinder is a deep metric learning (DML) framework designed to accurately detect flames, even when obscured by smoke, using thermal images from firefighter drones during wildfire monitoring. Traditional RGB cameras struggle in such conditions, but thermal cameras can capture smoke-obscured flame features. However, they lack absolute thermal reference points, leading to false positives.To address this issue, FlameFinder utilizes paired thermal-RGB images for training. By learning latent flame features from smoke-free samples, the model becomes less biased towards relative thermal gradients. In testing, it identifies flames in smoky patches by analyzing their equivalent thermal-domain distribution. This method improves performance using both supervised and distance-based clustering metrics.The framework incorporates a flame segmentation method and a DML-aided detection framework. This includes utilizing center loss (CL), triplet center loss (TCL), and triplet cosine center loss (TCCL) to identify optimal cluster representatives for classification. However, the dominance of center loss over the other losses leads to the model missing features sensitive to them. To address this limitation, an attention mechanism is proposed. This mechanism allows for non-uniform feature contribution, amplifying the critical role of cosine and triplet loss in the DML framework. Additionally, it improves interpretability, class discrimination, and decreases intra-class variance. As a result, the proposed model surpasses the baseline by 4.4% in the FLAME2 dataset and 7% in the FLAME3 dataset for unobscured flame detection accuracy. Moreover, it demonstrates enhanced class separation in obscured scenarios compared to VGG19, ResNet18, and three backbone models tailored for flame detection.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. F. Tedim, V. Leone, M. Amraoui, C. Bouillon, M. R. Coughlan, G. M. Delogu, P. M. Fernandes, C. Ferreira, S. McCaffrey, T. K. McGee, et al., “Defining extreme wildfire events: Difficulties, challenges, and impacts,” Fire, vol. 1, no. 1, p. 9, 2018.
  2. F. Afghah, A. Razi, J. Chakareski, and J. Ashdown, “Wildfire monitoring in remote areas using autonomous unmanned aerial vehicles,” in IEEE INFOCOM 2019-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pp. 835–840, IEEE, 2019.
  3. A. Shamsoshoara, F. Afghah, A. Razi, L. Zheng, P. Z. Fulé, and E. Blasch, “Aerial imagery pile burn detection using deep learning: The flame dataset,” Computer Networks, vol. 193, p. 108001, 2021.
  4. S. P. H. Boroujeni, A. Razi, S. Khoshdel, F. Afghah, J. L. Coen, L. O’Neill, P. Fule, A. Watts, N.-M. T. Kokolakis, and K. G. Vamvoudakis, “A comprehensive survey of research towards ai-enabled unmanned aerial systems in pre-, active-, and post-wildfire management,” Information Fusion, p. 102369, 2024.
  5. S. Khoshdel, Q. Luo, and F. Afghah, “Pyrotrack: Belief-based deep reinforcement learning path planning for aerial wildfire monitoring in partially observable environments,” 2024.
  6. A. Gaur, A. Singh, A. Kumar, A. Kumar, and K. Kapoor, “Video flame and smoke based fire detection algorithms: A literature review,” Fire technology, vol. 56, pp. 1943–1980, 2020.
  7. M. J. Sousa, A. Moutinho, and M. Almeida, “Thermal infrared sensing for near real-time data-driven fire detection and monitoring systems,” Sensors, vol. 20, no. 23, p. 6803, 2020.
  8. X. Chen, B. Hopkins, H. Wang, L. O’Neill, F. Afghah, A. Razi, P. Fulé, J. Coen, E. Rowell, and A. Watts, “Wildland fire detection and monitoring using a drone-collected rgb/ir image dataset,” IEEE Access, vol. 10, pp. 121301–121317, 2022.
  9. S. Treneska and B. R. Stojkoska, “Wildfire detection from uav collected images using transfer learning,” in Proceedings of the 18th International Conference on Informatics and Information Technologies, Skopje, North Macedonia, pp. 6–7, 2021.
  10. M. Kaya and H. Ş. Bilge, “Deep metric learning: A survey,” Symmetry, vol. 11, no. 9, p. 1066, 2019.
  11. A. Farahani, S. Voghoei, K. Rasheed, and H. R. Arabnia, “A brief review of domain adaptation,” Advances in data science and information engineering: proceedings from ICDATA 2020 and IKE 2020, pp. 877–894, 2021.
  12. J. Li, Y. Wu, J. Zhao, and K. Lu, “Low-rank discriminant embedding for multiview learning,” IEEE Transactions on Cybernetics, vol. 47, no. 11, pp. 3516–3529, 2017.
  13. J. Yu, X. Yang, F. Gao, and D. Tao, “Deep multimodal distance metric learning using click constraints for image ranking,” IEEE Transactions on Cybernetics, vol. 47, no. 12, pp. 4014–4024, 2017.
  14. Y. Chen, Y. Zhang, J. Xin, G. Wang, L. Mu, Y. Yi, H. Liu, and D. Liu, “Uav image-based forest fire detection approach using convolutional neural network,” in 2019 14th IEEE conference on industrial electronics and applications (ICIEA), pp. 2118–2123, IEEE, 2019.
  15. S. C. Sethuraman, G. R. Tadkapally, S. P. Mohanty, and A. Subramanian, “idrone: Iot-enabled unmanned aerial vehicles for detecting wildfires using convolutional neural networks,” SN Computer Science, vol. 3, no. 3, p. 242, 2022.
  16. A. Nguyen, H. Nguyen, V. Tran, H. X. Pham, and J. Pestana, “A visual real-time fire detection using single shot multibox detector for uav-based fire surveillance,” in 2020 IEEE Eighth International Conference on Communications and Electronics (ICCE), pp. 338–343, IEEE, 2021.
  17. J. Ryu and D. Kwak, “Flame detection using appearance-based pre-processing and convolutional neural network,” Applied Sciences, vol. 11, no. 11, p. 5138, 2021.
  18. D. Shen, X. Chen, M. Nguyen, and W. Q. Yan, “Flame detection using deep learning,” in 2018 4th International Conference on Control, Automation and Robotics (ICCAR), pp. 416–420, 2018.
  19. W. Li and Z. Yu, “A lightweight convolutional neural network flame detection algorithm,” in 2021 IEEE 11th International Conference on Electronics Information and Emergency Communication (ICEIEC) 2021 IEEE 11th International Conference on Electronics Information and Emergency Communication (ICEIEC), pp. 83–86, IEEE, 2021.
  20. L. Karlinsky, J. Shtok, S. Harary, E. Schwartz, A. Aides, R. Feris, R. Giryes, and A. M. Bronstein, “Repmet: Representative-based metric learning for classification and few-shot object detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5197–5206, 2019.
  21. H. Rajoli, F. Lotfi, A. Atyabi, and F. Afghah, “Triplet loss-less center loss sampling strategies in facial expression recognition scenarios,” in 2023 57th Annual Conference on Information Sciences and Systems (CISS), pp. 1–6, IEEE, 2023.
  22. B. Hong, Y. Zhou, H. Qin, Z. Wei, H. Liu, and Y. Yang, “Few-shot object detection using multimodal sensor systems of unmanned surface vehicles,” Sensors, vol. 22, no. 4, p. 1511, 2022.
  23. Y. Lu, X. Chen, Z. Wu, and J. Yu, “Decoupled metric network for single-stage few-shot object detection,” IEEE Transactions on Cybernetics, vol. 53, no. 1, pp. 514–525, 2022.
  24. P. Wu, S. C. H. Hoi, P. Zhao, C. Miao, and Z.-Y. Liu, “Online multi-modal distance metric learning with application to image retrieval,” IEEE Transactions on Knowledge and Data Engineering, vol. 28, no. 2, pp. 454–467, 2016.
  25. K. Roth, T. Milbich, S. Sinha, P. Gupta, B. Ommer, and J. P. Cohen, “Revisiting training strategies and generalization performance in deep metric learning,” in Proceedings of the 37th International Conference on Machine Learning (H. D. III and A. Singh, eds.), vol. 119 of Proceedings of Machine Learning Research, pp. 8242–8252, PMLR, 13–18 Jul 2020.
  26. D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” arXiv preprint arXiv:1409.0473, 2014.
  27. W. Kim, B. Goyal, K. Chawla, J. Lee, and K. Kwon, “Attention-based ensemble for deep metric learning,” in Proceedings of the European conference on computer vision (ECCV), pp. 736–751, 2018.
  28. X. Wang, Y. Hua, E. Kodirov, G. Hu, and N. M. Robertson, “Deep metric learning by online soft mining and class-aware attention,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 5361–5368, 2019.
  29. D. Kotovenko, P. Ma, T. Milbich, and B. Ommer, “Cross-image-attention for conditional embeddings in deep metric learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11070–11081, 2023.
  30. J. D. Seidenschwarz, I. Elezi, and L. Leal-Taixé, “Learning intra-batch connections for deep metric learning,” in International Conference on Machine Learning, pp. 9410–9421, PMLR, 2021.
  31. Y. Li, S. Kan, and Z. He, “Unsupervised deep metric learning with transformed attention consistency and contrastive clustering loss,” in European Conference on Computer Vision, pp. 141–157, Springer, 2020.
  32. P. Yang, Y. Zhai, L. Li, H. Lv, J. Wang, C. Zhu, and R. Jiang, “A deep metric learning approach for histopathological image retrieval,” Methods, vol. 179, pp. 14–25, 2020.
  33. H. Dong, K. Song, Q. Wang, Y. Yan, and P. Jiang, “Deep metric learning-based for multi-target few-shot pavement distress classification,” IEEE Transactions on Industrial Informatics, vol. 18, no. 3, pp. 1801–1810, 2021.
  34. H. Coskun, D. J. Tan, S. Conjeti, N. Navab, and F. Tombari, “Human motion analysis with deep metric learning,” in Proceedings of the European conference on computer vision (ECCV), pp. 667–683, 2018.
  35. E. Buza and A. Akagic, “Unsupervised method for wildfire flame segmentation and detection,” IEEE Access, vol. 10, pp. 55213–55225, 2022.
  36. D. Dzigal, A. Akagic, E. Buza, A. Brdjanin, and N. Dardagan, “Forest fire detection based on color spaces combination,” in 2019 11th International Conference on Electrical and Electronics Engineering (ELECO), pp. 595–599, IEEE, 2019.
  37. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241, Springer, 2015.
  38. A. H. Farzaneh and X. Qi, “Facial expression recognition in the wild via deep attentive center loss,” in Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 2402–2411, 2021.
  39. P. Fule, A. Watts, F. Afghah, B. Hopkins, L. O. O’Neill, A. Razi, and J. Coen, “Flame 2: Fire detection and modeling: Aerial multi-spectral image dataset,” IEEE DataPort, 2022.
  40. R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in Proceedings of the IEEE international conference on computer vision, pp. 618–626, 2017.
  41. E. Zachariadis, M. Antonakakis, and M. Zervakis, “Wildforest fire detection with shre-xception network on aerial optical and infrared images,” in 2023 IEEE International Conference on Imaging Systems and Techniques (IST), pp. 1–6, IEEE, 2023.
  42. R. Ghali, M. A. Akhloufi, and W. S. Mseddi, “Deep learning and transformer approaches for uav-based wildfire detection and segmentation,” Sensors, vol. 22, no. 5, p. 1977, 2022.
  43. R. Ghali and M. A. Akhloufi, “Ct-fire: a cnn-transformer for wildfire classification on ground and aerial images,” International Journal of Remote Sensing, vol. 44, no. 23, pp. 7390–7415, 2023.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets