Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
131 tokens/sec
GPT-4o
10 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

CrackNex: a Few-shot Low-light Crack Segmentation Model Based on Retinex Theory for UAV Inspections (2403.03063v1)

Published 5 Mar 2024 in cs.CV

Abstract: Routine visual inspections of concrete structures are imperative for upholding the safety and integrity of critical infrastructure. Such visual inspections sometimes happen under low-light conditions, e.g., checking for bridge health. Crack segmentation under such conditions is challenging due to the poor contrast between cracks and their surroundings. However, most deep learning methods are designed for well-illuminated crack images and hence their performance drops dramatically in low-light scenes. In addition, conventional approaches require many annotated low-light crack images which is time-consuming. In this paper, we address these challenges by proposing CrackNex, a framework that utilizes reflectance information based on Retinex Theory to help the model learn a unified illumination-invariant representation. Furthermore, we utilize few-shot segmentation to solve the inefficient training data problem. In CrackNex, both a support prototype and a reflectance prototype are extracted from the support set. Then, a prototype fusion module is designed to integrate the features from both prototypes. CrackNex outperforms the SOTA methods on multiple datasets. Additionally, we present the first benchmark dataset, LCSD, for low-light crack segmentation. LCSD consists of 102 well-illuminated crack images and 41 low-light crack images. The dataset and code are available at https://github.com/zy1296/CrackNex.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. Y. Yan, S. Zhu, S. Ma, Y. Guo, and Z. Yu, “Cycleadc-net: A crack segmentation method based on multi-scale feature fusion,” Measurement, vol. 204, p. 112107, 2022.
  2. W. Choi and Y.-J. Cha, “Sddnet: Real-time crack segmentation,” IEEE Transactions on Industrial Electronics, vol. 67, no. 9, pp. 8016–8025, 2019.
  3. Y. Liu, J. Yao, X. Lu, R. Xie, and L. Li, “Deepcrack: A deep hierarchical feature learning architecture for crack segmentation,” Neurocomputing, vol. 338, pp. 139–153, 2019.
  4. D. Kang, S. S. Benipal, D. L. Gopal, and Y.-J. Cha, “Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning,” Automation in Construction, vol. 118, p. 103291, 2020.
  5. J. Liu, X. Yang, S. Lau, X. Wang, S. Luo, V. C.-S. Lee, and L. Ding, “Automated pavement crack detection and segmentation based on two-step convolutional neural network,” Computer-Aided Civil and Infrastructure Engineering, vol. 35, no. 11, pp. 1291–1305, 2020.
  6. A. Rezaie, R. Achanta, M. Godio, and K. Beyer, “Comparison of crack segmentation using digital image correlation measurements and deep learning,” Construction and Building Materials, vol. 261, p. 120474, 2020.
  7. Z. Zheng, X. Ying, Z. Yao, and M. C. Chuah, “Robustness of trajectory prediction models under map-based attacks,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 4541–4550.
  8. Y.-T. Hsieh, K. Anjum, S. Huang, I. Kulkarni, and D. Pompili, “Neural network design via voltage-based resistive processing unit and diode activation function-a new architecture,” in 2021 IEEE International Midwest Symposium on Circuits and Systems (MWSCAS).   IEEE, 2021, pp. 59–62.
  9. Z. Wang, L. Zhang, L. Wang, and M. Zhu, “Landa: Language-guided multi-source domain adaptation,” arXiv preprint arXiv:2401.14148, 2024.
  10. Y. Zhu, Y. Qiu, Q. Wu, F. L. Wang, and Y. Rao, “Topic driven adaptive network for cross-domain sentiment classification,” Information Processing & Management, vol. 60, no. 2, p. 103230, 2023.
  11. W. Wang and C. Su, “Automatic concrete crack segmentation model based on transformer,” Automation in Construction, vol. 139, p. 104275, 2022.
  12. L. Fan, S. Li, Y. Li, B. Li, D. Cao, and F.-Y. Wang, “Pavement cracks coupled with shadows: A new shadow-crack dataset and a shadow-removal-oriented crack detection approach,” IEEE/CAA Journal of Automatica Sinica, vol. 10, no. 7, pp. 1593–1607, 2023.
  13. W. Li, Z. Shen, and P. Li, “Crack detection of track plate based on yolo,” in 2019 12th international symposium on computational intelligence and design (ISCID), vol. 2.   IEEE, 2019, pp. 15–18.
  14. G. Li, Q. Liu, W. Ren, W. Qiao, B. Ma, and J. Wan, “Automatic recognition and analysis system of asphalt pavement cracks using interleaved low-rank group convolution hybrid deep network and segnet fusing dense condition random field,” Measurement, vol. 170, p. 108693, 2021.
  15. Y. Wang, K. Song, J. Liu, H. Dong, Y. Yan, and P. Jiang, “Renet: Rectangular convolution pyramid and edge enhancement network for salient object detection of pavement cracks,” Measurement, vol. 170, p. 108698, 2021.
  16. J. Dong, N. Wang, H. Fang, Q. Hu, C. Zhang, B. Ma, D. Ma, and H. Hu, “Innovative method for pavement multiple damages segmentation and measurement by the road-seg-capsnet of feature fusion,” Construction and Building Materials, vol. 324, p. 126719, 2022.
  17. E. H. Land, “The retinex theory of color vision,” Scientific american, vol. 237, no. 6, pp. 108–129, 1977.
  18. C. Wei, W. Wang, W. Yang, and J. Liu, “Deep retinex decomposition for low-light enhancement,” arXiv preprint arXiv:1808.04560, 2018.
  19. N. Zhao, T.-S. Chua, and G. H. Lee, “Few-shot 3d point cloud semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 8873–8882.
  20. K. Wang, J. H. Liew, Y. Zou, D. Zhou, and J. Feng, “Panet: Few-shot image semantic segmentation with prototype alignment,” in proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 9197–9206.
  21. N. Dong and E. P. Xing, “Few-shot semantic segmentation with prototype learning.” in BMVC, vol. 3, no. 4, 2018.
  22. J. Min, D. Kang, and M. Cho, “Hypercorrelation squeeze for few-shot segmentation,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 6941–6952.
  23. B. Mao, X. Zhang, L. Wang, Q. Zhang, S. Xiang, and C. Pan, “Learning from the target: Dual prototype network for few shot semantic segmentation,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 2, 2022, pp. 1953–1961.
  24. L. Sun, C. Li, X. Ding, Y. Huang, Z. Chen, G. Wang, Y. Yu, and J. Paisley, “Few-shot medical image segmentation using a global correlation network with discriminative embedding,” Computers in biology and medicine, vol. 140, p. 105067, 2022.
  25. P. Pan, Z. Fan, B. Y. Feng, P. Wang, C. Li, and Z. Wang, “Learning to estimate 6dof pose from limited data: A few-shot, generalizable approach using rgb images,” arXiv preprint arXiv:2306.07598, 2023.
  26. L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam, “Rethinking atrous convolution for semantic image segmentation,” arXiv preprint arXiv:1706.05587, 2017.
  27. C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in International conference on machine learning.   PMLR, 2017, pp. 1126–1135.
  28. V. Garcia and J. Bruna, “Few-shot learning with graph neural networks,” arXiv preprint arXiv:1711.04043, 2017.
  29. T. Munkhdalai and H. Yu, “Meta networks,” in International conference on machine learning.   PMLR, 2017, pp. 2554–2563.
  30. Z. Wang, M. Ye, X. Zhu, L. Peng, L. Tian, and Y. Zhu, “Metateacher: Coordinating multi-model domain adaptation for medical image classification,” Advances in Neural Information Processing Systems, vol. 35, pp. 20 823–20 837, 2022.
  31. C. Li, X. Lin, Y. Mao, W. Lin, Q. Qi, X. Ding, Y. Huang, D. Liang, and Y. Yu, “Domain generalization on medical imaging classification using episodic training with task augmentation,” Computers in biology and medicine, vol. 141, p. 105144, 2022.
  32. T. Elsken, B. Staffler, J. H. Metzen, and F. Hutter, “Meta-learning of neural architectures for few-shot learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 12 365–12 375.
  33. B. Zoph and Q. V. Le, “Neural architecture search with reinforcement learning,” arXiv preprint arXiv:1611.01578, 2016.
  34. S. Baik, J. Choi, H. Kim, D. Cho, J. Min, and K. M. Lee, “Meta-learning with task-adaptive loss function for few-shot learning,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 9465–9474.
  35. X. Ying, X. Li, and M. C. Chuah, “Weakly-supervised object representation learning for few-shot semantic segmentation,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 1497–1506.
  36. Q. Fan, W. Pei, Y.-W. Tai, and C.-K. Tang, “Self-support few-shot semantic segmentation,” in European Conference on Computer Vision.   Springer, 2022, pp. 701–719.
  37. A. Okazawa, “Interclass prototype relation for few-shot segmentation,” in European Conference on Computer Vision.   Springer, 2022, pp. 362–378.
  38. S. L. Lau, E. K. Chong, X. Yang, and X. Wang, “Automated pavement crack segmentation using u-net-based convolutional neural network,” Ieee Access, vol. 8, pp. 114 892–114 899, 2020.
  39. J. König, M. D. Jenkins, M. Mannion, P. Barrie, and G. Morison, “Optimized deep encoder-decoder methods for crack segmentation,” Digital Signal Processing, vol. 108, p. 102907, 2021.
  40. Y. Qiu, Y. Shen, Z. Sun, Y. Zheng, X. Chang, W. Zheng, and R. Wang, “Sats: Self-attention transfer for continual semantic segmentation,” Pattern Recognition, vol. 138, p. 109383, 2023.
  41. K. Liu, X. Han, and B. M. Chen, “Deep learning based automatic crack detection and segmentation for unmanned aerial vehicle inspections,” in 2019 IEEE international conference on robotics and biomimetics (ROBIO).   IEEE, 2019, pp. 381–387.
  42. C. Sun, S. Huang, and D. Pompili, “Hmaac: Hierarchical multi-agent actor-critic for aerial search with explicit coordination modeling,” in 2023 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2023, pp. 7728–7734.
  43. Y. Wei, Z. Wei, Y. Rao, J. Li, J. Zhou, and J. Lu, “Lidar distillation: Bridging the beam-induced domain gap for 3d object detection,” in European Conference on Computer Vision.   Springer, 2022, pp. 179–195.
  44. B. Dang, D. Ma, S. Li, X. Dong, H. Zang, and R. Ding, “Enhancing kitchen independence: Deep learning-based object detection for visually impaired assistance,” Academic Journal of Science and Technology, vol. 9, no. 2, pp. 180–184, 2024.
  45. D. Ma, B. Dang, S. Li, H. Zang, and X. Dong, “Implementation of computer vision technology based on artificial intelligence for medical image analysis,” International Journal of Computer Science and Information Technology, vol. 1, no. 1, pp. 69–76, 2023.
  46. J.-A. Sarmiento, “Pavement distress detection and segmentation using yolov4 and deeplabv3 on pavements in the philippines,” arXiv preprint arXiv:2103.06467, 2021.
  47. C. Xiang, J. Guo, R. Cao, and L. Deng, “A crack-segmentation algorithm fusing transformers and convolutional neural networks for complex detection scenarios,” Automation in Construction, vol. 152, p. 104894, 2023.
  48. X. Ying, X. Li, and M. C. Chuah, “Weakly-supervised object representation learning for few-shot semantic segmentation,” in Proceedings of the IEEE WACV, 2020.
  49. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition.   Ieee, 2009, pp. 248–255.
  50. S. Kulkarni, S. Singh, D. Balakrishnan, S. Sharma, S. Devunuri, and S. C. R. Korlapati, “Crackseg9k: a collection and benchmark for crack segmentation datasets and frameworks,” in European Conference on Computer Vision.   Springer, 2022, pp. 179–195.
  51. S. W. Zamir, A. Arora, S. Khan, M. Hayat, F. S. Khan, and M.-H. Yang, “Restormer: Efficient transformer for high-resolution image restoration,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 5728–5739.
  52. X. Ying, B. Lang, Z. Zheng, and M. C. Chuah, “Delving into light-dark semantic segmentation for indoor scenes understanding,” in Proceedings of the 1st Workshop on Photorealistic Image and Environment Synthesis for Multimedia Experiments, 2022, pp. 3–9.
  53. S. Hong, S. Cho, J. Nam, S. Lin, and S. Kim, “Cost aggregation with 4d convolutional swin transformer for few-shot segmentation,” in European Conference on Computer Vision.   Springer, 2022, pp. 108–126.
  54. L. Yang, W. Zhuo, L. Qi, Y. Shi, and Y. Gao, “Mining latent classes for few-shot segmentation,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 8721–8730.
  55. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  56. J. Kiefer and J. Wolfowitz, “Stochastic estimation of the maximum of a regression function,” The Annals of Mathematical Statistics, pp. 462–466, 1952.
  57. J. Peng, Y. Liu, S. Tang, Y. Hao, L. Chu, G. Chen, Z. Wu, Z. Chen, Z. Yu, Y. Du et al., “Pp-liteseg: A superior real-time semantic segmentation model,” arXiv preprint arXiv:2204.02681, 2022.
  58. M. Fan, S. Lai, J. Huang, X. Wei, Z. Chai, J. Luo, and X. Wei, “Rethinking bisenet for real-time semantic segmentation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 9716–9725.
  59. Y. Li, R. Ma, H. Liu, and G. Cheng, “Real-time high-resolution neural network with semantic guidance for crack segmentation,” Automation in Construction, vol. 156, p. 105112, 2023.
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now