Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
166 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Cycle-YOLO: A Efficient and Robust Framework for Pavement Damage Detection (2405.17905v1)

Published 28 May 2024 in cs.CV, cs.AI, cs.CY, and cs.LG

Abstract: With the development of modern society, traffic volume continues to increase in most countries worldwide, leading to an increase in the rate of pavement damage Therefore, the real-time and highly accurate pavement damage detection and maintenance have become the current need. In this paper, an enhanced pavement damage detection method with CycleGAN and improved YOLOv5 algorithm is presented. We selected 7644 self-collected images of pavement damage samples as the initial dataset and augmented it by CycleGAN. Due to a substantial difference between the images generated by CycleGAN and real road images, we proposed a data enhancement method based on an improved Scharr filter, CycleGAN, and Laplacian pyramid. To improve the target recognition effect on a complex background and solve the problem that the spatial pyramid pooling-fast module in the YOLOv5 network cannot handle multiscale targets, we introduced the convolutional block attention module attention mechanism and proposed the atrous spatial pyramid pooling with squeeze-and-excitation structure. In addition, we optimized the loss function of YOLOv5 by replacing the CIoU with EIoU. The experimental results showed that our algorithm achieved a precision of 0.872, recall of 0.854, and mean average [email protected] of 0.882 in detecting three main types of pavement damage: cracks, potholes, and patching. On the GPU, its frames per second reached 68, meeting the requirements for real-time detection. Its overall performance even exceeded the current more advanced YOLOv7 and achieved good results in practical applications, providing a basis for decision-making in pavement damage detection and prevention.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (79)
  1. W. Ye, W. Jiang, Z. Tong, D. Yuan, and J. Xiao, “Convolutional neural network for pothole detection in asphalt pavement,” Road materials and pavement design, vol. 22, no. 1, pp. 42–58, 2021.
  2. M. Bock, A. Cardazzi, and B. R. Humphreys, “Where the rubber meets the road: Pavement damage reduces traffic safety and speed,” National Bureau of Economic Research, Tech. Rep., 2021.
  3. L. Abdullah, S. Singh, S. Abdullah, A. Azman, and A. Ariffin, “Fatigue reliability and hazard assessment of road load strain data for determining the fatigue life characteristics,” Engineering Failure Analysis, vol. 123, p. 105314, 2021.
  4. S. A. Hassan, H. A. Amlan, N. E. Alias, M. A. Ab-Kadir, and N. S. A. Sukor, “Vulnerability of road transportation networks under natural hazards: A bibliometric analysis and review,” International Journal of Disaster Risk Reduction, vol. 83, p. 103393, 2022.
  5. H. Maeda, T. Kashiyama, Y. Sekimoto, T. Seto, and H. Omata, “Generative adversarial network for road damage detection,” Computer-Aided Civil and Infrastructure Engineering, vol. 36, no. 1, pp. 47–60, 2021.
  6. D. Arya, H. Maeda, S. K. Ghosh, D. Toshniwal, and Y. Sekimoto, “Rdd2020: An annotated image dataset for automatic road damage detection using deep learning,” Data in brief, vol. 36, p. 107133, 2021.
  7. S. Shim, J. Kim, S.-W. Lee, and G.-C. Cho, “Road damage detection using super-resolution and semi-supervised learning with generative adversarial network,” Automation in construction, vol. 135, p. 104139, 2022.
  8. S. Wang, Y. Tang, X. Liao, J. He, H. Feng, H. Jiao, X. Su, and Q. Yuan, “An ensemble learning approach with multi-depth attention mechanism for road damage detection,” in 2022 IEEE International Conference on Big Data (Big Data).   IEEE, 2022, pp. 6439–6444.
  9. W.-W. Jin, G.-H. Chen, Z. Chen, Y.-L. Sun, J. Ni, H. Huang, W.-H. Ip, and K.-L. Yung, “Road pavement damage detection based on local minimum of grayscale and feature fusion,” Applied Sciences, vol. 12, no. 24, p. 13006, 2022.
  10. V. Gagliardi, F. Bella, G. Sansonetti, R. Previti, and L. Menghini, “Automatic damage detection of bridge joints and road pavements by artificial neural networks anns,” in Earth Resources and Environmental Remote Sensing/GIS Applications XIII, vol. 12268.   SPIE, 2022, pp. 89–100.
  11. E. F. Muttaqiy, “Deteksi kerusakan jalan menggunakan metode deep learning ssd mobilenet v2 fpnlite,” Ph.D. dissertation, Institut Teknologi Sepuluh Nopember, 2024.
  12. R. Palani, N. Puviarasan et al., “Road surface damage detection with ensemble techniques,” SJIS-P, vol. 35, no. 1, pp. 1403–1412, 2023.
  13. F. A. L. N. Mouzinho and H. Fukai, “Hierarchical semantic segmentation based approach for road surface damages and markings detection on paved road,” in 2021 8th International Conference on Advanced Informatics: Concepts, Theory and Applications (ICAICTA).   IEEE, 2021, pp. 1–5.
  14. S. Shim, “Detection algorithm of road surface damage using adversarial learning,” The Journal of The Korea Institute of Intelligent Transport Systems, vol. 20, no. 4, pp. 95–105, 2021.
  15. D. Dube, S. Sonawane, S. Nawale, S. Jangam, and J. Niyajali, “Advancing road safety through pothole detection system: A deep learning approach.”
  16. R. Bibi, Y. Saeed, A. Zeb, T. M. Ghazal, T. Rahman, R. A. Said, S. Abbas, M. Ahmad, and M. A. Khan, “Edge ai-based automated detection and classification of road anomalies in vanet using deep learning,” Computational intelligence and neuroscience, vol. 2021, pp. 1–16, 2021.
  17. Y. Ge, Q. Zhang, T.-Z. Xiang, C. Zhang, and H. Bi, “Tcnet: Co-salient object detection via parallel interaction of transformers and cnns,” IEEE Transactions on Circuits and Systems for Video Technology, 2022.
  18. G. Chen, F. Shao, X. Chai, H. Chen, Q. Jiang, X. Meng, and Y.-S. Ho, “Modality-induced transfer-fusion network for rgb-d and rgb-t salient object detection,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 4, pp. 1787–1801, 2022.
  19. L. Zhao, J. Guo, D. Xu, and L. Sheng, “Transformer3d-det: Improving 3d object detection by vote refinement,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 12, pp. 4735–4746, 2021.
  20. Z. Liu, Y. Tan, Q. He, and Y. Xiao, “Swinnet: Swin transformer drives edge-aware rgb-d and rgb-t salient object detection,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 7, pp. 4486–4497, 2021.
  21. Y. Yang, Q. Qin, Y. Luo, Y. Liu, Q. Zhang, and J. Han, “Bi-directional progressive guidance network for rgb-d salient object detection,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 8, pp. 5346–5360, 2022.
  22. C. Chen, J. Song, C. Peng, G. Wang, and Y. Fang, “A novel video salient object detection method via semisupervised motion quality perception,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 5, pp. 2732–2745, 2021.
  23. Q. Wang, J. Mao, X. Zhai, J. Gui, W. Shen, and Y. Liu, “Improvements of yolov3 for road damage detection,” in Journal of Physics: Conference Series, vol. 1903, no. 1.   IOP Publishing, 2021, p. 012008.
  24. F. Wan, C. Sun, H. He, G. Lei, L. Xu, and T. Xiao, “Yolo-lrdd: A lightweight method for road damage detection based on improved yolov5s,” EURASIP Journal on Advances in Signal Processing, vol. 2022, no. 1, p. 98, 2022.
  25. C. He, J. Wu, and Y. Yang, “Research on expressway pavement crack detection based on improved yolov5s,” Frontiers in Computing and Intelligent Systems, vol. 5, no. 3, pp. 121–127, 2023.
  26. J. Yang and Y. Li, “Research on road damage detection based on improved yolov8,” Academic Journal of Science and Technology, vol. 9, no. 3, pp. 162–167, 2024.
  27. W. Qian, S. Chen, Y. Huang, X. Xu, F. Shi, H. Wan, and Z. Lu, “Vehicle-mounted road damage detection method using yolov5s framework,” in 2023 IEEE 23rd International Conference on Communication Technology (ICCT).   IEEE, 2023, pp. 204–209.
  28. Y. Huang, X. Xu, Z. He, Y. Wang, Z. Lu, F. Shi, H. Wan, and G. Gui, “A lightweight road crack and damage detection method using yolov5s for iot applications,” in 2023 IEEE/CIC International Conference on Communications in China (ICCC).   IEEE, 2023, pp. 1–6.
  29. Z. Zhang, X. Lu, G. Cao, Y. Yang, L. Jiao, and F. Liu, “Vit-yolo: Transformer-based yolo for object detection,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 2799–2808.
  30. H. Zhou, S. Chen, G. Yan, Y. Huang, and Z. Lu, “An intelligent road damage detection method based on yolov7-tiny framework,” in 2023 IEEE 23rd International Conference on Communication Technology (ICCT).   IEEE, 2023, pp. 286–290.
  31. A. Singh, A. Mehta, A. A. Padaria, N. K. Jadav, R. Geddam, and S. Tanwar, “Enhanced pothole detection using yolov5 and federated learning,” in 2024 14th International Conference on Cloud Computing, Data Science & Engineering (Confluence).   IEEE, 2024, pp. 549–554.
  32. S. Kamalakannan, S. Navaneethan, Y. S. Deshmukh, V. Sujay, V. Ramalingam, D. Jagadeesan, C. Venkatesh et al., “A novel pothole detection model based on yolo algorithm for vanet,” International Journal of Intelligent Systems and Applications in Engineering, vol. 12, no. 11s, pp. 56–61, 2024.
  33. M. Khan, M. A. Raza, G. Abbas, S. Othmen, A. Yousef, and T. A. Jumani, “Pothole detection for autonomous vehicles using deep learning: a robust and efficient solution,” Frontiers in Built Environment, vol. 9, p. 1323792, 2024.
  34. A. Parmar, R. Gaiiar, and N. Gajjar, “Drone based potholes detection using machine learning on various edge ai devices in real-time,” in 2023 IEEE International Symposium on Smart Electronic Systems (iSES).   IEEE, 2023, pp. 22–26.
  35. S. Kumari, A. Gautam, S. Basak, and N. Saxena, “Yolov8 based deep learning method for potholes detection,” in 2023 IEEE International Conference on Computer Vision and Machine Intelligence (CVMI).   IEEE, 2023, pp. 1–6.
  36. C. Wei, Q. Zhang, X. Zhang, Y. Yang, J. Zhang, and D. Zhai, “Yolov5s-bss: A novel deep neural network for crack detection of road damage,” in 2023 IEEE International Conference on Big Data (BigData).   IEEE, 2023, pp. 1550–1554.
  37. C. Ruseruka, J. Mwakalonge, G. Comert, S. Siuhi, F. Ngeni, and Q. Anderson, “Augmenting roadway safety with machine learning and deep learning: Pothole detection and dimension estimation using in-vehicle technologies,” Machine Learning with Applications, vol. 16, p. 100547, 2024.
  38. S. V. Hong Pham, K. V. Tien Nguyen, H. Quang Le, and P. L. H. Tran, “Road surface damages allocation with rti-ims software based on yolo v5 model,” Sustainable and Resilient Infrastructure, vol. 9, no. 3, pp. 242–261, 2024.
  39. N. K. Rout, G. Dutta, V. Sinha, A. Dey, S. Mukherjee, and G. Gupta, “Improved pothole detection using yolov7 and esrgan,” arXiv preprint arXiv:2401.08588, 2023.
  40. F.-J. Du and S.-J. Jiao, “Improvement of lightweight convolutional neural network model based on yolo algorithm and its research in pavement defect detection,” Sensors, vol. 22, no. 9, p. 3537, 2022.
  41. J. Chen, X. Yu, Q. Li, W. Wang, and B.-G. He, “Lag-yolo: Efficient road damage detector via lightweight attention ghost module,” Journal of Intelligent Construction, vol. 2, no. 1, 2024.
  42. T. Fujii, R. Jinki, and Y. Horita, “Performance evaluation of detection model for road surface damage using yolo,” in 2023 IEEE 12th Global Conference on Consumer Electronics (GCCE).   IEEE, 2023, pp. 216–217.
  43. M. Yang, R. Lv, Z. Cao, X. Wang, and K. Sheng, “Roadyolo: An improved fast and accurate yolo model for road damage detection by global attention mechanism and lightweight modules,” Available at SSRN 4597443.
  44. G. Guo and Z. Zhang, “Road damage detection algorithm for improved yolov5,” Scientific reports, vol. 12, no. 1, p. 15523, 2022.
  45. V. Pham, D. Nguyen, and C. Donan, “Road damage detection and classification with yolov7,” in 2022 IEEE International Conference on Big Data (Big Data).   IEEE, 2022, pp. 6416–6423.
  46. H.-T. GENG, Z.-Y. LIU, J. JIANG, Z.-C. FAN, and J.-X. LI, “Embedded road crack detection algorithm based on improved yolov8,” Journal of Computer Applications, p. 0, 2023.
  47. V. H. S. Pham, V. T. K. Nguyen, and Q. H. Le, “Developing rti-ims software to identify road surface damages,” in The International Conference on Sustainable Civil Engineering and Architecture.   Springer, 2023, pp. 453–461.
  48. A. M. Okran, M. Abdel-Nasser, H. A. Rashwan, and D. Puig, “Effective deep learning-based ensemble model for road crack detection,” in 2022 IEEE international conference on big data (Big Data).   IEEE, 2022, pp. 6407–6415.
  49. K. Hacıefendioğlu and H. B. Başağa, “Concrete road crack detection using deep learning-based faster r-cnn method,” Iranian Journal of Science and Technology, Transactions of Civil Engineering, vol. 46, no. 2, pp. 1621–1633, 2022.
  50. S. Parvathavarthini, M. Shreekanth, S. Vigneshkumar, and N. Santhosh, “Road damage detection using deep learning,” in 2023 7th International Conference on Computing Methodologies and Communication (ICCMC).   IEEE, 2023, pp. 314–318.
  51. D. B. Naik and V. Dondeti, “Pothole detection and classification in vehicular networks using resnet-18,” in 2023 IEEE 7th Conference on Information and Communication Technology (CICT).   IEEE, 2023, pp. 1–6.
  52. C.-B. Lin and Y. Liu, “Research on potholes detection based on improved mask rcnn algorithms,” in 2023 12th International Conference on Awareness Science and Technology (iCAST).   IEEE, 2023, pp. 50–54.
  53. B. Kulambayev, M. Nurlybek, G. Astaubayeva, G. Tleuberdiyeva, S. Zholdasbayev, and A. Tolep, “Real-time road surface damage detection framework based on mask r-cnn model,” International Journal of Advanced Computer Science and Applications, vol. 14, no. 9, 2023.
  54. S. Li and Y. Huang, “Damage detection algorithm based on faster-rcnn,” in 2023 5th International Conference on Electronics and Communication, Network and Computer Technology (ECNCT).   IEEE, 2023, pp. 177–180.
  55. M. K. K. Reddy, T. Bhavana, B. Tejaswini, P. Mallesh, and A. sharath Chandra, “A deep learning approach for pothole detection,” Journal of Survey in Fisheries Sciences, pp. 2582–2585, 2023.
  56. A. K. Alsharafi and M. N. O. Zahid, “Investigation on a vision-based approch for smart pothole detection using deep learning based on fast cnn,” MEKATRONIKA, vol. 5, no. 2, pp. 87–99, 2023.
  57. X. Cui, Q. Wang, J. Dai, R. Zhang, and S. Li, “Intelligent recognition of erosion damage to concrete based on improved yolo-v3,” Materials Letters, vol. 302, p. 130363, 2021.
  58. P. K. Saha, D. Arya, and Y. Sekimoto, “Federated learning–based global road damage detection,” Computer-Aided Civil and Infrastructure Engineering, 2024.
  59. Z. Li, Y. Xie, X. Xiao, L. Tao, J. Liu, and K. Wang, “An image data augmentation algorithm based on yolov5s-da for pavement distress detection,” in 2022 5th International Conference on Pattern Recognition and Artificial Intelligence (PRAI).   IEEE, 2022, pp. 891–895.
  60. L. Zhu, C. Lu, Z. Y. Dong, and C. Hong, “Imbalance learning machine-based power system short-term voltage stability assessment,” IEEE Transactions on Industrial Informatics, vol. 13, no. 5, pp. 2533–2543, 2017.
  61. G. E. Batista, R. C. Prati, and M. C. Monard, “A study of the behavior of several methods for balancing machine learning training data,” ACM SIGKDD explorations newsletter, vol. 6, no. 1, pp. 20–29, 2004.
  62. D. Elreedy and A. F. Atiya, “A comprehensive analysis of synthetic minority oversampling technique (smote) for handling class imbalance,” Information Sciences, vol. 505, pp. 32–64, 2019.
  63. Y. Li, Y. Chai, Y. Hu, and H. Yin, “Review of imbalanced data classification methods,” Control and decision, vol. 34, no. 4, pp. 673–688, 2019.
  64. S. G. Müller and F. Hutter, “Trivialaugment: Tuning-free yet state-of-the-art data augmentation,” 2021.
  65. E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le, “Autoaugment: Learning augmentation policies from data,” 2019.
  66. J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2223–2232.
  67. J. Weickert and H. Scharr, “A scheme for coherence-enhancing diffusion filtering with optimized rotation invariance,” Journal of Visual Communication and Image Representation, vol. 13, no. 1-2, pp. 103–118, 2002.
  68. P. J. Burt and E. H. Adelson, “The laplacian pyramid as a compact image code,” in Readings in computer vision.   Elsevier, 1987, pp. 671–679.
  69. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020.
  70. K. He, X. Zhang, S. Ren, and J. Sun, “Spatial pyramid pooling in deep convolutional networks for visual recognition,” IEEE transactions on pattern analysis and machine intelligence, vol. 37, no. 9, pp. 1904–1916, 2015.
  71. L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 4, pp. 834–848, 2017.
  72. J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7132–7141.
  73. S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “Cbam: Convolutional block attention module,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 3–19.
  74. M. Tan, R. Pang, and Q. V. Le, “Efficientdet: Scalable and efficient object detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 10 781–10 790.
  75. X. Glorot, A. Bordes, and Y. Bengio, “Deep sparse rectifier neural networks,” in Proceedings of the fourteenth international conference on artificial intelligence and statistics.   JMLR Workshop and Conference Proceedings, 2011, pp. 315–323.
  76. G. Cybenko, “Approximation by superpositions of a sigmoidal function,” Mathematics of control, signals and systems, vol. 2, no. 4, pp. 303–314, 1989.
  77. Z. Zheng, P. Wang, D. Ren, W. Liu, R. Ye, Q. Hu, and W. Zuo, “Enhancing geometric factors in model learning and inference for object detection and instance segmentation,” IEEE Transactions on Cybernetics, vol. 52, no. 8, pp. 8574–8586, 2021.
  78. M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes (voc) challenge,” International journal of computer vision, vol. 88, pp. 303–338, 2010.
  79. Y.-F. Zhang, W. Ren, Z. Zhang, Z. Jia, L. Wang, and T. Tan, “Focal and efficient iou loss for accurate bounding box regression,” Neurocomputing, vol. 506, pp. 146–157, 2022.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets