YOLO-Vehicle-Pro: A Cloud-Edge Collaborative Framework for Object Detection in Autonomous Driving under Adverse Weather Conditions (2410.17734v1)
Abstract: With the rapid advancement of autonomous driving technology, efficient and accurate object detection capabilities have become crucial factors in ensuring the safety and reliability of autonomous driving systems. However, in low-visibility environments such as hazy conditions, the performance of traditional object detection algorithms often degrades significantly, failing to meet the demands of autonomous driving. To address this challenge, this paper proposes two innovative deep learning models: YOLO-Vehicle and YOLO-Vehicle-Pro. YOLO-Vehicle is an object detection model tailored specifically for autonomous driving scenarios, employing multimodal fusion techniques to combine image and textual information for object detection. YOLO-Vehicle-Pro builds upon this foundation by introducing an improved image dehazing algorithm, enhancing detection performance in low-visibility environments. In addition to model innovation, this paper also designs and implements a cloud-edge collaborative object detection system, deploying models on edge devices and offloading partial computational tasks to the cloud in complex situations. Experimental results demonstrate that on the KITTI dataset, the YOLO-Vehicle-v1s model achieved 92.1% accuracy while maintaining a detection speed of 226 FPS and an inference time of 12ms, meeting the real-time requirements of autonomous driving. When processing hazy images, the YOLO-Vehicle-Pro model achieved a high accuracy of 82.3% mAP@50 on the Foggy Cityscapes dataset while maintaining a detection speed of 43 FPS.
- S. Liang, H. Wu, L. Zhen, Q. Hua, S. Garg, G. Kaddoum, M. M. Hassan, and K. Yu, “Edge yolo: Real-time intelligent object detection system based on edge-cloud cooperation in autonomous vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 12, pp. 25 345–25 360, 2022.
- N. S. Rajput, U. Singh, A. Dua, N. Kumar, J. J. P. C. Rodrigues, S. Sisodia, M. Elhoseny, and Y. Lakys, “Amalgamating vehicular networks with vehicular clouds, ai, and big data for next-generation its services,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 1, pp. 869–883, 2024.
- Q. Zhang, R. Cong, C. Li, M.-M. Cheng, Y. Fang, X. Cao, Y. Zhao, and S. Kwong, “Dense attention fluid network for salient object detection in optical remote sensing images,” IEEE Transactions on Image Processing, vol. 30, pp. 1305–1317, 2021.
- H. Ding, J. Gao, Y. Yuan, and Q. Wang, “An end-to-end contrastive license plate detector,” IEEE Trans. Intell. Transp. Syst., vol. 25, no. 1, pp. 503–516, 2024. [Online]. Available: https://doi.org/10.1109/TITS.2023.3304816
- C. Zhang, S. Zheng, H. Wu, Z. Gu, W. Sun, and L. Yang, “Attentiontrack: Multiple object tracking in traffic scenarios using features attention,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 2, pp. 1661–1674, 2024.
- C. Li, L. Li, H. Jiang, K. Weng, Y. Geng, L. Li, Z. Ke, Q. Li, M. Cheng, W. Nie, Y. Li, B. Zhang, Y. Liang, L. Zhou, X. Xu, X. Chu, X. Wei, and X. Wei, “Yolov6: A single-stage object detection framework for industrial applications,” CoRR, vol. abs/2209.02976, 2022. [Online]. Available: https://doi.org/10.48550/arXiv.2209.02976
- C. Wang, A. Bochkovskiy, and H. M. Liao, “Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023. IEEE, 2023, pp. 7464–7475. [Online]. Available: https://doi.org/10.1109/CVPR52729.2023.00721
- Q. Chen, Y. Wang, T. Yang, X. Zhang, J. Cheng, and J. Sun, “You only look one-level feature,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 13 034–13 043.
- T. Cheng, L. Song, Y. Ge, W. Liu, X. Wang, and Y. Shan, “Yolo-world: Real-time open-vocabulary object detection,” in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 16 901–16 911.
- L. Tian, H. Zhang, B. Liu, J. Zhang, N. Duan, A. Yuan, and Y. Huo, “Vmf-ssd: A novel v-space based multi-scale feature fusion ssd for apple leaf disease detection,” IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 20, no. 3, pp. 2016–2028, 2023.
- J. Ni, K. Shen, Y. Chen, and S. X. Yang, “An improved ssd-like deep network-based object detection method for indoor scenes,” IEEE Transactions on Instrumentation and Measurement, vol. 72, pp. 1–15, 2023.
- M. He, H. Wang, F. Zhang, and Y. Xiang, “Exploring accurate invariants on polar harmonic fourier moments in polar coordinates for robust image watermarking,” IEEE Transactions on Multimedia, vol. 26, pp. 5435–5449, 2024.
- X. Sun, P. Wang, W. Lu, Z. Zhu, X. Lu, Q. He, J. Li, X. Rong, Z. Yang, H. Chang, Q. He, G. Yang, R. Wang, J. Lu, and K. Fu, “Ringmo: A remote sensing foundation model with masked image modeling,” IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1–22, 2023.
- Z. Tian, C. Shen, H. Chen, and T. He, “Fcos: A simple and strong anchor-free object detector,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 4, pp. 1922–1933, 2022.
- K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel prior,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 12, pp. 2341–2353, 2011. [Online]. Available: https://doi.org/10.1109/TPAMI.2010.168
- T. M. Bui and W. Kim, “Single image dehazing using color ellipsoid prior,” IEEE Transactions on Image Processing, vol. 27, no. 2, pp. 999–1009, 2018.
- Q. Liu, X. Gao, L. He, and W. Lu, “Single image dehazing with depth-aware non-local total variation regularization,” IEEE Transactions on Image Processing, vol. 27, no. 10, pp. 5178–5191, 2018.
- B. Li, X. Peng, Z. Wang, J. Xu, and D. Feng, “Aod-net: All-in-one dehazing network,” in 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 4780–4788.
- X. Liu, Z. Shi, Z. Wu, J. Chen, and G. Zhai, “Griddehazenet+: An enhanced multi-scale network with intra-task knowledge transfer for single image dehazing,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 1, pp. 870–884, 2023.
- Y. Wang, X. Yan, F. L. Wang, H. Xie, W. Yang, X.-P. Zhang, J. Qin, and M. Wei, “Ucl-dehaze: Toward real-world image dehazing via unsupervised contrastive learning,” IEEE Transactions on Image Processing, vol. 33, pp. 1361–1374, 2024.
- W. Ren, L. Ma, J. Zhang, J. Pan, X. Cao, W. Liu, and M.-H. Yang, “Gated fusion network for single image dehazing,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 3253–3261.
- Y. Li, R. Lan, H. Huang, H. Zhou, Z. Liu, C. Pang, and X. Luo, “Single traffic image deraining via similarity-diversity model,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 1, pp. 90–103, 2024.
- L. Li, Y. Dong, W. Ren, J. Pan, C. Gao, N. Sang, and M.-H. Yang, “Semi-supervised image dehazing,” IEEE Transactions on Image Processing, vol. 29, pp. 2766–2779, 2020.
- Z. Chen, Z. He, and Z.-M. Lu, “Dea-net: Single image dehazing based on detail-enhanced convolution and content-guided attention,” IEEE Transactions on Image Processing, vol. 33, pp. 1002–1015, 2024.
- G. J. et al., “Yolov8,” Available: https://github.com/ultralytics/ultralytics, 2023.
- X. Sun, P. Zhang, P. Zhang, H. Shah, K. Saenko, and X. Xia, “Dime-fm : Distilling multimodal and efficient foundation models,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 15 475–15 487.
- A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervision,” in Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Eds., vol. 139. PMLR, 2021, pp. 8748–8763. [Online]. Available: http://proceedings.mlr.press/v139/radford21a.html
- C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, “Scaled-yolov4: Scaling cross stage partial network,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 13 024–13 033.
- J. Liu, Y. Qiu, X. Ni, B. Shi, and H. Liu, “Fast detection of railway fastener using a new lightweight network op-yolov4-tiny,” IEEE Trans. Intell. Transp. Syst., vol. 25, no. 1, pp. 133–143, 2024. [Online]. Available: https://doi.org/10.1109/TITS.2023.3305300
- Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 9992–10 002.
- V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” in Proceedings of the 27th International Conference on Machine Learning (ICML-10), June 21-24, 2010, Haifa, Israel, J. Fürnkranz and T. Joachims, Eds. Omnipress, 2010, pp. 807–814. [Online]. Available: https://icml.cc/Conferences/2010/papers/432.pdf
- A. Sekar and V. Perumal, “Cfc-gan: Forecasting road surface crack using forecasted crack generative adversarial network,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 11, pp. 21 378–21 391, 2022.
- Z. Zheng, P. Wang, W. Liu, J. Li, R. Ye, and D. Ren, “Distance-iou loss: Faster and better learning for bounding box regression,” in The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020. AAAI Press, 2020, pp. 12 993–13 000. [Online]. Available: https://doi.org/10.1609/aaai.v34i07.6999
- J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 779–788.
- E. J. C. Nacpil, J. Han, and I. Jeon, “Artificial intelligence-assisted robustness of optoelectronics for automated driving: A review,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 1, pp. 57–73, 2024.
- Y. Huang, S. Yang, L. Wang, K. Yuan, H. Zheng, and H. Chen, “An efficient self-evolution method of autonomous driving for any given algorithm,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 1, pp. 602–612, 2024.