Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Importance-Aware Image Segmentation-based Semantic Communication for Autonomous Driving (2401.10153v1)

Published 16 Jan 2024 in cs.NI and cs.CV

Abstract: This article studies the problem of image segmentation-based semantic communication in autonomous driving. In real traffic scenes, detecting the key objects (e.g., vehicles, pedestrians and obstacles) is more crucial than that of other objects to guarantee driving safety. Therefore, we propose a vehicular image segmentation-oriented semantic communication system, termed VIS-SemCom, where image segmentation features of important objects are transmitted to reduce transmission redundancy. First, to accurately extract image semantics, we develop a semantic codec based on Swin Transformer architecture, which expands the perceptual field thus improving the segmentation accuracy. Next, we propose a multi-scale semantic extraction scheme via assigning the number of Swin Transformer blocks for diverse resolution features, thus highlighting the important objects' accuracy. Furthermore, the importance-aware loss is invoked to emphasize the important objects, and an online hard sample mining (OHEM) strategy is proposed to handle small sample issues in the dataset. Experimental results demonstrate that the proposed VIS-SemCom can achieve a coding gain of nearly 6 dB with a 60% mean intersection over union (mIoU), reduce the transmitted data amount by up to 70% with a 60% mIoU, and improve the segmentation intersection over union (IoU) of important objects by 4%, compared to traditional transmission scheme.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. S. Chen, J. Hu, Y. Shi, L. Zhao, and W. Li, “A vision of c-v2x: Technologies, field testing, and challenges with chinese development,” IEEE Internet of Things Journal, vol. 7, no. 5, pp. 3872–3881, 2020.
  2. H. Zhou, W. Xu, J. Chen, and W. Wang, “Evolutionary v2x technologies toward the internet of vehicles: Challenges and opportunities,” Proceedings of the IEEE, vol. 108, no. 2, pp. 308–323, 2020.
  3. J. Choi, V. Va, N. Gonzalez-Prelcic, R. Daniels, C. R. Bhat, and R. W. Heath, “Millimeter-wave vehicular communication to support massive automotive sensing,” IEEE Communications Magazine, vol. 54, no. 12, pp. 160–167, 2016.
  4. L. Kong, M. K. Khan, F. Wu, G. Chen, and P. Zeng, “Millimeter-wave wireless communications for iot-cloud supported autonomous vehicles: Overview, design, and challenges,” IEEE Communications Magazine, vol. 55, no. 1, pp. 62–68, 2017.
  5. A. Memedi and F. Dressler, “Vehicular visible light communications: A survey,” IEEE Communications Surveys & Tutorials, vol. 23, no. 1, pp. 161–181, 2020.
  6. M. Almarashli and S. Lindenmeier, “Evaluation of vehicular 4g/5g-mimo antennas via data-rate measurement in an emulated urban test drive,” in 2018 48th European Microwave Conference (EuMC).   IEEE, 2018, pp. 300–303.
  7. J. Bao, P. Basu, M. Dean, C. Partridge, A. Swami, W. Leland, and J. A. Hendler, “Towards a theory of semantic communication,” in 2011 IEEE Network Science Workshop.   IEEE, 2011, pp. 110–117.
  8. E. Calvanese Strinati and S. Barbarossa, “6g networks: Beyond shannon towards semantic and goal-oriented communications,” Computer Networks, vol. 190, p. 107930, 2021.
  9. P. Zhang, W. Xu, H. Gao, K. Niu, X. Xu, X. Qin, C. Yuan, Z. Qin, H. Zhao, J. Wei, and F. Zhang, “Toward wisdom-evolutionary and primitive-concise 6g: A new paradigm of semantic communication networks,” Engineering, vol. 8, no. 1, pp. 60–73, 2022.
  10. H. Xie, Z. Qin, G. Y. Li, and B.-H. Juang, “Deep learning enabled semantic communication systems,” IEEE Transactions on Signal Processing, vol. 69, pp. 2663–2675, 2021.
  11. H. Xie and Z. Qin, “A lite distributed semantic communication system for internet of things,” IEEE Journal on Selected Areas in Communications, vol. 39, no. 1, pp. 142–153, 2020.
  12. P. Jiang, C.-K. Wen, S. Jin, and G. Y. Li, “Deep source-channel coding for sentence semantic transmission with harq,” IEEE transactions on communications, vol. 70, no. 8, pp. 5225–5240, 2022.
  13. Z. Weng, Z. Qin, and G. Y. Li, “Semantic communications for speech signals,” in ICC 2021-IEEE International Conference on Communications.   IEEE, 2021, pp. 1–6.
  14. Z. Weng and Z. Qin, “Semantic communication systems for speech transmission,” IEEE Journal on Selected Areas in Communications, vol. 39, no. 8, pp. 2434–2444, 2021.
  15. H. Tong, Z. Yang, S. Wang, Y. Hu, W. Saad, and C. Yin, “Federated learning based audio semantic communication over wireless networks,” in 2021 IEEE Global Communications Conference (GLOBECOM), 2021, pp. 1–6.
  16. E. Bourtsoulatze, D. B. Kurka, and D. Gündüz, “Deep joint source-channel coding for wireless image transmission,” IEEE Transactions on Cognitive Communications and Networking, vol. 5, no. 3, pp. 567–579, 2019.
  17. D. B. Kurka and D. Gündüz, “Deep joint source-channel coding of images with feedback,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).   IEEE, 2020, pp. 5235–5239.
  18. M. Jankowski, D. Gündüz, and K. Mikolajczyk, “Joint device-edge inference over wireless links with pruning,” in 2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).   IEEE, 2020, pp. 1–5.
  19. X. Kang, B. Song, J. Guo, Z. Qin, and F. R. Yu, “Task-oriented image transmission for scene classification in unmanned aerial systems,” IEEE Transactions on Communications, vol. 70, no. 8, pp. 5181–5192, 2022.
  20. P. Jiang, C.-K. Wen, S. Jin, and G. Y. Li, “Wireless semantic communications for video conferencing,” IEEE Journal on Selected Areas in Communications, vol. 41, no. 1, pp. 230–244, 2023.
  21. S. Wang, J. Dai, Z. Liang, K. Niu, Z. Si, C. Dong, X. Qin, and P. Zhang, “Wireless deep video semantic transmission,” IEEE Journal on Selected Areas in Communications, vol. 41, no. 1, pp. 214–229, 2023.
  22. C. Liang, X. Deng, Y. Sun, R. Cheng, L. Xia, D. Niyato, and M. A. Imran, “Vista: Video transmission over a semantic communication approach,” in 2023 IEEE International Conference on Communications Workshops (ICC Workshops), 2023, pp. 1777–1782.
  23. J. Chen, C. Guo, C. Feng, and C. Liu, “Resource allocation for semantic communication in intelligent networked environment,” Chinese Journal on Internet of Things, vol. 6, no. 3, pp. 47–57, 2022.
  24. W. Xu, Y. Zhang, F. Wang, Z. Qin, C. Liu, and P. Zhang, “Semantic communication for the internet of vehicles: A multiuser cooperative approach,” IEEE Vehicular Technology Magazine, vol. 18, no. 1, pp. 100–109, 2023.
  25. L. Xia, Y. Sun, D. Niyato, D. Feng, L. Feng, and M. A. Imran, “xurllc-aware service provisioning in vehicular networks: A semantic communication perspective,” IEEE Transactions on Wireless Communications, pp. 1–1, 2023.
  26. J. Su, Z. Liu, Y.-a. Xie, K. Ma, H. Du, J. Kang, and D. Niyato, “Semantic communication-based dynamic resource allocation in d2d vehicular networks,” IEEE Transactions on Vehicular Technology, vol. 72, no. 8, pp. 10 784–10 796, 2023.
  27. Q. Hu, G. Zhang, Z. Qin, Y. Cai, G. Yu, and G. Y. Li, “Robust semantic communications against semantic noise,” in 2022 IEEE 96th Vehicular Technology Conference (VTC2022-Fall), 2022, pp. 1–6.
  28. H. Yoo, L. Dai, S. Kim, and C.-B. Chae, “On the role of vit and cnn in semantic communications: Analysis and prototype validation,” IEEE Access, vol. 11, pp. 71 528–71 541, 2023.
  29. K. Yang, S. Wang, J. Dai, K. Tan, K. Niu, and P. Zhang, “Witt: A wireless image transmission transformer for semantic communications,” in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023, pp. 1–5.
  30. B. Chen, C. Gong, and J. Yang, “Importance-aware semantic segmentation for autonomous vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 1, pp. 137–148, 2019.
  31. Q. Pan, H. Tong, J. Lv, T. Luo, Z. Zhang, C. Yin, and J. Li, “Image segmentation semantic communication over internet of vehicles,” in 2023 IEEE Wireless Communications and Networking Conference (WCNC), 2023, pp. 1–6.
  32. Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 9992–10 002.
  33. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015.   Springer, 2015, pp. 234–241.
  34. L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 4, pp. 834–848, 2017.
  35. M. Contributors, “MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark,” https://github.com/open-mmlab/mmsegmentation, 2020.
  36. M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 3213–3223.
  37. Y. Yuan, X. Chen, and J. Wang, “Object-contextual representations for semantic segmentation.” in European Conference on Computer Vision – ECCV 2020.   Springer, 2020, pp. 173–190.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Jie Lv (5 papers)
  2. Haonan Tong (15 papers)
  3. Qiang Pan (2 papers)
  4. Zhilong Zhang (20 papers)
  5. Xinxin He (5 papers)
  6. Tao Luo (149 papers)
  7. Changchuan Yin (64 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets