Swin Transformer-Based Dynamic Semantic Communication for Multi-User with Different Computing Capacity (2307.03402v1)
Abstract: Semantic communication has gained significant attention from researchers as a promising technique to replace conventional communication in the next generation of communication systems, primarily due to its ability to reduce communication costs. However, little literature has studied its effectiveness in multi-user scenarios, particularly when there are variations in the model architectures used by users and their computing capacities. To address this issue, we explore a semantic communication system that caters to multiple users with different model architectures by using a multi-purpose transmitter at the base station (BS). Specifically, the BS in the proposed framework employs semantic and channel encoders to encode the image for transmission, while the receiver utilizes its local channel and semantic decoder to reconstruct the original image. Our joint source-channel encoder at the BS can effectively extract and compress semantic features for specific users by considering the signal-to-noise ratio (SNR) and computing capacity of the user. Based on the network status, the joint source-channel encoder at the BS can adaptively adjust the length of the transmitted signal. A longer signal ensures more information for high-quality image reconstruction for the user, while a shorter signal helps avoid network congestion. In addition, we propose a hybrid loss function for training, which enhances the perceptual details of reconstructed images. Finally, we conduct a series of extensive evaluations and ablation studies to validate the effectiveness of the proposed system.
- L. X. Nguyen, Y. K. Tun, T. N. Dang, Y. M. Park, Z. Han, and C. S. Hong, “Dependency tasks offloading and communication resource allocation in collaborative UAV networks: A metaheuristic approach,” IEEE Internet of Things Journal, vol. 10, no. 10, pp. 9062–9076, Jan. 2023.
- X. Luo, H.-H. Chen, and Q. Guo, “Semantic communications: Overview, open issues, and future research directions,” IEEE Wireless Communications, vol. 29, no. 1, pp. 210–219, Feb. 2022.
- W. Jiang, B. Han, M. A. Habibi, and H. D. Schotten, “The road towards 6G: A comprehensive survey,” IEEE Open Journal of the Communications Society, vol. 2, pp. 334–366, Feb. 2021.
- T. N. Dang, L. X. Nguyen, H. Q. Le, K. Kim, S. A. Kazmi, S.-B. Park, E.-N. Huh, and C. S. Hong, “Semantic communication for ar-based services in 5g and beyond,” in Proc. IEEE International Conference on Information Networking (ICOIN), Bangkok, Thailand, Jan. 2023.
- Z. Weng and Z. Qin, “Semantic communication systems for speech transmission,” IEEE Journal on Selected Areas in Communications, vol. 39, no. 8, pp. 2434–2444, Aug. 2021.
- T. Han, Q. Yang, Z. Shi, S. He, and Z. Zhang, “Semantic-preserved communication system for highly efficient speech transmission,” IEEE Journal on Selected Areas in Communications, vol. 41, no. 1, pp. 245–259, Jan. 2023.
- H. Xie, Z. Qin, G. Y. Li, and B.-H. Juang, “Deep learning enabled semantic communication systems,” IEEE Transactions on Signal Processing, vol. 69, pp. 2663–2675, Apr. 2021.
- X. Peng, Z. Qin, D. Huang, X. Tao, J. Lu, G. Liu, and C. Pan, “A robust deep learning enabled semantic communication system for text,” in Proc. IEEE Global Communications Conference (GLOBECOM), Brazil, Dec. 2022.
- J. Kang, H. Du, Z. Li, Z. Xiong, S. Ma, D. Niyato, and Y. Li, “Personalized saliency in task-oriented semantic communications: Image transmission and performance analysis,” IEEE Journal on Selected Areas in Communications, vol. 41, no. 1, pp. 186–201, Nov. 2022.
- W. Zhang, Y. Wang, M. Chen, T. Luo, and D. Niyato, “Optimization of image transmission in a cooperative semantic communication networks,” IEEE Transactions on Wireless Communications, Early Access, Jun. 2023, DOI: 10.1109/TWC.2023.3282906.
- Q. Hu, G. Zhang, Z. Qin, Y. Cai, G. Yu, and G. Y. Li, “Robust semantic communications with masked VQ-VAE enabled codebook,” IEEE Transactions on Wireless Communications, Early Access, Apr. 2023, DOI: 10.1109/TWC.2023.3265201.
- H. Hu, X. Zhu, F. Zhou, W. Wu, R. Q. Hu, and H. Zhu, “One-to-many semantic communication systems: Design, implementation, performance evaluation,” IEEE Communications Letters, vol. 26, no. 12, pp. 2959–2963, Sep. 2022.
- H. Xie, Z. Qin, and G. Y. Li, “Task-oriented multi-user semantic communications for VQA,” IEEE Wireless Communications Letters, vol. 11, no. 3, pp. 553–557, Dec. 2021.
- E. Bourtsoulatze, D. B. Kurka, and D. Gündüz, “Deep joint source-channel coding for wireless image transmission,” IEEE Transactions on Cognitive Communications and Networking, vol. 5, no. 3, pp. 567–579, May. 2019.
- M. K. Farshbafan, W. Saad, and M. Debbah, “Common language for goal-oriented semantic communications: A curriculum learning framework,” in Proc. IEEE International Conference on Communications-(ICC), Seoul, Korea, Aug. 2022.
- Y. Wang, M. Chen, T. Luo, W. Saad, D. Niyato, H. V. Poor, and S. Cui, “Performance optimization for semantic communications: An attention-based reinforcement learning approach,” IEEE Journal on Selected Areas in Communications, vol. 40, no. 9, pp. 2598–2613, Jul. 2022.
- C. K. Thomas and W. Saad, “Neuro-symbolic causal reasoning meets signaling game for emergent semantic communications,” arXiv preprint arXiv:2210.12040, Oct. 2022.
- J. Xu, B. Ai, W. Chen, A. Yang, P. Sun, and M. Rodrigues, “Wireless image transmission using deep source channel coding with attention modules,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 4, pp. 2315–2328, May. 2021.
- D. B. Kurka and D. Gündüz, “DeepJSCC-f: Deep joint source-channel coding of images with feedback,” IEEE Journal on Selected Areas in Information Theory, vol. 1, no. 1, pp. 178–193, Apr. 2020.
- Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature, vol. 521, no. 7553, pp. 436–444, May. 2015.
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Proc. Advances in neural information processing systems, vol. 30, Long Beach, CA, Dec. 2017.
- A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, Oct. 2020.
- H. Xie, Z. Qin, X. Tao, and K. B. Letaief, “Task-oriented multi-user semantic communications,” IEEE Journal on Selected Areas in Communications, vol. 40, no. 9, pp. 2584–2597, Sep. 2022.
- K. Yang, S. Wang, J. Dai, K. Tan, K. Niu, and P. Zhang, “WITT: A wireless image transmission transformer for semantic communications,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May. 2023.
- Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF international conference on computer vision, Montreal, Canada, Oct. 2021.
- C. Zhu, W. Ping, C. Xiao, M. Shoeybi, T. Goldstein, A. Anandkumar, and B. Catanzaro, “Long-short transformer: Efficient transformers for language and vision,” Proc. Advances in Neural Information Processing Systems(NeurIPS), vol. 34, Dec. 2021.
- Z. Wang, A. C. Bovik, and E. P. Simoncelli, “Structural approaches to image quality assessment,” Handbook of image and video processing, vol. 7, no. 18, Jun. 2005.
- Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing, vol. 13, no. 4, pp. 600–612, Apr. 2004.
- Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structural similarity for image quality assessment,” in The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, vol. 2, Pacific Grove, CA, Nov. 2003.
- H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for image restoration with neural networks,” IEEE Transactions on computational imaging, vol. 3, no. 1, pp. 47–57, Dec. 2016.
- A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems, vol. 32, Vancouver, Canada, Dec. 2019.
- E. Agustsson and R. Timofte, “Ntire 2017 challenge on single image super-resolution: Dataset and study,” in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, Honolulu, HI, Jul. 2017.
- R. Franzen, “Kodak lossless true color image suite,” source: http://r0k. us/graphics/kodak, vol. 4, no. 2, Nov. 1999.
- D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. International Conference on Learning Representations (ICLR), San Diega, CA, May. 2015.
- Loc X. Nguyen (12 papers)
- Ye Lin Tun (16 papers)
- Yan Kyaw Tun (37 papers)
- Minh N. H. Nguyen (17 papers)
- Chaoning Zhang (66 papers)
- Zhu Han (431 papers)
- Choong Seon Hong (165 papers)