End-to-End Generative Semantic Communication Powered by Shared Semantic Knowledge Base (2405.05738v1)
Abstract: Semantic communication has drawn substantial attention as a promising paradigm to achieve effective and intelligent communications. However, efficient image semantic communication encounters challenges with a lower testing compression ratio (CR) compared to the training phase. To tackle this issue, we propose an innovative semantic knowledge base (SKB)-enabled generative semantic communication system for image classification and image generation tasks. Specifically, a lightweight SKB, comprising class-level information, is exploited to guide the semantic communication process, which enables us to transmit only the relevant indices. This approach promotes the completion of the image classification task at the source end and significantly reduces the transmission load. Meanwhile, the category-level knowledge in the SKB facilitates the image generation task by allowing controllable generation, making it possible to generate favorable images in resource-constrained scenarios. Additionally, semantic accuracy is introduced as a new metric to validate the performance of semantic transmission powered by the SKB. Evaluation results indicate that the proposed method outperforms the benchmarks and achieves superior performance with minimal transmission overhead, especially in the low SNR regime.
- K. B. Letaief, Y. Shi, J. Lu, and J. Lu, “Edge artificial intelligence for 6G: Vision, enabling technologies, and applications,” IEEE Journal on Selected Areas in Communications, vol. 40, no. 1, pp. 5–36, 2022.
- W. Yang, H. Du, Z. Q. Liew, W. Y. B. Lim, Z. Xiong, D. Niyato, X. Chi, X. Shen, and C. Miao, “Semantic communications for future Internet: Fundamentals, applications, and challenges,” IEEE Communications Surveys & Tutorials, vol. 25, no. 1, pp. 213–250, 2023.
- F. Zhou, Y. Li, X. Zhang, Q. Wu, X. Lei, and R. Q. Hu, “Cognitive semantic communication systems driven by knowledge graph,” in IEEE International Conference on Communications (ICC), Seoul, South Korea, 2022.
- X. Xu, H. Xiong, Y. Wang, S. Han, B. Wang, and P. Zhang, “Knowledge-enhanced semantic communication system with OFDM transmissions,” Science China Information Sciences, vol. 66, no. 7, pp. 172 302:1–17, 2023.
- P. Yi, Y. Cao, X. Kang, and Y.-C. Liang, “Semantic communication systems with a shared knowledge base,” in IEEE International Conference on Communications Workshops (ICC Workshops), Rome, Italy, 2023.
- H. Zhang, M. Tao, and S. Shao, “A wyner-ziv coding-based semantic communication approach with a shared semantic codebook,” in IEEE Global Communications Conference Workshops (GC Wkshps), Kuala Lumpur, Malaysia, 2023.
- Q. Hu, G. Zhang, Z. Qin, Y. Cai, G. Yu, and G. Y. Li, “Robust semantic communications with masked VQ-VAE enabled codebook,” IEEE Transactions on Wireless Communications, vol. 22, no. 12, pp. 8707–8722, 2023.
- Y. Sun, H. Chen, X. Xu, P. Zhang, and S. Cui, “Zero-shot multi-level feature transmission policy powered by semantic knowledge base,” in IEEE Global Communications Conference (GLOBECOM), Kuala Lumpur, Malaysia, 2023.
- H. Zhang, S. Shao, M. Tao, X. Bi, and K. B. Letaief, “Deep learning-enabled semantic communication systems with task-unaware transmitter and dynamic data,” IEEE Journal on Selected Areas in Communications, vol. 41, no. 1, pp. 170–185, 2023.
- S. Tang, Q. Yang, L. Fan, X. Lei, Y. Deng, and A. Nallanathan, “Contrastive learning based semantic communication for wireless image transmission,” in IEEE 98th Vehicular Technology Conference (VTC2023-Fall), Hong Kong, China, 2023.
- A. Bandi, P. V. S. R. Adapa, and Y. E. V. P. K. Kuchi, “The power of generative AI: A review of requirements, models, input-output formats, evaluation metrics, and challenges,” Future Internet, vol. 15, no. 8, 2023.
- E. Erdemir, T.-Y. Tung, P. L. Dragotti, and D. Gündüz, “Generative joint source-channel coding for semantic image transmission,” IEEE Journal on Selected Areas in Communications, vol. 41, no. 8, pp. 2645–2657, 2023.
- E. Grassucci, J. Park, S. Barbarossa, S.-L. Kim, J. Choi, and D. Comminiello, “Generative AI meets semantic communication: Evolution and revolution of communication tasks,” arXiv preprint arXiv:2401.06803, 2024.
- W. Xu, Y. Xian, J. Wang, B. Schiele, and Z. Akata, “Attribute prototype network for zero-shot learning,” in Advances in Neural Information Processing Systems (NeurIPS), 2020.
- A. Vahdat and J. Kautz, “NVAE: A deep hierarchical variational autoencoder,” in Advances in Neural Information Processing Systems (NeurIPS), 2020.
- C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, “The caltech-ucsd birds-200-2011 dataset,” technical Report, 2011.
- M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “GANs trained by a two time-scale update rule converge to a local nash equilibrium,” in Advances in Neural Information Processing Systems (NeurIPS), 2017.
- Shuling Li (2 papers)
- Yaping Sun (25 papers)
- Jinbei Zhang (6 papers)
- Kechao Cai (8 papers)
- Shuguang Cui (275 papers)
- Xiaodong Xu (268 papers)