Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Semantic Information Recovery in Wireless Networks (2204.13366v4)

Published 28 Apr 2022 in cs.IT, cs.AI, cs.LG, eess.SP, math.IT, and stat.ML

Abstract: Motivated by the recent success of Machine Learning (ML) tools in wireless communications, the idea of semantic communication by Weaver from 1949 has gained attention. It breaks with Shannon's classic design paradigm by aiming to transmit the meaning of a message, i.e., semantics, rather than its exact version and thus allows for savings in information rate. In this work, we extend the fundamental approach from Basu et al. for modeling semantics to the complete communications Markov chain. Thus, we model semantics by means of hidden random variables and define the semantic communication task as the data-reduced and reliable transmission of messages over a communication channel such that semantics is best preserved. We cast this task as an end-to-end Information Bottleneck problem, allowing for compression while preserving relevant information most. As a solution approach, we propose the ML-based semantic communication system SINFONY and use it for a distributed multipoint scenario: SINFONY communicates the meaning behind multiple messages that are observed at different senders to a single receiver for semantic recovery. We analyze SINFONY by processing images as message examples. Numerical results reveal a tremendous rate-normalized SNR shift up to 20 dB compared to classically designed communication systems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. C. E. Shannon, “A Mathematical Theory of Communication,” The Bell System Technical Journal, vol. 27, no. 3, pp. 379–423, Jul. 1948.
  2. W. Weaver, “Recent Contributions to the Mathematical Theory of Communication,” in The Mathematical Theory of Communication, 1949, vol. 10, pp. 261–281.
  3. M. Gastpar, B. Rimoldi, and M. Vetterli, “To code, or not to code: lossy source-channel communication revisited,” IEEE Trans. Inf. Theory, vol. 49, no. 5, pp. 1147–1158, May 2003.
  4. T. O’Shea and J. Hoydis, “An Introduction to Deep Learning for the Physical Layer,” IEEE Trans. on Cogn. Commun. Netw., vol. 3, no. 4, pp. 563–575, Dec. 2017.
  5. O. Simeone, “A Very Brief Introduction to Machine Learning with Applications to Communication Systems,” IEEE Trans. on Cogn. Commun. Netw., vol. 4, no. 4, pp. 648–664, Dec. 2018.
  6. E. Beck, C. Bockelmann, and A. Dekorsy, “CMDNet: Learning a Probabilistic Relaxation of Discrete Variables for Soft Detection With Low Complexity,” IEEE Trans. Commun., vol. 69, no. 12, pp. 8214–8227, Dec. 2021.
  7. P. Popovski, O. Simeone, F. Boccardi, D. Gündüz, and O. Sahin, “Semantic-Effectiveness Filtering and Control for Post-5G Wireless Connectivity,” Journal of the Indian Institute of Science, vol. 100, no. 2, pp. 435–443, Apr. 2020.
  8. E. C. Strinati and S. Barbarossa, “6G networks: Beyond Shannon towards semantic and goal-oriented communications,” Computer Networks, vol. 190, p. 107930, May 2021.
  9. Q. Lan, D. Wen, Z. Zhang, Q. Zeng, X. Chen, P. Popovski, and K. Huang, “What is Semantic Communication? A View on Conveying Meaning in the Era of Machine Intelligence,” Journal of Communications and Information Networks, vol. 6, no. 4, pp. 336–371, Dec. 2021.
  10. E. Uysal, O. Kaya, A. Ephremides, J. Gross, M. Codreanu, P. Popovski, M. Assaad, G. Liva, A. Munari, B. Soret, T. Soleymani, and K. H. Johansson, “Semantic Communications in Networked Systems: A Data Significance Perspective,” IEEE/ACM Trans. Netw., vol. 36, no. 4, pp. 233–240, Jul. 2022.
  11. D. Gündüz, Z. Qin, I. E. Aguerri, H. S. Dhillon, Z. Yang, A. Yener, K. K. Wong, and C.-B. Chae, “Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications,” IEEE J. Sel. Areas Commun., vol. 41, no. 1, pp. 5–41, Jan. 2023.
  12. L. Floridi, “Philosophical Conceptions of Information,” in Formal Theories of Information: From Shannon to Semantic Information Theory and General Concepts of Information, ser. Lecture Notes in Computer Science, 2009, pp. 13–53.
  13. J. Bao, P. Basu, M. Dean, C. Partridge, A. Swami, W. Leland, and J. A. Hendler, “Towards a theory of semantic communication,” in 2011 IEEE Network Science Workshop (NSW), West Point, NY, USA, Jun. 2011, pp. 110–117.
  14. P. Basu, J. Bao, M. Dean, and J. Hendler, “Preserving quality of information by using semantic relationships,” Pervasive and Mobile Computing, vol. 11, pp. 188–202, Apr. 2014.
  15. B. Güler, A. Yener, and A. Swami, “The Semantic Communication Game,” IEEE Trans. on Cogn. Commun. Netw., vol. 4, no. 4, pp. 787–802, Dec. 2018.
  16. N. Farsad, M. Rao, and A. Goldsmith, “Deep Learning for Joint Source-Channel Coding of Text,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, Canada, Apr. 2018, pp. 2326–2330.
  17. H. Xie, Z. Qin, G. Y. Li, and B.-H. Juang, “Deep Learning based Semantic Communications: An Initial Investigation,” in 2020 IEEE Global Communications Conference (GLOBECOM), Tapei, Taiwan, Dec. 2020, pp. 1–6.
  18. ——, “Deep Learning Enabled Semantic Communication Systems,” IEEE Transactions on Signal Processing, vol. 69, pp. 2663–2675, 2021.
  19. Z. Weng, Z. Qin, and G. Y. Li, “Semantic Communications for Speech Signals,” in 2021 IEEE International Conference on Communications (ICC), Virtual Conference, Jun. 2021, pp. 1–6.
  20. J. Shao, Y. Mao, and J. Zhang, “Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach,” IEEE Journal on Selected Areas in Communications, vol. 40, no. 1, pp. 197–211, Jan. 2022.
  21. E. Bourtsoulatze, D. B. Kurka, and D. Gündüz, “Deep Joint Source-Channel Coding for Wireless Image Transmission,” IEEE Trans. on Cogn. Commun. Netw., vol. 5, no. 3, pp. 567–579, Sep. 2019.
  22. N. Tishby, F. C. Pereira, and W. Bialek, “The information bottleneck method,” in 37th Annual Allerton Conference on Communication, Control and Computing, Sep. 1999, pp. 368–377.
  23. S. Hassanpour, T. Monsees, D. Wübben, and A. Dekorsy, “Forward-Aware Information Bottleneck-Based Vector Quantization for Noisy Channels,” IEEE Trans. Commun., vol. 68, no. 12, pp. 7911–7926, 2020.
  24. S. Hassanpour, D. Wübben, and A. Dekorsy, “Forward-Aware Information Bottleneck-Based Vector Quantization: Multiterminal Extensions for Parallel and Successive Retrieval,” IEEE Trans. Commun., vol. 69, no. 10, pp. 6633–6646, Jul. 2021.
  25. E. Beck, B.-S. Shin, S. Wang, T. Wiedemann, D. Shutin, and A. Dekorsy, “Swarm Exploration and Communications: A First Step towards Mutually-Aware Integration by Probabilistic Learning,” Electronics, vol. 12, no. 8, p. 1908, Apr. 2023.
  26. I. E. Aguerri and A. Zaidi, “Distributed Variational Representation Learning,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 1, pp. 120–138, Jan. 2021.
  27. J. Shao, Y. Mao, and J. Zhang, “Task-Oriented Communication for Multidevice Cooperative Edge Inference,” IEEE Transactions on Wireless Communications, vol. 22, no. 1, pp. 73–87, Jan. 2023.
  28. O. Simeone, “A Brief Introduction to Machine Learning for Engineers,” Foundations and Trends® in Signal Processing, vol. 12, no. 3-4, pp. 200–431, Aug. 2018.
  29. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol, “Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion,” Journal of Machine Learning Research, vol. 11, no. 110, pp. 3371–3408, 2010.
  30. M. Sana and E. C. Strinati, “Learning Semantics: An Opportunity for Effective 6G Communications,” in 2022 IEEE 19th Annual Consumer Communications Networking Conference (CCNC), Virtual Conference, Jan. 2022, pp. 631–636.
  31. N. Farsad, N. Shlezinger, A. J. Goldsmith, and Y. C. Eldar, “Data-Driven Symbol Detection Via Model-Based Machine Learning,” in 2021 IEEE Statistical Signal Processing Workshop (SSP), Virtual Conference, Jul. 2021, pp. 571–575.
  32. Z. Goldfeld and Y. Polyanskiy, “The Information Bottleneck Problem and its Applications in Machine Learning,” IEEE Journal on Selected Areas in Information Theory, vol. 1, no. 1, pp. 19–38, May 2020.
  33. A. Zaidi, I. Estella-Aguerri, and S. Shamai (Shitz), “On the Information Bottleneck Problems: Models, Connections, Applications and Information Theoretic Views,” Entropy, vol. 22, no. 2, p. 151, Feb. 2020.
  34. B. M. Kurkoski and H. Yagi, “Quantization of binary-input discrete memoryless channels,” IEEE Transactions on Information Theory, vol. 60, no. 8, pp. 4544–4552, 2014.
  35. J. Lewandowsky and G. Bauch, “Information-optimum ldpc decoders based on the information bottleneck method,” IEEE Access, vol. 6, pp. 4054–4071, 2018.
  36. A. A. Alemi, I. Fischer, J. V. Dillon, and K. Murphy, “Deep Variational Information Bottleneck,” in 5th International Conference on Learning Representations (ICLR), Toulon, France, Apr. 2017, pp. 1–19.
  37. M. I. Belghazi, A. Baratin, S. Rajeswar, S. Ozair, Y. Bengio, A. Courville, and R. D. Hjelm, “MINE: Mutual Information Neural Estimation,” in Proc. of the 35th International Conference on Machine Learning (PMLR), Stockholm, Sweden, Jun. 2018.
  38. K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, Jun. 2016, pp. 770–778.
  39. ——, “Identity Mappings in Deep Residual Networks,” in European Conference on Computer Vision 2016 (ECCV), ser. Lecture Notes in Computer Science, Amsterdam, Netherlands, Oct. 2016, pp. 630–645.
  40. E. Beck, “Semantic Information Transmission and Recovery (SINFONY) Software,” Jun. 2023, Zenodo. [Online]. Available: https://doi.org/10.5281/zenodo.8006567
  41. K. He, X. Zhang, S. Ren, and J. Sun, “Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification,” in IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, Dec. 2015, pp. 1026–1034.
  42. J. Hoydis, S. Cammerer, F. A. Aoudia, A. Vem, N. Binder, G. Marcus, and A. Keller, “Sionna: An Open-Source Library for Next-Generation Physical Layer Research,” arXiv:2203.11854 [cs, math], Mar. 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Edgar Beck (8 papers)
  2. Carsten Bockelmann (22 papers)
  3. Armin Dekorsy (43 papers)
Citations (18)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now