Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

One-Shot Coding over General Noisy Networks (2402.06021v2)

Published 8 Feb 2024 in cs.IT and math.IT

Abstract: We present a unified one-shot coding framework designed for the communication and compression of messages among multiple nodes across a general acyclic noisy network. Our setting can be seen as a one-shot version of the acyclic discrete memoryless network studied by Lee and Chung, and noisy network coding studied by Lim, Kim, El Gamal and Chung. We design a proof technique, called the exponential process refinement lemma, that is rooted in the Poisson matching lemma by Li and Anantharam, and can significantly simplify the analyses of one-shot coding over multi-hop networks. Our one-shot coding theorem not only recovers a wide range of existing asymptotic results, but also yields novel one-shot achievability results in different multi-hop network information theory problems, such as compress-and-forward and partial-decode-and-forward bounds for a one-shot (primitive) relay channel, and a bound for one-shot cascade multiterminal source coding. In a broader context, our framework provides a unified one-shot bound applicable to any combination of source coding, channel coding and coding for computing problems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Y. Polyanskiy, H. V. Poor, and S. Verdú, “Channel coding rate in the finite blocklength regime,” IEEE Trans. Inf. Theory, vol. 56, no. 5, pp. 2307–2359, 2010.
  2. V. Kostina and S. Verdú, “Lossy joint source-channel coding in the finite blocklength regime,” IEEE Transactions on Information Theory, vol. 59, no. 5, pp. 2545–2575, 2013.
  3. D. Wang, A. Ingber, and Y. Kochman, “The dispersion of joint source-channel coding,” in 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton), Sep. 2011, pp. 180–187.
  4. V. Y. Tan and O. Kosut, “On the dispersions of three network information theory problems,” IEEE Transactions on Information Theory, vol. 60, no. 2, pp. 881–903, 2013.
  5. A. Feinstein, “A new basic theorem of information theory,” IRE Trans. Inf. Theory, no. 4, pp. 2–22, 1954.
  6. C. E. Shannon, “Certain results in coding theory for noisy channels,” Information and control, vol. 1, no. 1, pp. 6–25, 1957.
  7. M. Hayashi, “Information spectrum approach to second-order coding rate in channel coding,” IEEE Transactions on Information Theory, vol. 55, no. 11, pp. 4947–4966, 2009.
  8. S. Verdú, “Non-asymptotic achievability bounds in multiuser information theory,” in 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).   IEEE, 2012, pp. 1–8.
  9. M. H. Yassaee, M. R. Aref, and A. Gohari, “A technique for deriving one-shot achievability results in network information theory,” in 2013 IEEE International Symposium on Information Theory.   IEEE, 2013, pp. 1287–1291.
  10. J. Liu, P. Cuff, and S. Verdú, “Resolvability in Eγsubscript𝐸𝛾E_{\gamma}italic_E start_POSTSUBSCRIPT italic_γ end_POSTSUBSCRIPT with applications to lossy compression and wiretap channels,” in 2015 IEEE International Symposium on Information Theory (ISIT).   IEEE, 2015, pp. 755–759.
  11. ——, “One-shot mutual covering lemma and marton’s inner bound with a common message,” in 2015 IEEE International Symposium on Information Theory (ISIT).   IEEE, 2015, pp. 1457–1461.
  12. E. C. Song, P. Cuff, and H. V. Poor, “The likelihood encoder for lossy compression,” IEEE Transactions on Information Theory, vol. 62, no. 4, pp. 1836–1849, 2016.
  13. S. Watanabe, S. Kuzuoka, and V. Y. F. Tan, “Nonasymptotic and second-order achievability bounds for coding with side-information,” IEEE Trans. Inf. Theory, vol. 61, no. 4, pp. 1574–1605, April 2015.
  14. M. H. Yassaee, M. R. Aref, and A. Gohari, “Non-asymptotic output statistics of random binning and its applications,” in 2013 IEEE International Symposium on Information Theory.   IEEE, 2013, pp. 1849–1853.
  15. C. T. Li and V. Anantharam, “A unified framework for one-shot achievability via the poisson matching lemma,” IEEE Transactions on Information Theory, vol. 67, no. 5, pp. 2624–2651, 2021.
  16. S. Verdú and T. S. Han, “A general formula for channel capacity,” IEEE Trans. Inf. Theory, vol. 40, no. 4, pp. 1147–1157, 1994.
  17. C. T. Li and A. El Gamal, “Strong functional representation lemma and applications to coding theorems,” IEEE Transactions on Information Theory, vol. 64, no. 11, pp. 6967–6978, 2018.
  18. S.-H. Lee and S.-Y. Chung, “A unified random coding bound,” IEEE Transactions on Information Theory, vol. 64, no. 10, pp. 6779–6802, 2018.
  19. Y.-H. Kim, “Coding techniques for primitive relay channels,” in Proc. Forty-Fifth Annual Allerton Conf. Commun., Contr. Comput, 2007, p. 2007.
  20. M. Mondelli, S. H. Hassani, and R. Urbanke, “A new coding paradigm for the primitive relay channel,” Algorithms, vol. 12, no. 10, p. 218, 2019.
  21. A. El Gamal, A. Gohari, and C. Nair, “Achievable rates for the relay channel with orthogonal receiver components,” in 2021 IEEE Information Theory Workshop (ITW).   IEEE, 2021, pp. 1–6.
  22. ——, “A strengthened cutset upper bound on the capacity of the relay channel and applications,” IEEE Transactions on Information Theory, vol. 68, no. 8, pp. 5013–5043, 2022.
  23. S. I. Gel’fand and M. S. Pinsker, “Coding for channel with random parameters,” Probl. Contr. and Inf. Theory, vol. 9, no. 1, pp. 19–31, 1980.
  24. A. El Gamal and N. Hassanpour, “Relay-without-delay,” in Proceedings. International Symposium on Information Theory, 2005. ISIT 2005.   IEEE, 2005, pp. 1078–1080.
  25. A. El Gamal, N. Hassanpour, and J. Mammen, “Relay networks with delays,” IEEE Transactions on Information Theory, vol. 53, no. 10, pp. 3413–3431, 2007.
  26. A. Wyner and J. Ziv, “The rate-distortion function for source coding with side information at the decoder,” IEEE Transactions on information Theory, vol. 22, no. 1, pp. 1–10, 1976.
  27. A. D. Wyner, “The rate-distortion function for source coding with side information at the decoder-ii. general sources,” Information and control, vol. 38, no. 1, pp. 60–80, 1978.
  28. H. Yamamoto, “Wyner-ziv theory for a general function of the correlated sources (corresp.),” IEEE Transactions on Information Theory, vol. 28, no. 5, pp. 803–807, 1982.
  29. R. Ahlswede, “Multi-way communication channels,” in 2nd Int. Symp. Inform. Theory, Tsahkadsor, Armenian SSR, 1971, pp. 23–52.
  30. H. Liao, “Multiple access channels,” Ph.D. dissertation, University of Hawaii, Honolulu, HI, 1972.
  31. R. Ahlswede, “The capacity region of a channel with two senders and two receivers,” The annals of probability, vol. 2, no. 5, pp. 805–814, 1974.
  32. K. Marton, “A coding theorem for the discrete memoryless broadcast channel,” IEEE Transactions on Information Theory, vol. 25, no. 3, pp. 306–311, 1979.
  33. C. E. Shannon, “Two-way communication channels,” in Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, vol. 4.   University of California Press, 1961, pp. 611–645.
  34. S. H. Lim, Y.-H. Kim, A. El Gamal, and S.-Y. Chung, “Noisy network coding,” IEEE Transactions on Information Theory, vol. 57, no. 5, pp. 3132–3152, 2011.
  35. E. C. Van Der Meulen, “Three-terminal communication channels,” Advances in applied Probability, vol. 3, no. 1, pp. 120–154, 1971.
  36. T. Cover and A. E. Gamal, “Capacity theorems for the relay channel,” IEEE Transactions on information theory, vol. 25, no. 5, pp. 572–584, 1979.
  37. C. H. Bennett, P. W. Shor, J. Smolin, and A. V. Thapliyal, “Entanglement-assisted capacity of a quantum channel and the reverse Shannon theorem,” IEEE Trans. Inf. Theory, vol. 48, no. 10, pp. 2637–2655, 2002.
  38. P. Cuff, “Distributed channel synthesis,” IEEE Transactions on Information Theory, vol. 59, no. 11, pp. 7071–7096, 2013.
  39. P. Cuff, H. Permuter, and T. M. Cover, “Coordination capacity,” IEEE Trans. Inf. Theory, vol. 56, no. 9, pp. 4181–4206, Sept 2010.
  40. J. Scarlett, “On the dispersions of the gel’fand–pinsker channel and dirty paper coding,” IEEE Transactions on Information Theory, vol. 61, no. 9, pp. 4569–4586, 2015.
  41. C. T. Li, “An automated theorem proving framework for information-theoretic results,” IEEE Transactions on Information Theory, vol. 69, no. 11, pp. 6857–6877, 2023.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now