Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

The Duality Upper Bound for Finite-State Channels with Feedback (2401.05102v2)

Published 10 Jan 2024 in cs.IT and math.IT

Abstract: This paper investigates the capacity of finite-state channels (FSCs) with feedback. We derive an upper bound on the feedback capacity of FSCs by extending the duality upper bound method from mutual information to the case of directed information. The upper bound is expressed as a multi-letter expression that depends on a test distribution on the sequence of channel outputs. For any FSC, we show that if the test distribution is structured on a $Q$-graph, the upper bound can be formulated as a Markov decision process (MDP) whose state being a belief on the channel state. In the case of FSCs and states that are either unifilar or have a finite memory, the MDP state simplifies to take values in a finite set. Consequently, the MDP consists of a finite number of states, actions, and disturbances. This finite nature of the MDP is of significant importance, as it ensures that dynamic programming algorithms can solve the associated BeLLMan equation to establish analytical upper bounds, even for channels with large alphabets. We demonstrate the simplicity of computing bounds by establishing the capacity of a broad family of Noisy Output is the State (NOST) channels as a simple closed-form analytical expression. Furthermore, we introduce novel, nearly optimal analytical upper bounds on the capacity of the Noisy Ising channel.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. O. Sabag and H. H. Permuter, “The duality upper bound for unifilar finite-state channels with feedback,” in International Zurich Seminar on Information and Communication (IZS), Feb. 2020, pp. 68–72.
  2. B. McMillan, “The basic theorems of information theory,” Ann. Math. Statist, vol. 24, no. 2, pp. 196–219, Jun. 1953.
  3. C. E. Shannon, “Certain results in coding theory for noisy channels,” Inf. Control, vol. 1, pp. 6–25, Sep. 1957.
  4. D. Blackwell, L. Breiman, and A. Thomasian, “Proof of Shannon’s transmission theorem for finite-state indecomposable channels,” Ann. Math. Stat, vol. 29, pp. 1209–1220, 1958.
  5. P. Sadegh, R. A. Kennedy, P. B. Rapajic, and R. Shams, “Finite-state Markov modeling of fading channels - a survey of principles and applications,” IEEE Signal Process. Mag., vol. 25, no. 5, pp. 57–80, Sep. 2008.
  6. Q. Zhang and S. A. Kassam, “Finite-state Markov model for Rayleigh fading channels,” IEEE Trans. Commun., vol. 47, no. 11, pp. 1688–1692, Nov. 1999.
  7. H. S. Wang and N. Moayeri, “Finite-state markov channel-a useful model for radio communication channels,” IEEE Trans. Inf. Theory, vol. 44, pp. 163–171, 1995.
  8. K. A. S. Immink, P. H. Siegel, and J. K. Wolf, “Codes for digital recorders,” IEEE Trans. Inf. Theory, vol. 44, no. 6, pp. 2260–2299, Oct. 1998.
  9. L. Galluccio, A. Lombardo, G. Morabito, S. Palazzo, C. Panarello, and G. Schembra, “Capacity of a binary droplet-based microfluidic channel with memory and anticipation for flow-induced molecular communications,” IEEE Transactions on Communications, vol. 66, no. 1, pp. 194–208, Jan 2018.
  10. N. Farsad, H. B. Yilmaz, A. Eckford, C. Chae, and W. Guo, “A comprehensive survey of recent advancements in molecular communication,” IEEE Communications Surveys Tutorials, vol. 18, no. 3, pp. 1887–1919, thirdquarter 2016.
  11. H. H. Permuter, P. W. Cuff, B. Van-Roy, and T. Weissman, “Capacity of the trapdoor channel with feedback,” IEEE Trans. Inf. Theory, vol. 54, pp. 3150–3165, 2008.
  12. S. Yang, A. Kavc̆ić, and S. Tatikonda, “Feedback capacity of finite-state machine channels,” IEEE Trans. Inf. Theory, vol. 51, no. 3, pp. 799–810, Mar. 2005.
  13. S. Tatikonda and S. Mitter, “The capacity of channels with feedback,” IEEE Trans. Inf. Theory, vol. 55, no. 1, pp. 323–349, Jan. 2009.
  14. J. Chen and T. Berger, “The capacity of finite-state Markov channels with feedback,” IEEE Trans. Inf. Theory, vol. 51, pp. 780–789, Mar. 2005.
  15. H. H. Permuter, P. Cuff, B. V. Roy, and T. Weissman, “Capacity of the Trapdoor channel with feedback,” IEEE Trans. Inf. Theory, vol. 54, no. 7, pp. 3150–3165, Jul. 2008.
  16. O. Elishco and H. Permuter, “Capacity and coding for the Ising channel with feedback,” IEEE Trans. Inf. Theory, vol. 60, no. 9, pp. 5138–5149, Sep. 2014.
  17. O. Sabag, H. Permuter, and N. Kashyap, “The feedback capacity of the binary erasure channel with a no-consecutive-ones input constraint,” IEEE Trans. Inf. Theory, vol. 62, no. 1, pp. 8–22, Jan. 2016.
  18. J. Wu and A. Anastasopoulos, “On the capacity of the chemical channel with feedback,” in Proc. of IEEE Int. Symp. on Inf. Theory (ISIT), Jul. 2016, pp. 295–299.
  19. A. Sharov and R. M. Roth, “On the capacity of generalized Ising channels,” IEEE Trans. Inf. Theory, vol. 63, no. 4, pp. 2338–2356, Apr. 2017.
  20. O. Sabag, H. H. Permuter, and N. Kashyap, “Feedback capacity and coding for the BIBO channel with a no-repeated-ones input constraint,” IEEE Trans. on Inf. Theory, vol. 64, no. 7, pp. 4940–4961, July 2018.
  21. O. Sabag, H. H. Permuter, and H. D. Pfister, “A single-letter upper bound on the feedback capacity of unifilar finite-state channels,” IEEE Trans. Inf. Theory, vol. 63, no. 3, pp. 1392–1409, Mar. 2017.
  22. O. Sabag, B. Huleihel, and H. H. Permuter, “Graph-based encoders and their performance for finite-state channels with feedback,” IEEE Trans. Commun., vol. 68, no. 4, pp. 2106–2117, Apr. 2020.
  23. Z. Aharoni, O. Sabag, and H. H. Permuter, “Feedback capacity of ising channels with large alphabet via reinforcement learning,” IEEE Trans. Inf. Theory, vol. 68, no. 9, pp. 5637–5656, 2022.
  24. A. Mohanan and A. Thangaraj, “Dual capacity upper bounds for binary-input single-tap ISI channels,” IEEE Trans. Commun., vol. 67, no. 10, pp. 6725–6734, Oct. 2019.
  25. A. Lapidoth and S. M. Moser, “Capacity bounds via duality with applications to multiple-antenna systems on flat-fading channels,” IEEE Trans. Inf. Theory, vol. 49, no. 10, pp. 2426–2467, Oct. 2003.
  26. A. Thangaraj, “Dual capacity upper bounds for noisy runlength constrained channels,” IEEE Trans. Inf. Theory, vol. 63, no. 11, pp. 7052–7065, Nov. 2017.
  27. A. Thangaraj, G. Kramer, and G. Böcherer, “Capacity bounds for discrete-time, amplitude-constrained, additive white Gaussian noise channels,” IEEE Trans. Inf. Theory, vol. 63, no. 7, pp. 4172–4182, Jul. 2017.
  28. P. O. Vontobel and D. M. Arnold, “An upper bound on the capacity of channels with memory and constraint input,” in Proceedings IEEE Inf. Theory Workshop, Sep. 2001, pp. 147–149.
  29. B. Huleihel, O. Sabag, H. H. Permuter, N. Kashyap, and S. Shamai, “Computable upper bounds on the capacity of finite-state channels,” IEEE Transactions on Information Theory, pp. 1–1, 2021.
  30. E. Shemuel, O. Sabag, and H. H. Permuter, “The feedback capacity of noisy output is the state (NOST) channels,” IEEE Trans. Inf. Theory, vol. 68, no. 8, pp. 5044–5059, Apr. 2022.
  31. T. Berger and F. Bonomi, “Capacity and zero-error capacity of Ising channels,” IEEE Trans. Inf. Theory, vol. 36, no. 1, pp. 173–180, Jan. 1990.
  32. G. Kramer, “Directed information for channels with feedback,” Ph.D. Dissertation, Swiss Federal Institute of Technology (ETH) Zurich, 1998.
  33. Y.-H. Kim, “A coding theorem for a class of stationary channels with feedback,” IEEE Trans. Inf. Theory., vol. 54, no. 4, pp. 1488–1499, Apr. 2008.
  34. F. Topsøe, “An information theoretical identity and a problem involving capacity,” Studia Scientiarum Mathematicarum Hungarica, vol. 2, pp. 291–292, 1967.
  35. I. Csiszár, “Informationstheoretische Konvergenzbegriffe im Raum der Wahrscheinlichkeitsverteilungen,” Publ. Math. Inst. Hungar. Acad., vol. 7, pp. 137–158, 1962.
  36. A. Arapostathis, V. S. Borkar, E. Fernández-Gaucherand, M. K. Ghosh, and S. I. Marcus, “Discrete-time controlled Markov processes with average cost criterion: a survey,” Siam Journal on Control and Optimization, vol. 31, pp. 282–344, 1993.
  37. A. Arapostathis, V. S. Borkar, E. Fernandez-Gaucherand, M. K. Ghosh, and S. Marcus, “Discrete time controlled Markov processes with average cost criterion - a survey,” SIAM Journal of Control and Optimization, vol. 31, no. 2, pp. 282–344, 1993.
  38. H. Permuter, H. Asnani, and T. Weissman, “Capacity of a POST channel with and without feedback,” IEEE Trans. Inf. Theory, vol. 60, no. 10, pp. 6041–6057, Oct. 2014.
  39. M. Fekete, “Über die verteilung der wurzeln bei gewissen algebraischen gleichungen mit ganzzahligen koeffizienten,” Mathematische Zeitschrift, vol. 17, no. 1, pp. 228–249, Dec. 1923.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now