Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Feedback Capacity of the Continuous-Time ARMA(1,1) Gaussian Channel (2302.13073v2)

Published 25 Feb 2023 in cs.IT and math.IT

Abstract: We consider the continuous-time ARMA(1,1) Gaussian channel and derive its feedback capacity in closed form. More specifically, the channel is given by $\boldsymbol{y}(t) =\boldsymbol{x}(t) +\boldsymbol{z}(t)$, where the channel input ${\boldsymbol{x}(t) }$ satisfies average power constraint $P$ and the noise ${\boldsymbol{z}(t)}$ is a first-order {\em autoregressive moving average} (ARMA(1,1)) Gaussian process satisfying $$ \boldsymbol{z}\prime(t)+\kappa \boldsymbol{z}(t)=(\kappa+\lambda)\boldsymbol{w}(t)+\boldsymbol{w}\prime(t), $$ where $\kappa>0,~\lambda\in\mathbb{R}$ and ${\boldsymbol{w}(t) }$ is a white Gaussian process with unit double-sided spectral density. We show that the feedback capacity of this channel is equal to the unique positive root of the equation $$ P(x+\kappa)2 = 2x(x+\vert \kappa+\lambda\vert)2 $$ when $-2\kappa<\lambda<0$ and is equal to $P/2$ otherwise. Among many others, this result shows that, as opposed to a discrete-time additive Gaussian channel, feedback may not increase the capacity of a continuous-time additive Gaussian channel even if the noise process is colored. The formula enables us to conduct a thorough analysis of the effect of feedback on the capacity for such a channel. We characterize when the feedback capacity equals or doubles the non-feedback capacity; moreover, we disprove continuous-time analogues of the half-bit bound and Cover's $2P$ conjecture for discrete-time additive Gaussian channels.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (67)
  1. C. E. Shannon, “Communication in the presence of noise,” Proceedings of the IRE, vol. 37, no. 1, pp. 10–21, 1949.
  2. L. Koralov and Y. G. Sinai, Theory of Probability and Random Processes. Springer Science & Business Media, 2007.
  3. Academic Press, 2014.
  4. N. Obata, White Noise Calculus and Fock Space. Springer, 2006.
  5. World Scientific, 1993.
  6. T. Kadota, M. Zakai, and J. Ziv, “Mutual information of the white Gaussian channel with and without feedback,” IEEE Transactions on Information Theory, vol. 17, no. 4, pp. 368–371, 1971.
  7. G. Han and S. Shamai, “On sampling continuous-time AWGN channels,” IEEE Transactions on Information Theory, vol. 68, no. 2, pp. 782–794, 2021.
  8. X. Liu and G. Han, “On continuous-time Gaussian channels,” Entropy, vol. 21, no. 1, pp. 1–51, 2019.
  9. T. H. Ng and G. Han, “Relative entropy convergence under Picard’s iteration for stochastic differential equations,” ArXiv:1710.05277, 2017.
  10. Springer Science & Business Media, 1991.
  11. R. Huang and R. Johnson, “Information capacity of time-continuous channels,” IRE Transactions on Information Theory, vol. 8, no. 5, pp. 191–198, 1962.
  12. R. Huang and R. Johnson, “Information transmission with time-continuous random processes,” IEEE Transactions on Information Theory, vol. 9, no. 2, pp. 84–94, 1963.
  13. M. Hitsuda and S. Ihara, “Gaussian channels and the optimal coding,” Journal of Multivariate Analysis, vol. 5, no. 1, pp. 106–118, 1975.
  14. S. Ihara, “On the capacity of the continuous time Gaussian channel with feedback,” Journal of Multivariate Analysis, vol. 10, no. 3, pp. 319–331, 1980.
  15. S. Butman, “A general formulation of linear feedback communication systems with solutions,” IEEE Transactions on Information Theory, vol. 15, no. 3, pp. 392–400, 1969.
  16. M. S. Pinsker, “The probability of error in block transmission in a memoryless Gaussian channel with feedback,” Problemy Peredachi Informatsii, vol. 4, no. 4, pp. 3–19, 1968.
  17. S. Ihara, “Capacity of mismatched Gaussian channels with and without feedback,” Probability Theory and Related Fields, vol. 84, no. 4, pp. 453–471, 1990.
  18. I. M. Gelfand and A. M. Yaglom, Calculation of the amount of information about a random function contained in another such function. American Mathematical Society Providence, 1959.
  19. M. S. Pinsker, Information and Information Stability of Random Variables and Processes. Holden-Day, 1964.
  20. C. Baker, “Capacity of the mismatched Gaussian channel,” IEEE Transactions on Information Theory, vol. 33, no. 6, pp. 802–812, 1987.
  21. C. R. Baker, “Channel models and their capacity,” Sen, P.K. (ed.) Contributions to Statistics; Essays in Honor of Norman L. Johnson, pp. 1–16, 1983.
  22. C. R. Baker and S. Ihara, “Information capacity of the stationary Gaussian channel,” IEEE Transactions on Information Theory, vol. 37, no. 5, pp. 1314–1326, 1991.
  23. R. M. Fano, “Transmission of information: A statistical theory of communications,” American Journal of Physics, vol. 29, no. 11, pp. 793–794, 1961.
  24. Springer, 1968.
  25. C. E. Shannon, “A mathematical theory of communication,” The Bell System Technical Journal, vol. 27, no. 3, pp. 379–423, 1948.
  26. S. Ihara, “Optimal coding in white Gaussian channel with feedback,” Lecture Notes Math, vol. 330, pp. 120–123, 1973.
  27. R. S. Liptser, “Optimal encoding and decoding for transmission of a Gaussian markov signal in a noiseless-feedback channel,” Problemy Peredachi Informatsii, vol. 10, no. 4, pp. 3–15, 1974.
  28. S. Ihara, “Coding theory in white Gaussian channel with feedback,” Journal of Multivariate Analysis, vol. 4, no. 1, pp. 74–87, 1974.
  29. P. J. Brockwell and J. Hannig, “CARMA(p,q)𝑝𝑞(p,q)( italic_p , italic_q ) generalized random processes,” Journal of Statistical Planning and Inference, vol. 140, no. 12, pp. 3613–3618, 2010.
  30. R. A. Maller, G. Müller, and A. Szimayer, “Ornstein–Uhlenbeck processes and extensions,” Handbook of Financial Time Series, pp. 421–437, 2009.
  31. P. J. Brockwell, “Lévy–driven continuous–time ARMA processes,” in Handbook of Financial Time Series, pp. 457–480, Springer, 2009.
  32. P. Brockwell, “Recent results in the theory and applications of CARMA processes,” Annals of the Institute of Statistical Mathematics, vol. 66, pp. 647–685, 2014.
  33. P. J. Brockwell and R. A. Davis, Time Series: Theory and Methods. Springer Science & Business Media, 2009.
  34. Fowler, Statistical Mechanics. CUP Archive, 1967.
  35. E. Bibbona, G. Panfilo, and P. Tavella, “The Ornstein–Uhlenbeck process as a model of a low pass filtered white noise,” Metrologia, vol. 45, no. 6, p. S117, 2008.
  36. T. Liu and G. Han, “Feedback capacity of stationary Gaussian channels further examined,” IEEE Transactions on Information Theory, vol. 65, no. 4, pp. 2492–2506, 2018.
  37. T. Liu and G. Han, “The ARMA(k)𝑘(k)( italic_k ) Gaussian feedback capacity,” in 2017 IEEE International Symposium on Information Theory (ISIT), pp. 211–215, IEEE, 2017.
  38. G. Han and T. Liu, “ARMA(1) Gaussian feedback capacity revisited,” in 2016 International Symposium on Information Theory and Its Applications (ISITA), pp. 111–115, IEEE, 2016.
  39. Y.-H. Kim, “Feedback capacity of stationary Gaussian channels,” IEEE Transactions on Information Theory, vol. 56, no. 1, pp. 57–85, 2009.
  40. Y.-H. Kim, “Feedback capacity of the first-order moving average Gaussian channel,” IEEE Transactions on Information Theory, vol. 52, no. 7, pp. 3063–3079, 2006.
  41. S. Ihara, “On the feedback capacity of the first-order moving average Gaussian channel,” Japanese Journal of Statistics and Data Science, vol. 2, pp. 491–506, 2019.
  42. O. Sabag, V. Kostina, and B. Hassibi, “Feedback capacity of MIMO Gaussian channels,” in 2021 IEEE International Symposium on Information Theory (ISIT), pp. 7–12, IEEE, 2021.
  43. A. Gattami, “Feedback capacity of Gaussian channels revisited,” IEEE Transactions on Information Theory, vol. 65, no. 3, pp. 1948–1960, 2018.
  44. S. Fang and Q. Zhu, “Feedback capacity of parallel ACGN channels and Kalman filter: Power allocation with feedback,” ArXiv:2102.02730, 2021.
  45. M. Hitsuda, “Representation of Gaussian processes equivalent to Wiener process,” Osaka Journal of Mathematics, vol. 5, no. 2, pp. 299–312, 1968.
  46. M. Hitsuda, “Mutual information in Gaussian channels,” Journal of Multivariate Analysis, vol. 4, no. 1, pp. 66–73, 1974.
  47. J. Schalkwijk and T. Kailath, “A coding scheme for additive noise channels with feedback–I: No bandwidth constraint,” IEEE Transactions on Information Theory, vol. 12, no. 2, pp. 172–182, 1966.
  48. J. Schalkwijk, “Center-of-gravity information feedback,” IEEE Transactions on Information Theory, vol. 14, no. 2, pp. 324–331, 1968.
  49. J. Schalkwijk, “A coding scheme for additive noise channels with feedback–II: Band-limited signals,” IEEE Transactions on Information Theory, vol. 12, no. 2, pp. 183–189, 1966.
  50. F. Smithies, Integral Equations. CUP Archive, 1958.
  51. S. Ihara, “Coding theorems for the continuous time Gaussian channel with feedback II,” Probability Theory and Mathematical Statistics: Proceedings of the Sixth USSR-Japan Symposium, Kiev, USSR, pp. 132–142, 1991.
  52. T. E. Duncan, “On the calculation of mutual information,” SIAM Journal on Applied Mathematics, vol. 19, no. 1, pp. 215–220, 1970.
  53. P. Ebert, “The capacity of the Gaussian channel with feedback,” Bell System Technical Journal, vol. 49, no. 8, pp. 1705–1712, 1970.
  54. M. Markakis, “Closed-form solutions of certain Abel equations of the first kind,” Applied Mathematics Letters, vol. 22, no. 9, pp. 1401–1405, 2009.
  55. Springer Science & Business Media, 2013.
  56. T. M. Cover and S. Pombra, “Gaussian feedback capacity,” IEEE Transactions on Information Theory, vol. 35, no. 1, pp. 37–43, 1989.
  57. S. Yang, A. Kavcic, and S. Tatikonda, “On the feedback capacity of power-constrained Gaussian noise channels with memory,” IEEE Transactions on Information Theory, vol. 53, no. 3, pp. 929–954, 2007.
  58. M. S. Derpich and J. Østergaard, “Comments on “feedback capacity of stationary Gaussian channels”,” IEEE Transactions on Information Theory, vol. 70, no. 3, pp. 1848–1851, 2024.
  59. Springer, 1977.
  60. B. Oksendal, Stochastic Differential Equations: An Introduction with Applications. Springer Science & Business Media, 2013.
  61. M. Pinsker, “in Talk Delivered at the Soviet Information Theory Meeting,” No abstract published, 1969.
  62. H. W. Chen and K. Yanagi, “Refinements of the half-bit and factor-of-two bounds for capacity in Gaussian channel with feedback,” IEEE Transactions on Information Theory, vol. 45, no. 1, pp. 319–325, 1999.
  63. S. Ihara, “Mutual information and capacity of the continuous time Gaussian channel with feedback,” Gaussian Random Fields, World Scientific, pp. 242–256, 1991.
  64. T. M. Cover, “Conjecture: Feedback doesn’t help much,” Open Problems in Communication and Computation, pp. 70–71, 1987.
  65. Y.-H. Kim, “A counterexample to Cover’s 2⁢P2𝑃2P2 italic_P conjecture on Gaussian feedback capacity,” IEEE Transactions on Information Theory, vol. 52, no. 8, pp. 3792–3793, 2006.
  66. Walter de gruyter, 2011.
  67. G. Harris and C. Martin, “Shorter notes: The roots of a polynomial vary continuously as a function of the coefficients,” Proceedings of the American Mathematical Society, pp. 390–392, 1987.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets