Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Derivation of Mutual Information and Linear Minimum Mean-Square Error for Viterbi Decoding of Convolutional Codes Using the Innovations Method (2403.05740v1)

Published 9 Mar 2024 in cs.IT and math.IT

Abstract: We see that convolutional coding/Viterbi decoding has the structure of the Kalman filter (or the linear minimum variance filter). First, we calculate the covariance matrix of the innovation (i.e., the soft-decision input to the main decoder in a Scarce-State-Transition (SST) Viterbi decoder). Then a covariance matrix corresponding to that of the one-step prediction error in the Kalman filter is obtained. Furthermore, from that matrix, a covariance matrix corresponding to that of the filtering error in the Kalman filter is derived using the formula in the Kalman filter. As a result, the average mutual information per branch for Viterbi decoding of convolutional codes is given using these covariance matrices. Also, the trace of the latter matrix represents the linear minimum mean-square error (LMMSE). We show that an approximate value of the average mutual information is sandwiched between half the SNR times the average filtering and one-step prediction LMMSEs. In the case of QLI codes, from the covariance matrix of the soft-decision input to the main decoder, we can get a matrix. We show that the trace of this matrix has some connection with the linear smoothing error.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (21)
  1. T. E. Duncan, “On the calculation of mutual information,” SIAM J. Appl. Math., vol. 19, pp. 215–220, Jul. 1970.
  2. R. Esposito, “On a relation between detection and estimation in decision theory,” Inform. Contr., vol. 12, pp. 116-120, 1968.
  3. G. D. Forney, Jr., “Convolutional codes I: Algebraic structure,” IEEE Trans. Inf. Theory, vol. IT-16, no. 6, pp. 720–738, Nov. 1970.
  4. P. A. Frost and T. Kailath, “An innovations approach to least-squares estimation–Part III: Nonlinear estimation in white Gaussian noise,” IEEE Trans. Automatic Control, vol. AC-16, no. 3, pp. 217–226, June 1971.
  5. D. Guo, S. Shamai (Shitz), and S. Verdú, “Mutual information and minimum mean-square error in Gaussian channels,” IEEE Trans. Inf. Theory, vol. 51, no. 4, pp. 1261–1282, Apr. 2005.
  6. J. A. Heller and I. M. Jacobs, “Viterbi decoding for satellite and space communication,” IEEE Trans. Commun. Technol., vol. COM-19, no. 5, pp. 835–848, Oct. 1971.
  7. T. T. Kadota, M. Zakai, and J. Ziv, “Mutual information of the white Gaussian channel with and without feedback,” IEEE Trans. Inf. Theory, vol. IT-17, no. 4, pp. 368–371, Jul. 1971.
  8. T. Kailath, “An innovations approach to least-squares estimation–Part I: Linear filtering in additive white noise,” IEEE Trans. Automatic Control, vol. AC-13, no. 6, pp. 646–655, Dec. 1968.
  9. T. Kailath and P. Frost, “An innovations approach to least-squares estimation–Part II: Linear smoothing in additive white noise,” IEEE Trans. Automatic Control, vol. AC-13, no. 6, pp. 655–660, Dec. 1968.
  10. T. Kailath, “A note on least-squares estimates from likelihood ratios,” Inform. Contr., vol. 13, pp. 534–540, 1968.
  11. T. Kailath, “A general likelihood-ratio formula for random signals in Gaussian noise,” IEEE Trans. Inf. Theory, vol. IT-15, no. 3, pp. 350–361, May 1969.
  12. T. Kailath, “The innovations approach to detection and estimation theory,” Proc. IEEE, vol. 58, no. 5, pp. 680–695, May 1970.
  13. T. Kailath, “A view of three decades of linear filtering theory (Invited paper),” IEEE Trans. Inf. Theory, vol. IT-20, no. 2, pp. 146–181, Mar. 1974.
  14. T. Kailath and H. V. Poor, “Detection of stochastic processes (Invited paper),” IEEE Trans. Inf. Theory, vol. 44, no. 6, pp. 2230–2259, Oct. 1998.
  15. S. Kubota, S. Kato, and T. Ishitani, “Novel Viterbi decoder VLSI implementation and its performance,” IEEE Trans. Commun., vol. 41, no. 8, pp. 1170–1178, Aug. 1993.
  16. J. L. Massey and D. J. Costello, Jr., “Nonsystematic convolutional codes for sequential decoding in space applications,” IEEE Trans. Commun. Technol., vol. COM-19, no. 5, pp. 806–813, Oct. 1971.
  17. S. Ping, Y. Yan, and C. Feng, “An effective simplifying scheme for Viterbi decoder,” IEEE Trans. Commun., vol. 39, no. 1, pp. 1–3, Jan. 1991.
  18. M. Tajima, K. Shibata, and Z. Kawasaki, “On the equivalence between Scarce-State-Transition Viterbi decoding and syndrome decoding of convolutional codes,” IEICE Trans. Fundamentals, vol. E86-A, no. 8, pp. 2107–2116, Aug. 2003.
  19. M. Tajima, “An innovations approach to Viterbi decoding of convolutional codes,” IEEE Trans. Inf. Theory, vol. 65, no. 5, pp. 2704–2722, May 2019.
  20. M. Tajima, “Corrections to “An innovations approach to Viterbi decoding of convolutional codes”,” IEEE Trans. Inf. Theory, vol. 67, no. 5, pp. 3183–3185, May 2021.
  21. M. Zakai, “On mutual information, likelihood ratios, and estimation error for the additive Gaussian channel,” IEEE Trans. Inf. Theory, vol. 51, no. 9, pp. 3017–3024, Sep. 2005.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now