Cosine Annealing Optimized Denoising Diffusion Error Correction Codes (2405.03638v1)
Abstract: To address the issue of increased bit error rates during the later stages of linear search in denoising diffusion error correction codes, we propose a novel method that optimizes denoising diffusion error correction codes (ECC) using cosine annealing. In response to the challenge of decoding long codewords, the proposed method employs a variance adjustment strategy during the reverse diffusion process, rather than maintaining a constant variance. By leveraging cosine annealing, this method effectively lowers the bit error rate and enhances decoding effciency. This letter extensively validates the approach through experiments and demonstrates signifcant improvements in bit error rate reduction and iteration effciency compared to existing methods. This advancement offers a promising solution for improving ECC decoding performance, potentially impacting secure digital communication practices.
- M. Ibnkahla, “Applications of neural networks to digital communications–a survey,” Signal processing, vol. 80, no. 7, pp. 1185–1215, 2000.
- T. Gruber, S. Cammerer, J. Hoydis, and S. Ten Brink, “On deep learning-based channel decoding,” in 2017 51st annual conference on information sciences and systems (CISS). IEEE, 2017, pp. 1–6.
- S. Cammerer, T. Gruber, J. Hoydis, and S. Ten Brink, “Scaling deep learning-based decoding of polar codes via partitioning,” in GLOBECOM 2017-2017 IEEE global communications conference. IEEE, 2017, pp. 1–6.
- H. Kim, Y. Jiang, R. Rana, S. Kannan, S. Oh, and P. Viswanath, “Communication algorithms via deep learning,” arXiv preprint arXiv:1805.09317, 2018.
- A. Bennatan, Y. Choukroun, and P. Kisilev, “Deep learning for decoding of linear codes-a syndrome-based approach,” in 2018 IEEE International Symposium on Information Theory (ISIT). IEEE, 2018, pp. 1595–1599.
- J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in International conference on machine learning. PMLR, 2015, pp. 2256–2265.
- J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840–6851, 2020.
- Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” arXiv preprint arXiv:2011.13456, 2020.
- J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” arXiv preprint arXiv:2010.02502, 2020.
- A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion probabilistic models,” in International conference on machine learning. PMLR, 2021, pp. 8162–8171.
- F. Bao, C. Li, J. Sun, J. Zhu, and B. Zhang, “Estimating the optimal covariance with imperfect mean in diffusion probabilistic models,” arXiv preprint arXiv:2206.07309, 2022.
- Y. Choukroun and L. Wolf, “Error correction code transformer,” Advances in Neural Information Processing Systems, vol. 35, pp. 38 695–38 705, 2022.
- Choukroun, Yoni and Wolf, Lior, “Denoising diffusion error correction codes,” arXiv preprint arXiv:2209.13533, 2022.
- Y. Choukroun and L. Wolf, “Deep quantum error correction,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 1, 2024, pp. 64–72.
- D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
- R. Xiong, Y. Yang, D. He, K. Zheng, S. Zheng, C. Xing, H. Zhang, Y. Lan, L. Wang, and T. Liu, “On layer normalization in the transformer architecture,” in International Conference on Machine Learning. PMLR, 2020, pp. 10 524–10 533.