Universal Slepian-Wolf coding for individual sequences
Abstract: We establish a coding theorem and a matching converse theorem for separate encodings and joint decoding of individual sequences using finite-state machines. The achievable rate region is characterized in terms of the Lempel-Ziv (LZ) complexities, the conditional LZ complexities and the joint LZ complexity of the two source sequences. An important feature that is needed to this end, which may be interesting on its own right, is a certain asymptotic form of a chain rule for LZ complexities, which we establish in this work. The main emphasis in the achievability scheme is on the universal decoder and its properties. We then show that the achievable rate region is universally attainable by a modified version of Draper's universal incremental Slepian-Wolf (SW) coding scheme, provided that there exists a low-rate reliable feedback link.
- S. C. Draper, “Universal incremental Slepian–Wolf coding,” Proc. Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL, October 2004.
- A. N. Kolmogorov, “Logical basis for information theory and probability theory,” IEEE Trans. Inform. Theory, vol. IT-14, no. 5, pp. 662-664, September 1968.
- N. Merhav, “Universal detection of messages via finite–state channels,” IEEE Trans. Inform. Theory, vol. 46, no. 6, pp. 2242–2246, September 2000.
- N. Merhav, “Universal decoding for source-channel coding with side information,” Communications in Information and Systems, vol. 16, no. 1, pp. 17-58, 2016.
- N. Merhav, “A universal ensemble for sample-wise lossy compression,” Entropy, 2023, 25(8), 1199; https://doi.org/10.3390/e25081199, August 2023.
- N. Shulman and M. Feder, “Source broadcasting with an unknown amount of receiver side information,” Proc. 2002 IEEE Information Theory Workshop, Bangalore, India, pp. 127–130, October 2002.
- D. Slepian and J. K. Wolf, “Noiseless coding of correlated information sources,” IEEE Trans. Inform. Theory, vol. IT-19, pp. 471–480, July 1973.
- T. Uyematsu and S. Kuzuoka, “Conditional Lempel-Ziv complexity and its application to source coding theorem with side information,” IEICE Trans. Fundamentals, Vol. E86-A, no. 10, pp. 2615–2617, October 2003.
- J. Ziv, “Fixed-rate encoding of individual sequences with side information”, IEEE Transactions on Information Theory, vol. IT–30, no. 2, pp. 348–452, March 1984.
- J. Ziv, “Universal decoding for finite-state channels,” IEEE Trans. Inform. Theory, vol. IT–31, no. 4, pp. 453–460, July 1985.
- J. Ziv and A. Lempel, “Compression of individual sequences via variable-rate coding,” IEEE Trans. Inform. Theory, vol. IT–24, no. 5, pp. 530–536, September 1978.
- A. K. Zvonkin and L. A. Levin, “The complexity of finite objects and the development of the concepts of information and randomness by means of the theory of algorithms,” Russian Mathematical Surveys, vol. 25, no. 6, pp. 83–124, 1970.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
