A New Approach to Compute Information Theoretic Outer Bounds and Its Application to Regenerating Codes (2205.01612v1)

Abstract: The study of the fundamental limits of information systems is a central theme in information theory. Both the traditional analytical approach and the recently proposed computational approach have significant limitations, where the former is mainly due to its reliance on human ingenuity, and the latter due to its exponential memory and computational complexity. In this work, we propose a new computational approach to tackle the problem with much lower memory and computational requirements, which can naturally utilize certain intuitions, but also can maintain the strong computational advantage of the existing computational approach. A reformulation of the underlying optimization problem is first proposed, which converts the large linear program to a maximin problem. This leads to an iterative solving procedure, which uses the LP dual to carry over learned evidence between iterations. The key in the reformulated problem is the selection of good information inequalities, with which a relaxed LP can be formed. A particularly powerful intuition is a potentially optimal code construction, and we provide a method that directly utilizes it in the new algorithm. As an application, we derive a tighter outer bound for the storage-repair tradeoff for the $(6,5,5)$ regenerating code problem, which involves at least 30 random variables and is impossible to compute with the previously known computational approach.