A Dual Approach for Hierarchical Information-Theoretic Tree Abstractions

Abstract: In this paper, we consider establishing a formal connection between two distinct tree-abstraction problems inspired by the information-bottleneck (IB) method. Specifically, we consider the hard- and soft-constrained formulations that have recently appeared in the literature to determine the conditions for which the two approaches are equivalent. Our analysis leverages concepts from Lagrangian relaxation and duality theory to relate the dual function of the hard-constrained problem to the Q-function employed in Q-tree search and shows the connection between tree phase transitions and solutions to the dual problem obtained by exploiting the problem structure. An algorithm is proposed that employs knowledge of the tree phase transitions to find a setting of the dual variable that solves the dual problem. Furthermore, we present an alternative approach to select the dual variable that leverages the integer programming formulation of the hard-constrained problem and the strong duality of linear programming. To obtain a linear program, we establish that a relaxation of the integer programming formulation of the hard-constrained tree-search problem has the integrality property by showing that the program constraint matrix is totally unimodular. Empirical results that corroborate the theoretical developments are presented and discussed throughout.