Directed Information on Abstract spaces: Properties and Extremum Problems

Abstract: This paper describes a framework in which directed information is defined on abstract spaces. The framework is employed to derive properties of directed information such as convexity, concavity, lower semicontinuity, by using the topology of weak convergence of probability measures on Polish spaces. Two extremum problems of directed information related to capacity of channels with memory and feedback, and non-anticipative and sequential rate distortion are analyzed showing existence of maximizing and minimizing distributions, respectively.