Generalizing Geometric Partition Entropy for the Estimation of Mutual Information in the Presence of Informative Outliers (2410.17367v2)

Abstract: The recent introduction of geometric partition entropy brought a new viewpoint to non-parametric entropy quantification that incorporated the impacts of informative outliers, but its original formulation was limited to the context of a one-dimensional state space. A generalized definition of geometric partition entropy is now provided for samples within a bounded (finite measure) region of a d-dimensional vector space. The basic definition invokes the concept of a Voronoi diagram, but the computational complexity and reliability of Voronoi diagrams in high dimension make estimation by direct theoretical computation unreasonable. This leads to the development of approximation schemes that enable estimation that is faster than current methods by orders of magnitude. The partition intersection ($\pi$) approximation, in particular, enables direct estimates of marginal entropy in any context resulting in an efficient and versatile mutual information estimator. This new measure-based paradigm for data driven information theory allows flexibility in the incorporation of geometry to vary the representation of outlier impact, which leads to a significant broadening in the applicability of established entropy-based concepts. The incorporation of informative outliers is illustrated through analysis of transient dynamics in the synchronization of coupled chaotic dynamical systems.