pyBregMan: A Python library for Bregman Manifolds (2408.04175v1)

Abstract: A Bregman manifold is a synonym for a dually flat space in information geometry which admits as a canonical divergence a Bregman divergence. Bregman manifolds are induced by smooth strictly convex functions like the cumulant or partition functions of regular exponential families, the negative entropy of mixture families, or the characteristic functions of regular cones just to list a few such convex Bregman generators. We describe the design of pyBregMan, a library which implements generic operations on Bregman manifolds and instantiate several common Bregman manifolds used in information sciences. At the core of the library is the notion of Legendre-Fenchel duality inducing a canonical pair of dual potential functions and dual Bregman divergences. The library also implements the Fisher-Rao manifolds of categorical/multinomial distributions and multivariate normal distributions. To demonstrate the use of the pyBregMan kernel manipulating those Bregman and Fisher-Rao manifolds, the library also provides several core algorithms for various applications in statistics, machine learning, information fusion, and so on.

• The paper presents pyBregMan, a Python library that implements operations on Bregman manifolds using Legendre-Fenchel duality.
• It provides tools for constructing and visualizing both Bregman and Fisher-Rao manifolds, facilitating key statistical and machine learning applications.
• The library supports practical use cases including Gaussian, categorical, and matrix analyses with extensible APIs for seamless Python integration.

A Python Library for Bregman Manifolds

This paper introduces pyBregMan, a Python library designed to support operations and manipulations on Bregman manifolds with applications in statistics, machine learning, and information sciences. Bregman manifolds, also termed dually flat spaces, arise from a smooth, strictly convex generator function and play a crucial role in information geometry.

Core Concepts

The library centers around key operations within Bregman manifolds, leveraging the Legendre-Fenchel duality to manage dual potential functions and dual Bregman divergences. Additionally, it includes implementations for Fisher-Rao manifolds, fundamental in categorical and multivariate normal distributions. These structures enable a variety of core algorithms within pyBregMan, facilitating diverse applications from clustering to information fusion.

Design and Implementation

• Manifold Construction: Bregman and Fisher-Rao manifolds can be constructed for bivariate normal distributions. This involves defining manifold structures capable of calculating Kullback-Leibler divergences and various centroids, as evidenced by visual centroid examples.
• Library Installation: The library is available through pip or can be cloned directly from the GitHub repository, ensuring accessible integration with Python development environments.
• Core Algorithms and Visualizations: By defining Bregman manifolds and associating geometric objects, the library enables straightforward computations and visualizations, critical for understanding manifold properties and relationships.

Detailed Review of Bregman and Fisher-Rao Manifolds

Bregman Manifolds

A Bregman manifold derives its structure from a convex generator function $F$. The divergence induced by this function, Bregman divergence, measures dissimilarity between points. When the generator is a Legendre-type function, it supports a dually flat space structure characterized by flat affine connections and a coupled metric tensor. This duality facilitates representation in both $\theta$-coordinates and dual $\eta$-coordinates.

Fisher-Rao Manifolds

Fisher-Rao manifolds extend the concept by incorporating the Fisher information metric, producing a Riemannian manifold with geodesic distance known as Rao's distance. In cases where the Fisher information metric is Hessian, the manifold can be treated as a Bregman manifold. The practical use of Fisher-Rao manifolds often focuses on Gaussian distributions, benefiting from known geodesics and efficient distance approximation algorithms.

Divergence-Based Information Geometry

Eguchi's divergence-based manifold framework leverages smooth divergences to induce dualistic structures, exemplified by Bregman and Burbea-Rao divergences. The Zhang $\alpha$-divergence extends this framework, parameterizing divergences such that both primal and dual structures are directly modeled.

Library Goals and Structure

pyBregMan aims to provide:

1. A Geometric Kernel: For the implementation, visualization, and export of foundational concepts in Bregman manifold geometry.
2. APIs for Fisher-Rao Manifolds: Supporting manipulation of statistical models, particularly those where the Fisher information metric is of the Hessian type.
3. Demonstration of Practical Use Cases: Addressing real-world applications, such as computing Chernoff information, matrix geometric means, and discrete Jensen-Shannon centroids.

The library's structure is divided into several submodules, each offering primitives and application-specific classes:

• Primitives: Foundational classes in bregman.base and bregman.manifold enable definition and manipulation of data points on manifolds.
• Geometric Objects: Including geodesics, bisectors, and balls, allowing sophisticated geometric calculations.
• Application Manifolds: Concrete implementations for various distributions and matrix manifolds, such as multivariate normal and categorical distributions.
• Visualization: MatplotlibVisualizer for creating visual representations of manifold objects, integrating smoothly with existing Python plotting libraries.

Examples of Applications

Positive Semi-Definite Matrices

An iterative algorithm to compute the geometric mean of symmetric positive-definite matrices is visualized, illustrating the convergence of arithmetic and harmonic means to the Riemannian geodesic mid-point.

Categorical Distributions

Histograms of pixel intensity distributions from images are used to calculate different centroids (Jensen-Shannon and Jeffreys), demonstrating the library's capabilities in practical data analysis tasks.

Gaussian Distributions

The calculation of the Chernoff point in a univariate Gaussian manifold is shown, highlighting applications in hypothesis testing and information fusion.

Extending pyBregMan

The paper concludes by showcasing how users can extend the library with new application manifolds and geometric objects. This includes defining new Bregman generators using automatic differentiation and implementing customized manifold-specific objects.

Implications and Future Developments

pyBregMan significantly eases the implementation of complex geometric and statistical operations on Bregman manifolds, with broad implications for machine learning and statistical analysis. Future developments could include expanding manifold types, optimizing numerical precision, and integrating with broader machine learning frameworks for deeper analytics and visualization capabilities.