Geometric PID: Bivariate Info Decomposition

• Geometric PID is an information-theoretic framework that decomposes shared, unique, and synergistic contributions of two sources to a target using KL divergence and convex geometry.
• It employs projections onto convex hulls in probability simplices, offering a clear geometric interpretation with a rigorous axiomatic foundation.
• While ensuring nonnegativity and interpretability, its restriction to bivariate systems and computational overhead highlight challenges for higher-dimensional generalizations.

Geometric Partial Information Decomposition (PID) is an information-theoretic framework designed to disentangle the contributions of multiple information sources to a target variable in terms of redundancy, unique information, and synergy. The Geometric PID formalism offers an operational and mathematically principled construction of redundancy for bivariate systems, rooted in the geometry of probability distributions and Kullback–Leibler (KL) projections. It is notable for its rigorous axiomatic foundation, clear geometric interpretation, and explicit computability, though it is inherently restricted to systems with exactly two sources (Liardi et al., 3 Mar 2026).

1. Formal Definition of Geometric PID

Consider two discrete source variables $X_1$, $X_2$ and a target $Y$ with joint distribution $P(x_1, x_2, y)$. For each $x_1$ in the support of $X_1$, the conditional distribution $p_{y|x_1} := P(Y=y|X_1=x_1)$ is viewed as a point in the probability simplex $\Delta_Y$. Similarly, for $x_2$ in the support of $X_2$, $X_2$0 is defined.

Define for $X_2$1 the convex hull $X_2$2 and analogously $X_2$3 for $X_2$4. The information projection (I-projection) of $X_2$5 onto $X_2$6 is

$X_2$7

yielding the projected conditional $X_2$8. The directed projected information from $X_2$9 into $Y$0 is then

$Y$1

Redundant information is given by

$Y$2

The atoms of the PID lattice are then given by Möbius inversion:

• Redundancy: $Y$3
• Unique information of $Y$4: $Y$5
• Unique information of $Y$6: $Y$7
• Synergy: $Y$8

2. Geometric Interpretation and Information Projection

Each conditional $Y$9 ($P(x_1, x_2, y)$0) can be interpreted as a point on the $P(x_1, x_2, y)$1-simplex. The set of conditionals $P(x_1, x_2, y)$2 spans a convex polytope $P(x_1, x_2, y)$3. The projection $P(x_1, x_2, y)$4 finds the point in $P(x_1, x_2, y)$5 that is closest (in the KL sense) to $P(x_1, x_2, y)$6. Intuitively, this projects the information that $P(x_1, x_2, y)$7 has about $P(x_1, x_2, y)$8 onto the “statistical structure” available from $P(x_1, x_2, y)$9. This geometry underpins the “shared” content: only information already expressible by $x_1$0 conditionals is counted as redundant.

Symmetry is enforced by minimizing the directed projections in both possible directions.

3. Computational Workflow

Computation of the Geometric PID proceeds as follows (Liardi et al., 3 Mar 2026):

1. Marginal and Conditional Computation: Compute $x_1$1, $x_1$2, $x_1$3, then the conditionals $x_1$4 and $x_1$5.
2. Convex Hull Construction: Form $x_1$6 as the convex hull of $x_1$7. For each $x_1$8, solve the convex projection (e.g., using Blahut–Arimoto or gradient methods) to find $x_1$9.
3. Projected Information Calculation: Compute $X_1$0 using the projected conditionals.
4. Symmetry Step: Repeat for $X_1$1.
5. Redundancy and Atom Derivation: Assign $X_1$2 and derive $X_1$3 as above.

The following table summarizes the definitions of the bivariate PID atoms:

Atom	Formula	Description
Redundancy	$X_1$4	Information shared by $X_1$5, $X_1$6 about $X_1$7
Unique $X_1$8	$X_1$9	Unique information of $p_{y|x_1} := P(Y=y|X_1=x_1)$0
Unique $p_{y|x_1} := P(Y=y|X_1=x_1)$1	$p_{y|x_1} := P(Y=y|X_1=x_1)$2	Unique information of $p_{y|x_1} := P(Y=y|X_1=x_1)$3
Synergy	$p_{y|x_1} := P(Y=y|X_1=x_1)$4	Information only available jointly

4. Axiomatic Properties and Limiting Results

The Geometric redundancy $p_{y|x_1} := P(Y=y|X_1=x_1)$5 satisfies the following axioms and properties:

• Self-redundancy (SR): $p_{y|x_1} := P(Y=y|X_1=x_1)$6
• (Weak) Symmetry (S₀): Invariant under swapping $p_{y|x_1} := P(Y=y|X_1=x_1)$7
• (Weak) Monotonicity (M₀): Redundancy does not increase when adding a source, $p_{y|x_1} := P(Y=y|X_1=x_1)$8
• Subset-Equality (SE): If $p_{y|x_1} := P(Y=y|X_1=x_1)$9 then $\Delta_Y$0
• Nonnegativity (GP): $\Delta_Y$1
• Local Positivity (LP): All PID atoms are nonnegative
• Identity (ID): For $\Delta_Y$2, $\Delta_Y$3
• Independent-Identity (IID): If $\Delta_Y$4, $\Delta_Y$5
• Lower Bound (LB): Redundancy lower-bounded by less-informative surrogates
• Equivalence-Invariance (EI): Invariant to relabeling of variable values

Crucially, several no-go results establish that Geometric PID cannot be consistently extended to more than two sources while retaining all the aforementioned properties plus chain-rule (TC) or target monotonicity (TM). Indeed, Geometric PID fails TM/TC: adding more of $\Delta_Y$6 can decrease redundancy.

5. Illustrative Example: XOR Gate

For $\Delta_Y$7 and $\Delta_Y$8 independent fair bits, $\Delta_Y$9:

• $x_2$0 for $x_2$1; thus $x_2$2 for all $x_2$3 is the simplex center.
• The projections $x_2$4, so $x_2$5.
• $x_2$6, $x_2$7, $x_2$8 bit, so $x_2$9 bit: all information is synergistic, no redundancy. This aligns with the expected behavior for the XOR structure (Liardi et al., 3 Mar 2026).

6. Advantages, Limitations, and Applications

Advantages

• Identity Validity: Satisfies the ID axiom; independent copies do not yield spurious redundancy.
• Nonnegativity and Interpretability: All PID atoms are nonnegative and possess a geometric interpretation as KL projections.
• Label Invariance: Equivalence-invariant under invertible relabeling of variable values.

Limitations

• Bivariate Only: Formalism is restricted to two sources; no generalization exists to higher dimensions that preserves all core properties and ID.
• Violation of Target-Monotonicity: Adding more of $X_2$0 can decrease redundancy (TM fails).
• Computational Overhead: For large support on $X_2$1, repeated convex optimizations may become computationally expensive.

Use Cases

• Bivariate PID: Settings where two source variables are analyzed for contributions to a target.
• Contexts Demanding Identity and Nonnegativity: Experimental systems needing strict adherence to these axioms.
• Low-dimensional Targets: These facilitate practical geometric projection computation.

7. Relation to Alternative Geometric PID Approaches

A related but distinct geometric PID approach leverages information geometry over partially ordered sets (posets) of variable subsets (Sugiyama et al., 2016). This framework generalizes Amari's hierarchy to enable decomposition on structured spaces, constructing a dually-flat manifold (with $X_2$2- and $X_2$3-coordinates) for arbitrary posets and deriving PID atoms through Möbius inversion on KL divergence projections. While more general and multivariate, the practical and conceptual constraints differ from the bivariate-focused Geometric PID defined by Harder et al. Thus, users should be cautious to distinguish these two flavors of "geometric" PID, as only the latter corresponds precisely to the KL-projection and simplex geometry described in (Liardi et al., 3 Mar 2026).