---
title: Unique Information Decomposition
url: https://www.emergentmind.com/topics/unique-information-decomposition
type: topic
---

# Unique Information Decomposition

Unique information decomposition is a framework for dissecting the mutual information between a set of predictor (source) variables and a target variable into constituent components—specifically, into unique, shared (redundant), and synergistic (complementary) parts. This structural disaggregation addresses fundamental questions about how information is distributed, lost, or synergistically generated among multiple variables, with profound implications for neuroscience, cryptography, machine learning, communication theory, and the foundations of information theory. Theoretical and algorithmic advances have been largely centered on the bivariate case (two predictors) due to deep challenges in extending to higher-order decompositions.

## 1. Formal Framework: Bivariate Decomposition

Consider three discrete random variables $X$, $Y$, and $Z$ with joint law $P(X,Y,Z)$. The unique information of $Y$ about $X$ with respect to $Z$, denoted $\widetilde{UI}(X\,;\,Y \setminus Z)$, is defined as the minimum conditional mutual information $MI_Q(X:Y|Z)$ over the set of joint distributions $\Delta_P$ that preserve the observed $(X,Y)$ and $(X,Z)$ pairwise marginals:
\[
\widetilde{UI}(X\,;\,Y \setminus Z)
=
\min_{Q \in \Delta_P} MI_Q(X:Y|Z)
\]
where
\[
\Delta_P
=
\left\{
Q : Q(X,Y) = P(X,Y),\ Q(X,Z) = P(X,Z)
\right\}
\]
Analogously, shared (redundant) and complementary (synergistic) information are defined by:
\[
\begin{aligned}
\widetilde{SI}(X\,;\,Y,Z) &= \max_{Q\in\Delta_P}\; CoI_Q(X;Y;Z) \\
\widetilde{CI}(X\,;\,Y,Z) &= MI(X:(Y,Z)) - \min_{Q\in\Delta_P} MI_Q(X:(Y,Z))
\end{aligned}
\]
where $CoI_Q$ denotes coinformation, and $MI(\cdot\,:\,\cdot)$ denotes mutual information.

The decomposition holds:
\[
MI(X:(Y,Z)) = \widetilde{SI} + \widetilde{UI}(X:Y \setminus Z) + \widetilde{UI}(X:Z \setminus Y) + \widetilde{CI}
\]
with the corresponding consistency equations for each marginal.

This framework, due to Bertschinger et al. [1311.2852][1404.3146], is both operationally and structurally motivated.

## 2. Key Properties and Interpretation

The unique, shared, and synergistic information quantities exhibit several essential properties [1404.3146]:

- **Nonnegativity**: All four quantities ($\widetilde{SI}$, $\widetilde{UI}$, $\widetilde{CI}$) are nonnegative.
- **Marginal-invariance**: Unique and shared information depend only on the observed pairwise marginals $(X,Y)$ and $(X,Z)$.
- **Monotonicity**: Unique information is monotonic in each argument—enlarging $Z$ (adding predictors) cannot increase the unique information of $Y$, while enlarging $Y$ or $X$ cannot decrease it.
- **Special cases**:
  - For $X=Y\oplus Z$ (XOR of independent bits): $\widetilde{SI} = \widetilde{UI} = 0$, $\widetilde{CI} = \mathrm{MI}(X:(Y,Z)) = 1$ bit (pure synergy).
  - For $Y$ determines $X$ and $Z$ is independent: all information unique to $Y$, with $\widetilde{UI}(X;Y\setminus Z) = \mathrm{MI}(X:Y)$.

Operationally, the unique information quantifies the advantage held in a decision problem where an agent has access to $Y$ in place of $Z$, or vice versa, formalized via the decision-theoretic Blackwell order [1311.2852].

## 3. Algorithmic Computation and Numerical Methods

The unique information involves a constrained optimization (minimization of $MI_Q(X:Y|Z)$) over the polytope $\Delta_P$, which is typically high-dimensional [1709.07487]. Convex optimization methods—including alternating divergence minimization and generalized iterative scaling—have been developed for efficient, convergent computation:
```python
# Pseudocode (admUI) outline [1709.07487]
Input: p_{XY}, p_{XZ}
Initialize R^{(0)}(y,z)>0 uniformly
repeat
    for each x ∈ supp(p_X) in parallel:
        Q_{YZ|x}^{(i+1)} ← I-projection of R^{(i)} to satisfy marginals
    R^{(i+1)}(y,z) ← ∑_x p_X(x) Q_{YZ|x}^{(i+1)}(y,z)
until convergence
Return q^*(x,y,z)=p_X(x) Q_{YZ|x}^{(i)}(y,z)
```
This method guarantees global convergence and is substantially faster than unconstrained solvers for system sizes up to $|X|,|Y|,|Z|\approx 12$.

For continuous or parametric distributions (e.g., Gaussian, Poisson, multinomial), analytic solutions or adaptations exist when the structure of the marginal polytope allows one unique information term to vanish, massively simplifying the decomposition [2305.07013].

## 4. Theoretical Limitations and Multivariate Decomposition

A central result shows that any attempt to extend the bivariate SI/UI/CI decomposition to more than two predictors via the Williams–Beer partial information lattice leads to an incompatibility: no redundancy function $I_\cap$ can simultaneously satisfy nonnegativity, monotonicity, symmetry, and the identity axiom for $n\geq3$ [1404.3146]. 

**Theorem (Rauh et al.)**: For $n=3$, any redundancy function $I_\cap$ designed to generalize bivariate SI/UI/CI with nonnegative atomic decomposition must violate at least one natural axiom.

This precludes a nonnegative information decomposition situated on the PI-lattice for $n>2$. The precise combinatorial structure enabling a consistent multivariate unique information decomposition remains an open problem.

Nevertheless, pairwise ("partitioned") unique information can still be meaningfully defined, and nonnegative inequalities such as
\[
UI(X:Y \setminus \{Z,W\}) + UI(X:Z \setminus \{Y,W\}) \leq UI(X:\{Y,Z\} \setminus W)
\]
are conjectured to hold, offering a possible route to reconstructing atomic structure in larger systems [1404.3146].

## 5. Operational and Decision-Theoretic Foundations

The decision-theoretic underpinning of unique information is rooted in the Blackwell order: $Y$ carries no unique information about $X$ relative to $Z$ if and only if $Z$ is a garbling (randomized copy) of $Y$ in the sense of statistical decision theory [1311.2852][1404.3146]. More specifically, unique information vanishes precisely when no decision-maker observing $Y$ can improve upon optimal choices based solely on $Z$ for any loss function.

The structure of the optimization set $\Delta_P$ ensures that $\widetilde{UI}$, $\widetilde{SI}$, and $\widetilde{CI}$ depend only on the two relevant decision problems defined by the pairwise marginals, and that synergy is forced to zero for some $Q\in\Delta_P$, capturing the channel degradation structure [1404.3146].

## 6. Examples and Interpretation of Decomposition

The following table summarizes several archetypal examples:

| Distribution            | $\widetilde{SI}$ | $\widetilde{UI}$ (both) | $\widetilde{CI}$ | Key Feature                |
|-------------------------|------------------|------------------------|------------------|----------------------------|
| $X=(Y,Z)$ (COPY)        | $I(Y:Z)$         | $H(Y|Z), H(Z|Y)$       | $0$              | All redundancy/unique      |
| $X=Y\oplus Z$ (XOR)     | $0$              | $0$                    | $1$              | Pure synergy               |
| $X=Y\land Z$ (AND)      | $\approx 0.311$  | $0$                    | $\approx 0.5$    | Partial redundancy/synergy |

These confirm the interpretive design: UI captures only information exclusive to a given predictor, SI the redundancy, and CI the higher-order synergy.

## 7. Practical Considerations and Open Directions

Practically, the unique information framework is robust in the bivariate setting, with efficient algorithms and a clear operational meaning. However, current limitations include:

- **Non-uniqueness of solutions**: For certain marginal cardinalities or conditional independence cases, the optimizer for $\widetilde{UI}$ is not unique, resulting in ambiguity in the attribution of unique information [1912.12505].
- **Incomplete multivariate extensions**: There is no general nonnegative decomposition into unique, redundant, and synergistic parts for more than two predictors based on the PI-lattice. Structuring the appropriate combinatorial or geometric foundation for the multivariate case remains unresolved [1404.3146].
- **Application to practice**: Despite these theoretical obstacles, bivariate unique information can be computed for arbitrary partitions in multivariate systems, providing a partial picture of information flow in complex systems.

Current research seeks to overcome these barriers via richer combinatorial structures, and to establish new nonnegative atomic decompositions compatible with natural axioms for systems with three or more predictors [1404.3146].

---

**References:**
- "Quantifying unique information" [1311.2852]
- "Reconsidering unique information: Towards a multivariate information decomposition" [1404.3146]
- "Computing the Unique Information" [1709.07487]
- "Properties of Unique Information" [1912.12505]

Source: https://www.emergentmind.com/topics/unique-information-decomposition