Partial Information Rate Decomposition

• Partial Information Rate Decomposition is a framework that decomposes the rate of information transferred among stationary stochastic processes into unique, redundant, and synergistic components.
• The method extends static PID by operationalizing information rates through spectral analysis and lattice-theoretic Möbius inversion, particularly for Gaussian processes.
• Its frequency-resolved approach and rigorous mathematical foundation make PIRD valuable for analyzing dynamic interdependencies in fields like neuroscience, climate science, and physiology.

Partial Information Rate Decomposition (PIRD) is a mathematical framework for decomposing the information rate that a set of stochastic processes (sources) conveys about a target process into unique, redundant, and synergistic channels. PIRD generalizes the static Partial Information Decomposition (PID) from random variables to stationary stochastic processes by operationalizing decomposition in the context of information rates rather than instantaneous mutual information. This extension enables rigorous analysis of high-order interdependencies and dynamical correlations in networks of time series, particularly for Gaussian processes and systems with significant temporal structure (Faes et al., 6 Feb 2025, Sparacino et al., 6 Feb 2025).

1. Mathematical Foundation and Formal Construction

Let $Y = \{Y_t\}_{t \in \mathbb{Z}}$ denote a stationary target process and $X_1,\ldots,X_n$ be $n$ stationary source processes, written collectively as $X = (X_1,\ldots,X_n)$. The mutual information rate (MIR) between $Y$ and a source collection $X_S$ ($S \subseteq \{1,\ldots,n\}$) is defined by

$$\dot{I}(Y; X_S) \equiv \lim_{m \to \infty} \tfrac1m I\big( (Y_1,\ldots,Y_m); (X_{S,1},\ldots,X_{S,m}) \big),$$

or equivalently using entropy rates,

$$\dot{I}(Y; X_S) = \dot{H}(Y) + \dot{H}(X_S) - \dot{H}(Y, X_S).$$

This object quantifies the average information flow per time step.

The core of PIRD is a lattice-theoretic expansion. The power set $L = \{ p \subseteq \{1,\ldots,n\} \}$, ordered by set inclusion, forms the basis for organizing possible source combinations. The redundancy rate function $\dot{I}^\cap(X_p ; Y)$ is posited on the lattice nodes, subject to the relation

$$\dot{I}^\cap(X_p ; Y) = \sum_{q \preceq p} \dot{\Delta}_q,$$

where $\dot{\Delta}_q$ is the "partial information rate atom" for $q$. Möbius inversion on this poset yields

$$\dot{\Delta}_p = \sum_{q \preceq p} \mu(q, p) \cdot \dot{I}^\cap(X_q ; Y),$$

with $\mu(q, p)$ the standard Möbius function, $\mu(q,p) = (-1)^{|p| - |q|}$.

The total information rate decomposes as

$$\dot{I}(Y;\,X_1,\ldots,X_n) = \sum_{p \subseteq \{1,\ldots,n\}} \dot{\Delta}_p,$$

where the atoms $\dot{\Delta}_p$ individuate unique, redundant, or synergistic rate contributions, according to the cardinality and structure of $p$ (Faes et al., 6 Feb 2025, Sparacino et al., 6 Feb 2025).

2. Lattice Structure, Parthood, and Redundancy Measures

PIRD inherits from PID the use of distributive lattices to organize the decomposition, but crucially tracks the flow of information per time step rather than over static variables. The redundancy function must satisfy symmetry, self-redundancy, and monotonicity, as originally formalized by Williams and Beer. In the PIRD context, these lattice-theoretic and "parthood-distribution" principles assure the unique attribution of rate-level atoms to each manner in which source sets can inform the target (Gutknecht et al., 2020).

The Möbius inversion is fundamental to this attribution. Given redundancy rates on subsets/antichains, Möbius inversion uniquely determines the set of atomic rate contributions. There are three isomorphic views for this structure: parthood functions, antichain labeling, and logical-statement representations (e.g., with source-value conjunctions/disjunctions) (Gutknecht et al., 2020).

3. Spectral Methods and the Gaussian Case

For jointly stationary Gaussian processes, all mutual information rates admit a spectral decomposition. If $S(\omega)$ is the $(n+1)\times(n+1)$ power spectral density matrix of $(X_1,\ldots,X_n,Y)$, the spectral density of mutual information is

$$i_{X_S;Y}(\omega) = \frac{1}{2} \log \frac{\det S_Y(\omega)}{\det S_{Y|X_S}(\omega)},$$

where $S_{Y|X_S}(\omega)$ denotes the conditional spectrum. Thus,

$$\dot{I}(Y; X_S) = \frac{1}{2\pi} \int_{-\pi}^\pi i_{X_S;Y}(\omega) d\omega.$$

PIRD operationalizes redundancy for each source subset via a frequency-specific minimum-MIR (MMI) principle: $$i^\cap_{X_p;Y}(\omega) = \min_{q \preceq p, |q|=1} i_{X_q;Y}(\omega),$$ and applies spectral Möbius inversion to yield spectral atoms,

$$\dot{\Delta}_p(\omega) = \sum_{q \preceq p} \mu(q,p) \cdot i^\cap_{X_q;Y}(\omega).$$

Integration over frequency recovers time-domain atoms. This enables the identification and quantification of redundancy and synergy as a function of frequency, critical in domains such as neuroscience, physiology, and climate science (Faes et al., 6 Feb 2025, Sparacino et al., 6 Feb 2025).

4. Implementation and Algorithmic Considerations

The standard PIRD estimation pipeline for Gaussian (or approximately Gaussian) processes is as follows:

• Fit a high-order vector autoregressive (VAR) model to $(X_1, ..., X_n, Y)$, or estimate the spectral density matrix $S(\omega)$ using multitaper/Welch’s method.
• Compute marginal and conditional spectra at each $\omega$ for all source combinations.
• Calculate spectral mutual information densities via the log-det formula.
• Compute frequency-resolved redundancy rates and effect Möbius inversion on the lattice to obtain spectral atoms.
• Integrate atoms over frequency to yield final decomposition into unique, redundant, and synergistic information rate channels.
• Check sum-consistency: the sum of all atomic rates exactly recovers the total MIR from sources to target (Faes et al., 6 Feb 2025, Sparacino et al., 6 Feb 2025).

This algorithm has been validated in simulations and real-world networks. Practical efficiency is ensured by the explicit structure of spectral quantities in the Gaussian case.

5. Theoretical Properties and Generalizations

PIRD preserves critical theoretical guarantees:

• Spectral–time consistency: Integrals of spectral atoms recover time-domain atoms, ensuring that decompositions are invariant under Fourier transformation.
• Nonnegativity and uniqueness: For Gaussian processes and the MMI spectral redundancy, the lattice Möbius inversion yields a nonnegative allocation of total MIR to unique, redundant, and synergistic atoms. The decomposition is unique once the redundancy measure is fixed.
• Axiomatic soundness: The PIRD decomposition, under the MMI or channel-order approaches, meets Williams–Beer axioms (symmetry, self-redundancy, monotonicity, and order-equality on sources) (Gutknecht et al., 2020, Gomes et al., 2023).

PIRD can be explicitly generalized beyond Gaussianity and to arbitrary redundancy functions, including those derived from channel orders such as Blackwell, less-noisy, or more-capable (see preorders in (Gomes et al., 2023, Kolchinsky, 2019)). Such generalizations provide families of PIRD variants, each associated with different operational or decision-theoretic semantics.

6. Illustrative Applications

Table: Example Applications of PIRD

Domain	Description	Key Insights
Simulated VAR Networks	Multivariate Gaussian time series with causal or lagged coupling	Captures dynamic emergence of synergy/redundancy missed by PID
Climate Oscillations	Decomposition of SOI with sources (NINO34, TSA, PDO, NTA)	PIRD uncovers high-order dynamic effects invisible to static PID
Physiological Networks	Cardiovascular data (mean flow, arterial pressure, etc.)	Frequency-resolved redundancy buffers observed under stress

In simulated Gaussian networks, PIRD detects synergy or redundancy arising from dynamic (lagged) couplings that static PID at lag 0 misses. In climate network analysis, PIRD reveals high-order contributions related to genuine dynamical correlations, supported by surrogate-data null experiments. In multivariate physiological recordings, PIRD exposes frequency-specific dynamics, such as redundant buffering among cardiovascular regulators and emergent respiratory influence before syncope (Faes et al., 6 Feb 2025, Sparacino et al., 6 Feb 2025).

7. Connections to General Information Decomposition and Open Problems

PIRD extends the algebraic and logical structure of PID to the time or rate domain. The transition from static to dynamic decomposition is principled: per-symbol (local) mutual information atoms are averaged to yield rates, and the classical inclusion–exclusion/Möbius framework remains intact (Gutknecht et al., 2020, Makkeh et al., 2020).

Cryptographic operationalizations—such as those based on secret-key agreement rates—have highlighted interpretational issues in PID for unique and redundant channel decomposition, suggesting directions for operational rate-based definitions that are consistent and continuous (James et al., 2018). Channel-order-based methods introduce further flexibility and allow adaptation to non-Gaussian and non-Shannon-theoretic contexts (Gomes et al., 2023, Kolchinsky, 2019).

The computation of redundancy measures via channel orders remains algorithmically nontrivial, and further research targets scalable approximation schemes, analysis of continuity properties, and domain-specific operational benchmarks.

---

References:

(Faes et al., 6 Feb 2025, Sparacino et al., 6 Feb 2025, Gutknecht et al., 2020, Gomes et al., 2023, Makkeh et al., 2020, Kolchinsky, 2019, James et al., 2018)