Directed Information Transfer

• Directed information transfer is a metric that quantifies the directional flow of entropy between measurable state-space partitions in dynamical systems.
• It employs finite-dimensional approximations and transfer operators to compute causal influence and optimize sensor and actuator placement.
• The framework extends classical information theory to classify ergodicity and mixing, proving essential for control in high-dimensional non-equilibrium settings.

Directed information transfer quantifies the directional flow of information between subsystems in a dynamical system, typically formalized via measurable events (Borel sets) under the evolution of a nonsingular transformation. This framework—based on the transfer of Shannon entropy as the system evolves—provides a rigorously defined, operationally computable, and causally interpretable metric for influence between state-space partitions, extending classical information-theoretic and control-theoretic approaches to non-equilibrium and high-dimensional settings. Recent advances by Sinha, Vaidya, and Yeung characterize this transfer between arbitrary sets as a means for the classification of ergodicity and mixing and enable efficient computation via finite-dimensional transfer operators, facilitating optimal actuator and sensor placement in control applications (Sinha et al., 2019).

1. Formal Framework: Information Transfer Between Measurable Sets

Let $(X,\mathcal B,\mu)$ be a probability space with normalized total measure $\mu(X)=1$, and let $T:X\to X$ be a nonsingular measurable map (i.e., $\mu(A)=0\implies \mu(T^{-1}A)=0$). Consider a measurable partition labeled by sets $\{A_i\}, A_i\in\mathcal B$, each with $\mu(A_i)>0$.

Define the one-step information transfer from set $A_i$ to set $A_j$ as: $$\mu_{ij} = \frac{ \mu( T^{-1}(A_j) \cap A_i ) }{ \mu(A_i) }$$

$\mathcal T_{i\to j}^\mu = - \mu_{ij} \log \mu_{ij}$

For $\mu(X)=1$0-step transfer, define: $\mu(X)=1$1 and the total transfer up to time $\mu(X)=1$2 as $\mu(X)=1$3 (Sinha et al., 2019).

This quantifies the transfer of entropy from $\mu(X)=1$4 to $\mu(X)=1$5 under the dynamics. For invertible $\mu(X)=1$6, $\mu(X)=1$7 equates to $\mu(X)=1$8.

2. Causality, Directionality, and Zero-Transfer Conditions

Key properties:

• Nonnegativity: $\mu(X)=1$9
• Zero-transfer: $T:X\to X$0 iff $T:X\to X$1, signifying complete absence of causal transfer from $T:X\to X$2 to $T:X\to X$3.
• Asymmetry: Generally, $T:X\to X$4, because the relevant pre-images may not align.
• Causal interpretation: $T:X\to X$5 means $T:X\to X$6 can influence $T:X\to X$7, and $T:X\to X$8 reflects causal independence (Sinha et al., 2019).

3. Dynamical Systems Applications: Ergodicity, Mixing, and Classification

Directed information transfer provides classification criteria for ergodicity and strong mixing:

• Ergodicity: A measure-preserving map $T:X\to X$9 is ergodic iff, for every $\mu(A)=0\implies \mu(T^{-1}A)=0$0 with positive measure, the total transfer from $\mu(A)=0\implies \mu(T^{-1}A)=0$1 to $\mu(A)=0\implies \mu(T^{-1}A)=0$2 is nonzero for some $\mu(A)=0\implies \mu(T^{-1}A)=0$3. That is: $\mu(A)=0\implies \mu(T^{-1}A)=0$4
• Mixing: $\mu(A)=0\implies \mu(T^{-1}A)=0$5 is (strongly) mixing iff $\mu(A)=0\implies \mu(T^{-1}A)=0$6. In this case the long-term $\mu(A)=0\implies \mu(T^{-1}A)=0$7-step transfer approaches the full entropy: $\mu(A)=0\implies \mu(T^{-1}A)=0$8 where $\mu(A)=0\implies \mu(T^{-1}A)=0$9 (Sinha et al., 2019). Thus, maximal directed transfer characterizes mixing.

4. Finite-Dimensional Approximations and Computation

To enable practical computation, partition the state space into disjoint cells $\{A_i\}, A_i\in\mathcal B$0, and approximate the Perron–Frobenius operator by the transition matrix: $\{A_i\}, A_i\in\mathcal B$1 where $\{A_i\}, A_i\in\mathcal B$2 is typically Lebesgue measure.

Information transfer between cells is then: $\{A_i\}, A_i\in\mathcal B$3

$\{A_i\}, A_i\in\mathcal B$4

An efficient computational algorithm proceeds by: selecting a partition, sampling trajectories, estimating transition frequencies, constructing $\{A_i\}, A_i\in\mathcal B$5, computing matrix powers, and evaluating the transfer. Complexity is dominated by trajectory sampling $\{A_i\}, A_i\in\mathcal B$6 and matrix multiplication $\{A_i\}, A_i\in\mathcal B$7, but can be accelerated via sparsity (Sinha et al., 2019).

5. Sensor and Actuator Placement via Information Transfer

Directed information transfer supports system-theoretic control objectives. Optimal placement tasks are formulated as convex relaxations:

• Actuator placement: Select sets of cells (actuator locations) so that each cell is accessible via nonzero transfer—i.e., minimize actuator “support” under transfer coverage constraints.
• Sensor placement: Select cells to maximize “coarse observability,” ensuring that transfer from the rest of the domain to these cells is nonzero (reverse transfer coverage).

Empirical studies:

• In a time-periodic double-gyre flow, actuators optimally cover the mixing region; transfer plots reveal controllers' reachability.
• For Navier–Stokes-driven indoor airflow, optimally placed sensors (6 locations) ensure nonzero transfer capture over nearly all state space, matching physical intuition about observability (Sinha et al., 2019).

6. Algorithmic Summary and Illustrative Example Table

The operational algorithm can be summarized as follows:

Step	Description	Complexity
1	Choose state-space partition	user-specified
2	Sample L i.i.d. points/cell	O(NL)
3	Apply T, assign to destination	O(NL)
4	Count transitions, build $\{A_i\}, A_i\in\mathcal B$8	O(NL)
5	Compute $\{A_i\}, A_i\in\mathcal B$9	O(N^3 \log n)
6	Compute transfer metrics	O(N^2)

Table: Main algorithm steps for directed information transfer computation (Sinha et al., 2019).

For a $\mu(A_i)>0$0 discretization of a flow field, the method reveals transfer connectivity structure, facilitates sensor placement matching separating streamlines, and provides log-scale visualization of transfer distributions—crucial for analyzing non-equilibrium transport in high-dimensional domains.

7. Implications and Significance for Dynamical and Controlled Systems

Directed information transfer combines rigorous causal directionality, nonnegativity, and algorithmic accessibility, distinguishing it from more classical measures (e.g., Granger causality, transfer entropy) that may fail to respect underlying dynamical structure or lack operationally interpretable transfer asymmetry.

Its ability to partition transfer, formalize absence of causal effect, and quantify reachability or observability through entropy flow makes it a foundational tool for classification of system regimes (ergodic, mixing), design of optimal control and estimation architectures, and empirical studies in complex, possibly non-smooth, dynamical settings (Sinha et al., 2019).