Granger causality and transfer entropy are equivalent for Gaussian variables

Abstract: Granger causality is a statistical notion of causal influence based on prediction via vector autoregression. Developed originally in the field of econometrics, it has since found application in a broader arena, particularly in neuroscience. More recently transfer entropy, an information-theoretic measure of time-directed information transfer between jointly dependent processes, has gained traction in a similarly wide field. While it has been recognized that the two concepts must be related, the exact relationship has until now not been formally described. Here we show that for Gaussian variables, Granger causality and transfer entropy are entirely equivalent, thus bridging autoregressive and information-theoretic approaches to data-driven causal inference.

• The paper presents a formal proof that for Gaussian systems, Granger causality is equivalent to twice the transfer entropy.
• The study bridges autoregressive models with information‐theoretic measures, enabling new insights through spectral domain applications.
• The work advances causal inference by aligning statistical and data-driven methods, while highlighting the limitations of Gaussian assumptions.

Equivalence of Granger Causality and Transfer Entropy for Gaussian Variables

The paper by Barnett, Barrett, and Seth investigates the relationship between Granger causality and transfer entropy, presenting a formal proof of their equivalence when applied to Gaussian variables. Their work provides a bridge between the autoregressive and information-theoretic approaches to data-driven causal inference, which could harmonize the application of these methods across various domains, notably in fields like neuroscience and econometrics.

Granger causality is traditionally used to understand causal influence in time series data by determining whether past values of one variable can predict future values of another. Transfer entropy, on the other hand, is an information-theoretic measure capturing the amount of directed (time-asymmetric) information transfer between two jointly dependent processes. While these concepts have been intuitively linked, this paper is the first to formally demonstrate their equivalence for Gaussian variables.

The main result of the paper is captured in the equation that expresses Granger causality as twice the transfer entropy for Gaussian systems. This equivalence is observed under Gaussian assumptions, where any finite subset of variable components is jointly Gaussian distributed. The researchers achieve this by expressing transfer entropy in terms of the covariance matrix of residuals from linear regression models. They extend a contention by Geweke, suggesting that Granger causality can be calculated using the determinants of covariance matrices, which are then shown to correspond to expressions for transfer entropy.

Implications stemming from this equivalence include potential advancements in the spectral domain of transfer entropy as seen in Granger causality, allowing a transformation of insights from one domain to apply to the other. The spectral decomposition available in Granger causality could be adapted to transfer entropy, enriching its applicability. Conversely, transfer entropy's invariant properties under general nonlinear transformations might provide pathways to identify suitable nonlinear autoregressive models.

The Gaussian assumption is a significant aspect of this research. Although Gaussian processes are frequently used in disciplines such as neuroscience and econometrics due to their analytical tractability, real-world data complexities might limit these assumptions' validity. Future research could address how this equivalence might degrade with deviations from Gaussian behavior, potentially enhancing the practical applicability and robustness of these methods in non-Gaussian contexts.

From a methodological perspective, while Granger causality typically benefits from the structured modeling environment of multivariate autoregressive (MVAR) models, transfer entropy’s lack of requisite distributional assumptions paves the way for empirical application across varied datasets. Thus, researchers have the flexibility to select whichever method aligns best with their data's characteristics, leveraging the analytical insights provided by the equivalence outlined in this paper.

In conclusion, Barnett, Barrett, and Seth provide a mathematically rigorous basis for understanding the equivalence between Granger causality and transfer entropy in Gaussian settings. This contribution enhances the theoretical foundation of causal inference methodologies and invites further exploration into applied settings where Gaussian assumptions may not hold. Future work could elaborate on this foundation to develop even more robust causal inference tools for practical applications across scientific disciplines.