An inferential measure of dependence between two systems using Bayesian model comparison (2412.06478v2)

Abstract: We propose to quantify dependence between two systems $X$ and $Y$ in a dataset $D$ based on the Bayesian comparison of two models: one, $H_0$, of statistical independence and another one, $H_1$, of dependence. In this framework, dependence between $X$ and $Y$ in $D$, denoted $B(X,Y|D)$, is quantified as $P(H_1|D)$, the posterior probability for the model of dependence given $D$, or any strictly increasing function thereof. It is therefore a measure of the evidence for dependence between $X$ and $Y$ as modeled by $H_1$ and observed in $D$. We review several statistical models and reconsider standard results in the light of $B(X,Y|D)$ as a measure of dependence. Using simulations, we focus on two specific issues: the effect of noise and the behavior of $B(X,Y|D)$ when $H_1$ has a parameter coding for the intensity of dependence. We then derive some general properties of $B(X,Y|D)$, showing that it quantifies the information contained in $D$ in favor of $H_1$ versus $H_0$. While some of these properties are typical of what is expected from a valid measure of dependence, others are novel and naturally appear as desired features for specific measures of dependence, which we call inferential. We finally put these results in perspective; in particular, we discuss the consequences of using the Bayesian framework as well as the similarities and differences between $B(X,Y|D)$ and mutual information.

• The paper introduces a Bayesian method comparing dependence and independence hypotheses to quantify the relationship between two systems.
• It demonstrates robust asymptotic behavior and noise resilience through extensive simulation studies.
• The framework is applied to EEG data, highlighting its potential for practical use in neuroimaging and other fields.

An Inferential Measure of Dependence Using Bayesian Model Comparison

The paper by Marrelec and Giron introduces a novel approach to measure the dependence between two systems using Bayesian model comparison. The proposed method quantifies dependence between two systems, X and Y, given a dataset, D, by comparing two models: one that assumes statistical independence (H0) and another that incorporates potential dependence (H1). This approach directly challenges traditional dependence measures by providing a framework with both theoretical and practical implications for understanding dependencies in data.

Core Concepts and Methodology

Central to this approach is the Bayesian measure, denoted as B(X,Y|D), which represents the statistical evidence supporting the dependence hypothesis, H1, over H0 given the dataset D. This measure can take the form of a posterior probability or any strictly increasing function thereof. The paper argues that B(X,Y|D) serves as a legitimate measure of dependence, leveraging its interpretation from a Bayesian perspective as quantifying the credibility of dependence present in the observed dataset.

The paper outlines several key aspects of this methodology:

• Bayesian Model Comparison: The comparison hinges on the calculation of the marginal likelihood of data under each hypothesis, which involves integrating over potential parameter values that model dependence.
• Asymptotic Properties: B(X,Y|D) is shown to display intuitive asymptotic behavior—decreasing under independence (converging to −∞) and increasing under dependence (converging to +∞) as the dataset size N grows.
• Comparison with Classical Measures: The connection between B(X,Y|D) and traditional measures like mutual information and correlation is explored, highlighting similarities and addressing differences in interpretation.
• Robustness to Model Misspecification: The approach demonstrates resilience to model specification errors, behaving as if the closer model to the true generative model in terms of Kullback-Leibler divergence was correct.

Simulation Studies and Real-Life Applications

The paper conducts extensive simulation studies to validate the method:

• Noise Influence: In scenarios with varying noise levels and dataset sizes, B(X,Y|D) shows expected trends in its behavior—decreasing with noise in independent systems and increasing with dataset size in dependent systems.
• Intensity of Dependence: When dependence intensity is encoded as a parameter, the measure increases correspondingly, affirming its sensitivity to underlying dependency structures.
• Real-World Utility: A practical application showcases B(X,Y|D) in analyzing phase consistency in EEG data during event-related protocols, suggesting its utility in neuroimaging and neuroscience.

Implications and Future Directions

The implications of this work extend beyond immediate applications, suggesting future developments and research directions:

• Versatility in Application: The framework's ability to incorporate different dependency structures (e.g., through copulas or nested models) positions it as a versatile tool for various fields requiring dependence analysis, including finance, neuroimaging, and genomics.
• Role of Priors: While the method requires careful selection of priors—affecting the measure significantly in Bayesian contexts—its adaptability can optimize the inference process across disciplines by tailoring these choices to particular datasets or hypotheses.
• Comparative Analysis: Future work can expand on comparative analysis within Bayesian contexts or against classical approaches, establishing further empirical and theoretical backing for its broad adoption.

In sum, the paper's proposed framework offers a robust and nuanced measure of dependence leveraging Bayesian inference. While expanding the theoretical discourse around dependence measures, it holds considerable promise for application across diverse scientific fields. As research progresses, continued refinement and application of B(X,Y|D) will likely enhance understanding of complex dependency networks present in real-world data.