Kernel-based Tests for Joint Independence (1603.00285v3)

Abstract: We investigate the problem of testing whether $d$ random variables, which may or may not be continuous, are jointly (or mutually) independent. Our method builds on ideas of the two variable Hilbert-Schmidt independence criterion (HSIC) but allows for an arbitrary number of variables. We embed the $d$-dimensional joint distribution and the product of the marginals into a reproducing kernel Hilbert space and define the $d$-variable Hilbert-Schmidt independence criterion (dHSIC) as the squared distance between the embeddings. In the population case, the value of dHSIC is zero if and only if the $d$ variables are jointly independent, as long as the kernel is characteristic. Based on an empirical estimate of dHSIC, we define three different non-parametric hypothesis tests: a permutation test, a bootstrap test and a test based on a Gamma approximation. We prove that the permutation test achieves the significance level and that the bootstrap test achieves pointwise asymptotic significance level as well as pointwise asymptotic consistency (i.e., it is able to detect any type of fixed dependence in the large sample limit). The Gamma approximation does not come with these guarantees; however, it is computationally very fast and for small $d$, it performs well in practice. Finally, we apply the test to a problem in causal discovery.

• The paper proposes the dHSIC framework, an extension of HSIC, for directly testing the joint independence of any number of variables using kernel methods.
• Numerical simulations demonstrate that dHSIC tests effectively detect complex dependencies and are applicable to problems like validating independence in causal models.
• The kernel-based approach provides a theoretically sound, robust, and flexible tool for multivariate independence testing that surpasses traditional pairwise methods.

An Expert Overview of "Kernel-based Tests for Joint Independence"

The paper "Kernel-based Tests for Joint Independence," authored by Niklas Pfister, Peter Bühlmann, Bernhard Schölkopf, and Jonas Peters, explores a novel approach to testing the joint independence of multivariate random variables. The authors extend the Hilbert-Schmidt Independence Criterion (HSIC) to accommodate an arbitrary number of variables, thus formulating the d-variable Hilbert-Schmidt independence criterion (dHSIC). This extension facilitates a direct examination of mutual independence rather than relying solely on pairwise independence assessments, which can be insufficient in certain statistical models, particularly in causal inference scenarios.

Technical Contributions and Methodology

The central contribution of this research is the development of the dHSIC framework, which measures the squared distance between the joint distribution and the product of marginals within a reproducing kernel Hilbert space (RKHS). For kernels deemed characteristic, dHSIC upholds the property that its value is zero if and only if the variables are jointly independent. This rigorous mathematical formulation underlies three proposed nonparametric hypothesis tests — a permutation test, a bootstrap test, and a test based on a Gamma distribution approximation.

In extending HSIC, the authors leverage statistical advantages inherent in kernel methods such as flexibility and robustness to assumptions about data distributions. Theoretical results assert the asymptotic consistency of the permutation and bootstrap-based procedures. Notably, the bootstrap test is shown to achieve pointwise asymptotic level and consistency, making it a robust choice from a theoretical standpoint despite potentially high computational demands.

Numerical Results and Applications

The numerical simulations illustrate the performance of the proposed tests under various scenarios, showing that dHSIC effectively assesses joint independence across both simulated and real datasets. The paper notably addresses causal discovery problems, highlighting how dHSIC could assist in validating the independence of residuals in structural equation models, which is crucial for inferring causal structures from observational data.

The empirical results underscore dHSIC's superiority in detecting complex dependency structures compared to traditional pairwise methods and Bonferroni-corrected multiple tests. The tests exhibit computational efficiency, with dHSIC's computational complexity being quadratic concerning the data size, thereby rendering it viable for moderate to large datasets.

Theoretical Implications and Future Directions

The research offers significant theoretical contributions, including the formulation of general results concerning V-statistics, which have broad applicability beyond the context of this paper. The theoretical groundwork laid by the authors fosters potential extensions of kernel-based methods to more complex data structures or alternative methodological frameworks in independence testing.

Future developments could aim at enhancing the practical feasibility of the proposed tests, focusing on optimizing computational procedures for large-scale implementation. Moreover, further exploration into refining the kernel choice and parameter selection could lead to more powerful and adaptable testing procedures suitable for a broader range of applications, especially those involving high-dimensional or non-Euclidean data structures.

In summary, this paper advances the statistical toolkit available for independence testing in multivariate settings, offering a robust, versatile, and theoretically sound framework that has practical implications for fields such as causal inference and beyond.