Robust Kernel Hypothesis Testing under Data Corruption (2405.19912v1)

Abstract: We propose two general methods for constructing robust permutation tests under data corruption. The proposed tests effectively control the non-asymptotic type I error under data corruption, and we prove their consistency in power under minimal conditions. This contributes to the practical deployment of hypothesis tests for real-world applications with potential adversarial attacks. One of our methods inherently ensures differential privacy, further broadening its applicability to private data analysis. For the two-sample and independence settings, we show that our kernel robust tests are minimax optimal, in the sense that they are guaranteed to be non-asymptotically powerful against alternatives uniformly separated from the null in the kernel MMD and HSIC metrics at some optimal rate (tight with matching lower bound). Finally, we provide publicly available implementations and empirically illustrate the practicality of our proposed tests.

• The paper introduces two robust methods for kernel hypothesis tests that ensure non-asymptotic validity and minimax optimal power even under data corruption.
• It leverages differential privacy by injecting Laplace noise into test statistics to adjust significance levels according to corruption severity.
• The study also proposes a direct correction approach that adjusts critical values without randomness, ensuring reproducibility and high power in practical settings.

Robust Kernel Hypothesis Testing Under Data Corruption

The paper "Robust Kernel Hypothesis Testing Under Data Corruption" by Antonin Schrab and Ilmun Kim addresses the challenge of conducting hypothesis tests in scenarios where the data may be subject to corruption. Hypothesis testing is a cornerstone of statistical analysis, but real-world data often contain noise, outliers, or even deliberate adversarial modifications. Thus, developing robust tests that maintain their validity in the presence of such corruptions is critical for practical applications.

Contributions and Methodology

The authors introduce two general methods for constructing robust permutation tests that can withstand data corruption. These methods aim to achieve multiple objectives: ease of implementation, applicability to a wide range of non-parametric testing problems, non-asymptotic validity under data corruption, and minimax optimal power under certain conditions.

Differential Privacy-Based Tests (DP Tests)

The first method leverages differential privacy (DP) principles to construct robust tests. Differential privacy ensures that the output of a statistical analysis is not overly sensitive to any single data point, a property closely related to robustness under data corruption. The authors adapt the DP test of Kim et al. (2023), which injects Laplace noise into the test statistics to achieve privacy guarantees while ensuring non-asymptotic validity.

The DP test scales the significance level α according to the corruption level r and the privacy parameter ε. The test remains consistent, maintaining a control over type I error and achieving high power against fixed alternatives.

Direct Correction-Based Tests (DC Tests)

The second method foregoes noise injection and instead directly adjusts the critical value of the test statistic to account for the maximal possible corruption. This DC test is advantageous in settings where reproducibility is crucial because it avoids the randomness introduced by noise injection.

DC tests also control non-asymptotic type I error and achieve high power, but they require careful tuning to balance robustness and power. The test becomes consistent under minimal conditions regarding the growth rates of the sample size and the number of permutations.

Application to Kernel-Based Hypothesis Testing

The authors apply their robust testing framework specifically to two prominent kernel-based hypothesis testing scenarios:

• Two-sample Testing (MMD): The Maximum Mean Discrepancy (MMD) statistic is used to test if two samples come from the same distribution. The robust tests, dpMMD, and dcMMD, are shown to control type I error under corruption and are consistent against fixed alternatives.
• Independence Testing (HSIC): The Hilbert-Schmidt Independence Criterion (HSIC) statistic tests the independence of paired samples. The tests, dpHSIC and dcHSIC, provide type I error control and consistency under corrupted data.

Theoretical Guarantees and Empirical Validation

The paper demonstrates that both dpMMD and dcMMD tests are minimax optimal, achieving the best possible separation rate in terms of the MMD metric. Similarly, dpHSIC and dcHSIC tests attain minimax optimal power in terms of the HSIC metric. This indicates their robustness and efficacy even in highly corrupted data scenarios.

The authors validate their methods through extensive experiments, including synthetic data and real-world datasets (IMDb movie reviews). The results show that the proposed tests effectively control type I error under various corruption levels and achieve high test power, significantly improving over traditional tests that do not account for data corruption.

Implications and Future Directions

This work makes significant strides in robust statistical testing, particularly under adversarial conditions. The implications are far-reaching, especially for applications in fields like clinical trials, model validation, and data-driven decision-making, where data integrity cannot always be assured.

Future developments may explore more sophisticated corruption models, such as Huber's $\epsilon$-contamination, and extend the proposed methods to a broader class of test statistics. Additionally, optimizing these tests for computational efficiency remains an open area, potentially leveraging recent advances in scalable kernel methods.

Conclusion

Antonin Schrab and Ilmun Kim's work on robust kernel hypothesis testing provides robust, theoretically sound, and practical solutions for hypothesis testing in the presence of data corruption. The methods not only guarantee performance under worst-case scenarios but also maintain high power in real-world applications, marking a significant advancement in the field of robust statistics.