NAMMD-Based Two-Sample Test

• The NAMMD-based two-sample test is a kernel method that adaptively normalizes the MMD statistic using RKHS norms to capture both mean and concentration differences.
• It improves statistical power over standard MMD by detecting subtle distributional differences even in high-dimensional or non-Euclidean contexts.
• The method employs permutation tests for type-I error control and is applicable to tasks like anomaly detection, dataset shift, and model validation.

The norm-adaptive maximum mean discrepancy (NAMMD)-based two-sample test is a kernel-based, nonparametric statistical test for assessing whether two probability distributions are identical, with particular emphasis on adaptively scaling the test statistic to the concentration (RKHS norm) of the distributions involved (Zhou et al., 17 Jul 2025). This approach generalizes the widely used maximum mean discrepancy (MMD), providing increased sensitivity in scenarios where the two distributions differ not only in location but also in spread or concentration as captured by the reproducing kernel Hilbert space (RKHS) structure. The NAMMD framework has been shown theoretically and empirically to enhance test power over classical MMD-based procedures, especially in high-dimensional and non-Euclidean data contexts.

1. Definition and Theoretical Motivation

The NAMMD statistic is a normalized version of MMD that incorporates the RKHS norms of the distributions under comparison. For a bounded, positive-definite kernel $\kappa(x, y)$ with $0 \leq \kappa(x,y) \leq K$, and two probability distributions $P$ and $Q$ defined on a measurable space $\mathcal{X}$, the associated kernel mean embeddings are

$$\mu_P = \mathbb{E}_{x \sim P}[ \kappa(\cdot, x) ], \qquad \mu_Q = \mathbb{E}_{y \sim Q}[ \kappa(\cdot, y) ].$$

The classical (squared) MMD is

$$\operatorname{MMD}^2(P, Q; \kappa) = \|\mu_P - \mu_Q\|^2_{\mathcal{H}},$$

where $\mathcal{H}$ is the RKHS associated with $\kappa$.

NAMMD rescales this measure to account for the concentration of the distributions in RKHS via

$$\operatorname{NAMMD}(P, Q; \kappa) = \frac{\operatorname{MMD}^2(P, Q; \kappa)}{4K - \|\mu_P\|^2_{\mathcal{H}} - \|\mu_Q\|^2_{\mathcal{H}}}.$$

The denominator, $4K - \|\mu_P\|^2 - \|\mu_Q\|^2$, is strictly positive and decreases as the distributions become more concentrated (i.e., when their RKHS norms increase). For translation-invariant kernels (e.g., Gaussian, Laplacian), this norm reflects the "spread" of $P$ and $Q$ under the kernel metric; thus, the normalization adaptively adjusts the test's sensitivity.

A key property is that, for characteristic kernels, $\operatorname{NAMMD}(P, Q; \kappa) = 0$ if and only if $P = Q$.

2. Empirical Estimation and Statistical Procedure

Given i.i.d. samples $X = \{x_1, ..., x_m\} \sim P$ and $Y = \{y_1, ..., y_m\} \sim Q$, the empirical NAMMD statistic is computed as

$$\widehat{\operatorname{NAMMD}}(X, Y; \kappa) = \frac{ \sum_{i \neq j} \left[ \kappa(x_i, x_j) + \kappa(y_i, y_j) - \kappa(x_i, y_j) - \kappa(y_i, x_j) \right] }{ \sum_{i \neq j} \left[ 4K - \kappa(x_i, x_j) - \kappa(y_i, y_j) \right] }.$$

The null hypothesis $H_0$: $P = Q$ is tested using a permutation framework. The procedure shuffles the pooled data into two groups multiple times, computes the statistic each time, and defines the (empirical) $1-\alpha$ quantile as the rejection threshold for level-$\alpha$ testing.

Asymptotic Properties

Under $H_0$, the null distribution of $$m \cdot \widehat{\operatorname{NAMMD}}(X, Y; \kappa)$$ converges in distribution to a weighted sum of centered chi-squared variables: $$m \cdot \widehat{\operatorname{NAMMD}}(X,Y; \kappa) \xrightarrow{d} \frac{ \sum_i \lambda_i (Z_i^2 - 2) }{ 4K - \| (\mu_P + \mu_Q)/\sqrt{2} \|^2_{\mathcal{H}} },$$ where $\{\lambda_i\}$ are the eigenvalues of the covariance operator of the centered kernel and each $Z_i \sim \mathcal{N}(0,2)$.

Under $H_1$: $P \neq Q$, the estimator is asymptotically normal: $$\sqrt{m}\left(\widehat{\operatorname{NAMMD}}(X,Y; \kappa) - \operatorname{NAMMD}(P, Q; \kappa)\right) \xrightarrow{d} \mathcal{N}(0, \sigma^2),$$ where the variance $\sigma^2$ is explicitly characterized in the paper.

Because the null distribution depends on the underlying distributions and is not simple to estimate analytically, permutation tests provide reliable finite-sample control for the type-I error rate.

3. Theoretical Advantages Over Standard MMD

The NAMMD statistic provides several key advantages:

• Adaptive Sensitivity: For two pairs of distributions with the same MMD but different RKHS norms, NAMMD increases the statistic when the distributions are more concentrated. This is achieved via the denominator's dependence on the sum of the norms, which is smaller for concentrated (less dispersed) distributions.
• Increased Power: Under alternatives where differences in concentration or spread matter, NAMMD can be more sensitive than MMD, leading to higher probability of rejecting $H_0$ (see Theorem comp_tst). There are situations where MMD-based tests may not reject $H_0$ but NAMMD-based tests do.
• Theoretical Guarantees: The sample complexity for detecting a fixed alternative (at prescribed risk) scales as $$O(1/(\operatorname{NAMMD}(P,Q;\kappa) - \epsilon)^2)$$. Large deviation results and type-I error bounds are established under both the permutation and asymptotic frameworks.

4. Experimental Results

A range of experiments validates the improved power of the NAMMD-based two-sample test across synthetic and real-world datasets:

• Synthetic datasets: Gaussian mixture models in low (2d, "blob") and high (10d, "hdgm") dimensions demonstrate that NAMMD-based tests achieve higher power than standard MMD at the same sample size and kernel.
• Tabular data: On the Higgs dataset—with complex high-dimensional structure—the enhanced sensitivity manifests as better detection of shifts between background and signal samples.
• Image data: Experiments on MNIST and CIFAR test for differences between image classes or perturbed vs. nominal distributions. In these applications, NAMMD-based tests produce lower $p$-values and higher rejection rates when RKHS norm differences dominate.

The test's behavior is especially pronounced when comparing multiple pairs of distributions with similar MMDs but different RKHS norms, illustrating its adaptive power.

A multiple-kernel fusion variant ("NAMMDFuse") further improves robustness by aggregating evidence across several kernels, often outperforming state-of-the-art alternatives.

5. Applications and Implications

The NAMMD-based two-sample test is applicable to a wide variety of modern statistical tasks:

• Dataset shift and drift detection: In monitoring streaming or time-evolving data in high dimensions, the test detects distributional changes related to both mean and concentration.
• Model assessment without labels: By computing NAMMD between training and test data, practitioners can estimate distributional mismatches that correlate with downstream prediction errors.
• Anomaly and adversarial detection: The sensitivity to both location and spread renders NAMMD effective in detecting adversarial perturbations and subtle anomalies in complex domains (e.g. images).
• Model criticism and data integration: In multi-domain or multi-source data, the method offers a rigorous means of assessing compatibility or identity between distributions.

The test's nonparametric, kernel-based nature makes it widely applicable across domains such as bioinformatics, computer vision, domain adaptation, and privacy-preserving data analysis.

6. Comparison to Other Approaches

NAMMD generalizes and refines kernel two-sample testing in several directions:

• Versus standard MMD: While classical MMD is only sensitive to the distance between means in the RKHS, NAMMD incorporates norm-based scaling, crucial in many high-dimensional and manifold-structured problems.
• Versus closeness testing and other metrics: NAMMD is designed to uniformly increase power in tasks where not only the mean but also concentration/structure matters. It delivers a more informative ranking of the "closeness" of distributions than non-adaptive MMD, addressing scenarios where MMD is insufficiently discriminative.
• Implementation: The permutation-based version is computationally practical and requires only a simple plug-in estimate of the NAMMD numerator and denominator from pairwise kernel evaluations. Theoretical guarantees are provided for both null and alternative cases.

7. Limitations and Considerations

While NAMMD-based tests provide improved sensitivity and theoretical guarantees, several considerations are relevant:

• The method assumes access to a bounded, characteristic kernel and may require careful handling in extremely large-scale scenarios.
• Calculation of the statistic is quadratic in sample size due to pairwise kernel evaluations, but can be accelerated using approximations if needed.
• The denominator scaling may, in rare cases, be less effective if RKHS norms are poorly estimated from very limited data.
• In the presence of heavily imbalanced or nonstationary data, the interpretation of "spread" via RKHS norms should be considered in context.

Summary Table: NAMMD-based versus MMD-based Two-Sample Testing

Aspect	NAMMD-Based Test	MMD-Based Test
Sensitivity	Adaptive to both mean and concentration	Mean only
Statistic	$$\frac{\text{MMD}^2}{4K - \|\mu_P\|^2 - \|\mu_Q\|^2}$$	$\|\mu_P - \mu_Q\|^2$
Null distribution	Weighted chi-square (via permutation threshold)	Weighted chi-square
Power under alternatives	Higher, especially when norms differ	May miss norm differences
Implementation	Empirical, permutation test	Empirical/permutation
Applications	Distribution shift, high-dim data, anomaly detect	Broad, but less adaptive

In conclusion, the NAMMD-based two-sample test advances the kernel-based hypothesis testing framework by adaptively scaling the MMD statistic, providing heightened sensitivity to a wider class of distributional differences while maintaining rigorous control of error rates and practical implementability (Zhou et al., 17 Jul 2025).