Two-Cluster Test: Theory & Practice

• The paper introduces two-cluster tests as formal methods to distinguish between homogeneous and bimodal data structures using rigorous hypothesis testing.
• It employs diverse methodologies—minimax-optimal techniques, selective inference, and boundary-based methods—to ensure precise control of type-I errors across various data settings.
• Applications span clustering validation in genomics, error estimation in regression models, and spectral analysis in graph theory, highlighting practical and theoretical advancements.

A two-cluster test refers to a family of methodologies for formally assessing whether data or partitions exhibit two distinct clusters versus a simpler alternative such as homogeneity or a single-cluster structure. These tests span diverse data settings: Euclidean or high-dimensional spaces, graphs, regression models, distributional data, and more. The central aim is to provide statistically principled, often minimax-optimal, and type-I error controlled procedures for distinguishing between the presence and the absence of two-cluster structure, addressing limitations of classical two-sample or clustering-validity tests.

1. Formal Problem Statements and Hypotheses

Two-cluster tests can be formulated in a range of settings, but several canonical forms recur:

• Gaussian Mixture Equivalence: Given two independent samples,

$$X_1, ..., X_{n_1} \sim \text{i.i.d. } N(z_i \theta, I_p), \quad Y_1, ..., Y_{n_2} \sim \text{i.i.d. } N(\sigma_j \eta, I_p),$$

with latent labels $z_i, \sigma_j \in \{-1, +1\}$ and mean vectors $\theta, \eta \in \mathbb{R}^p$, the test is

$$H_0: \ell(z, \sigma) = 0 \text{ (identical clustering up to sign)} \quad \text{vs.} \quad H_1: \ell(z, \sigma) > \epsilon$$

where $\ell$ is a minimum label disagreement metric (Gao et al., 2019).

• Cluster Structure of Graphs: For a graph $G = (V, E)$, test whether it is $(2, \phi)$-clusterable (can be partitioned into two vertex sets with conductance at least $\phi$) versus being $\epsilon$-far from any such partition (Silwal et al., 2018).
• Feature Difference after Clustering: Given a data matrix $X \in \mathbb{R}^{n \times q}$, after clustering into two groups $z_i, \sigma_j \in \{-1, +1\}$0 and $z_i, \sigma_j \in \{-1, +1\}$1, test for a mean difference in a fixed feature $z_i, \sigma_j \in \{-1, +1\}$2 via

$z_i, \sigma_j \in \{-1, +1\}$3

with conditioning to control selective error (Chen et al., 2023, Yun et al., 2024).

• Testing in Regression Models: Is variance estimation adequately captured by assuming “fine" clustering, or is “coarse”/higher-level clustering necessary? $z_i, \sigma_j \in \{-1, +1\}$4: finer clustering sufficient; $z_i, \sigma_j \in \{-1, +1\}$5: need coarser clustering (MacKinnon et al., 2023, Davezies et al., 25 Jun 2025).
• Nonparametric/Flexible Settings: Is a data partition into two candidate clusters supported versus the hypothesis that the data is single-cluster? Settings include vector data (distance-based tests) (Modak, 20 May 2026), labeled graph data (planarity under cluster constraints) (Fulek et al., 2013), and partitioning of $z_i, \sigma_j \in \{-1, +1\}$6 distributions into two homogeneous groups (Kumar et al., 9 Dec 2025).

In all cases, the null hypothesis encodes the simplest (single-cluster or identical partition) structure, while the alternative posits meaningful two-group heterogeneity.

2. Methodological Frameworks

(a) Minimax and Detection-Boundary Theory

For high-dimensional Gaussian mixtures, the two-cluster equivalence test establishes the detection boundary $z_i, \sigma_j \in \{-1, +1\}$7 in the space of signal-to-noise ratio and imbalance, with higher-criticism (HC)-type statistics achieving both lower and upper bounds. Tests involve univariate projections of the data, explicit computation of one-dimensional statistics, and adaptive procedures in the presence of unknown means:

• Known means: HC statistics on $z_i, \sigma_j \in \{-1, +1\}$8 attain the phase boundary.
• Unknown means: Three-fold sample splitting, PCA-based parameter estimation, and plug-in HC statistics adapted for estimation variability (Gao et al., 2019).

(b) Selective Inference for Cluster Validity

Classical tests (e.g., $z_i, \sigma_j \in \{-1, +1\}$9-tests) applied to post-clustering partitions do not control type-I error due to selection bias. Modern two-cluster validity tests (as in selective inference) condition on the observed (data-dependent) clustering assignments:

• Derivation of polyhedral or quadratic selection regions for $\theta, \eta \in \mathbb{R}^p$0-means/hierarchical clustering.
• Construction of selective $\theta, \eta \in \mathbb{R}^p$1-values via truncated normal/F-distributions, exact finite-sample control of the conditional type I error (Chen et al., 2023, Yun et al., 2024).

(c) Boundary-Point and Nonparametric Methods

For data clusters produced via clustering algorithms, the “Boundary-based Two-Cluster Test” (BTCT) uses only near-boundary points. Each such point, identified via mutual nearest neighbors across clusters, gives rise to a Binomial$\theta, \eta \in \mathbb{R}^p$2 test for label-homogeneity among its neighborhood, and combined $\theta, \eta \in \mathbb{R}^p$3-values (via Fisher's method) yield valid global inference. This approach avoids classical two-sample test's selection bias (Liu et al., 11 Jul 2025).

Distance-based two-cluster tests (e.g., (Modak, 20 May 2026)) construct per-observation group-difference $\theta, \eta \in \mathbb{R}^p$4-values based on chi-squared or permutation statistics for the interpoint distance distributions versus respective clusters. An aggregate metric (the average $\theta, \eta \in \mathbb{R}^p$5-value) serves as a global test statistic.

(d) Graph- and Planarity-Based Tests

In graphs, the two-cluster test leverages spectral properties, specifically the structure of Laplacian eigenvectors. A graph is $\theta, \eta \in \mathbb{R}^p$6-clusterable if random walk endpoint distributions from each vertex are collinear (rank-one minors in the spectral matrix). The test uses random walk sampling, $\theta, \eta \in \mathbb{R}^p$7-norm and inner-product testers, and checks 2x2 minors for (non-)collinearity (Silwal et al., 2018). Clustered planarity is tested via solving linear systems over $\theta, \eta \in \mathbb{R}^p$8 for the parity of independent edge crossings, leveraging extensions of the Hanani–Tutte theorem (Fulek et al., 2013).

3. Statistical Properties and Theoretical Guarantees

Test Class	Type I Control	Consistency/Power	Minimax/Optimality
Higher-criticism/Minimax	Valid under thresholding/HC	Consistent below detection boundary	Optimal in $\theta, \eta \in \mathbb{R}^p$9 regime (Gao et al., 2019)
Selective inference	Exact conditional on clustering	Consistent/power increases with effect size	Valid under arbitrary partition selection (Chen et al., 2023, Yun et al., 2024)
Nonparametric BTCT	Calibrated at nominal $$H_0: \ell(z, \sigma) = 0 \text{ (identical clustering up to sign)} \quad \text{vs.} \quad H_1: \ell(z, \sigma) > \epsilon$$0	Power close to classical two-sample for real clusters	Maintains level under selection
Distance-aggregation	Controls level by aggregate $$H_0: \ell(z, \sigma) = 0 \text{ (identical clustering up to sign)} \quad \text{vs.} \quad H_1: \ell(z, \sigma) > \epsilon$$1-value	Consistent with increasing separation	Nonparametric, minimal tuning
Graph spectral	Guarantees (completeness, soundness) hold relative to conductance gap $$H_0: \ell(z, \sigma) = 0 \text{ (identical clustering up to sign)} \quad \text{vs.} \quad H_1: \ell(z, \sigma) > \epsilon$$2	Near-optimal query/sample complexity	Phase transition in eigenvalue gaps (Silwal et al., 2018)

In particular, classical two-sample tests, when applied post-clustering, can yield dramatically inflated Type I error (empirical Type I $$H_0: \ell(z, \sigma) = 0 \text{ (identical clustering up to sign)} \quad \text{vs.} \quad H_1: \ell(z, \sigma) > \epsilon$$3 for $$H_0: \ell(z, \sigma) = 0 \text{ (identical clustering up to sign)} \quad \text{vs.} \quad H_1: \ell(z, \sigma) > \epsilon$$4) (Liu et al., 11 Jul 2025). Selective inference frameworks and boundary-point methods address this selection bias rigorously.

4. Implementation and Algorithmic Procedures

• Higher-Criticism (HC) and Adaptive Procedures: Compute projections using estimated means or principal components, evaluate univariate summary statistics on a held-out sample fold, and threshold using higher-criticism or multiple-testing criteria (Gao et al., 2019).
• Selective Tests for Clustering-Derived Pairs: Represent the selection event (from clustering) as a system of quadratic inequalities in the test statistic’s space, compute truncated law intervals, and evaluate exact or approximate tail probabilities (Chen et al., 2023, Yun et al., 2024).
• BTCT Algorithm: Identify boundary points from mutual nearest neighbors. For each, count label-matching neighbors; compute binomial $$H_0: \ell(z, \sigma) = 0 \text{ (identical clustering up to sign)} \quad \text{vs.} \quad H_1: \ell(z, \sigma) > \epsilon$$5-values and combine via Fisher’s method to get an overall $$H_0: \ell(z, \sigma) = 0 \text{ (identical clustering up to sign)} \quad \text{vs.} \quad H_1: \ell(z, \sigma) > \epsilon$$6-value (Liu et al., 11 Jul 2025).
• Distance-Aggregate Test: For each observation, compare its within-cluster and between-cluster normalized distances by chi-squared or permutation, aggregate $$H_0: \ell(z, \sigma) = 0 \text{ (identical clustering up to sign)} \quad \text{vs.} \quad H_1: \ell(z, \sigma) > \epsilon$$7-values, and set rejection via a mean threshold (Modak, 20 May 2026).
• Spectral Method for Graphs: Approximate endpoint distributions by repeated random walks, estimate $$H_0: \ell(z, \sigma) = 0 \text{ (identical clustering up to sign)} \quad \text{vs.} \quad H_1: \ell(z, \sigma) > \epsilon$$8-norms and inner-products, compute eigenvalues of constructed minors, and reject the clusterable hypothesis if eigenvalues exceed a threshold (Silwal et al., 2018).
• ANOCVA: For population clustering comparisons (e.g., in neuroimaging), average dissimilarity matrices by group, compute silhouette features, form an omnibus deviation statistic, and bootstrap the null by resampling subject-level matrices to obtain $$H_0: \ell(z, \sigma) = 0 \text{ (identical clustering up to sign)} \quad \text{vs.} \quad H_1: \ell(z, \sigma) > \epsilon$$9-values (Fujita et al., 2013).

5. Applications and Empirical Lessons

• Clustering Validation and Significance: Selective two-cluster tests are critical in post-hoc validation of clustering solutions, single-cell omics (detecting gene expression differences between inferred cell subpopulations), and interpretable tree-based/disjunctive clustering (Chen et al., 2023, Liu et al., 11 Jul 2025).
• Decision-Tree and Hierarchical Clustering: Incorporating two-cluster significance tests controls over-splitting, resulting in interpretable trees with the correct (or near-correct) number of clusters. BTCT avoids the rampant false discoveries typical with unadjusted two-sample procedures (Liu et al., 11 Jul 2025).
• Regression and Clustered Inference: Deciding the appropriate level of clustering in regression errors is addressed by score-variance tests, with wild bootstrap calibration recommended in small-cluster regimes (MacKinnon et al., 2023). Analytic min–max corrections provide uniform validity in non-Gaussian clustering scenarios (Davezies et al., 25 Jun 2025).
• Graph Theory and Planarity: Testing for two-cluster planarity is efficiently characterized via parity-vector solvability over $\ell$0 and is fundamentally easier and more stable than higher $\ell$1 (Fulek et al., 2013).
• Statistical Genetics, fMRI, Ecology: ANOCVA and population clustering structure tests offer formal detection and localization of group-specific structure in multi-subject/multi-feature settings (Fujita et al., 2013).

6. Limitations, Assumptions, and Future Directions

• Assumptions: Many two-cluster tests require assumptions such as Gaussianity (or its analogue, e.g., exchangeable dissimilarity in ANOCVA), cluster separation, or bounded cluster size relative to sample size. Some approaches require known variance (Yun et al., 2024), finite moments and independence (Patton et al., 2019), or fixed clustering algorithms.
• Selective Conditioning Necessity: Conditioning on the exact observed clustering is necessary for valid inference, as sample splitting or unconditional permutation approaches either lose power or fail to control the selective type I error (Chen et al., 2023, Yun et al., 2024).
• Computation: Quadratic inequality representation and interval computation for conditional tests can be computationally intensive for very large $\ell$2, though parallelization and pruning can substantially mitigate cost (Chen et al., 2023).
• Multi-Cluster Generalization: Extensions to $\ell$3-cluster settings, as in the distribution testing literature (Kumar et al., 9 Dec 2025), typically involve more complex selection and type I error control schemes.

A plausible implication is that rigorous two-cluster testing methodologies enable valid post-selection inference in clustering and partitioning problems across a wide spectrum of statistical and data-scientific domains, offering principled error control as well as insights into partition structure and group differences not attainable with naive or classical approaches.