Pairwise Similarity Distribution Clustering

• PSDC is a clustering framework that uses full pairwise similarity or discrepancy distributions to drive robust and fair cluster assignments.
• It integrates spectral methods, Gaussian mixture models, graph-based shifts, and Bayesian priors to address noisy-label learning and bias correction.
• Practical applications of PSDC include data debiasing, balanced graph clustering, and functional data partitioning, with strong empirical performance on fairness and accuracy benchmarks.

Pairwise Similarity Distribution Clustering (PSDC) refers to a family of clustering frameworks unified by the centrality of pairwise similarity (or discrepancy) distributions as the mathematical foundation for cluster assignment. Across current literature, PSDC encompasses deterministic spectral methods for data debiasing, robust sample assignment under label noise, regularized graph partitioning, and nonparametric Bayesian models incorporating similarity kernels. Its implementations target problems from fairness-aware data preprocessing and sample-cleansing in noisy-label learning, to functional-data partitioning and balanced graph clustering. This entry synthesizes PSDC methodologies as documented in contemporary research, including algorithmic protocol, mathematical formalism, hyperparameter governance, experimental benchmarks, and noted strengths and limitations.

1. Fundamental Methodologies in PSDC

The defining element in PSDC is leveraging the full distribution (or structure) of pairwise similarities or discrepancies among subjects, features, or datasets to drive cluster formation. The core approaches include:

• Affinity-based Spectral Clustering using Maximum Mean Discrepancy (MMD): For multi-dataset bias correction, PSDC starts by evaluating all-pairs distributional discrepancies using MMD, a kernel-based nonparametric distance, to construct a symmetric distance matrix. MMD is given by

$$\mathrm{MMD}^2(P, Q) = \lVert \mu_P - \mu_Q \rVert^2_{\mathcal{H}},$$

with practical computation via empirical kernel averages. Distance matrices are then transformed through a Gaussian kernel to define affinities.

• Sample-level Pairwise Similarity and Gaussian Mixture Model (GMM): In noisy-label learning, PSDC computes pairwise similarities (cosine affinities in feature space) among samples with the same observed label. The row-summed “total-affinity” vector for each sample is modeled using a 2-component GMM, segmenting samples by whether their similarity to the putative cluster is statistically robust.
• Shifted Similarity for Graph-clustering: PSDC regularizes clustering via direct shifts to the pairwise similarity matrix, resulting in balanced cluster granularity. The shift may be adaptive, per-edge, ensuring all rows and columns sum to zero, creating an equivalence with Correlation Clustering.
• Bayesian Nonparametric PSDC via Similarity-weighted Random Partitions: The Similarity-based Generalized Dirichlet Process (SGDP) introduces pairwise similarities as stochastic weights directly within the probability of cluster assignments, linking the random partition probability to the underlying pairwise similarity structure.

2. Algorithmic Schemes and Theoretical Underpinnings

All PSDC variants implement a multi-step protocol in which pairwise measurements inform affinity (or discrepancy) matrices that are subsequently used in clustering or assignment, often with embedded regularization or probabilistic modeling:

• Spectral PSDC Pipeline (data debiasing): Compute pairwise discrepancies, convert to affinities, build a graph Laplacian, perform eigendecomposition, select the number of clusters by the largest spectral gap, embed clusters, and execute k-means in embedded space. Clustering result justifies data augmentation by borrowing real samples from similar clusters, effectively balancing group representation (Ghodsi et al., 2023).
• PSDC for Noisy-Label Learning: Feature-extracted representations are grouped per class, cosine similarities yield affinity matrices, row-sums define affinity scores, and GMM splits clean from noisy samples robustly. Theoretical backing is provided by a central limit theorem for the sum of affinities and mixture separation guarantees (Bai, 2 Apr 2024).
• Shifted Min-Cut PSDC: The clustering objective is the sum of Min-Cut and a cluster-size penalty, algebraically equivalent to a Min-Cut on a shifted similarity matrix. Adaptive shifts enforce row/column centralization, eliminating singleton or superclusters, and admitting efficient local search or Frank–Wolfe optimization with $O(1/t)$ convergence to stationary points (Chehreghani, 2021).
• Similarity-weighted Priors in Bayesian PSDC: Cluster assignment probabilities in the SGDP prior are proportional to weights derived from normalized pairwise similarities, allowing arbitrary similarity kernels. The Gibbs sampling updates cycle through clusters, assignments, and hyperparameters, maintaining full Bayesian uncertainty (Wakayama et al., 2023).

3. Implementational Features, Hyperparameterization, and Procedural Details

Specific algorithmic steps and critical hyperparameters vary by use-case:

PSDC Variant	Pairwise Metric	Key Parameters/Steps
MMD-based Spectral	MMD (linear/RBF)	Kernel choice, bandwidth $\gamma$ by median
Noisy-label GMM	Cosine similarity	GMM cutoff $d_\text{cutoff}$, Mixup $\alpha$
Shifted Graph-based	Arbitrary similarity	Shift $\alpha$ (mean or adaptive), cluster $K$
Bayesian SGDP	Any kernel	$(\alpha, \beta)$, similarity kernel $\lambda$

Other salient details:

• Spectral methods often set $\gamma=1/\text{median}(W_{ij}^2)$.
• GMM partitioning thresholds (e.g., $d_\text{cutoff}=0.9$) may be bootstrapped early with alternative criteria until features mature.
• Local-search in shifted-similarity methods is made computationally efficient by incremental updates, allowing $O(n^2)$ per pass.
• In SGDP, adjacency or other domain-relevant kernels can be incorporated, with mixing weights for composite similarities.

4. Empirical Findings and Performance Benchmarks

Empirical investigations across the surveyed PSDC methods demonstrate substantial practical benefits:

• Data Debiasing: Applied to U.S. Adult census state-level tasks, PSDC achieves ideal (33%) group ratios across racial groups post-augmentation, with associated improvements in Statistical Parity, Disparate Impact, and Equalized Odds. PSDC generally outperforms or matches established augmentation baselines (geographic neighbors, SMOTE, RUS) in fairness metrics and test accuracy (Ghodsi et al., 2023).
• Noisy-Label Learning: On CIFAR-10/100 and Clothing1M, PSDC surpasses DivideMix and JSD-based selection in robust test accuracy under both symmetric and asymmetric synthetic and real-world label noise. Ablations confirm that modeling the distribution of pairwise similarities is integral—naive k-means or per-feature GMM are less effective (Bai, 2 Apr 2024).
• Graph-Based Clustering: Across 11 UCI datasets and large document clustering tasks, Shifted-Min-Cut PSDC is consistently top-performing on adjusted Mutual Information and Rand Index, with superior runtime to standard k-means or GMM in moderate-to-large $n$ (Chehreghani, 2021).
• Functional Data (SGDP-PSDC): On spatiotemporal population flow in Tokyo, PSDC identifies semantically coherent, spatially meaningful clusters matching urban typologies, controls cluster overabundance, and achieves parity with or improvement over standard GDP/SDP random partition models (Wakayama et al., 2023).

5. Limitations, Extensions, and Applicability

Key caveats and possible extensions for PSDC techniques are documented:

• Covariate Shift Assumption: For MMD-based methods, the assumption that $P(Y|X)$ is stable is critical. Concept drift or label shift requires joint embedding or joint divergence computation (Ghodsi et al., 2023).
• Computational Complexity: Quadratic scaling in the number of samples or tasks for pairwise affinity computation is an issue; scalable variants include linear-time approximations or random features (Ghodsi et al., 2023, Bai, 2 Apr 2024).
• Cluster Granularity: Unnormalized Laplacians can yield unbalanced clusters; normalization or minimum cluster size enforcement may be necessary (Ghodsi et al., 2023).
• Hyperparameter Sensitivity: Especially for regularization strength and GMM thresholds; careful selection or adaptive/parameter-free strategies (as in adaptive shifting) are preferred (Chehreghani, 2021).
• Bayesian and Online Extensions: Online updating and incorporating multiple kernels are supported in SGDP-PSDC, enabling temporal or streaming data applicability (Wakayama et al., 2023).

A plausible implication is that PSDC methods are adaptable to arbitrary domains where a meaningful pairwise similarity or discrepancy can be defined, including but not limited to text, genetics, and functional data.

6. Connections to Broader Clustering Paradigms

PSDC is conceptually linked to numerous established frameworks:

• Spectral clustering (with affinity matrices and Laplacian eigenmaps),
• Correlation Clustering (through shifted similarity matrix equivalence),
• Mixture models on similarity distributions (for sample-level partitioning under noise),
• Random partition models (through similarity-modulated Dirichlet processes),
• and Data augmentation under fairness constraints (using cluster structure to guide authentic data borrowing).

The unifying principle is the operationalization of pairwise similarity or discrepancy distributions in a central, mathematically explicit role for cluster structure discovery, either in deterministic, probabilistic, or Bayesian frameworks.

---

For reference to core contributions and empirical benchmarks on PSDC, see (Ghodsi et al., 2023, Bai, 2 Apr 2024, Chehreghani, 2021, Wakayama et al., 2023).