Hierarchical Cluster-Class Matching (HCCM)

• Hierarchical Cluster-Class Matching (HCCM) is a method that translates nested speaker recognition embeddings into semantic labels such as gender, nationality, and identity.
• It employs a greedy matching algorithm using F-score and Liebig’s score to balance precision and recall when associating hierarchical clusters with predefined semantic classes and conjunctive divisions.
• The approach reveals a multi-layered structure in embeddings, from broad gender splits to finer identity groupings, offering actionable insights for explainable AI in speaker recognition.

Hierarchical Cluster-Class Matching (HCCM) is a cluster–class matching algorithm introduced for explainable analysis of speaker-recognition embeddings. In the formulation developed in "Explainable AI in Speaker Recognition -- Making Latent Representations Understandable," HCCM performs one-to-one matching between predefined semantic classes and hierarchical representation clusters produced by SLINK or HDBSCAN, thereby translating unlabeled hierarchical clusters of embeddings into human semantic concepts such as gender, nationality, identity, and conjunctions such as "male and UK" or "female and Ireland" (Xu et al., 25 Apr 2026). The method is paired with Liebig’s score, a diagnostic metric defined as the minimum of precision and recall, so that a matched cluster can be interpreted not only semantically but also in terms of the factor that most strongly limits match quality.

1. Problem setting and conceptual role

The method is introduced in a study of a speaker recognition network built from ResNet34, prototypical contrastive loss, and VoxCeleb2, with speaker embeddings extracted from the penultimate layer (Xu et al., 25 Apr 2026). Applying hierarchical clustering to those embeddings reveals what the paper calls inner hierarchical clustering: embeddings do not merely form flat clusters, but nested cluster trees. This shifts the central explainability question from whether clustering exists to what the hierarchical clusters mean in terms of known semantic attributes.

Earlier analyses of network representations had used algorithms such as t-distributed Stochastic Neighbour Embedding and K-means to study flat clustering phenomena. HCCM is designed for the distinct case in which the representation space exhibits hierarchical clustering phenomena. Its purpose is therefore not global evaluation alone, but cluster-level semantic interpretation. In that role it extends the cluster-class matching perspective associated with CCM by assigning semantic labels to individual hierarchical clusters rather than only reporting a single overall alignment score.

A central feature of the method is that it treats semantic interpretation as an assignment problem between two kinds of objects: predefined semantic divisions derived from dataset labels, and hierarchical clusters derived from the latent representation space. The output is a partly semantically labeled hierarchy in which matched clusters can correspond either to individual semantic classes or to conjunctions of semantic classes.

2. Semantic classes, conjunctive classes, and hierarchical clusters

The semantic side of HCCM is defined from the VoxCeleb1 test set, which provides labels for speaker identity, gender, and nationality (Xu et al., 25 Apr 2026). The individual semantic classes comprise 40 identity classes, 2 gender classes, and 12 nationality classes, for a total of 54 individual classes. For any semantic class $i$, the predefined representation division is

$C = \{ c_1, \dots, c_{n_{\text{class}}} \},$

where each $c_i$ is the set of indices of representations in the dataset $DB$ whose input utterances have class label $i$.

HCCM also admits and-logic-based conjunctive semantic classes. Starting from the index set of individual classes $I = \{1,\dots,n_{\text{class}}\}$, the conjunctive classes are defined as pairwise intersections:

$$\mathcal{C} = \bigl\{\, c_T \mid T \subseteq I,\; |T| = 2,\; c_T \neq \varnothing \bigr\}, \quad c_T = \bigcap_{t \in T} c_t.$$

In the experiments, $\mathcal{C}$ is constructed by intersecting 2 gender classes with 12 nationality classes, yielding up to 24 non-empty gender$\times$nationality conjunctive classes. The full semantic inventory used by HCCM is therefore 54 individual classes plus 24 non-empty conjunctive classes, for a total of 78 semantic divisions.

The cluster side of HCCM is derived from penultimate-layer embeddings extracted at four audio lengths: 0.2 s, 1 s, 2 s, and 4 s. For each duration, the study applies SLINK and HDBSCAN. SLINK is implemented via HDBSCAN with minPts=0, which reduces to plain SLINK with Euclidean distance, while HDBSCAN is evaluated for minPts values $2,4,6,8,12,16,21,27$ and is described as MST-based SLINK in mutual reachability distance space (Xu et al., 25 Apr 2026). Both methods induce nested clusters parameterized by a threshold $C = \{ c_1, \dots, c_{n_{\text{class}}} \},$0, or equivalently $C = \{ c_1, \dots, c_{n_{\text{class}}} \},$1. Pruning MST edges longer than $C = \{ c_1, \dots, c_{n_{\text{class}}} \},$2 gives a flat clustering; as $C = \{ c_1, \dots, c_{n_{\text{class}}} \},$3 decreases, clusters split, and the sequence of splits defines the hierarchy.

The set of all hierarchical clusters is written as

$C = \{ c_1, \dots, c_{n_{\text{class}}} \},$4

where each $C = \{ c_1, \dots, c_{n_{\text{class}}} \},$5 is a set of indices of embeddings in $C = \{ c_1, \dots, c_{n_{\text{class}}} \},$6 that form a hierarchical cluster at some level of the tree. Because $C = \{ c_1, \dots, c_{n_{\text{class}}} \},$7, there are many more hierarchical clusters than semantic classes. The paper emphasizes that HCCM does not directly use parent–child structure in the matching algorithm; each hierarchical cluster is treated as a set $C = \{ c_1, \dots, c_{n_{\text{class}}} \},$8. Nevertheless, the matched clusters, when plotted on the dendrogram, expose a hierarchy of semantic granularity in which higher levels can correspond to gender, intermediate levels to conjunctions such as nationality and gender, and lower levels potentially to identity.

3. Greedy matching procedure and scoring functions

HCCM takes the union of individual and conjunctive semantic divisions,

$C = \{ c_1, \dots, c_{n_{\text{class}}} \},$9

and greedily selects cluster–class pairs that maximize a cluster–class matching score (Xu et al., 25 Apr 2026). The formal procedure is

$c_i$0

This yields a sequence of best class–cluster pairs $c_i$1. The assignment is one-to-one in the class dimension: each semantic class is used at most once. A cluster can, in principle, be reused across multiple classes, although the interpretation focuses on distinct pairs with high match quality.

The matching step uses precision, recall, and F-score. For a class division $c_i$2 and a cluster $c_i$3,

$c_i$4

Precision measures the proportion of embeddings in cluster $c_i$5 that belong to class $c_i$6, whereas recall measures the proportion of class-$c_i$7 embeddings captured by cluster $c_i$8. The F-score is

$c_i$9

Because it balances precision and recall, F-score is used for the greedy selection step.

The paper argues, however, that F-score is hard to interpret semantically, and therefore introduces Liebig’s score as a reporting and diagnosis metric:

$DB$0

Liebig’s score is the minimum of recall and precision. Its interpretation is explicitly diagnostic: if $DB$1 equals precision, the limiting factor is over-inclusion of irrelevant samples in the cluster; if it equals recall, the limiting factor is under-coverage of the class. The paper’s illustrative example matches "cluster 5" to the UK class with precision $DB$2, recall $DB$3, F-score approximately $DB$4, and L-score $DB$5, making recall the limiting factor and indicating that 27% of UK samples are not retrieved by the cluster. In the dendrogram visualization, only cluster–class pairs with $DB$6-score $DB$7 are shown.

4. Integration with SLINK and HDBSCAN

The experimental pipeline begins with embedding extraction from the penultimate layer of the trained ResNet34 for 0.2 s, 1 s, 2 s, and 4 s audio using the VoxCeleb1 test set, with each embedding inheriting speaker ID, gender, and nationality labels (Xu et al., 25 Apr 2026). Hierarchical clustering is then performed with SLINK and HDBSCAN across the specified minPts settings.

Before HCCM is applied, the paper uses CCM as a global evaluation step. For each clustering result, the overall matching degree is computed between hierarchical clusters and predefined divisions of identity classes, nationality classes, and gender classes, using both F-score-based and L-score-based CCM overall matching degrees. The reported result is that the best scores are consistently for SLINK on 4-second audio embeddings, often near 1.0 for identity and gender. That case is then selected for further interpretation with HCCM.

HCCM is applied to the SLINK hierarchy on 4-second embeddings using all 78 semantic divisions and the complete set of hierarchical clusters $DB$8. Best class–cluster pairs are selected by F-score, then interpreted with L-score. For readability, the dendrogram visualization keeps only hierarchical clusters containing at least 800 embeddings and annotates clusters with icons for matched class type and text labels showing whether precision or recall is the limiting factor.

The resulting dendrogram reveals a layered structure. At the top level, the root cluster splits into two large clusters matched to the individual gender classes male and female. The male cluster has an L-score limited by precision, with 2% of members not male, corresponding to precision around 0.98; the female cluster is similarly limited by precision, with 1% of members not female, corresponding to precision around 0.99. The paper therefore interprets the highest-level grouping as gender.

Below the male cluster, HCCM identifies mid-level clusters matched to conjunctive classes such as India & male and USA & male. The USA & male cluster has an L-score limited by precision at around 0.65, meaning that 35% of cluster members are not USA & male. Below that cluster, further splits are matched to Mexico & male, Ireland & male, UK & male, and Canada & male. The UK & male cluster has L-score $DB$9, limited by precision, while the Canada & male cluster has L-score $i$0, limited by recall. One cluster matches Ireland & male perfectly, with L-score $i$1, so both precision and recall are $i$2.

Below the female cluster, one child cluster is matched as USA & female. The paper notes an asymmetry: unlike UK & male, which appears nested under USA & male, the UK & female cluster does not appear under USA & female but elsewhere in the hierarchy. The UK & female cluster has L-score around $i$3, limited by recall. The study also reports union-like cases in which a single cluster is simultaneously the best match for France & female and Norway & female, and another shared cluster for Canada & female and Ireland & female. These are interpreted as cases where the cluster behaves like a union of multiple conjunctive classes rather than a clean realization of a single semantic class.

5. Interpretation, limitations, and possible extensions

The principal interpretive claim of HCCM is that the speaker-recognition embedding space exhibits a non-trivial hierarchical organization whose levels align with semantic attributes (Xu et al., 25 Apr 2026). The observed layering is summarized as high-level gender structure near the root, mid-level gender$i$4nationality conjunctive structure beneath it, and lower-level groupings that are plausibly related to identity and finer distinctions. The explicit inclusion of conjunctive classes is critical in this interpretation, because many mid-level clusters cannot be well characterized by gender-only or nationality-only labels, yet align well with combinations such as Ireland & male.

Liebig’s score adds a diagnostic dimension to that interpretation. A cluster with L-score limited by precision is broad or impure; a cluster with L-score limited by recall captures only part of the intended class. The dendrogram annotations therefore do not merely name clusters, but state why the match is imperfect. This suggests a form of cluster-level semantic auditing that is different in kind from global clustering scores.

The study also reports a correlation between task performance and hierarchical clustering quality. The best hierarchical clustering, as judged by CCM and HCCM, occurs for 4-second audio, which is also where the network has the best recognition performance, described as the lowest EER. For 0.2-second audio, where EER is high, hierarchical clustering quality is poor. A plausible implication is that better recognition performance is associated with more coherent and semantically structured inner hierarchical clustering.

The paper identifies several limitations. Not all hierarchical clusters are interpretable using the chosen semantic classes, partly because $i$5. Some classes are poorly captured, as illustrated by the UK & female case with L-score around $i$6 limited by recall. Some clusters correspond to unions of multiple conjunctive classes, so a single semantic label is ambiguous. The results are also sensitive to clustering algorithm and hyperparameters: SLINK with Euclidean distance gives better alignment than HDBSCAN with mutual reachability distance, and increasing HDBSCAN’s minPts tends to decrease overall matching degree. Finally, the greedy matching rule does not guarantee a globally optimal assignment across all classes, and the semantic coverage is restricted to gender, nationality, and identity rather than other latent factors such as accent, recording conditions, or age.

Possible extensions mentioned in the study include applying HCCM to other tasks or models, using more complex conjunctive or higher-order semantic classes, and connecting the resulting hierarchical interpretations with human studies in psychology, linguistics, or vocal pedagogy.

6. Position in the literature and acronym ambiguity

Within the explainable-AI landscape, HCCM is positioned as a representation-space method rather than an input-saliency or output-logit method (Xu et al., 25 Apr 2026). It analyzes latent embeddings, serves as a hierarchical counterpart to flat-clustering analyses based on k-means and t-SNE or UMAP, and refines CCM by moving from global matching degrees to cluster-wise semantic interpretation. Unlike t-SNE or UMAP visualization alone, it directly ties clusters to semantic labels with explicit scores; unlike attention or saliency methods such as Grad-CAM, it explains how representations are organized rather than which input features are important for a particular decision.

The acronym is not unique across recent arXiv literature. In "HCCM: Hierarchical Cross-Granularity Contrastive and Matching Learning for Natural Language-Guided Drones," HCCM denotes Hierarchical Cross-Granularity Contrastive and Matching learning, a vision-language training framework built around Region-Global Image-Text Contrastive Learning, Region-Global Image-Text Matching, and Momentum Contrast and Distillation rather than cluster–class matching in the speaker-recognition sense (Ruan et al., 29 Aug 2025). In "Hierarchical clustered multiclass discriminant analysis via cross-validation," a conceptually related class-merging procedure appears in an LDA framework, where classes are agglomeratively merged into metaclasses using leave-one-out cross-validation error as the dissimilarity measure (Hirose et al., 2021). By contrast, "Hierarchical Conformal Classification" develops hierarchical prediction sets with conformal coverage guarantees and is concerned with node-valued prediction sets over class hierarchies rather than cluster–class matching of latent representations (Hengst et al., 18 Aug 2025).

In the strict sense defined by the speaker-recognition study, Hierarchical Cluster-Class Matching names a greedy matching procedure for assigning semantic meaning to hierarchical clusters of neural embeddings. Its distinctive elements are the use of predefined individual and conjunctive semantic divisions, class-wise one-to-one greedy assignment by F-score, and post hoc diagnosis by Liebig’s score. In that formulation, HCCM is a method for labeling and interpreting hierarchical clustering phenomena in latent spaces, rather than a general synonym for any hierarchical matching method.