Topeax Model: Robust Topic Discovery

• Topeax is a topic modeling framework that integrates density-peak clustering with a hybrid lexical-semantic scoring algorithm to extract stable and interpretable topics.
• The method projects dense document embeddings to 2D with t-SNE and applies kernel density estimation followed by Gaussian mixture modeling to reliably recover natural cluster structures.
• By fusing semantic similarity with NPMI-based lexical importance, Topeax produces contextually relevant keywords, addressing limitations seen in models like Top2Vec and BERTopic.

Topeax is a topic modeling and text clustering framework that advances over prevalent embedding-based clustering models—such as Top2Vec and BERTopic—by introducing both robust, hyperparameter-stable cluster discovery via density-peak analysis in 2D embedding space and a hybrid lexical-semantic keyword scoring algorithm. The approach is positioned to resolve core limitations in the reliability and interpretability of unsupervised topic structures in large corpora, facilitating more faithful recovery of natural clusters and extractive cluster description (Kardos, 29 Jan 2026).

1. Motivations and Problem Setting

Contemporary topic modeling pipelines, such as Top2Vec and BERTopic, deploy dense document embeddings (typically from Sentence-Transformers) and discover clusters/topics by projecting these embeddings to low-dimensional spaces (using UMAP or t-SNE) followed by density-based clustering (e.g., HDBSCAN). These methods then describe clusters using keywords selected by distinct, often one-sided criteria: centroid proximity for Top2Vec, and class-based TF–IDF (c-TF–IDF) for BERTopic.

These models, however, exhibit two principal deficiencies:

• Erratic topic count discovery: The number of clusters found by conventional methods is highly sensitive to embedding manifold hyperparameters, HDBSCAN settings (e.g., min_cluster_size), and sample size. This frequently yields dramatic overestimation or, less commonly, underestimation of the true number of clusters.
• Unidimensional term importance: Top2Vec’s use of centroid proximity fails to capture corpus-level word frequency, often introducing “junk” or stop words. BERTopic’s c-TF–IDF approach, conversely, ignores the semantic relatedness of words to the cluster centroid, yielding less coherent, sometimes semantically irrelevant keywords.

Topeax is designed to rectify these issues by (i) replacing HDBSCAN with a density-peak discovery and mixture modeling strategy that is robust to sample size and parameter choice, and (ii) fusing lexical and semantic scoring of terms to achieve balanced, contextually relevant cluster descriptors (Kardos, 29 Jan 2026).

2. Density Peak-Based Cluster Detection

The core clustering engine in Topeax is the Peax algorithm, which proceeds via the following stages:

2.1 Dimensionality Reduction

Dense document embeddings $x_1,\dots,x_n\in \mathbb{R}^d$ (e.g., all-MiniLM-L6-v2) are projected to 2D using t-SNE configured with cosine distance and a perplexity parameter typically set near 50.

2.2 Kernel Density Estimation (KDE)

A Gaussian kernel is placed over each point in this 2D space. The density at location $x$ is estimated by

$$\hat{f}(x) = \frac{1}{n h^d} \sum_{i=1}^n K\Bigl(\frac{x-x_i}{h}\Bigr)$$

with $K(u) = (2\pi)^{-d/2}e^{-\|u\|^2/2}$ (Gaussian), $h$ chosen via Scott’s rule.

2.3 Density Peak Detection

The estimated density field is evaluated on a $100\times 100$ grid. A local-maximum filter identifies grid cells where density values are maximal within a 5-unit neighborhood (radius, connectivity 25). Peaks below a threshold $\tau$ may be pruned.

2.4 Gaussian Mixture Approximation and Assignment

Each density peak $\mu_k$ anchors a component in a $K$-component Gaussian mixture model (GMM, with fixed means $\mu_k$ and shared diagonal covariance). Documents $x$0 are assigned to the topic with maximum posterior responsibility, but the soft-responsibility structure enables modeling of non-spherical cluster shapes and better reflects underlying corpus structure than HDBSCAN thresholding or centroid-based assignments.

Comparative Table: Clustering Approaches

Model	Topic Count Discovery	Cluster Assignment
Top2Vec	HDBSCAN (low-dim, UMAP)	K-means/centroid
BERTopic	HDBSCAN (low-dim, UMAP)	Majority assignment
Topeax	KDE density peaks (t-SNE)	GMM (fixed means)

3. Lexical-Semantic Term Importance

After clusters are discovered, Topeax computes topic-descriptive keywords for each cluster by integrating both semantic proximity and lexical distinctiveness.

3.1 Semantic Importance

For each topic $x$1, a prototype vector $x$2 is computed as the responsibility-weighted average of document embeddings: $x$3 where $x$4 denotes the posterior responsibility of component $x$5 for document $x$6. For term $x$7 (with embedding $x$8), semantic importance is

$x$9

3.2 Lexical Importance via NPMI

Let $$\hat{f}(x) = \frac{1}{n h^d} \sum_{i=1}^n K\Bigl(\frac{x-x_i}{h}\Bigr)$$0 be the total occurrences of term $$\hat{f}(x) = \frac{1}{n h^d} \sum_{i=1}^n K\Bigl(\frac{x-x_i}{h}\Bigr)$$1, $$\hat{f}(x) = \frac{1}{n h^d} \sum_{i=1}^n K\Bigl(\frac{x-x_i}{h}\Bigr)$$2 its count in topic $$\hat{f}(x) = \frac{1}{n h^d} \sum_{i=1}^n K\Bigl(\frac{x-x_i}{h}\Bigr)$$3. With a symmetric Dirichlet$$\hat{f}(x) = \frac{1}{n h^d} \sum_{i=1}^n K\Bigl(\frac{x-x_i}{h}\Bigr)$$4 prior ($$\hat{f}(x) = \frac{1}{n h^d} \sum_{i=1}^n K\Bigl(\frac{x-x_i}{h}\Bigr)$$5 = 2), unigram marginals are estimated: $$\hat{f}(x) = \frac{1}{n h^d} \sum_{i=1}^n K\Bigl(\frac{x-x_i}{h}\Bigr)$$6 where $$\hat{f}(x) = \frac{1}{n h^d} \sum_{i=1}^n K\Bigl(\frac{x-x_i}{h}\Bigr)$$7 are corpus, vocabulary, and topic-level counts, respectively. Pointwise mutual information is

$$\hat{f}(x) = \frac{1}{n h^d} \sum_{i=1}^n K\Bigl(\frac{x-x_i}{h}\Bigr)$$8

normalized to [–1,1]: $$\hat{f}(x) = \frac{1}{n h^d} \sum_{i=1}^n K\Bigl(\frac{x-x_i}{h}\Bigr)$$9

3.3 Geometric Fusion

Both $K(u) = (2\pi)^{-d/2}e^{-\|u\|^2/2}$0 and $K(u) = (2\pi)^{-d/2}e^{-\|u\|^2/2}$1 are min–max normalized to $K(u) = (2\pi)^{-d/2}e^{-\|u\|^2/2}$2. The overall importance score is

$K(u) = (2\pi)^{-d/2}e^{-\|u\|^2/2}$3

with $K(u) = (2\pi)^{-d/2}e^{-\|u\|^2/2}$4, $K(u) = (2\pi)^{-d/2}e^{-\|u\|^2/2}$5 the normalized scores. Ranking terms by $K(u) = (2\pi)^{-d/2}e^{-\|u\|^2/2}$6 selects cluster keywords that are simultaneously semantically proximal to the topic centroid and lexically distinctive.

4. End-to-End Pipeline

The Topeax workflow comprises the following steps:

1. Input raw corpus.
2. Compute document embeddings using a Sentence-Transformer.
3. Project embeddings to 2D via t-SNE (cosine distance, perplexity ≈ 50).
4. Estimate KDE over the 2D space (Gaussian kernel, bandwidth by Scott’s rule).
5. Detect density peaks via local-max filtering.
6. Fit Gaussian mixture with peaks as fixed means; compute responsibilities.
7. Assign each document to its top-responsibility topic.
8. Compute topic prototype vectors.
9. For all vocabulary terms, compute semantic similarity scores. 10. Compute empirical term frequencies and NPMI scores.
10. Min–max normalize and geometrically fuse scores to compute $K(u) = (2\pi)^{-d/2}e^{-\|u\|^2/2}$7.
11. Extract top-N terms per topic based on $K(u) = (2\pi)^{-d/2}e^{-\|u\|^2/2}$8 (Kardos, 29 Jan 2026).

5. Quantitative Evaluation

5.1 Cluster Recovery

Performance on seven gold-labeled corpora from the MTEB v2 suite indicates Topeax achieves a mean Fowlkes–Mallows index (FMI) of ≈0.45, markedly higher than Top2Vec (≈0.30) and BERTopic (≈0.32).

5.2 Topic Count Estimation

Topeax exhibits lower mean absolute percentage error (MAPE) in the number of class predictions (≈60.5%, SD 26.2), in contrast to much higher error from Top2Vec (≈1797%, SD 2622) and BERTopic (≈2439%, SD 3012), signaling reduced “topic inflation” and improved correspondence to true class structure.

5.3 Topic Quality Metrics

Assessments of internal coherence $K(u) = (2\pi)^{-d/2}e^{-\|u\|^2/2}$9 (GloVe on-corpus), external coherence $h$0 (word2vec-GoogleNews), diversity $h$1, and interpretability $h$2 ($h$3) yield:

Model	$h$4	$h$5	$h$6	$h$7
Topeax	$h$8	$h$9	$100\times 100$0	$100\times 100$1
Top2Vec	$100\times 100$2	$100\times 100$3	$100\times 100$4	$100\times 100$5
BERTopic	$100\times 100$6	$100\times 100$7	$100\times 100$8	$100\times 100$9

6. Sensitivity, Strengths, and Limitations

Hyperparameter Sensitivity

Topeax’s density-peak and fusion pipeline stabilizes topic count and interpretability at small sample sizes ($\tau$0) and modest t-SNE perplexity settings (convergence by perplexity $\tau$1 for topic count), outperforming contemporary alternatives which experience proliferating topic inflation as corpus size or manifold parameters grow.

Strengths

• Principled cluster count discovery via density peaks, reducing reliance on unstable HDBSCAN heuristics.
• Topic keywords reflect joint lexical and semantic discriminability.
• Robustness to hyperparameter and sample size variation, advantageous for non-manual and automated workflows.

Limitations

• Systematic underestimation of the number of topics versus gold-label counts, though unions often reflect meaningful merges of related categories.
• Not directly applicable to online or streaming data; proper operation requires batch computation of t-SNE and KDE.

Potential Extensions

A plausible implication is that adoption of incremental or parametric t-SNE/UMAP, adaptive density thresholds, and user/semi-supervised intervention in peak merging/splitting could address present limitations and widen use cases (Kardos, 29 Jan 2026).

7. Summary and Context

Topeax supersedes the dominant UMAP + HDBSCAN-based topic modeling frameworks by integrating a density-peak-based cluster discovery apparatus with a geometric fusion of NPMI-based lexical and embedding-based semantic term importance. The resulting model demonstrates state-of-the-art performance on unsupervised cluster fidelity, topic description, and interpretability metrics across a range of benchmark datasets while retaining practical robustness against key sensitivity factors that undermine existing approaches (Kardos, 29 Jan 2026).