BERTopic: Neural Topic Modeling Framework

• BERTopic is a neural topic modeling framework that uses transformer-based embeddings and c-TF-IDF for context-aware topic extraction.
• It integrates UMAP for dimensionality reduction and HDBSCAN for flexible, non-parametric clustering, enabling robust analysis of complex datasets.
• Its modular design supports diverse applications, from financial text analysis to multilingual and low-resource topic modeling.

BERTopic is a neural topic modeling framework that leverages transformer-based LLMs and a class-based TF-IDF procedure to derive coherent and interpretable topics from large text corpora. Unlike earlier bag-of-words-based topic models, BERTopic decouples semantic representation from clustering and topic rendering, allowing for robust and context-aware analysis even in heterogeneous, dynamic, and domain-specific datasets.

1. Theoretical Foundations and Core Architecture

BERTopic was designed to address fundamental limitations of classical topic models such as Latent Dirichlet Allocation (LDA) and Non-Negative Matrix Factorization (NMF), which represent documents as samples from word co-occurrence distributions. Instead, BERTopic models document semantics using vector representations produced by pre-trained transformer-based LLMs (e.g., Sentence-BERT). This enables the capture of syntactic and semantic structure, idiomatic expressions, and domain-specific jargon in ways that traditional models cannot.

The architecture of BERTopic consists of three modular, sequential steps:

1. Semantic Embedding: Texts are converted into dense, context-rich embeddings using a transformer-based sentence encoder.
2. Dimensionality Reduction and Clustering: High-dimensional embeddings are reduced using Uniform Manifold Approximation and Projection (UMAP), followed by density-based clustering (HDBSCAN) to form topical clusters without a priori assumptions on the number or shape of clusters.
3. Topic Representation: Each cluster is treated as a “class document” for a class-based TF-IDF (c-TF-IDF) process that extracts representative words for interpretability.

Formally, the c-TF-IDF procedure substitutes standard document frequency with an inverse class frequency (across clusters). Specifically, for term $t$ and cluster $c$:

$$W_{t,c} = \mathrm{tf}_{t,c} \cdot \log\left(1 + \frac{A}{\mathrm{tf}_t}\right)$$

where $\mathrm{tf}_{t,c}$ is the term frequency in cluster $c$, $A$ is the average number of words per class, and $\mathrm{tf}_t$ is the global cluster frequency of $t$.

2. Representation and Clustering Methodologies

Semantic Embeddings

Document representation is achieved via transformer encoders (e.g., Sentence-BERT, RoBERTa, FinBERT for finance, HindSBERT for Hindi, etc.). These models yield high-dimensional dense embeddings, typically ranging from 384 to 1,024 dimensions. The modularity of BERTopic enables the use of domain-specific or multilingual models, which is critical for effective topic modeling in low-resource or morphologically rich languages and specialized contexts.

Dimensionality Reduction

UMAP projects these embeddings to a lower-dimensional space (frequently 5–100 dimensions). This preserves the topological and global structure necessary for effective clustering while combating the curse of dimensionality that plagues high-dimensional similarity measures. Key UMAP hyperparameters (e.g., n_neighbors, n_components, min_dist, metric) require tuning for balance between local and global semantic structure, as confirmed in multilingual and domain-specific settings (Schäfer et al., 11 Jul 2024).

Unsupervised Clustering

HDBSCAN is used post-reduction for density-based, non-parametric clustering. Unlike k-Means, which assumes spherical clusters of equal variance and requires a predefined number of clusters, HDBSCAN can handle clusters of varying shapes and densities, automatically identify outliers, and does not assign every document to a cluster—an important property when filtering noise or non-topical content. Parameter settings such as min_cluster_size and min_samples determine the granularity and sensitivity.

It has been noted that HDBSCAN’s aggressive outlier assignment can result in high document exclusion (e.g., up to 74% on short texts) (Groot et al., 2022); alternatives like k-Means can improve coverage at the expense of lower topic coherence and diversity.

3. Topic Construction, Evaluation, and Optimization

c-TF-IDF Topic Representation

After clustering, the c-TF-IDF procedure treats each cluster as a bag-of-words “class document.” This quantifies the importance of each word in the cluster, normalized by its presence across clusters. Mathematically:

$$c\text{-}TF\text{-}IDF(t,c) = TF(t, c) \times \log\left(\frac{N_c}{DF(t)}\right)$$

where $TF(t,c)$ is the term frequency in cluster $c$, $DF(t)$ is the number of clusters containing $t$, and $N_c$ is the total term count in cluster $c$ (Kandala et al., 20 Apr 2025). This yields interpretable topic-keyword tuples, facilitating downstream qualitative or quantitative analysis.

For enhanced topic diversity, techniques inspired by KeyBERT or maximal marginal relevance may be used to extract a diverse set of high-relevance keywords (Opu et al., 13 Jun 2025).

Topic Quality Metrics

Topic quality is typically assessed via:

• Topic Coherence (TC): Quantified using normalized pointwise mutual information (NPMI), c_v, u_mass, or c_uci, which measure the pairwise semantic similarity of topic keywords. For NPMI:

$$NPMI(w_i, w_j) = \frac{\log \frac{P(w_i, w_j)}{P(w_i)P(w_j)}}{-\log P(w_i, w_j)}$$

where $P(w_i), P(w_j)$ are marginal probabilities and $P(w_i, w_j)$ is the joint probability.

• Topic Diversity (TD): Fraction of unique words in the top-n words of each topic; higher is better.
• Silhouette Score: For clustering evaluation, measuring cohesion versus separation.

Recent work also uses contextual LLM-based coherence metrics (e.g., CTC_Intrusion, CTC_Rating via ChatGPT) for more human-aligned assessment (Schäfer et al., 11 Jul 2024).

4. Practical Applications and Extensions

BERTopic has demonstrated consistent performance and adaptability across diverse domains:

• Financial Texts: Use of FinBERT or FinTextSim embeddings yields well-separated, domain-specific topic clusters in consumer complaints and 10-K filings, outperforming generic models on metrics such as intra-topic similarity and coherence (Sangaraju et al., 2022, Jehnen et al., 22 Apr 2025).
• Short Text and Low-resource Languages: BERTopic’s reliance on contextual embeddings allows effective modeling even with minimal preprocessing in morphologically rich and low-resource settings such as Serbian and Hindi, outperforming LDA/NMF in both coherence and diversity (Medvecki et al., 5 Feb 2024, Mutsaddi et al., 7 Jan 2025).
• Hybrid and Hierarchical Frameworks: Multi-scale hybrid models combine NMF for coarse topic assignment and BERTopic for fine-grained, semantically-rich subtopic extraction, enabling multi-label topic assignments and substantially reducing computational cost (Cheng et al., 2022).
• Strategic and Personalized Recommendation: BERTopic facilitates program recommendations by mapping student interests (via user-chosen topic keywords) onto a knowledge graph of course-program relationships, enabling fair and explainable recommendations (Hill et al., 11 Jan 2025).
• Sentiment-augmented Prediction: Topic-level sentiment features derived from BERTopic improve downstream tasks such as stock price predictions when fused with deep learning models (Zhu et al., 2 Apr 2024).
• Domain-specific Maintenance: The approach is scalable, enabling unsupervised structuring, evolution analysis, and tool development insights in domains such as open source blockchain software and aviation safety (Opu et al., 13 Jun 2025, Nanyonga et al., 30 May 2025).

5. Comparative Performance and Empirical Findings

Across multiple benchmarks and domains, BERTopic demonstrates superior or competitive topic coherence and interpretability relative to classical probabilistic models (LDA, PLSA) and matrix factorization approaches (NMF):

• On CFPB data, BERTopic yielded higher topic coherence and distinctiveness, with domain-specific embeddings (FinBERT) further enhancing performance (Sangaraju et al., 2022).
• In aviation safety narratives, it achieved a c_v topic coherence score of 0.41 compared to 0.37 for PLSA and demonstrated better interpretability and scalability in expert validation (Nanyonga et al., 30 May 2025).
• For open-ended, morphologically complex corpora (e.g., Belgian Dutch), BERTopic produced more culturally and contextually resonant topics as confirmed by human evaluation, highlighting the limitations of co-occurrence-based evaluation metrics in such settings (Kandala et al., 20 Apr 2025).
• User studies in educational recommendation settings reported alignment of over 98% between program recommendations and user interests, with fairness and personalization metrics supporting system robustness (Hill et al., 11 Jan 2025).

6. Technical Innovations, Enhancements, and Current Limitations

Embedding Layer Optimization

Alternative embedding extraction strategies—such as aggregation or pooling from intermediate transformer layers—provide measurable gains in topic coherence and diversity across datasets, outperforming the default (last layer, mean pooling) configuration. Stop word removal prior to embedding enhances signal quality and further improves results. Configurations such as sum/concatenation of several recent layers with mean/max pooling often yielded the highest scores, while CLS pooling was consistently suboptimal (Koterwa et al., 10 May 2025).

Parameter Tuning

Performance is highly sensitive to hyperparameters throughout the pipeline:

• UMAP’s n_neighbors and n_components impact local/global structural preservation.
• HDBSCAN’s min_cluster_size and cluster_selection_method modulate granularity and outlier rates, requiring validation (e.g., via DBCV) for interpretable results.
• c-TF-IDF’s vocabulary restriction and outlier reduction steps have notable effects on topic granularity and coverage (Medvecki et al., 5 Feb 2024, Schäfer et al., 11 Jul 2024).

Integration with LLMs

BERTopic can be coupled with LLM-based summarization for automatic labeling and summarizing of topic clusters, yielding concise, human-interpretable topic titles and overviews. GPT-4 integration using prompt engineering has demonstrated significant improvements in both topic coherence and human-judged summary quality, with more distinct topic clusters and increased summarization relevance (Azhar et al., 8 Mar 2025).

Known Limitations

• Assigns each document to a single topic; multi-label and soft-assignments require hybrid frameworks or post-processing (Cheng et al., 2022).
• Topic representation still relies on bag-of-words aggregation, potentially underutilizing the full potential of contextual embeddings.
• HDBSCAN’s aggressive outlier filtering can result in excessive data exclusion on short or sparse texts; replacing with k-Means removes outliers but slightly reduces topic quality (Groot et al., 2022).

7. Research Directions and Implications

Ongoing and future work includes:

• Improved handling for documents with multiple overlapping topics through multi-label clustering or topic assignment smoothing (Schäfer et al., 11 Jul 2024).
• Robust embedding selection, layer aggregation, and adaptive preprocessing strategies for maximizing performance across languages and domains (Koterwa et al., 10 May 2025).
• Integration of hierarchical and hybrid modeling (e.g., NMF-BERTopic multi-scale approaches) to capture both broad and fine-grained thematic insights (Cheng et al., 2022).
• Domain-adaptation through further fine-tuning of transformer encoders (e.g., FinTextSim for finance) to maximize intra-topic cohesion and inter-topic separation (Jehnen et al., 22 Apr 2025).
• Event and time-evolving topic models leveraging dynamic BERTopic for tracking thematic developments (Grootendorst, 2022).

In summary, BERTopic represents a paradigm shift in unsupervised topic modeling, combining state-of-the-art transformer embeddings, nonparametric clustering, and an interpretable c-TF-IDF topic representation. Its flexible, modular structure and empirically validated adaptability position it as a central technique for large-scale, multilingual, and domain-specific text mining in contemporary NLP research and applied machine learning.