---
title: 'BERTopic: Neural Topic Modeling'
url: https://www.emergentmind.com/topics/bertopic-neural-topic-modeling
type: topic
---

# BERTopic: Neural Topic Modeling

BERTopic is a neural topic modeling framework that combines contextual transformer-based document embeddings, non-linear dimensionality reduction, density-based clustering, and class-based TF-IDF for interpretable topic representation. Unlike probabilistic generative models such as LDA or NMF, BERTopic leverages sentence-level semantic information through pre-trained transformer language models and thus enables flexible, high-coherence, and high-diversity discovery of latent topics, especially in short-text, low-resource, and multilingual settings [2203.05794][2502.02100][2402.03067][2505.06696].

## 1. Methodological Foundations and Model Pipeline

The BERTopic pipeline consists of four sequential stages:

1. **Contextual Document Embedding**  
   Each input document $d$ (sentence, paragraph, tweet, etc.) is mapped to a dense vector $\mathbf{e}_d \in \mathbb{R}^D$ using a pre-trained transformer such as Sentence-BERT (SBERT). Typical backbones include “all-mpnet-base-v2” (768-dim) or “all-MiniLM-L6-v2” (384-dim), with pooling across token representations (usually mean-pooling, but max-pooling and CLS pooling are options) [2505.06696]. The embedding model can be domain-specific, monolingual, or multilingual depending on the language and availability [2502.02100][2402.03067][2510.22904]. For morphologically rich or low-resource languages, monolingual or well-trained cross-lingual SBERT variants are preferred [2502.02100][2402.03067].

2. **Dimensionality Reduction (UMAP)**  
   Since transformer embeddings are high-dimensional, BERTopic applies UMAP (Uniform Manifold Approximation and Projection) to reduce $\{\mathbf{e}_d\}$ from $D$-dimensional space to $\mathbb{R}^d$ with $5 \leq d \leq 50$ (often $d=2,5,10$), preserving local structure. UMAP constructs high- and low-dimensional fuzzy simplicial sets and minimizes cross-entropy between them [2203.05794][2511.18843]. Hyperparameters include $n_\text{neighbors}$ (default 15), $min_\text{dist}$ (default 0.1), and $n_\text{components}$ [2511.18843][2402.03067][2407.08417].  

3. **Density-Based Clustering (HDBSCAN)**  
   Reduced embeddings are clustered using HDBSCAN, a hierarchical, density-based method able to detect clusters of varying density and shape without requiring a fixed number of clusters. The key parameter is $min\_cluster\_size$ (often 5–15), which controls minimal cluster size. Documents that do not fit any cluster with sufficient density are labeled as noise/outliers [2203.05794][2502.02100][2402.03067][2511.18843].

4. **Topic Representation via Class-Based TF‑IDF (c-TF‑IDF)**  
   For each cluster $c$, all assigned documents are concatenated into a pseudo-document. c‑TF‑IDF ranks terms as follows:
   \[
   \text{c-TF-IDF}_{t,c} = \mathrm{TF}_{t,c} \cdot \log \frac{K}{df_t}
   \]
   where $\mathrm{TF}_{t,c}$ is the class term frequency, $K$ is the number of clusters, and $df_t$ the number of clusters containing term $t$ [2203.05794][2502.02100][2402.03067]. The top $M$ terms (usually 5–20) are used as topic keywords.

This modular pipeline is extensible: any embedding model is supported, UMAP can be replaced by other nonlinear/projection methods, and clustering is decoupled from embedding generation [2203.05794][2505.06696]. The overall workflow is:

```
Preprocess text → Embed (BERT/SBERT) → UMAP → HDBSCAN → c-TF-IDF topic extraction
```

## 2. Embedding Choices and Layer Strategies

BERTopic’s performance is sensitive to the embedding layer, pooling strategy, and choice of model:

- **Transformer Backbone**: Monolingual models capture morphological nuance; multilingual SBERTs are vital for lower-resource or cross-lingual corpora [2502.02100][2402.03067][2407.08417]. Larger models (mpnet-base-v2) consistently perform well, but smaller distilled models (MiniLM, distilUSE) are fast and robust after strong normalization [2402.03067].

- **Pooling and Hidden State Selection**:  
  Recent work demonstrates that representations from intermediate layers can outperform the default last-layer mean-pooled setup. Max pooling or aggregating (sum/concat) the last four layers can yield substantial gains in topic diversity and coherence (up to 10–20% relative improvement) [2505.06696]. CLS pooling should be avoided for this task. Empirical results indicate that stop-word removal prior to embedding further boosts performance in almost all cases.

- **Short Text and Morphological Richness**:  
  On short, morphologically complex texts, embeddings from well-trained multilingual transformers suffice even without lemmatization; topic quality does not degrade significantly if only minimal preprocessing (lowercasing, punctuation removal) is performed [2402.03067].

## 3. Hyperparameter Selection and Stability

BERTopic introduces critical hyperparameters at multiple stages:

| Stage                    | Key Hyperparameters         | Typical Values/Effects                        |
|--------------------------|----------------------------|-----------------------------------------------|
| Embedding                | Model, layer, pooling      | MPNet, MiniLM; mean/max pooling; see above    |
| Dimensionality Reduction | n_neighbors, min_dist, d   | 10–30, 0–0.1, 2–50                            |
| Clustering               | min_cluster_size, epsilon  | 5–15 (tighter = more/finer topics, noisier)   |
| Keyword Extraction       | Top-M words; c-TF-IDF      | M = 5–20; see pipeline above                  |

Fine-tuning is essential. Systematic grid or random search over UMAP and HDBSCAN parameters improves cluster stability and interpretability [2511.18843][2402.03067][2407.08417]. Researchers evaluate clustering robustness through bootstrapped resampling and metrics such as Adjusted Rand Index, Normalized Mutual Information, and Variation of Information [2511.18843]. Stability and coherence are often in tension; hierarchical merging of fine-grained topics into super-topics can balance these objectives [2511.18843].

## 4. Evaluation Metrics and Empirical Benchmarks

Evaluation focuses on two primary metrics:

- **Topic Coherence** ($C_v$, NPMI, UMass): Measures the semantic consistency of top keywords for each topic, typically using variants of normalized pointwise mutual information (NPMI) or sliding-window co-occurrence [2203.05794][2502.02100][2402.03067][2511.18843]. Higher values indicate more meaningful, interpretable topics.
- **Topic Diversity** ($TD$): Fraction of unique words among top words per topic, quantifying redundancy [2203.05794][2502.02100][2402.03067].
  \[
  \mathrm{Diversity} = \frac{|\bigcup_{t} w_t|}{K \cdot M}
  \]

Empirical results establish several regularities:

- On benchmarking datasets (20 Newsgroups, BBC News, Trump Tweets), BERTopic with MPNET or similar SBERTs consistently outperforms LDA and NMF in both coherence ($TC \approx 0.166$ vs. $0.058$ for LDA on 20 Newsgroups) and diversity ($TD \approx 0.85$ vs. $0.75$) [2203.05794].
- On low-resource, morphologically rich languages (Marathi, Serbian), BERTopic (with monolingual or robust multilingual SBERT) outperforms LDA and NMF in coherence ($C_v \approx 0.7$–$0.82$ for Marathi vs. $0.34$–$0.55$ for LDA) and delivers higher diversity ($>0.75$) [2502.02100][2402.03067].
- Preprocessing depth (lemmatization vs. raw text) has only marginal impact when strong contextual embeddings are used [2402.03067].
- Model remains robust to embedding model choice, provided it is well-matched to the language/domain [2203.05794].

## 5. Applications: Dynamic Modeling, Multilinguality, Hierarchy

BERTopic is widely adopted for both static and dynamic topic modeling in various settings:

- **Dynamic Topic Evolution**:  
  By running the pipeline in time-sliced bins (e.g., monthly), BERTopic enables temporal analysis of topic evolution [2510.22904]. Topic continuity across time is tracked via Jaccard overlap or cosine similarity over c-TF‑IDF vectors. This framework has been applied to the study of political discourse on Twitter, including downstream alignment with domain lexica such as Moral Foundations Theory. Topic persistence, splits, and merges across time can be formally analyzed [2510.22904].

- **Multilingual and Cross-lingual Analysis**:  
  Capable of handling German, English, Indic languages, and more using appropriate SBERT backbones. Side-by-side modeling of multiple corpora (e.g., fake news from various countries) enables identification of cross-lingual thematic overlap [2407.08417][2502.02100][2402.03067].

- **Hierarchical and Hybrid Model Integration**:  
  In large-scale or multi-scale scenarios, hierarchical pipelines combine coarse, interpretable themes (via NMF) followed by fine-grained subtopic discovery (via BERTopic), improving both scalability and semantic resolution [2211.13496]. This approach retains NMF's ability to provide multi-label assignment at the broad level, followed by single-topic assignment at the fine level.

- **Qualitative Research and Human Validation**:  
  Domain experts routinely validate topic interpretability using Likert-scale ratings, weighted Cohen's kappa, and ICC. Hierarchical merging of initially fine-grained clusters can further improve both statistical coherence and human agreement [2511.18843].

## 6. Best Practices, Limitations, and Adaptations

**Best Practices**

- Always tune UMAP and HDBSCAN hyperparameters according to corpus size, document length, and desired granularity [2511.18843][2402.03067][2407.08417].
- For short documents, use SBERT models trained/fine-tuned on semantic similarity tasks (NLI, STS) [2502.02100].
- Leverage monolingual language models when available for morphologically complex or low-resource languages to capture fine nuances [2502.02100][2402.03067].
- Remove stop-words prior to embedding for increased coherence and diversity [2505.06696].
- Systematic grid search and stability evaluation are advised for reproducibility and interpretability [2511.18843].
- For large or hierarchical corpora, adopt staged (NMF → BERTopic) modeling pipelines [2211.13496].

**Limitations and Known Constraints**

- Each document is assigned to a single topic; HDBSCAN soft probabilities enable some post hoc flexibility but do not natively allow for multi-topic assignments [2203.05794][2211.13496].
- c-TF-IDF provides bag-of-words representations, which may include semantically redundant top words for each topic—post hoc reranking is sometimes necessary [2203.05794].
- Scalability on extremely large corpora is limited by memory and embedding computation time, though hierarchical and divide-and-conquer strategies mitigate this bottleneck [2211.13496].
- Optimal cluster granularity must balance interpretability and statistical stability; hierarchical merging can address this but requires careful calibration [2511.18843].

## 7. Comparison to Other Neural and Classical Topic Models

Relative to LDA, NMF, and autoregressive neural topic models (e.g., DocNADE), BERTopic delivers higher coherence, richer topic diversity, and better adaptability to short, informal, or non-English texts [2502.02100][2402.03067][2203.05794][2511.18843]. It is competitive with or complementary to other neural approaches such as Top2Vec and Contextualized Topic Models (CTM), and can be extended via transfer learning in multi-source neural topic architectures [2104.08551]. BERTopic's modular pipeline, decoupled from explicit generative assumptions, allows direct interoperability with dynamic modeling, domain-specific lexica, and supervised downstream analyses [2510.22904][2211.13496].

---

By grounding topic assignment in dense contextual embeddings, flexible non-linear manifold learning, density-based clustering, and cluster-centric term weighting, BERTopic defines a state-of-the-art paradigm for neural topic modeling with demonstrated effectiveness across domains, scripts, and languages [2203.05794][2502.02100][2402.03067][2511.18843][2505.06696][2407.08417].

Source: https://www.emergentmind.com/topics/bertopic-neural-topic-modeling