---
title: BERTopic Modeling Overview
url: https://www.emergentmind.com/topics/bertopic-modeling
type: topic
---

# BERTopic Modeling Overview

BERTopic is a neural topic modeling framework that aggregates recent advances in sentence embedding, manifold-based dimension reduction, density-based clustering, and interpretable topic representation. The method is distinguished by its use of Transformer-based language models for contextual embeddings, an unsupervised clustering approach in embedding space, and a class-based TF-IDF mechanism for robust topic labeling. BERTopic has achieved superior topic coherence and diversity across linguistic contexts, genres, and domain applications compared to classical approaches such as LDA and NMF.

## 1. Pipeline Architecture and Mathematical Foundations

BERTopic orchestrates topic modeling through four sequential modules: (1) document embedding, (2) dimensionality reduction, (3) clustering, and (4) cluster representation via class-based TF-IDF.

1. **Embedding Extraction**: Each document $d_i$ is transduced into a dense vector $e_i = \phi(d_i) \in \mathbb{R}^M$ where $\phi$ is a pre-trained Sentence Transformer (e.g. MiniLM, mpnet, distiluse) and $M$ is the latent dimension (384–768, depending on model) [2402.03067].

2. **Dimensionality Reduction (UMAP)**: The set of high-dimensional embeddings $\{e_i\}$ is projected into a manifold $\{u_i\} \subset \mathbb{R}^m$ ($m \ll M$) using UMAP. UMAP constructs fuzzy simplicial sets in both high- and low-dimensional spaces and minimizes the cross-entropy between them:
   $$
   \min_U \sum_{i\neq j} \left[ p_{ij} \log\left(\frac{p_{ij}}{q_{ij}}\right) + (1-p_{ij}) \log\left(\frac{1-p_{ij}}{1-q_{ij}}\right) \right]
   $$
   where $p_{ij}$ and $q_{ij}$ denote membership strengths in the original and reduced spaces, respectively [2203.05794].

3. **Clustering (HDBSCAN)**: HDBSCAN discovers clusters $\{C_1, ..., C_K\}$ in the reduced space by building a hierarchical tree of density-connected points and selecting maximally stable clusters. Outliers are labeled as noise (cluster $-1$). Key parameters include `min_cluster_size` (controls topic granularity) and `min_samples` [2402.03067].

4. **Topic Representation (c-TF-IDF)**: For each topic $k$, BERTopic computes class-based TF-IDF weights for term $t$:
   $$
   w_{k,t} = \frac{\mathrm{tf}(t, C_k)}{|C_k|} \cdot \log\left(\frac{N}{\mathrm{df}(t)}\right)
   $$
   where $|C_k|$ is the cluster size, $N$ is the number of documents, $\mathrm{df}(t)$ is the document frequency of $t$. The top-$n$ ranking terms by $w_{k, t}$ constitute the topic descriptor [2402.03067, 2203.05794].

## 2. Embedding Model Selection and Intermediate Layer Strategies

BERTopic is agnostic to the Transformer encoder used, allowing the integration of multilingual and domain-specific models. Key findings show that:

- **Multilingual Sentence Transformers**: When monolingual models are unavailable (e.g., Serbian), multilingual transformers pretrained on 50+ languages—such as paraphrase-multilingual-mpnet-base-v2 (768d), distiluse-base-multilingual-cased-v2 (512d), and paraphrase-multilingual-MiniLM-L12-v2 (384d)—yield high topic diversity and coherence. Among these, larger models (mpnet) tend to maximize topic coherence, especially on morphologically rich, partially processed texts [2402.03067].

- **Embedding Layer Selection**: Extracting representations from intermediate model layers and varying pooling methods (mean, max, CLS) serves as a powerful tuning strategy. Results show that using the embedding/token layer plus mean-pooling improved coherence on UN/Newsgroups, while max pooling on the sum of all encoder layers boosted coherence on short texts (Trump Tweets). CLS pooling underperformed, indicating [CLS] is insufficient for nuanced topic separation. Aggregating layers and max pooling tends to increase topic diversity, mean pooling favors coherence. Stop-word removal universally enhances performance [2505.06696].

## 3. Robustness Across Preprocessing Levels and Languages

BERTopic displays resilience to varying levels of text preprocessing:

- **Partial Preprocessing**: Even minimal normalization (transliteration, token filtering) without lemmatization yields interpretable topics. Full morphological normalization (lemmatization) delivers a slight gain in topic coherence but is not strictly required if contextual embeddings are used [2402.03067, 2504.14707].

- **Low-resource and Morphologically Rich Languages**: Despite the absence of monolingual SBERTs, BERTopic with multilingual models produces informative clusters in Serbian [2402.03067], Hindi [2501.03843], and Dutch [2504.14707]. In qualitative and quantitative benchmarks, BERTopic outperforms LDA, NMF, and Top2Vec on short and open-ended texts in terms of NPMI-based coherence and diversity, even when classical models are constrained to the same number of topics.

## 4. Hyperparameter Sensitivity and Optimization

BERTopic's performance pivots on several core hyperparameters:

- **UMAP**: `n_neighbors` determines structure preservation (default: 15), `min_dist` controls cluster tightness (default: 0.1). Increasing `n_neighbors` yields more global clusters, reducing it favors local specificity.
- **HDBSCAN**: `min_cluster_size` adjusts topic granularity; raising its value collapses small, noisy clusters into larger, interpretable ones. Automatic cluster count selection often produces overly fine topics; manual adjustment to domain-optimal values (e.g., 10–15) is common [2402.03067, 2506.11451].
- **Cluster Outlier Strategy**: HDBSCAN can discard >50% of points as noise. BERTopic's reduce_outliers method can assign noise points to topics via c-TF-IDF-based re-assignment, improving coverage [2402.03067].
- **Vocabulary Size**: When benchmarking against LDA/NMF, constraining vocabulary and topic counts to matched values enables fair comparison. BERTopic typically yields higher coherence and more granular subtopics [2402.03067, 2308.11520].

## 5. Quantitative Benchmarks: Coherence and Diversity Evaluation

Topic quality in BERTopic is routinely assessed using normalized pointwise mutual information (NPMI) for coherence and the fraction of unique top terms for diversity:

- **Topic Coherence (TC)**:
  $$
  \mathrm{TC} = \frac{1}{K} \sum_{k=1}^{K} \frac{2}{n(n-1)} \sum_{1\leq i<j \leq n} \mathrm{NPMI}(w_{k,i}, w_{k,j})
  $$
  where
  $$
  \mathrm{NPMI}(w_i, w_j) = \frac{\log \frac{P(w_i, w_j)}{P(w_i)P(w_j)}}{-\log P(w_i, w_j)}
  $$
  with probabilities estimated via document co-occurrences. TC ranges from $[-1, 1]$; higher is better [2203.05794, 2402.03067].

- **Topic Diversity (TD)**:
  $$
  \mathrm{TD} = \frac{\mid \bigcup_{k=1}^K W_k \mid}{K n}
  $$
  TD near 1 signifies highly varied topic term sets; benchmarks show BERTopic achieves TD > 0.85 for most datasets and languages.

BERTopic typically outperforms classical models (LDA, NMF) on coherence and at least matches in diversity. For example, in Serbian vaccine-hesitancy tweets, BERTopic's best tc = –0.054 (distiluse-base) trumps LDA at –0.104, and diversity remains high (.892 versus .897 for LDA) [2402.03067].

## 6. Qualitative Insights, Limitations, and Best Practices

- **Interpretability**: Topic structures remain meaningful under minimal preprocessing, and embeddings capture fine-grained phenomena (e.g., manufacturer-specific vaccine mistrust, conspiracies) beyond classical models' reach [2402.03067].
- **Parameter Tuning**: Default hyperparameters suffice as baseline, but granularity can be adjusted by tuning `nr_topics` or `min_cluster_size` to match domain requirements.
- **Outlier Handling**: Large fractions of points may be classified as noise; outlier reduction strategies should be employed, especially in short-text analysis.
- **Model Selection**: Larger transformer models (e.g., mpnet) increase coherence but incur higher computational cost. Layer-wise embedding selection offers additional gains if time and resources permit [2505.06696].
- **Recommendations**: Always set fixed random seeds for UMAP and HDBSCAN when analyzing topic assignments qualitatively.

Limitations include HDBSCAN's tendency to discard documents as outliers, the hard assignment of each document to a single topic, and the lack of probabilistic topic membership in the standard pipeline [2212.08459, 2402.03067]. Extensions incorporating soft clustering and multi-topic assignment have been identified as future directions.

## 7. Application Domains and Future Directions

BERTopic has been deployed in multilingual social media analytics, customer feedback mining, educational recommender systems, historical document analysis, aviation safety, and discourse study. Its architecture supports hybridized pipelines (e.g., integrating with NMF for hierarchical modeling [2211.13496]) and dynamic topic tracking (e.g., mapping the evolution of regulatory themes over time [2509.13387]).

Areas identified for future research include:
- Recursive or hierarchical topic modeling for subtopic discovery
- Integration of contextualized phrase ranking for improved topic labels
- Automated hyperparameter tuning based on coherence/diversity feedback
- Incorporation of soft topic assignments
- Domain-specific embedding fine-tuning for maximal performance in highly specialized corpora

In summary, BERTopic offers a robust, flexible neural topic modeling solution that consistently surpasses classical bag-of-words approaches in coherence, interpretability, and adaptability across a range of languages, genres, and application domains [2402.03067, 2505.06696, 2203.05794].

Source: https://www.emergentmind.com/topics/bertopic-modeling