BERTopic with Seed Words

• BERTopic with Seed Words is a topic modeling approach that integrates user-provided seed words within embeddings, clustering, and keyword extraction.
• It augments unsupervised learning by guiding cluster formation and tuning c-TF-IDF scores to improve thematic focus and interpretability.
• Practical applications include domain adaptation and multilingual analysis, leading to enhanced topic coherence and classification accuracy.

BERTopic with Seed Words refers to the integration of explicit user-provided terms (seed words) into the BERTopic neural topic modeling framework. BERTopic utilizes transformer-based sentence embeddings, dimensionality reduction, unsupervised clustering, and class-based TF-IDF keyword extraction to model topics from document corpora. Introducing seed words augments the unsupervised process by offering prior knowledge or domain interest, aiming to improve thematic focus, interpretability, and alignment with user objectives. Seed words can play multiple roles: anchors for clusters, constraints in keyword extraction, or guides for topic coherence in multilingual or specialized domains.

1. Foundational Principles of BERTopic and Seed Words

BERTopic’s methodology involves converting documents into dense vector embeddings using pre-trained transformer models (e.g., Sentence-BERT, jina-embeddings-v3, or multilingual transformer variants), followed by dimensionality reduction (typically UMAP), clustering (usually HDBSCAN, but sometimes k-Means), and extracting topic representations using a class-based TF-IDF (c-TF-IDF) weighting scheme (Grootendorst, 2022). Seed words, on the other hand, are human-curated terms intended to represent topics of interest or latent semantic categories.

While BERTopic’s classic pipeline is unsupervised, several recent studies and practical applications advocate the incorporation of seed words to improve topic specificity, guide cluster formation, or reweight key terms in topics (Zhang et al., 2022, Zhang et al., 2022).

2. Mechanisms for Incorporating Seed Words

Seed words can be integrated into BERTopic through multiple mechanisms, which may occur at different stages of the modeling pipeline:

Stage	Seed Word Role	Reference
Embedding	Influence embedding space via proximity weighting	(Hill et al., 11 Jan 2025)
Clustering	Anchor cluster centroids (e.g., LITA, KMeans variant)	(Chang et al., 17 Dec 2024, Groot et al., 2022)
Keyword Extraction	Boost TF, modify c-TF-IDF scores	(Grootendorst, 2022, Schäfer et al., 11 Jul 2024)
Postprocessing	Re-rank topic labels, enforce seed prominence	(Grootendorst, 2022, Zhang et al., 2022)

The most direct mathematical modification occurs in the c-TF-IDF weighting. Rather than

$$W_{t,c} = \text{tf}_{t,c} \cdot \log{\left(1+\frac{A}{\text{tf}_t}\right)}$$

one can augment the term frequency for seed words:

$$\tilde{W}_{t,c} = (\text{tf}_{t,c} + \lambda \cdot \mathbb{I}(t \in \mathcal{S}_c)) \cdot \log{\left(1+\frac{A}{\text{tf}_t}\right)}$$

where $\lambda$ is a tunable boost and $\mathcal{S}_c$ is the seed set for topic $c$ (Groot et al., 2022, Schäfer et al., 11 Jul 2024).

Alternatively, clustering can be seeded by initializing centroids with seed word embeddings (as in LITA (Chang et al., 17 Dec 2024)), or reassigning ambiguous documents based on their proximity to seed-guided centroids.

3. Seed Word Expansion, Selection, and Multilingual Context

Seed words need not be restricted to exact user input; expansion strategies can be employed to capture morphological variants (e.g., truncation, regex-based matching (Gretter et al., 2021)), semantically related terms via word2vec or transformer similarity (Zhang et al., 2022), or even subword tokenization for composite/OOV (out-of-vocabulary) seeds (Zhang et al., 2022). In multilingual contexts, cross-lingual word relationships can be modeled by inverting document/word roles to yield pseudo-documents for each word (see inverted topic models (Ma et al., 2016)), or by leveraging multilingual transformer models trained on broad lexical coverage (Medvecki et al., 5 Feb 2024, Kandala et al., 20 Apr 2025).

A plausible implication is that seed word expansion—through word embedding similarity or local corpus statistics—improves coverage and robustness, especially for technical domains or morphologically rich languages.

4. Practical Applications and Performance Impact

Seed-word-augmented BERTopic has demonstrated clear utility in domain adaptation, personalized recommendation, and multilingual topic modeling:

• Domain-specific ASR LM adaptation: Expanded seed glossaries select adaptation texts, reducing OOV rates and improving WER for technical terminology (Gretter et al., 2021).
• Academic program recommendation: Interest topics extracted via BERTopic guide students to programs matching their preferences. Seed word sets could refine the knowledge map for personalized recommendations (Hill et al., 11 Jan 2025).
• Fake news and open-ended narrative analysis: Proper tuning of seed word roles improves thematic specificity and interpretable clusters in multilingual data (Schäfer et al., 11 Jul 2024).

Empirical results across multiple papers show that seed-guided topic models—whether through lexical expansion, cluster anchoring, or keyword weighting—yield improvements in topic coherence, relevance, and classification accuracy, with metrics such as NPMI, Macro/Micro-F1, and human interpretability all benefiting from seed word intervention (Zhang et al., 2022, Schäfer et al., 11 Jul 2024, Chang et al., 17 Dec 2024).

5. Comparative Perspectives: Generative vs. Embedding-Based Topic Models

Seed word integration has a long precedent in probabilistic topic models (SeededLDA, Weakly Supervised Prototype Topic Models), where seeds modify Dirichlet priors to encode category relevance (Wang et al., 2021). BERTopic operates differently: topics are the result of embedding-cluster assignments and keyword extraction via c-TF-IDF rather than generative mixtures. However, guided topic discovery frameworks (SeedTopicMine, SeeTopic) demonstrate that ensemble approaches, blending local corpus statistics with transformer-based semantic representations, exploit seed words for more distinct and accurate topics (Zhang et al., 2022, Zhang et al., 2022).

Further, iterative frameworks such as LITA combine seed centroids with LLM-based reassignment for ambiguous samples, achieving superior topic coherence and reducing human annotation labor (Chang et al., 17 Dec 2024).

6. Methodological Considerations, Limitations, and Future Directions

Care should be taken to avoid overfitting or bias from seed matching. Debiasing strategies such as seed deletion or random token deletion help classifiers rely on broader contextual signals rather than just seed occurrence (Dong et al., 2023). Excessive reliance on seed words, or inappropriate selection, may reduce model generalizability.

Parameter sensitivity—such as vocabulary size, min_topic_size for clustering, deletion ratio in debiasing—must be tuned for optimal coherence and coverage. The stochastic nature of BERTopic warrants multiple runs and robust evaluation across diversity and coverage metrics (e.g., C_V, Gini, outlier count) (Compton, 26 Aug 2025).

Future work is likely to exploit hybrid architectures that combine seed expansion, dynamic reassignment (using LLMs), and post hoc human-in-the-loop supervision for domain-specific tailoring. Multilingual and cross-domain adaptation will benefit from sophisticated pseudo-document representations and probabilistic alignment as outlined in inverted topic models (Ma et al., 2016).

7. Reproducibility and Researcher Agency

Recent discourse analysis research advocates for transparent, reproducible modeling pipelines that combine lexical and semantic layers. Custom pipelines—using open tools (NLTK, spaCy, Sentence Transformers)—allow fine-grained control over seed word integration, preprocessing, and evaluation, heightening methodological transparency and agency (Compton, 26 Aug 2025). This approach supports triangulation and interpretability, mitigating the trade-offs between black-box unsupervised clustering and precise, seed-guided lexicon mining.

Conclusion

Integrating seed words into BERTopic enhances topic modeling by infusing domain knowledge, fostering thematic orientation, and improving interpretability. Proper exploitation requires judicious selection, expansion, weighting, and regularization of seed terms throughout the embedding, clustering, and keyword extraction stages. Ensemble and iterative frameworks further amplify performance, especially in specialized, multilingual, or morphologically complex settings. By aligning BERTopic’s unsupervised neural machinery with guided supervision, researchers achieve more robust, relevant, and precise topic models for advanced discourse and information mining tasks.