Document Embedding and Clustering

• Document embedding and clustering is a foundational process that converts texts and visuals into dense vectors, grouping them by semantic similarity.
• Hybrid methods combining text, structural, and visual data enhance the robustness, interpretability, and precision of clustering outcomes.
• Graph-based and density-driven algorithms optimize semantic partitions, facilitating efficient retrieval, analysis, and human-in-the-loop validation.

Document embedding and clustering constitute a foundational workflow for the semantic organization, retrieval, analysis, and exploration of document corpora, both in purely textual and multimodal (text-layout-vision) domains. Modern approaches operationalize this task via dense, low-dimensional representations—“embeddings”—learned from the document’s textual, structural, or multimodal content, which are subsequently partitioned into semantically coherent groups (clusters) using a range of clustering algorithms. Recent advances have emphasized hybridization of representation (e.g., combining LLM embeddings, structured triples, named-entity relations, and visual/layout cues), graph-based algorithms, and explainable embeddings to improve robustness, interpretability, and alignment with human categories (An et al., 2021, Keraghel et al., 2024, Sampaio et al., 13 Jun 2025, Starosta et al., 2023). The following sections give a comprehensive overview of theoretical underpinnings, embedding methodologies, clustering algorithms, evaluation metrics, and recent trends in document embedding and clustering.

1. Fundamental Principles and Theoretical Foundations

The central premise of document embedding is to map variable-length or multimodal artifacts (text passages, document images, layouts) into fixed-dimensional vectors, such that similarity in this space reflects semantic or topical relatedness. The clustering stage then groups these embeddings by minimizing intra-cluster distances or maximizing intra-cluster similarity under chosen metrics (cosine, Euclidean, Mahalanobis).

Key theoretical frameworks include:

• Distributed Representations: Paragraph Vector models such as PV-DM and PV-DBOW are trained to predict words or contexts from a latent document vector, yielding semantic representations that support both similarity-based and analogy-style reasoning (Dai et al., 2015).
• Markov Stability and Graph Partitioning: Constructing a similarity graph from document embeddings enables multiscale community detection, such as the Markov Stability framework, which scans over partition granularities by diffusing random walks of increasing “Markov time”—producing a quasi-hierarchical topic decomposition (Altuncu et al., 2018).
• Spectral and Kernel-Based Embeddings: Graph Laplacian-based eigenproblems define embeddings whose axes optimize specific clustering objectives (e.g., ratio cut, normalized cut). K-embeddings further bridge the gap to interpretable, term-based similarity by yielding coordinate distances exactly matching 1 – cosine(text vector), thus providing direct explainability for cluster assignment (Starosta et al., 2023).
• Bayesian Nonparametric Models: Mixtures with Dirichlet Process priors (e.g., SiDPMM) incorporate flexible cluster-number inference and allow joint modeling of word-counts, sequential (LSTM) features, and averaged word embeddings in a fully conjugate setting (Duan et al., 2018).

These foundations enable robust representation and flexible partitioning of document collections, often with guarantees about the relationship between the embedding geometry and underlying document semantics.

2. Document Embedding Methodologies

Modern document embedding approaches encompass a diverse set of techniques:

A. Purely Textual Embedding

• Bag-of-Words and TF–IDF: Sparse histograms capturing term frequencies, sometimes combined with dependency or entity graphs for structural sensitivity (Rafi et al., 2014).
• Paragraph/Document Vectors: PV-DM and PV-DBOW create unsupervised, distributed embeddings trained to predict local context or bag-of-document words via hierarchical softmax or negative sampling (Dai et al., 2015).
• Transformer and Sentence Embeddings: General-purpose (MiniLM, MPNet), domain-adapted (SciBERT, SPECTER), and fine-tuned (SentenceBERT, LASER) transformer models produce dense, context-aware embeddings by mean-pooling or [CLS]-pooling last-layer outputs (Arcan, 19 Dec 2025, An et al., 2021).
• Instruction and Aspect-Guided Embedding: Task-tailored encoders (e.g., multilingual-e5-large-instruct) adjust embeddings via explicit instructions or analytic “lenses” (“Identify the topic,” “Detect sentiment”) and by pre-processing via LLM rewriting (Fischer et al., 17 Feb 2026).

B. Hybrid and Knowledge-Infused Embedding

• Triples and Hybrid Representations: Subject-predicate-object extraction from abstracts, linearized as sentences, allow knowledge-infused vectors, either alone or concatenated/segmented with the raw abstract (Arcan, 19 Dec 2025).
• Named Entity–Aware Graph Embedding: NER extracts key entities, which are embedded (typically via word2vec CBOW) and used to define sparse document graphs (edges link documents with high entity-similarity) to complement LLM-based global embeddings (Keraghel et al., 2024).
• Topic and Graph Embedding (Top2Vec + Node2Vec): Unsupervised topic discovery via UMAP+HDBSCAN in embedding space is fused with document–topic bipartite graphs, which are embedded using Node2Vec. The resultant concatenated vectors encode both semantics and structural/topical proximity (Bastola et al., 31 Aug 2025).

C. Multimodal Embedding

• Text, Layout, Vision Fusion: LayoutLMv1/v3, DiT, Donut, and ColPali generate modality-specific embeddings (mean-pooled across last-layer sequence or patch tokens) and concatenate to form multimodal vectors. Text-only (SBERT), layout-only, and vision-only models have complementary strengths and weaknesses (Sampaio et al., 13 Jun 2025).
• Visual Feature Analysis for Template Clustering: Model-specific visual patches are pooled and dimensionally reduced (PCA/UMAP) to enable template-level discrimination, especially under real-world document perturbations (Rodrigo et al., 8 Jan 2026).

Each approach involves crucial design choices in pooling strategy, dimensionality, negative sampling or contrastive objectives, and the use of domain adaptation or instruction tuning.

3. Clustering Algorithms and Graph-Based Partitioning

The dominant clustering paradigms for document embeddings are:

• Centroid-based Algorithms: K-means (Euclidean/cosine/spherical) and its variants are standard in both purely textual and multimodal embedding spaces. Choice of $k$ (number of clusters) is determined via elbow plots, silhouette maximization, or in some studies, ground-truth labels (Arcan, 19 Dec 2025, Sampaio et al., 13 Jun 2025).
• Density-Based Methods: HDBSCAN is used for identifying clusters of variable shape/density, marking sparse regions as noise/outliers. UMAP is commonly used prior to HDBSCAN to reduce dimensionality and enhance cluster separability (Fischer et al., 17 Feb 2026).
• Spectral Clustering: Laplacian embeddings and their interpretable variants (K-embedding) enable the recovery of arbitrarily shaped clusters and directly connect to maximizing within-cluster similarity (Starosta et al., 2023).
• Markov Stability Community Detection: Random-walk diffusion across multiscale times enables characterization of the document graph’s community structure without fixing cluster count a priori, supporting discovery of fine-to-coarse topic hierarchies (Altuncu et al., 2018).
• Nonparametric Bayesian Mixture Models: Dirichlet Process Mixtures with fully collapsed Gibbs inference permit automatic cluster-number selection and joint modeling of multiple document views (Duan et al., 2018).
• Graph Neural Networks for Document Clustering: LLM embeddings and NER-derived edges define the input graph, over which a simple graph convolution (self-looped, normalized) aggregates signals prior to joint autoencoding and clustering in embedding space. Alternating optimization enforces orthogonality and cluster-assignments (Keraghel et al., 2024).

Clustering performance varies with algorithm choice; centroid methods generally suit high-dimensional dense embeddings, while nonparametric and graph-based methods offer flexibility and improved performance on structured or noisy corpora.

4. Evaluation Metrics and Empirical Results

Document clustering outcomes are rigorously assessed through internal and external validation metrics:

• Internal Metrics:
  • Silhouette Score: Measures separation of clusters relative to their cohesion.
  • Davies–Bouldin Index, Calinski–Harabasz Score: Evaluate compactness and separation (lower/higher is better, respectively).
• External Metrics:
  • Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), Purity, Homogeneity, Completeness, Fowlkes–Mallows Index: All compare predicted clusters to ground-truth labels.

Representative empirical results:

Representation	Model (Algorithm)	K	ARI	NMI	Silh.
Abstract	MPNet (KMeans)	6	0.4703	0.5511	0.0633
Triples	MiniLM (KMeans)	8	0.3451	0.4006	0.0340
Abstract+Triples	MiniLM (KMeans)	10	0.4549	0.5459	0.0452
Clean template images	Donut (k-Means)	50	0.9708	0.9921	0.6937
Legal docs (hybrid)	Top2Vec+Node2Vec	25	0.051	0.153	0.927
BBC (LLM+NER GCN)	GCC* (p=2, N=5)	5	0.967	0.951	–

Empirically, integrating knowledge (triples, NER, topical graphs) lifts ARI/NMI over text-only representations, while multimodal embeddings yield significant gains in visually-rich or structured documents (Arcan, 19 Dec 2025, Keraghel et al., 2024, Sampaio et al., 13 Jun 2025, Bastola et al., 31 Aug 2025). Contrastive pretraining with clustering-based stratification further improves retrieval and representation quality (Merrick, 2024).

5. Advances in Hybrid, Multimodal, and Explainable Clustering

Contemporary document clustering research emphasizes several converging directions:

• Hybrid Representations: Combining dense contextual embeddings with structural information (NER graphs, triples, topic-node bipartite graphs) synergistically enhances semantic discrimination and supports robust, low-entropy clusters (Keraghel et al., 2024, Arcan, 19 Dec 2025, Bastola et al., 31 Aug 2025).
• Multimodal Embedding: Integrating textual, layout, and visual features (via mean or hybrid pooling) achieves fine-grained discrimination (e.g., template-level forms) unattainable by text or vision alone, with robust performance under document perturbations (Sampaio et al., 13 Jun 2025, Rodrigo et al., 8 Jan 2026).
• Interactive and Human-in-the-Loop Clustering: The inclusion of instruction-based embedding, LLM rewriting, and interactive refinement/fine-tuning enables human analysts to steer cluster formation and realign representations with analytic intent, e.g., for digital humanities or exploratory research (Fischer et al., 17 Feb 2026).
• Explainability and Interpretability: K-embedding explicitly aligns cluster geometry with word-level term frequencies, facilitating direct explanation of cluster assignment back to textual content. Similarly, c-TF-IDF, keyword extraction, and centroid-based retrieval clarify cluster themes for domain experts (Starosta et al., 2023, Fischer et al., 17 Feb 2026).

These trends reflect an increased demand for semantically meaningful, robust, interpretable, and customizable clustering pipelines.

6. Limitations, Open Problems, and Best Practices

Notable limitations across current methodologies include:

• Cluster Number Selection: Many frameworks require $k$ to be preset, with fully automated ($k$-free) selection only available in Bayesian nonparametric or density/HDBSCAN approaches (Duan et al., 2018).
• Interpretability in Deep/Multimodal Models: Explaining decisions in Transformer-based or vision-LLMs is still challenging, though K-embedding and keyword-based labeling offer progress (Starosta et al., 2023, Fischer et al., 17 Feb 2026).
• Modality Sensitivity: Vision-only embedding is brittle under image perturbations; text-only embedding cannot distinguish layout/template variants (Sampaio et al., 13 Jun 2025).
• Hyperparameter and Model Overhead: Multimodal and graph-based methods often require extensive hyperparameter-tuning (graph construction, aggregation, pooling) and may not scale to very large corpora without approximation (Keraghel et al., 2024, Bastola et al., 31 Aug 2025).
• Domain Adaptation: General-purpose sentence embeddings often outperform domain-specific models for clustering, but further improvement is possible via pretraining/fine-tuning on in-domain corpora (Arcan, 19 Dec 2025).

Best practices identified include:

• Segmenting long documents into topical or entity-centered spans for embedding (rather than bag-of-sentences) (An et al., 2021).
• Using centroid-based clustering (K-means, GMM) in high-dimensional, dense spaces and density-based clustering when the true number of clusters is unknown or clusters have varying densities (Arcan, 19 Dec 2025, Sampaio et al., 13 Jun 2025).
• Hybridizing representations (semantic + structural + NER/graph) for improved discrimination, particularly in entity- or layout-rich domains (Keraghel et al., 2024, Bastola et al., 31 Aug 2025).
• Post-hoc or interactive refinement using keyword summaries, LLM-based labeling, or human-in-the-loop feedback to align clusters with user intent (Fischer et al., 17 Feb 2026).

7. Future Directions and Outlook

Emerging research avenues include:

• Smarter and Curriculum-Based Clustering: Dynamic, periodic re-embedding and re-clustering to adapt representation and cluster granularity during ongoing pretraining, as well as curriculum learning strategies (Merrick, 2024).
• Automated Model Selection: Data-driven methods for selecting the optimal number of clusters, diffusion timescales, and embedding dimensionality, particularly in graph and spectral frameworks (Starosta et al., 2023).
• Integrating External Knowledge and Structure: Further infusion of knowledge bases (e.g., Wikidata, domain ontologies), advanced NER/coreference, and ontological constraints into embedding construction and graph formation.
• Scaling and Real-World Deployment: Efficient approximation of graph and multimodal clustering algorithms for very large or streaming document corpora.
• Explainability for Black-Box Models: Bridging neural embedding geometry and text content in a way that supports actionable, decomposable, and faithful explanations, especially for multimodal and LLM-derived representations (Starosta et al., 2023, Fischer et al., 17 Feb 2026).

In summary, document embedding and clustering is a rapidly advancing domain driven by innovations in representation (hybrid/multimodal/knowledge-infused), graph and spectral theory, and human-centered design. Rigorous empirical validation demonstrates that combining complementary embeddings and clustering strategies—supplemented by explainable and interactive interfaces—yields highly coherent, trustworthy, and actionable partitions of diverse document corpora.