Neural Graph Topic Model (NGTM)

• Neural Graph Topic Model (NGTM) is a family of neural architectures that leverage graph structures to induce, constrain, or interpret topic distributions in varied formulations.
• Variants such as GTM, TEC, GTFormer, and the graph generation NGTM employ distinct graph substrates and inference methods to capture document and entity relations.
• Empirical evaluations show NGTM approaches yield improved topic coherence, enhanced discriminative capability, and controllable graph generation compared to traditional models.

Neural Graph Topic Model (NGTM) denotes a family of neural architectures in which topical structure is induced, constrained, or interpreted through graph structure. In document modeling, this can mean message passing on document–word or document–document graphs to infer document-topic mixtures, or the use of knowledge-graph-derived entity embeddings whose clusters are treated as topics. In a later and distinct usage, NGTM names a graph generator that represents each graph as a mixture of latent topics over substructures. The term is therefore not attached to one canonical architecture; it covers several related but materially different formulations that share the premise that graphs provide an informative inductive bias for topic discovery or topic-conditioned generation (Zhou et al., 2020, Loureiro et al., 2023, Zhang et al., 17 Feb 2025, Zhuang et al., 17 Jul 2025).

1. Terminological scope and conceptual lineage

The main documented usages of NGTM fall into four technically different lines.

Formulation	Graph substrate	Primary output
GTM	document–word bipartite graph	document-topic and topic-word distributions
TEC	Wikidata-derived entity graph plus entity descriptions	clusters of entities interpreted as topics
GTFormer	hierarchical document graph in hyperbolic space	hierarchical topic distributions and document representations
NGTM (2025)	graphs decomposed into substructures	generated graphs with topic-level control

The 2020 Graph Topic Model (GTM) is the clearest document-topic instance of an NGTM: it constructs a weighted document–word graph, applies a two-layer GCN encoder, and recovers document-topic distributions together with topic-word distributions (Zhou et al., 2020). The 2023 Topics as Entity Clusters (TEC) paper does not explicitly introduce a model named NGTM, but it is described as strongly analogous to one: topics are induced by clustering bimodal entity embeddings formed from SBERT textual descriptions and node2vec embeddings trained on a Wikidata-derived graph (Loureiro et al., 2023). The 2025 GTFormer work generalizes graph-aware topic modeling toward hierarchical document graphs in hyperbolic space, integrating topic hierarchy and graph hierarchy inside each Transformer layer (Zhang et al., 17 Feb 2025). A separate 2025 paper uses the exact name NGTM for an interpretable graph generator whose “topics” are distributions over graph substructures rather than over words or entities (Zhuang et al., 17 Jul 2025).

A common misconception is that NGTM names a standardized model family analogous to LDA. The current literature does not support that interpretation. This suggests that NGTM functions more as a descriptive umbrella for neural topic models that operationalize graph structure in different ways than as a fixed recipe.

2. Graph Topic Model as the canonical document-level formulation

GTM represents a corpus as an undirected, weighted, bipartite document–word graph with self-loops. Documents and words are nodes, and edges are induced by document–word co-occurrence using symmetric TF–IDF weights. The adjacency matrix is defined by

$$A_{i,j}= \begin{cases} \mathrm{TF\text{-}IDF}_{ij}, & i\in \mathcal{D},\ j\in \mathcal{V} \ \mathrm{TF\text{-}IDF}_{ji}, & i\in \mathcal{V},\ j\in \mathcal{D} \ 1, & i=j \ 0, & \text{otherwise.} \end{cases}$$

There are no explicit document–document or word–word edges; those relations arise implicitly through multi-hop propagation, so that with at least two convolutional layers, signals can traverse document $\rightarrow$ word $\rightarrow$ document (Zhou et al., 2020).

The encoder is a two-layer GCN with output dimensions $[100, K]$, LeakyReLU activations, and Batch Normalization after each graph convolution. Its propagation rule is

$$\mathbf{H}^{(l+1)}=\sigma\!\left(\mathbf{D}^{-\frac12}\mathbf{A}\mathbf{D}^{-\frac12}\mathbf{H}^{(l)}\mathbf{W}^{(l)}\right).$$

For each document node $d$, the final encoder state is mapped to a multinomial topic distribution through

$$\hat{\mathbf{z}}_d=\mathrm{softmax}\!\left(\mathbf{H}^{(L)}_d\right).$$

A two-layer MLP decoder then reconstructs TF–IDF word distributions,

$$\hat{\mathbf{x}}_d=\mathrm{softmax}\!\big(G(\hat{\mathbf{z}}_d)\big),$$

and topic-word distributions are obtained by decoding the identity basis vectors, $\Phi_i=G(\mathbf{e}_i)$.

GTM is trained in a Wasserstein Autoencoder framework. Rather than using a Gaussian or logistic-normal latent variable, it directly enforces a Dirichlet prior over document topics by minimizing an MMD penalty between inferred topic proportions and samples from $\mathrm{Dirichlet}(\boldsymbol{\alpha})$. Reconstruction uses cross-entropy on TF–IDF weights. Training uses RMSProp with learning rate $\rightarrow$0 for $\rightarrow$1 epochs, and scalability is achieved through subgraph mini-batching containing $\rightarrow$2 document nodes and all word nodes.

Empirically, GTM consistently achieves the best average topic coherence on 20Newsgroups, Grolier, and NYTimes across $\rightarrow$3. On 20Newsgroups it reports $\rightarrow$4, $\rightarrow$5, and $\rightarrow$6, exceeding LDA’s $\rightarrow$7 and W-LDA’s $\rightarrow$8. On Grolier it achieves $\rightarrow$9, and on NYTimes $\rightarrow$0, again surpassing the listed baselines (Zhou et al., 2020). Qualitatively, the reported topics are cleaner and more discriminative than those of GraphBTM, W-LDA, and LDA.

The significance of GTM lies in its explicit corpus graph construction. Topic inference is no longer conditioned only on a document’s own token counts but also on neighboring documents reachable through shared words. In this sense, GTM is the most direct instantiation of NGTM as a graph-neural topic model for textual corpora.

3. Entity-centric and knowledge-graph-driven variants

TEC redefines the basic unit of topic modeling from words to conceptual entities identified by Wikidata Q-IDs. Its motivation is that entities are interpretable, language-independent, and grounded in external knowledge resources. A single Q-ID denotes the same concept across languages, which eliminates what the paper describes as “language bifurcation” of topics. TEC is therefore a purely entity-driven neural topic modeling approach and is presented as strongly analogous to an NGTM, even though that name is not used in the original model definition (Loureiro et al., 2023).

The architecture is bimodal. Each entity receives an implicit textual embedding from SBERT by encoding its Wikipedia description with paraphrase-multilingual-mpnet-base-v2, and an explicit graph embedding from node2vec trained on a Wikidata-derived graph. The two normalized embeddings are fused by weighted concatenation:

$\rightarrow$1

where $\rightarrow$2 controls the relative weight of graph and language information. Setting $\rightarrow$3 yields LLM-only representations; $\rightarrow$4 yields graph-only representations.

Topic induction is geometric rather than probabilistic. TEC clusters entity embeddings with FAISS K-Means, interpreting centroids as topic centroids. Documents are mapped into the same space by TF-weighted averaging of entity embeddings,

$\rightarrow$5

and a soft document-topic mixture is inferred by inverse-distance-squared interpolation,

$\rightarrow$6

Because nearest-centroid ranking alone can miss corpus-level frequency structure in sparse entity corpora, the paper reranks topic entities by aggregating entity frequencies from documents assigned to their top-inferred topic.

The data pipeline is unusually elaborate. Entity extraction uses multilingual Aho–Corasick automata built from Wikidata and Wikipedia surface forms; language-specific lemmatizers include spaCy for English, German, Spanish, and French, pymorphy2 for Russian, and zeyrek for Turkish. Disambiguation uses cosine similarity between text spans and candidate entity-description embeddings. The knowledge graph is filtered to roughly the top one million entities by QRank and restricted to conceptual predicates such as “Instance Of,” “Subclass Of,” and “Facet Of,” while also retaining relations whose objects are valid entities in order to densify the graph.

A central technical clarification is that TEC’s graph component is node2vec, described in the paper as a shallow neural method based on biased random walks and a skip-gram objective. Classical message-passing update rules such as GCN or GAT equations are explicitly stated not to be part of TEC. This matters because NGTM, in this line of work, does not require message-passing GNN layers.

On WIKIPEDIA with $\rightarrow$7 topics, the best TEC configuration is graph-only: $\rightarrow$8, $\rightarrow$9, and $[100, K]$0, compared with LDA (Entities) at $[100, K]$1 and WikiPDA at $[100, K]$2. With $[100, K]$3 topics, graph-only TEC again performs best, with $[100, K]$4, $[100, K]$5, and $[100, K]$6. CC-NEWS and MLSUM show the same pattern: graph-only or fused embeddings outperform LLM-only, and performance improves as $[100, K]$7 increases (Loureiro et al., 2023).

The qualitative examples are also diagnostic. An LLM-only topic yields a “Big Tech” cluster centered on entities such as Google, Amazon, Microsoft, and Apple Inc., whereas a graph-only topic yields a tightly coherent boxing cluster including Mike Tyson, WBO, IBF, Floyd Mayweather, WBA, Tyson Fury, Manny Pacquiao, WBC, and Evander Holyfield. The reported interpretation is that graph-derived relational semantics improve centroid structure and top-entity selection, especially under sparse entity-document statistics.

4. Hierarchical graph topic modeling in hyperbolic space

GTFormer extends NGTM-style reasoning to hierarchical document graphs. The corpus is a document graph $[100, K]$8, and each document is assumed to participate simultaneously in textual topic hierarchy and graph hierarchy. To preserve both, GTFormer operates in the Hyperboloid model of hyperbolic space and inserts two hierarchy-aware signals into every Transformer layer: a hierarchical topic-tree embedding and a hyperbolic graph embedding (Zhang et al., 17 Feb 2025).

The topic side is organized as a latent tree. For each document, the model infers a path distribution $[100, K]$9 over root-to-leaf topic paths and a level distribution $$\mathbf{H}^{(l+1)}=\sigma\!\left(\mathbf{D}^{-\frac12}\mathbf{A}\mathbf{D}^{-\frac12}\mathbf{H}^{(l)}\mathbf{W}^{(l)}\right).$$0 over specificity levels. These induce a hierarchical topic distribution $$\mathbf{H}^{(l+1)}=\sigma\!\left(\mathbf{D}^{-\frac12}\mathbf{A}\mathbf{D}^{-\frac12}\mathbf{H}^{(l)}\mathbf{W}^{(l)}\right).$$1 over all tree nodes. Topic embeddings $$\mathbf{H}^{(l+1)}=\sigma\!\left(\mathbf{D}^{-\frac12}\mathbf{A}\mathbf{D}^{-\frac12}\mathbf{H}^{(l)}\mathbf{W}^{(l)}\right).$$2 are constructed by a Hyperbolic Doubly Recurrent Neural Network (HDRNN), which models both ancestral and fraternal relations. The document-level tree embedding is then

$$\mathbf{H}^{(l+1)}=\sigma\!\left(\mathbf{D}^{-\frac12}\mathbf{A}\mathbf{D}^{-\frac12}\mathbf{H}^{(l)}\mathbf{W}^{(l)}\right).$$3

The graph side is handled by a hyperbolic graph encoder with neighbor attention. At each Transformer layer, the CLS representation yields a document embedding $$\mathbf{H}^{(l+1)}=\sigma\!\left(\mathbf{D}^{-\frac12}\mathbf{A}\mathbf{D}^{-\frac12}\mathbf{H}^{(l)}\mathbf{W}^{(l)}\right).$$4, from which a graph embedding $$\mathbf{H}^{(l+1)}=\sigma\!\left(\mathbf{D}^{-\frac12}\mathbf{A}\mathbf{D}^{-\frac12}\mathbf{H}^{(l)}\mathbf{W}^{(l)}\right).$$5 is computed using hyperbolic aggregation over $$\mathbf{H}^{(l+1)}=\sigma\!\left(\mathbf{D}^{-\frac12}\mathbf{A}\mathbf{D}^{-\frac12}\mathbf{H}^{(l)}\mathbf{W}^{(l)}\right).$$6. The topic embedding $$\mathbf{H}^{(l+1)}=\sigma\!\left(\mathbf{D}^{-\frac12}\mathbf{A}\mathbf{D}^{-\frac12}\mathbf{H}^{(l)}\mathbf{W}^{(l)}\right).$$7, graph embedding $$\mathbf{H}^{(l+1)}=\sigma\!\left(\mathbf{D}^{-\frac12}\mathbf{A}\mathbf{D}^{-\frac12}\mathbf{H}^{(l)}\mathbf{W}^{(l)}\right).$$8, and token sequence $$\mathbf{H}^{(l+1)}=\sigma\!\left(\mathbf{D}^{-\frac12}\mathbf{A}\mathbf{D}^{-\frac12}\mathbf{H}^{(l)}\mathbf{W}^{(l)}\right).$$9 are concatenated:

$d$0

The ensuing hyperbolic Transformer layer uses asymmetric multi-head attention: queries come only from the original token stream, whereas keys and values include the injected hierarchy tokens. This design allows every token to attend to topic and graph hierarchy at each depth.

Topic modeling is implemented through reconstruction rather than through an ELBO or Dirichlet prior. Topic-word distributions are computed by

$d$1

and each document’s reconstructed bag-of-words vector is

$d$2

Training combines a topic reconstruction loss with a hyperbolic graph contrastive loss, and optionally a supervised classification loss. The topic tree is dynamically updated during training by adding or pruning branches according to their corpus-level usage.

The reported empirical scope is broad: DS, ML, PL, COVID, DBLP, and Web, with DBLP containing $d$3 documents and $d$4 links, and Web containing $d$5 documents and $d$6 links. In unsupervised classification, GTFormer reports Micro F1 scores of $d$7 on DS, $d$8 on ML, $d$9 on PL, and $$\hat{\mathbf{z}}_d=\mathrm{softmax}\!\left(\mathbf{H}^{(L)}_d\right).$$0 on COVID; in supervised classification the corresponding values are $$\hat{\mathbf{z}}_d=\mathrm{softmax}\!\left(\mathbf{H}^{(L)}_d\right).$$1, $$\hat{\mathbf{z}}_d=\mathrm{softmax}\!\left(\mathbf{H}^{(L)}_d\right).$$2, $$\hat{\mathbf{z}}_d=\mathrm{softmax}\!\left(\mathbf{H}^{(L)}_d\right).$$3, and $$\hat{\mathbf{z}}_d=\mathrm{softmax}\!\left(\mathbf{H}^{(L)}_d\right).$$4. Link prediction AUC reaches $$\hat{\mathbf{z}}_d=\mathrm{softmax}\!\left(\mathbf{H}^{(L)}_d\right).$$5 on DS, $$\hat{\mathbf{z}}_d=\mathrm{softmax}\!\left(\mathbf{H}^{(L)}_d\right).$$6 on ML, $$\hat{\mathbf{z}}_d=\mathrm{softmax}\!\left(\mathbf{H}^{(L)}_d\right).$$7 on PL, $$\hat{\mathbf{z}}_d=\mathrm{softmax}\!\left(\mathbf{H}^{(L)}_d\right).$$8 on COVID, $$\hat{\mathbf{z}}_d=\mathrm{softmax}\!\left(\mathbf{H}^{(L)}_d\right).$$9 on DBLP, and $$\hat{\mathbf{x}}_d=\mathrm{softmax}\!\big(G(\hat{\mathbf{z}}_d)\big),$$0 on Web. Topic coherence in NPMI is also reported as higher than the baselines, for example $$\hat{\mathbf{x}}_d=\mathrm{softmax}\!\big(G(\hat{\mathbf{z}}_d)\big),$$1 versus HGTM’s $$\hat{\mathbf{x}}_d=\mathrm{softmax}\!\big(G(\hat{\mathbf{z}}_d)\big),$$2 on DS and $$\hat{\mathbf{x}}_d=\mathrm{softmax}\!\big(G(\hat{\mathbf{z}}_d)\big),$$3 versus $$\hat{\mathbf{x}}_d=\mathrm{softmax}\!\big(G(\hat{\mathbf{z}}_d)\big),$$4 on Web (Zhang et al., 17 Feb 2025).

GTFormer is important because it makes hierarchy itself the core modeling object. Relative to earlier NGTMs, it is explicitly non-Euclidean, explicitly hierarchical, and more tightly integrated across token-level semantics and graph structure.

5. NGTM as an interpretable graph generator

A different meaning of NGTM appears in 2025, where the term names a generative framework for graphs rather than a model for document corpora. Here, each graph is treated as a mixture of latent topics, each topic defines a distribution over semantically meaningful substructures, and the final graph is assembled sequentially under a global structural guidance variable (Zhuang et al., 17 Jul 2025).

The generative story introduces $$\hat{\mathbf{x}}_d=\mathrm{softmax}\!\big(G(\hat{\mathbf{z}}_d)\big),$$5 topics with Gaussian priors over latent substructure embeddings, $$\hat{\mathbf{x}}_d=\mathrm{softmax}\!\big(G(\hat{\mathbf{z}}_d)\big),$$6. A graph-level topic proportion vector is produced by

$$\hat{\mathbf{x}}_d=\mathrm{softmax}\!\big(G(\hat{\mathbf{z}}_d)\big),$$7

and a global structural vector is sampled as

$$\hat{\mathbf{x}}_d=\mathrm{softmax}\!\big(G(\hat{\mathbf{z}}_d)\big),$$8

For each substructure occurrence $$\hat{\mathbf{x}}_d=\mathrm{softmax}\!\big(G(\hat{\mathbf{z}}_d)\big),$$9, the model samples a topic index $\Phi_i=G(\mathbf{e}_i)$0, then a latent substructure vector $\Phi_i=G(\mathbf{e}_i)$1, decodes a substructure adjacency matrix $\Phi_i=G(\mathbf{e}_i)$2, computes a soft mapping matrix $\Phi_i=G(\mathbf{e}_i)$3, and updates the graph by

$\Phi_i=G(\mathbf{e}_i)$4

This yields a traceable account of which topic introduced which substructure and where it was attached.

Training is variational. The ELBO decomposes over Gaussian latents for topic mixture, global structure, and topic-conditioned substructure variables. Reconstruction uses a micro–macro loss that aligns local patterns and global graph statistics. To encourage topic separation, the model penalizes alignment among topic means through

$\Phi_i=G(\mathbf{e}_i)$5

The final loss is

$\Phi_i=G(\mathbf{e}_i)$6

The architecture uses three GNN encoders—Topic Encoder, Global Encoder, and Structure Encoder—each with four GCN layers, Layer Normalization, and graph-level average pooling. Their outputs parameterize Gaussian means and variances through MLPs. The Structure Decoder is a three-layer MLP; the Node Position Encoder uses MultiheadAttention; the Mapping Network has three linear layers with LayerNorm. All hidden dimensions are $\Phi_i=G(\mathbf{e}_i)$7. Optimization uses Adam with learning rate $\Phi_i=G(\mathbf{e}_i)$8 for up to $\Phi_i=G(\mathbf{e}_i)$9 epochs, or $\mathrm{Dirichlet}(\boldsymbol{\alpha})$0 on ogbg-molbbbp.

The reported datasets are Lobster, MUTAG, PTC, and ogbg-molbbbp. On MUTAG, NGTM reports $\mathrm{Dirichlet}(\boldsymbol{\alpha})$1 and $\mathrm{Dirichlet}(\boldsymbol{\alpha})$2; on Lobster, $\mathrm{Dirichlet}(\boldsymbol{\alpha})$3 and $\mathrm{Dirichlet}(\boldsymbol{\alpha})$4; on PTC, $\mathrm{Dirichlet}(\boldsymbol{\alpha})$5 and $\mathrm{Dirichlet}(\boldsymbol{\alpha})$6; on ogbg-molbbbp, $\mathrm{Dirichlet}(\boldsymbol{\alpha})$7 and $\mathrm{Dirichlet}(\boldsymbol{\alpha})$8. Ablation shows that removing global guidance degrades performance sharply, for example on ogbg-molbbbp raising MMD to $\mathrm{Dirichlet}(\boldsymbol{\alpha})$9 and reducing F1 to $\rightarrow$00 (Zhuang et al., 17 Jul 2025).

This model also foregrounds controllability. Increasing Topic 1 reduces graph diameter, in one sweep from $\rightarrow$01 to $\rightarrow$02, while increasing Topic 3 raises modularity from $\rightarrow$03 to $\rightarrow$04. On PTC, increasing Topic 3’s weight raises the proportion of graphs classified as carcinogenic by a GIN trained on real data, reaching approximately $\rightarrow$05 when Topic 3 dominates; increasing Topic 1 produces the opposite tendency. In this usage, NGTM is no longer about discovering topics in text, but about using topic mixtures as interpretable controls over graph synthesis.

6. Comparative structure, misconceptions, and limitations

Across these formulations, “topic” refers to different semantic objects. In GTM it is a simplex distribution over vocabulary items mediated by a decoder; in TEC it is a centroid in an entity embedding space; in GTFormer it is a node in a learned topic tree with level-specific semantics; in the 2025 NGTM it is a distribution over latent graph substructures (Zhou et al., 2020, Loureiro et al., 2023, Zhang et al., 17 Feb 2025, Zhuang et al., 17 Jul 2025). This suggests that NGTM is best understood through structural analogy rather than through a fixed probabilistic definition.

The role of the graph also changes. GTM uses a bipartite document–word graph to propagate corpus context. TEC uses a knowledge graph as an external semantic prior for entity representation. GTFormer uses inter-document links and hyperbolic geometry to model graph hierarchy jointly with topic hierarchy. The graph-generation NGTM treats the graph itself as the output object and uses topics to organize reusable substructures. Consequently, graph awareness can mean message passing, knowledge-grounded embedding, hierarchical geometry, or interpretable assembly.

Several technical distinctions are recurrent sources of confusion. First, not every NGTM is generative in the classical topic-model sense: GTM uses WAE-style prior matching, TEC is clustering-based, GTFormer relies on reconstruction plus graph contrastive learning, and only the 2025 NGTM is a full graph generator. Second, not every NGTM uses message-passing GNNs: TEC explicitly relies on node2vec rather than GCN or GAT. Third, interpretability is not uniform. GTM exposes topic-word distributions, TEC exposes entity clusters indexed by Wikidata concepts, GTFormer exposes hierarchical topics and level/path distributions, and the graph-generation NGTM exposes topic-conditioned substructure placements.

The limitations are correspondingly model-specific. GTM can propagate noise if TF–IDF assigns nontrivial weight to spurious document–word associations, and deeper GCNs may over-smooth, which is why the paper fixes $\rightarrow$06. TEC depends on entity-rich documents, uses NPMI-based evaluation that may misalign with human judgments in sparse settings, and employs a shallow graph model whose replacement by multilayer GNNs is presented as future work. GTFormer introduces sensitivity to curvature choice, tree-update thresholds, and hyperbolic numerical stability. The graph-generation NGTM scales well for small-to-medium graphs but incurs roughly $\rightarrow$07 assembly cost and may suffer from topic mode collapse or overlapping soft placements if regularization is insufficient (Zhou et al., 2020, Loureiro et al., 2023, Zhang et al., 17 Feb 2025, Zhuang et al., 17 Jul 2025).

Taken together, these works define NGTM not as a single method but as a research direction: topic models whose latent semantic structure is derived from graph connectivity, graph hierarchy, or graph-part decomposition. The direction has already split into at least two mature branches—document modeling and graph generation—and the most plausible near-term implication is further diversification rather than convergence on one standard architecture.