---
title: Dynamic KeyNMF Framework
url: https://www.emergentmind.com/topics/dynamic-keynmf
type: topic
---

# Dynamic KeyNMF Framework

Dynamic KeyNMF is a topic modeling framework that unifies transformer-based keyword importance scoring with non-negative matrix factorization (NMF) to enable tracking the evolution of topics across contiguous time periods in large text corpora. The central innovation lies in constructing a non-negative “keyword-importance” matrix using contextualized embeddings from pretrained transformers, enabling more semantically meaningful topic induction compared to token-count-based approaches. Dynamic KeyNMF incorporates both static and dynamic phases, with optional temporal regularization, and was designed to support high-resolution analysis of topical information dynamics in settings such as Chinese diaspora media in the leadup to electoral events [2410.12791].

## 1. Construction of the Keyword-Importance Matrix

Given a corpus of $D$ documents and a vocabulary $V$ of candidate keywords, an embedding model $f: \text{text} \to \mathbb{R}^d$ (e.g., multilingual Sentence-BERT) generates dense representations for each document and each token. For each document $d$, its embedding $x_d = f(d)$ is computed, as well as $v_w = f(w)$ for candidate tokens $w$. The set $K_d$ of top-$N$ tokens for $d$ is selected to maximize the sum of cosine similarities $\cos(x_d, v_w)$. The $D \times V$ keyword-importance matrix $M$ is then defined by:

$$
M_{d, w} = 
\begin{cases}
\max\{0, \cos(x_d, v_w)\}, & w \in K_d \\
0, & \text{otherwise}
\end{cases}
$$

This matrix is non-negative by construction. Unlike traditional document-term matrices filled by TF–IDF or raw counts, $M$ encodes semantic strength directly via contextual similarities.

## 2. Static KeyNMF Decomposition

The static phase factorizes $M$ into $W \in \mathbb{R}_{\ge 0}^{D \times K}$ and $H \in \mathbb{R}_{\ge 0}^{K \times V}$, where $K$ is the number of topics. The factorization minimizes the Frobenius norm:

$$
\min_{W \ge 0, H \ge 0} \| M - W H \|_F^2
$$

Optimization is performed via non-negative least squares, using coordinate-descent updates. Convergence is declared when the relative change in loss drops below $\epsilon$ (e.g., $10^{-4}$).

The coordinate-descent update for $H$ (for fixed $W$) is:

$$
H = \max\{0, H - \eta [W^\top W H - W^\top M]\}
$$

where $\eta$ is a small step-size. Updates for $W$ are symmetric.

## 3. Dynamic Slicing and Topic Evolution

To track topic evolution over $T$ time slices, Dynamic KeyNMF operates as follows:

- Compute the static factorization over the entire corpus to obtain $W$ and $H$.
- For each time slice $t$, extract $M_t$ (documents from slice $t$) and the corresponding $W_t$ (rows of $W$).
- Fix $W_t$ and re-solve for slice-specific $H_t$:

  $$
  H_t = \arg\min_{H' \ge 0} \| M_t - W_t H' \|_F^2
  $$

- Define the raw “temporal importance” of topic $j$ at time $t$:

  $$
  I_{t j} = \sum_{d=1}^{D_t} (W_t)_{d j}
  $$

- Normalize $I_{t \bullet}$ to obtain a pseudo-distribution:

  $$
  \hat{P}_{t j} = \frac{I_{t j}}{\sum_{i=1}^K I_{t i}}
  $$

This pseudo-distribution enables downstream information dynamics analyses such as novelty, transience, and resonance.

## 4. Optional Temporal Smoothness Regularization

Dynamic KeyNMF supports, but does not require, temporal smoothness regularization over topic assignments across time slices. The joint dynamic-NMF objective adds slice-to-slice continuity penalties:

$$
\min_{\{W_t, H_t\} \ge 0}
\sum_{t=1}^T \| M_t - W_t H_t \|_F^2 +
\lambda_W \sum_{t=2}^T \| W_t - W_{t-1} \|_F^2 +
\lambda_H \sum_{t=2}^T \| H_t - H_{t-1} \|_F^2
$$

where $\lambda_W$ and $\lambda_H$ are non-negative smoothness weights, typically chosen small ($\approx 0.01$–$0.1$) via cross-validation or grid search. The objective is optimized by alternating projected-gradient or coordinate-descent updates.

## 5. Preprocessing, Hyperparameters, and Practical Pipeline

Preprocessing steps include tokenization (e.g., jieba for Chinese), stopword removal, and candidate vocabulary construction. The main pipeline follows:

1. **Embedding & Keyword Extraction:** For each $d$, compute $x_d = f(d)$ and $v_w = f(w)$ for all $w$. Select top $N$ tokens per doc by $\cos(x_d, v_w)$.
2. **Matrix Construction:** Assemble $M$ as above.
3. **Static NMF:** Initialize $W$ and $H$ randomly, update via coordinate descent to convergence.
4. **Dynamic Slicing:** For each time $t$, extract $M_t$, $W_t$, solve for $H_t$; compute $\hat{P}_{t \bullet}$.
5. **(Optional) Information Dynamics:** Compute windowed Jensen–Shannon divergence between topic distributions for novelty, transience, and resonance analysis.

Key hyperparameters include $K$ (topics, typically 10–50), $N$ (keywords per doc, typically 15), $\epsilon$ (convergence threshold), $\eta$ (step size), $\lambda_W$, $\lambda_H$ (smoothness), $n$ (window size for novelty calculations), and the smoothing span for dynamics curves.

## 6. Computational Complexity and Scaling

- **Embedding:** $O(D \cdot \text{cost}_f)$ for $D$ documents.
- **Keyword Selection:** $O(D \cdot |d| \cdot d)$.
- **Static NMF:** Each iteration $O(KDV)$, for $I$ iterations.
- **Dynamic H$_t$ Re-Estimation**: Per slice, $O(K D_t V)$.
- **Total:** $O(I K D V + T K \bar{D} V)$, with $D$ documents, $T$ time slices, average slice size $\bar{D}$, and vocabulary size $V$.

For corpora with $D \approx 10^4$, $T \approx 200$, and $V \approx 5 \times 10^3$, static NMF computations dominate resource usage.

## 7. Evaluation Metrics and Benchmarks

Dynamic KeyNMF is evaluated using several metrics:

| Metric                | Description                                                  | Computed As      |
|-----------------------|-------------------------------------------------------------|------------------|
| Topic-diversity ($d$) | Fraction of unique words in union of top $L$ topic words    | Standard toolkit |
| Internal coherence ($C_{in}$) | Avg. pairwise cosine among top $L$ topic words   | Standard toolkit |
| External coherence ($C_{ex}$) | Same as $C_{in}$ but using external embeddings   | Standard toolkit |
| (Optional) NPMI       | Normalized pointwise mutual information (coherence)         | Standard toolkit |

These allow comparison against LDA, CTM, BERTopic, Top2Vec, etc., using e.g., the topic-benchmark Python package.

Dynamic KeyNMF has demonstrated competitive performance on several Chinese datasets, substantiating its suitability for high-resolution analysis of topical information dynamics in multilingual and non-Western corpora [2410.12791].

Source: https://www.emergentmind.com/topics/dynamic-keynmf