--- title: Topical Hierarchical Recurrent Encoder Decoder url: https://www.emergentmind.com/topics/topical-hierarchical-recurrent-encoder-decoder-thred type: topic --- # Topical Hierarchical Recurrent Encoder Decoder A Topical Hierarchical Recurrent Encoder Decoder (THRED) is a class of dialogue generation architectures that augment hierarchical recurrent encoder–decoder models with mechanisms for topic modeling and topic-aware attention, enabling the generation of diverse, contextually appropriate, and topically coherent responses in multi-turn conversational settings. Multiple variants have been proposed; the most cited are (1) context-aware topical attention approaches that integrate LDA-derived topic signals via joint attention mechanisms and (2) topic-coherent diversification schemes combining global latent variables and word-level topic biases for further enhancing diversity without sacrificing topical relevance [1811.01063, 1910.11161]. ## 1. Core Architecture and Hierarchical Modeling The THRED architecture extends canonical sequence-to-sequence (Seq2Seq) models with hierarchical encoding and joint attention. In one formulation [1811.01063], the model encodes multi-turn dialogue by first processing each utterance via a (bi/unidirectional) GRU to obtain utterance-level representations, then passing these through a second-level GRU (the context encoder) to capture the flow of conversation across turns. This two-level hierarchical structure allows better modeling of discourse dependencies and conversational structure compared to flat encoders. A distinct line of work applies a similar hierarchical recurrent backbone but uses LSTMs at both token- and utterance-level, and further incorporates variational (global) latent variables, as in VHRED [1910.11161]. This supports richer, more diverse generation by enabling sampling of the global conversational context at inference time. ## 2. Joint Attention and Topical Signal Integration At decoding time, THRED models implement a multi-pronged attention scheme: - **Message-level (word) attention** attends to word states within each utterance to form utterance summaries. - **Context-level attention** attends over the sequence of past utterance summaries, aggregating information across dialogue turns. - **Topic-level attention** incorporates topical concept vectors extracted by unsupervised topic modeling (LDA or NMF-based). Specifically, the dialogue history is assigned to a most probable topic, from which the top-n topic words are selected and embedded. These topic embeddings are aggregated by attention, producing a topic-context vector that is fed into the decoder and/or biases the output distribution toward topical vocabulary [1811.01063]. In the diversification-oriented THRED variant, topical information is encoded as a dense matrix constructed via NMF on a word-word Positive Pointwise Mutual Information (PPMI) matrix, producing latent topic vectors for each word [1910.11161]. Per-turn topic distributions are then derived and injected into the decoder, enhancing topical control at generation time. ## 3. Mathematical Formulation Let $D = (U_1, \ldots, U_N)$ denote a multi-turn dialogue comprising $N$ utterances. ### Utterance Encoder (per [1811.01063]): For the $i$-th utterance: \[ h_{i, t} = \mathrm{GRU}(h_{i, t-1}, w_{i, t}), \qquad t = 1, \ldots, L_i \] where $w_{i, t}$ is the $t$-th word of $U_i$. ### Word (Message)-Level Attention: At decoder time $t$, for each utterance $i$: \[ e_{i, j, t} = \eta(s_{t-1}, h_{i, j}, c_{i-1, t}) \] \[ \alpha_{i, j, t} = \frac{\exp(e_{i, j, t})}{\sum_{k=1}^{L_i} \exp(e_{i, k, t})} \] \[ m_{i, t} = \sum_{j=1}^{L_i} \alpha_{i, j, t} h_{i, j} \] ### Context Encoder: \[ c_{i, t} = \mathrm{GRU}(c_{i-1, t}, m_{i, t}),\quad i=1,\dots,N \] ### Joint Attention in the Decoder: - **Context Attention**: \[ e'_{i, t} = \eta(s_{t-1}, c_{i, t}) \] \[ \gamma_{i, t} = \frac{\exp(e'_{i, t})}{\sum_{j=1}^N \exp(e'_{j, t})} \] \[ r_t = \sum_{i=1}^N \gamma_{i, t} c_{i, t} \] - **Topic Attention**: Let $\{t_1, \ldots, t_n\}$ be top-n topic word embeddings from the inferred topic. \[ \beta_{i, t} = \frac{\exp(\eta(s_{t-1}, t_i, c_{N, t}))}{\sum_{j=1}^n \exp(\eta(s_{t-1}, t_j, c_{N, t}))} \] \[ k_t = \sum_{i=1}^n \beta_{i, t} t_i \] Decoder state: \[ s_t = \mathrm{GRU}(s_{t-1}, w_{t-1}, r_t, k_t) \] Output distribution over response- and topic-vocabularies is the normalized sum of two MLP heads, biasing the generation toward on-topic content. In the diversification-oriented THRED [1910.11161], the decoder LSTM also receives a sampled global latent variable $z$ (from $p_\theta(z|s_M)$) and the per-utterance topic vector $\theta_M$: \[ h_n^{\mathrm{dec}} = \mathrm{LSTM}(h_{n-1}^{\mathrm{dec}}, [\mathbf{e}(w_{M, n-1}); z; \theta_M]) \] ## 4. Topical Concept Extraction and Topic Modeling In [1811.01063], Latent Dirichlet Allocation (LDA) with 150 topics is trained via collapsed Gibbs sampling on large conversational corpora (Reddit 1M dialogues and OpenSubtitles). For each new dialogue, the most probable topic $T$ is inferred; the top $n=100$ words by topic-word probability are selected, embedded, and used for topic-level attention during decoding. Alternatively, [1910.11161] constructs a word-topic matrix via NMF on a pointwise mutual information matrix, providing continuous-valued topic distributions for all vocabulary items. Local topic vectors for utterances are computed as: \[ \tilde\theta_m = \frac{1}{N_m} \sum_{i=1}^{N_m} T_{\mathrm{idx}(w_{m,i}), :} \] with normalization via softmax. ## 5. Training Objectives and Optimization In both THRED variants, the primary objective is maximum likelihood estimation (negative log-likelihood) over the training triples of (dialog history, inferred topic, next utterance): \[ \mathcal{L} = -\sum_{(D, T, U_{N+1})} \sum_{t=1}^{|U_{N+1}|} \log p(w_t | D, T, w_{