--- title: 'Plan-then-Generate: Two-Stage Process' url: https://www.emergentmind.com/topics/plan-then-generate type: topic --- # Plan-then-Generate: Two-Stage Process Searching arXiv for the specified paper and closely related plan-then-generate work to ground the article. Plan-then-generate is a family of generation architectures in which a model first produces an explicit intermediate plan and only then realizes a final output conditioned on that plan. Across the literature, the “plan” varies substantially in granularity and ontology: it may be a short-horizon alignment schedule for neural machine translation, an ordered list of slot keys or predicates for data-to-text generation, a sequence of question blueprints for grounded long-form writing, or a symbolic action sequence for automated planning. The common thread is a structural decomposition of generation into at least two stages—planning and realization—motivated by controllability, grounding, faithfulness, or long-horizon coordination rather than by direct end-to-end decoding alone [1706.05087], [2108.13740], [2404.03381]. ## 1. Conceptual scope and definitions In the narrow sense used by "Plan, Attend, Generate: Character-Level Neural Machine Translation with Planning" [1706.05087], plan-then-generate refers to planning **future source-target alignments** before or while generating the next target symbols. In that formulation, the plan is not a discourse outline or semantic sketch; it is a latent alignment schedule inside the decoder. The model first proposes a short-horizon alignment trajectory and then generates target characters while following or revising that plan [1706.05087]. In broader text-generation work, the intermediate plan is often more interpretable. "Plan-then-Generate: Controlled Data-to-Text Generation via Planning" [2108.13740] defines a content plan as an ordered list of slot keys or predicates, encoding both **what content should be mentioned** and **in what order it should be expressed**. "Learning to Plan and Generate Text with Citations" [2404.03381] conceptualizes plans as a sequence of questions serving as blueprints of generated content and its organization. "Text-Blueprint" [2305.00034] likewise uses a sequence of question-answer pairs, or in the interactive variant a sequence of questions, as a blueprint plan for query-focused summarization. A more formal variant appears in symbolic planning systems. "Plansformer" [2212.08681] treats classical automated planning as a sequence generation problem in which the explicit symbolic plan is itself the generated output. "Plan2Evolve" [2509.21543] places formal PDDL planning before natural-language rationale generation: a domain is generated, a symbolic plan is solved, a symbolic state-transition trace is constructed, and only then is a natural-language chain of thought produced. This suggests that plan-then-generate is better understood as a general design pattern for introducing a structured intermediate artifact between input and final output, rather than as a single architectural template. ## 2. Alignment planning in sequence-to-sequence models The most technically specific early neural instantiation is PAG, introduced for character-level neural machine translation in [1706.05087]. The underlying motivation is that standard attention-based encoder-decoder models compute alignments locally at each decoding step, which can be limiting for long character sequences because the decoder must repeatedly rediscover alignment structure online. The proposed response is a planning-ahead mechanism: before generating future target symbols, the decoder constructs a plan of future alignments and decides how long to commit to it [1706.05087]. PAG augments a standard attention-based sequence-to-sequence architecture with two planning states: an **alignment plan matrix** and a **commitment plan vector**. The alignment plan matrix stores proposed alignments for the current and several future decoding steps, $$ \mathbf{A}_t \in \mathbb{R}^{k \times |X|}, $$ where row \(0\) stores the alignment logits to use immediately and later rows encode a rolling horizon of future attentional decisions. The actual alignment is read from the first row, $$ \alpha_t = \text{softmax}(\mathbf{A}_t[0]). $$ The commitment vector $$ \mathbf{c}_t \in \mathbb{R}^k $$ governs whether the decoder follows the current plan or recomputes a new one [1706.05087]. The model’s recurrent backbone remains conventional: $$ \mathbf{h}_i = [\mathbf{h}_i^\rightarrow; \mathbf{h}_i^\leftarrow], \qquad \psi_t = \sum_i^{|X|}\alpha_{ti}\mathbf{h}_i, \qquad \mathbf{s}_t = f_{\text{dec-rnn}}(\mathbf{s}_{t-1}, \mathbf{y}_t, \psi_t). $$ What changes is the source of \(\alpha_t\): attention is no longer recomputed purely myopically but is read from a planned structure. The paper also defines a candidate alignment plan \(\bar{\mathbf{A}}_t\), row summaries \(\beta_t^i = f_r(\mathbf{A}_{t-1}[i])\), and update gates that interpolate between old and new plans, $$ \mathbf{u}_{ti} = f_{\text{up}}(\mathbf{h}_i, \mathbf{s}_{t-1}), $$ $$ \mathbf{A}_t[:,i] = (1-\mathbf{u}_{ti}) \odot \mathbf{A}_{t-1}[:,i] + \mathbf{u}_{ti} \odot \bar{\mathbf{A}}_t[:,i]. $$ This is not merely additional decoder state; it is an explicit rolling representation of future attentional actions [1706.05087]. The commitment mechanism discretizes a short planning schedule: $$ \mathbf{c}_t = \mathtt{gumbel\_softmax}(f_c(\mathbf{s}_{t-1})), \qquad \bar{\mathbf{c}}_t = \mathtt{one\_hot}(\mathbf{c}_t), $$ with the first element acting as a switch \(g_t = \bar{\mathbf{c}}_t[0]\). If \(g_t = 0\), the model follows the current plan by shifting the plan forward, $$ \mathbf{A}_t = \rho(\mathbf{A}_{t-1}), \qquad \mathbf{c}_t = \rho(\mathbf{c}_{t-1}). $$ If \(g_t = 1\), it recomputes the commitment vector and refreshes the alignment plan [1706.05087]. The paper regularizes this behavior with $$ \mathcal{L}_{\text{com}} = \lambda_{\text{com}} \sum_{t=1}^{|X|}\sum_{i=0}^{k}\left\| \frac{1}{k} - \mathbf{c}_{ti} \right\|_2^2, $$ so that the model does not immediately collapse back to per-step replanning. The paper also introduces **rPAG**, a cheaper variant that dispenses with the explicit \(k \times |X|\) plan matrix and simply reuses the current alignment until the commitment switch triggers recomputation: $$ \alpha_t = \alpha_{t-1} $$ when \(g_t=0\). PAG is thus an explicit short-horizon planner, whereas rPAG is a lighter reuse policy with an implicit planning mechanism [1706.05087]. ## 3. Training and empirical profile of early neural planning models PAG is trained end to end with conditional negative log-likelihood, $$ \mathcal{L} = -\frac{1}{N}\sum_{n=1}^N \log p_\theta(\mathbf{y}^{(n)} \mid \mathbf{x}^{(n)}), $$ augmented by the commitment penalty. The use of Gumbel-Softmax and a straight-through estimator makes the commitment behavior differentiable in practice. The authors note that they tried REINFORCE but found Gumbel-Softmax performed better [1706.05087]. The experiments are on WMT’15 character-level translation tasks—English \(\rightarrow\) German, English \(\rightarrow\) Czech, and English \(\rightarrow\) Finnish—with BPE on the source side and character generation on the target side [1706.05087]. The planning models generally outperform the baseline on almost all language pairs, and the paper emphasizes two efficiency properties: PAG and rPAG were trained for **350K** updates versus **680K** for the baseline, and used about **4M fewer** parameters than the baseline [1706.05087]. The reported qualitative signature is smoother and more intuitive alignments, with the baseline often placing too much attention on the final empty source token after the first few characters of a word, effectively switching into language-model mode once it has guessed the word. PAG and rPAG maintain more coherent source tracking [1706.05087]. The later paper "Plan, Attend, Generate: Planning for Sequence-to-Sequence Models" [1711.10462] extends the same core idea to broader sequence-to-sequence settings. Its abstract states that the model constructs a matrix of proposed future alignments and a commitment vector that governs whether to follow or recompute the plan; that the mechanism is inspired by the strategic attentive reader and writer (STRAW) model; and that it outperforms a strong baseline on character-level translation tasks from WMT’15, the algorithmic task of finding Eulerian circuits of graphs, and question generation from text [1711.10462]. However, the available supplementary fragment for [1711.10462] does not contain the architecture, update equations, objective functions, or numeric results; what it directly shows is that the proposed model, called PAG, learns different and apparently better alignments than a baseline and that these alignments correlate with stronger qualitative translation outputs. This suggests continuity with the explicit alignment-planning mechanism detailed in [1706.05087]. ## 4. Content planning for controllable generation A distinct line of work treats planning as an explicit content-ordering interface. "Plan-then-Generate: Controlled Data-to-Text Generation via Planning" [2108.13740] formulates generation from structured data as a two-stage pipeline. The input dataset is $$ \mathcal{D}=\{(T, C, S)_i\}_{i=1}^{|D|}, $$ where \(T\) is linearized structured data, \(C\) is a reference content plan, and \(S\) is the reference text. For tabular inputs, each \(t_i\) is a slot key-value pair \(\{k_i, v_i\}\); for graph inputs, each \(t_i\) is an RDF triple \(\{s_i, p_i, o_i\}\). The content plan $$ C=\{c_1,\ldots,c_{|C|}\} $$ is an ordered list of slot keys or predicates extracted heuristically by delexicalization, $$ C=\mathcal{F}(T,S). $$ This plan makes content selection and ordering explicit [2108.13740]. The planner encodes candidate plan tokens with BERT-base and predicts their ordering with a linear-chain CRF: $$ P_{\textup{CRF}}(Y|H_C)= \frac{1}{Z}\exp\left(\sum_{i=1}^{|C|}\Phi_{y_i}(h^c_i) + \sum_{i=2}^{|C|}M_{y_{i-1}, y_i}\right), $$ followed by $$ \tilde{Y} = \argmax_{Y^{\prime}} P_{\textup{CRF}}(Y^{\prime}|H_C). $$ The labels represent output positions, with \(\emptyset\) indicating omission of a candidate token. The generator, built on BART-base, conditions on the concatenation \([T:C]\) and is trained with $$ \mathcal{L}_{\textup{LM}} = -\sum_{i=1}^{|S|}\log P_{G}(S_i|S_{