--- title: Plan-Based Generation Overview url: https://www.emergentmind.com/topics/plan-based-generation type: topic --- # Plan-Based Generation Overview Plan-Based Generation encompasses a family of methods that explicitly decompose complex generation tasks (text, code, structured data, control sequences) into intermediate representations, called plans, prior to the final realization. This paradigm enhances controllability, transparency, factual accuracy, and modularity across a wide range of tasks, including natural language generation, data-to-text systems, code generation, robotics, and multi-agent planning. Plans may take the form of outlines, anchor words, question–answer blueprints, ordered token sequences, paragraph-level instructions, or graphical control-flow models, and typically structure “what to say” and “in what order” before the final output is rendered. ## 1. Formal Foundations and Taxonomy The essential principle of plan-based generation is the explicit factorization of the generative process into at least two stages: first, construction of an intermediate plan $B$ (also denoted $P$, $z$, or $C$ depending on context); second, realization of surface output $y$ from this plan. Typical probabilistic formalizations include: \[ P(B, y | x) = P(B | x) \times P(y | x, B) \] where $x$ denotes the input (which could be a user query, document set, knowledge graph, or other signal), $B$ the blueprint or plan (sequence of subgoals, questions, or content selectors), and $y$ the final output. The nature of $B$ defines the plan’s granularity: it can be a sequence of question–answer pairs [2305.00034], an ordered list of content units [2108.13740], a rooted tree structure [2310.08185], a set of anchor tokens [2010.03272], or graphical process models [2410.12870, 2504.01637]. Archetypal plan-based generation settings include: - **Conditional Text Generation**: Plan as blueprint QA pairs [2305.00034], content outlines, or paragraph structures [2408.10490, 2502.19103], narrative trees [2310.08185], or anchor word sequences [2010.03272]. - **Data-to-Text**: Plans as ordered lists of table slots or facts [2108.13740], or as sentence-structured graphs [1909.09986]. - **Retrieval-Augmented Generation (RAG) and Code Generation**: Plans guide retrieval and in-context selection of facts or code examples via pseudocode queries [2408.10490, 2412.12447]. - **Robotics and Multi-Agent Systems**: Plans as temporal event sequences, simple temporal networks, or agent–status graphs [2504.01637, 1804.00038, 2512.21293]. - **Skill Learning and Process Mining**: Structured process models as plans for parallelism and interpretability [2410.12870]. ## 2. Algorithmic Frameworks and Model Architectures Variants of plan-based generation diverge with respect to plan representation, generative architecture, supervision level, and degree of interactivity. **Text Generation**: Transformer encoder–decoders are standard. In “Text-Blueprint” [2305.00034], a LongT5 encoder–decoder generates QA-pair blueprints (plan $B$), then uses cross-attention to realize summaries conditioned on both input and blueprint. The “Plan-then-Generate” approach [2108.13740] uses a BERT-based content planner over structured input $T$ to produce a plan $C$, followed by a BART-based generator consuming $T$ and $C$. “Latent Anchor Plan” [2010.03272] introduces a latent variable $z$ (sequence of anchor tokens) sampled from a learned or inferred prior, conditioning generation at a per-sentence level via LSTM-based architectures, with learning handled by amortized variational inference. **Code Generation**: “Plan-As-Query Example Retrieval” (PERC) [2412.12447] operates by converting both code pool and queries into pseudocode plans, retrieving top-k in-plan examples, and assembling (description, plan, code) tuples for few-shot prompting of the code LLM. **Retrieval-Augmented Knowledge Generation**: “Retrieve-Plan-Generation" (RPG) [2406.14979] employs an explicit plan–answer loop. At each iteration, plan tokens are generated, inform fine-grained evidence selection, then the answer segment is generated, and the process repeats. Soft prompts and low-rank adapters enable multi-task conditioning (planning vs. answering) without full LLM fine-tuning. **Robotics and Control**: In “LLM-mediated Multi-Agent Planning” [2504.01637], environmental statuses and goals are encoded as discrete symbols, with a GPT-4o-based agent generator recursively constructing ($\text{Cond}_i$, $\text{Add}_i$, $\text{Del}_i$) triplets; plans are realized as edges in a directed status–agent graph. Hierarchical decompositions (task-level then motion-level) appear in multi-robot systems [1804.00038], with simple temporal networks employed for schedule feasibility. **Skill Learning**: Process mining is used to discover, store, and retrieve structured control-flow plans as reusable “skills,” augmenting LLM-based planners with the ability to parallelize and interpret action traces [2410.12870]. ## 3. Evaluation Protocols and Empirical Findings Comprehensive evaluation of plan-based generation spans automatic metrics, human judgments, and ablation studies. - **Factuality and Faithfulness**: Plan-guided generation substantially reduces hallucination rates, e.g., ROUGE-1/2/L improvement (+1.5/+1.2 over no-plan T5), hallucination reduction (–10–15 points), and human faithfulness preference (80% vs. 55% for non-plan baselines) [2305.00034, 2408.10490]. - **Controllability and Interpretability**: The explicit plan allows direct manipulation (add/remove/edit questions, reorder content units), supporting desired rhetorical flow and forcing inclusion/exclusion of facts [2305.00034, 2108.13740]. Human raters consistently prefer plan-based outputs for control and traceability [2305.00034, 2502.19103]. - **Retrieval Coverage and Attribution**: Plan-based retrieval settings show AIS_strict up to 90%, +15–25 points over one-shot retrieval; ROUGE-2 precision up ∼13 points [2408.10490]. Fine-grained evidence selection enabled by plan tokens increases precision and reduces off-topic drift [2406.14979]. - **Efficiency and Scalability**: Neural planners achieve several orders of magnitude speedup over exhaustive enumeration for data-to-text tasks, with linear rather than exponential scaling [1909.09986, 2108.13740]. - **Long-Form Generation**: Plan-based sectioning leads to improved length-following (Len >0.9 at each section) and +16 points overall quality over direct generation for long-form (arXiv, Wikipedia, Blogs) [2502.19103]. - **Robotics**: LLM-generated agent networks achieve 70%+ coverage of human-constructed planning graphs; naive expansion leads to combinatorial failures beyond 400 agents, suggesting a need for selective expansion heuristics [2504.01637]. Typical evaluation includes industry-standard metrics (BLEU, ROUGE, METEOR, BERTScore F1, self-BLEU/diversity, PARENT), human or LLM-based correctness and preference scoring, plan-adherence, and coverage/consistency checks tailored to application. ## 4. Design Variations and Use-Case Specialization Plan-based generation is instantiated with diverse plan representations, supervision schemes, and downstream realization modules. - **Plan Granularity**: Section- or paragraph-level plans (summaries or outlines) for long-form texts [2502.19103]; sentence-level anchor tokens for stories [2010.03272]; graph/tree/dependency structures for data-to-text [1909.09986, 2108.13740, 2310.08185]. - **Supervision**: Ranges from manual plan annotation or extraction (QA-based iterative plan extraction in EIPE-text [2310.08185]) to unsupervised, variationally-induced plans [2010.03272]. - **Domain Inputs**: Inclusion of retrieval pipelines for knowledge grounding (web search/bi-encoder/cross-encoder stacks), especially in code generation [2412.12447], clinical decision support [2503.17900], or scientific writing [2408.10490]. - **User Interactivity**: Web-based UIs for plan visualization and editing (Text-Blueprint [2305.00034]); multi-stage plan–edit–generate loops. - **Architectural Control**: From sequence-to-sequence Transformers with concatenated or prefixed plan tokens [2305.00034, 2108.13740] to hybrid systems with structured prompts or module chaining. ## 5. Applications and Impact Across Domains Plan-based generation enables transparent, configurable, and robust content creation in tasks requiring factual accuracy, procedural correctness, or human-AI collaboration. - **Text Summarization & Report Generation**: Query-focused multi-document summarization, legal/scientific reporting, interactive editorial workflows [2305.00034]. - **Long-Form and Narrative Construction**: Hierarchical planning yields more coherent and globally consistent novels, essays, and presentations [2310.08185, 2502.19103]. - **Data-to-Text Realization**: Explicit content plans allow rhythmically or factually controlled output from knowledge graphs or tables [1909.09986, 2108.13740]. - **Code Generation**: Plan-based retrieval and example selection outperform RAG baselines, especially in low-resource languages or cross-domain transfer [2412.12447]. - **Autonomous Robotics**: LLM-derived plans as structured agent graphs or simple temporal networks scale up multi-agent path planning, dynamic task decomposition, and real-world navigation [2504.01637, 1804.00038, 2512.21293]. - **Clinical Decision-Support**: Sequential plan architectures enhance medical reasoning workflows, aligning output with SOAP standards and integrating historical patient context [2503.17900]. - **Process Automation and Skill Learning**: Control-flow process models mined from plan traces bring parallelism, interpretability, and retrieval to LLM-based planners [2410.12870]. ## 6. Limitations, Challenges, and Future Directions Principal open issues in plan-based generation concern the following: - **Plan Quality Dependency**: The overall output fidelity depends on accurate and relevant initial plan generation; suboptimal plans propagate errors downstream [2305.00034, 2408.10490, 2108.13740]. - **Latency and Complexity**: Two-stage (or multi-stage) reasoning introduces additional computational steps. Iterative refinement and evidence selection can increase latency compared to direct generation [2305.00034, 2406.14979]. - **Error Propagation**: Early planning errors are persistent, especially in non-interactive or non-adaptive systems [2305.00034]. - **Scalability and Combinatorial Explosion**: Unconstrained expansion in symbolic or graph-based planning leads to combinatorial intractability in multi-agent or high-dimensional domains [2504.01637]. - **Plan Representation Selection**: The optimal granularity, structure, and induction method for plans remain highly context-dependent. Hierarchical or mixed-initiative planning and continual plan correction are active research areas [2305.00034, 2310.08185, 2502.19103]. - **Generalization and Domain Adaptability**: Some advances—especially in plan-as-query retrieval for code [2412.12447] and clinical-plan composition [2503.17900]—highlight strong cross-domain transfer, but robustness when mixing plan pools or scaling plan libraries requires further investigation. - **Integration with Retrieval and Knowledge**: Coupling plan-based decomposition with dynamic document retrieval, fine-grained evidence selection, and high-accuracy attribution is an ongoing challenge [2408.10490, 2406.14979]. Anticipated future work aims to develop hierarchical and mixed-initiative planning (both model- and user-driven), richer plan formalisms (semantic/discourse/graph-based), process-aware retrievers, interleaved retrieval–plan–generation loops, and real-world applications in collaborative and autonomous systems. --- **References** [2305.00034] Text-Blueprint: An Interactive Platform for Plan-based Conditional Generation [2408.10490] Analysis of Plan-based Retrieval for Grounded Text Generation [2108.13740] Plan-then-Generate: Controlled Data-to-Text Generation via Planning [1909.09986] Improving Quality and Efficiency in Plan-based Neural Data-to-Text Generation [2010.03272] Narrative Text Generation with a Latent Discrete Plan [2310.08185] EIPE-text: Evaluation-Guided Iterative Plan Extraction for Long-Form Narrative Text Generation [2502.19103] LongEval: A Comprehensive Analysis of Long-Text Generation Through a Plan-based Paradigm [2406.14979] Retrieve-Plan-Generation: An Iterative Planning and Answering Framework for Knowledge-Intensive LLM Generation [2412.12447] PERC: Plan-As-Query Example Retrieval for Underrepresented Code Generation [2504.01637] LLM-mediated Dynamic Plan Generation with a Multi-Agent Approach [1804.00038] Overview: A Hierarchical Framework for Plan Generation and Execution in Multi-Robot Systems [2410.12870] Skill Learning Using Process Mining for Large Language Model Plan Generation [2503.17900] MedPlan:A Two-Stage RAG-Based System for Personalized Medical Plan Generation [2512.21293] Quadrupped-Legged Robot Movement Plan Generation using Large Language Model Source: https://www.emergentmind.com/topics/plan-based-generation