Utility-Guided Dynamic Outline Optimization

• Utility-Guided Dynamic Outline Optimization is a method that dynamically refines hierarchical outlines using explicit utility signals to assess and guide structural edits.
• It employs a structured decision process with atomic operations—expansion, contraction, and revision—integrated into a closed-loop optimization framework.
• This approach enhances performance in applications like research report generation, mathematical reasoning, and adaptive UI design by combining retrieval, structure, and generation utilities.

Utility-guided dynamic outline optimization refers to a class of methods that dynamically construct and refine outline structures in reasoning, generation, or design tasks, guided by explicit utility signals that estimate the downstream value of outline modifications. These methods contrast with static or heuristic-based outlining approaches by embedding outline evolution within a closed-loop optimization process, where the agent assesses the incremental utility of structural edits and continually updates the scaffold to maximize composite objectives. This paradigm has been influential in domains such as open-ended deep research report generation, multi-path mathematical reasoning, and adaptive user interface (UI) design, establishing itself as a core mechanism for coordinated multi-agent, multi-stage, or RL-driven systems.

1. Formal Foundations and Decision Process

Utility-guided dynamic outline optimization is typically formalized as a structured decision process over outline states. The state at iteration $t$ consists of the current outline scaffold—often represented as a tree or set of nodes $T_t=(V_t, P_t, \Phi_t)$, where $V_t$ denotes outline nodes (sections/subsections), $P_t$ are parent–child relations, and $\Phi_t$ captures node attributes including intent, attached evidence, and utility statistics (Yang et al., 18 Jun 2026). At each step, the agent selects a target node $v_t$ and an atomic operation $o_t$ from {Expansion, Contraction, Revision}, yielding a transformed outline $T_{t+1}$ and computing the scalar utility $U_t$ of the edit.

Node selection commonly employs an upper-confidence bound (UCB) style policy balancing exploitation (edits on low-utility nodes) with exploration (rarely visited nodes):

$$v_t = \arg\max_v \left[ -\bar{u}(v) + c \sqrt{\ln N_t / n_v} \right]$$

where $T_t=(V_t, P_t, \Phi_t)$0 is the running mean utility for node $T_t=(V_t, P_t, \Phi_t)$1, $T_t=(V_t, P_t, \Phi_t)$2 its visitation count, $T_t=(V_t, P_t, \Phi_t)$3 the total edit count, and $T_t=(V_t, P_t, \Phi_t)$4 an exploration coefficient (Yang et al., 18 Jun 2026). The agent interleaves node selection, operation selection, and feedback-driven utility updating until outline improvement saturates, terminating when recent marginal utility falls below a threshold.

2. Utility Function Design

Utility signals are designed to measure the downstream benefit of specific outline modifications, integrating retrieval, structural, and generative criteria. In open-ended research settings (e.g., ScaffoldAgent), the composite utility function is:

$T_t=(V_t, P_t, \Phi_t)$5

with default weights $T_t=(V_t, P_t, \Phi_t)$6.

• Retrieval Utility ($T_t=(V_t, P_t, \Phi_t)$7): Assesses the relevance ($T_t=(V_t, P_t, \Phi_t)$8) and novelty ($T_t=(V_t, P_t, \Phi_t)$9) of new evidence retrieved as a result of the outline modification.

$V_t$0

with $V_t$1, $V_t$2 (Yang et al., 18 Jun 2026).

• Structure Utility ($V_t$3): Encodes semantic coherence between parent and child intents, structural balance (depth and branching regularity), and redundancy penalties.

$V_t$4

with $V_t$5, $V_t$6, $V_t$7.

• Generation Utility ($V_t$8): Evaluates NLI-based support for trial-generated claims, coverage of section intent, and avoidance of content redundancy.

$V_t$9

with $P_t$0, $P_t$1, $P_t$2.

This multi-factor utility more accurately tracks the value of edit actions than proxy metrics such as token counts or static LLM assessments, enabling precise identification of beneficial scaffold changes (Yang et al., 18 Jun 2026, Guo et al., 9 Feb 2026).

3. Outline Operations, Feedback, and Optimization Strategies

Outline evolution is operationalized via a finite set of atomic actions:

• Expansion: Splits an under-granular node into child nodes, typically after identifying evidence clusters or unresolved facets. Expansion invokes both retrieval to populate new nodes and trial writing to assess splitting utility.
• Contraction: Merges semantically overlapping or weakly differentiated sibling nodes, reducing scaffold fragmentation and enhancing coherence.
• Revision: Rewrites a node’s intent and refreshes its evidence without altering tree topology, correcting drift or factual outdatedness.

Every operation is followed by trial generation (writing the subtree or section), retrieval sampling, and structure/utility computation for immediate feedback. Statistics $P_t$3 are incrementally updated per node, and the policy is dynamically refocused on persistently weak outline regions. Termination occurs when average utility gain across the most recent $P_t$4 steps drops below $P_t$5 (Yang et al., 18 Jun 2026); this prevents overfitting to local scaffold microstructure.

The general process is summarized in the following pseudo-code:

$\Phi_t$1 (Yang et al., 18 Jun 2026)

4. Examples of Utility-Guided Dynamic Outline Optimization

Utility-guided dynamic outline optimization has been instantiated in several representative systems:

a. ScaffoldAgent for Open-Ended Deep Research

ScaffoldAgent implements utility-guided outline optimization for long-form, evidence-grounded report generation. The agent continually evolves an outline scaffold through atomic operations, each evaluated via composite utility derived from evidence retrieval, structure, and trial generation. Empirically, ScaffoldAgent outperforms prior strong baselines (e.g., IRCoT, WebShaper, EDR) on DeepResearch Bench and DeepResearch Gym, yielding improvements in overall report quality (RACE Overall +2.2 vs. best baseline), citation efficacy, and factual grounding (Yang et al., 18 Jun 2026). Ablations confirm that removing any operation (Expansion, Contraction, Revision) or any utility component degrades performance by 3–5 points, demonstrating their necessity.

b. Outline-Guided Path Exploration (OPE) in Mathematical Reasoning

OPE decomposes mathematical problem solving for large reasoning models into an outline-planning phase and a conditional path-exploration phase. Utility signals derived from binary task rewards propagate upstream to guide outline diversification, measured as “Planning Gain” (the rate at which outlines alone partition solution space to increase mutual information with correct answers). GRPO (group relative policy optimization) updates outline-generating policies toward regions of high downstream path utility. Empirical results show significant gains in pass@k and best-of-N accuracy across standard math benchmarks compared to naïve parallel thinking, and a documented breakage of the mutual information saturation bottleneck (Guo et al., 9 Feb 2026).

c. AutoOptimization of User Interface Layouts

In adaptive UI design, utility-guided dynamic optimization is instantiated as a multi-objective problem over layout variables, with objectives and weights automatically inferred from user verbal instructions. Optimization proceeds via evolutionary Pareto-front search (NSGA-III), with candidate solutions ranked and filtered according to utility determined by a Vision-LLM (VLM) that scores alignment with user intent. Although the outline is implicit in UI configuration, the process involves iterative refinement of candidate sets and their decomposition/aggregation according to multi-dimensional utility signals (Li et al., 13 Feb 2026).

5. Mutual Information Bottleneck and Diversity Metrics

A central challenge in multi-path or multi-agent settings is the mutual information (MI) bottleneck: as parallel paths are enumerated, the marginal information gain from each additional reasoning trajectory decays rapidly due to redundancy (“mutual information saturation”) (Guo et al., 9 Feb 2026). Naïve strategies maximize

$P_t$6

which saturates quickly as $P_t$7 increases. By contrast, utility-guided outline approaches explicitly optimize

$P_t$8

where $P_t$9, the “Planning Gain,” captures diversity and partitioning power of generated outlines. This joint optimization yields exploration paths that more effectively cover the solution space, increasing unique solution rate, output diversity, and success probability at higher $\Phi_t$0 (best-of-k metrics) (Guo et al., 9 Feb 2026).

6. Experimental Benchmarks and State-of-the-Art Results

Evaluation across diverse tasks demonstrates that utility-guided dynamic outline optimization consistently outperforms static or locally heuristic baselines:

• DeepResearch Bench & Gym (ScaffoldAgent): RACE Overall 44.70 (Qwen3-32B), effective citation rate 30.42, citation accuracy 54.32, with monotonic improvements across model backbones (Yang et al., 18 Jun 2026).
• Mathematical Reasoning (OPE): +3.38% best-of-N and +3.90% self-consistency average accuracy across six math benchmarks compared to naïve RL baselines; unique answer coverage and more concise proof chains (Guo et al., 9 Feb 2026).
• UI Layout (AutoOptimization): Efficient candidate filtering and UI adaptations without manual Pareto-front inspection or weight tuning; full pipeline from verbal preference to validated layout solution (Li et al., 13 Feb 2026).

Ablation studies uniformly confirm that dynamic, utility-driven outline operations and composite feedback signals are essential for optimal performance. When outline adaptation is disabled or reduced to static planning, substantial regressions are observed in both coverage and correctness metrics.

7. Illustrative Dynamics and Interpretations

Dynamic outline optimization in practice proceeds through cycles of refinement, focusing expansion on under-specified or weak regions, contracting redundant branches, and revising outdated scaffolds. For instance, in deep research, a query about “mRNA vaccine platforms” initiates with a coarse root node, followed by expansions into “Lipid nanoparticle delivery,” “Antigen design,” and “Manufacturing challenges.” Over subsequent iterations, fine-grained splitting, targeted merging based on redundancy, and title revisions informed by retrieval drive an increasingly high-utility, semantically organized outline, directly supporting downstream report coherence and factuality (Yang et al., 18 Jun 2026). This process avoids scaffold drift and ensures outline complexity tracks informativeness rather than token count.

A plausible implication is that utility-guided dynamic outline optimization provides a foundational general framework adaptable to any reasoning, generation, or synthesis problem that can be decomposed into hierarchical or modular subproblems, where expert guidance, diversity, and downstream outcome utility are required. As systems scale to more complex or open-ended tasks, such algorithms are likely to serve as primary coordination mechanisms for self-improving multi-agent systems, advanced composition tools, and interpretable RL pipelines.