---
title: LLM-Based Hierarchical TODO Decomposition
url: https://www.emergentmind.com/topics/llm-based-hierarchical-todo-decomposition
type: topic
---

# LLM-Based Hierarchical TODO Decomposition

LLM-based Hierarchical TODO Decomposition is a paradigm for orchestrating large language models (LLMs) and agent systems to robustly solve complex, ambiguous, or multi-stage problems by systematically splitting them into hierarchically structured sub-tasks (“TODOs”), routing these to specialized agents or tools, and aggregating the results. This methodology overcomes context window limitations, enables parallel and modular execution, and delivers improved solution quality. Modern designs are grounded in formalisms from automated planning, multi-agent systems, computational graph theory, and empirical workflow management.

## 1. Formal Models and Notation

Formally, the decomposition process begins with a high-level task $T \in \mathcal{T}$, which is transformed into a set or hierarchy of sub-tasks via a decomposition function:

\[
D(T) = \{ t_1, t_2, \ldots, t_n \}
\]

A directed acyclic dependency graph $\mathrm{Dep} \subseteq \{ (t_i \to t_j) \}$ encodes prerequisite relations between subtasks. Each sub-task $t_i$ is annotated with:

- $d(t_i)$: domain/expertise label (e.g., “math calculation”, “flight search”)
- $c(t_i)$: complexity estimate (e.g., token budget, number of steps)
- $agent(t_i)$: assigned agent (LLM specialist or external tool)
- $status(t_i) \in \{\text{Pending}, \text{In-Progress}, \text{Done}, \text{Failed}\}$
- $result(t_i)$: the output of solving $t_i$

The global solution is reconstructed as:

\[
S_\text{final} = \text{Aggregate}( \{ result(t_i) \mid t_i \in D(T) \} )
\]

In multi-agent or multi-LLM workflows, assignment and prioritization are governed by scoring functions:

\[
agent(t_i) = \arg\max_{A_j} \text{Score}(A_j, d(t_i)) \\
\text{Score}(A_j, d) = w_\text{domain} \cdot \text{Match}(A_j.\text{domain}, d) + w_\text{perf} \cdot \text{historical\_accuracy}(A_j, d)
\]

This abstraction generalizes to recursive and cross-domain scenarios, such as tree-based mission planning for robots [2501.16539], debate-based subtask planning for 6G management [2506.06519], and compositional workflows in code generation [2407.18276, 2412.05393].

## 2. Decomposition and Orchestration Algorithms

A canonical orchestration pipeline proceeds in five distinct phases [2402.16713]:

1. **Requirement Elicitation**: The orchestrator LLM interacts with the user, posing clarifying questions until the specification is sufficient. This leverages chain-of-thought prompting to uncover ambiguous or missing requirements.
2. **Task Decomposition**: The orchestrator applies an LLM-driven split to produce a structured TODO list, modeled as a tree or DAG with explicit dependencies and stepwise domain annotations.
3. **Agent Assignment**: Each subtask is routed to the agent or tool best suited by capability and prior observed accuracy (domain-specific routing).
4. **Parallel Subproblem Solving**: Using a dependency graph and work queue (priority determined by topological order and/or complexity), subtasks are dispatched to agents as soon as all dependencies are satisfied. Execution is asynchronous and exploits available parallelism.
5. **Aggregation**: Final solutions are synthesized by prompting the orchestrator LLM with the collection of subtask results to produce a coherent, user-facing response.

The following pseudocode from [2402.16713] exemplifies this loop, with additional application-specific modules—such as utility-based robot task allocation [2501.16539] and DSE-driven prompt generation for IC design [2412.05393]—refining assignment and aggregation strategies.

```python
# Simplified orchestrator pseudocode
context = user_input
while requirements_incomplete(context):
    q = Orch.generate_follow_up_question(context)
    user_answer = query_user(q)
    context |= user_answer
root_task = context
subtasks = Orch.decompose(root_task)
for t in subtasks:
    t.assignee = select_agent(t.domain)
    queue.add(t)
while queue:
    t = queue.pop_ready()
    t.status = "In-Progress"
    t.result = t.assignee.solve(t.description)
    t.status = "Done"
    queue.enqueue_ready_dependents(t)
S_final = Orch.aggregate({t.result for t in subtasks})
return S_final
```

## 3. Data Structures and Hierarchy Representation

Task hierarchies are predominantly managed as trees or DAGs. Each node represents a subproblem:

```python
class TaskNode:
    id: str
    description: str
    domain: str
    complexity: float
    deps: List[str]
    assignee: AgentHandle
    status: Enum("Pending", "InProgress", "Done", "Failed")
    result: Optional[Any]
```

Orchestrators maintain a mapping of task IDs to TaskNodes, a dependency graph (adjacency list), and a priority work queue.

In multi-agent or modular agent systems, the entire workflow is a tree of services [2510.10890]:

- Root: high-level task (e.g., “Survey Generation”)
- Intermediate: phase/functional modules (e.g., AnalysisPhase, SkeletonPhase)
- Leaves: atomic LLM or tool servers (e.g., SearchServer, DigestServer)

Each module exposes one or more functions as standard protocols (e.g., MCP tool calls), facilitating distributed orchestration and plug-and-play module insertion.

## 4. Mathematical Criteria for Split, Assignment, and Aggregation

Although not always formalized as explicit closed-form equations, the decomposition process is driven by:

- **Chain-of-Thought (CoT) Decomposition**: $T \Rightarrow_{link} D(T)$, where $\Rightarrow_{link}$ models LLM-generated, stepwise breaking-down via CoT prompting.
- **Complexity Measures**: Subtasks are defined so as to keep $c(t_i)$ (tokens or steps per subtask) below agent-specific thresholds, ensuring each is LLM-manageable [2407.14788]. Theoretical analysis relates depth $D$, branching $b$, and per-node error $\varepsilon_D$ to overall workflow accuracy:

\[
E_0 \leq b^D \cdot \varepsilon_D \\
C_{total} \leq \sum_{l=0}^{D-1} b^l \cdot [C_{pre}(L_{sys}+m_l) + C_{dec}(L_{sys}+m_l, L_{dec}(m_l))]
\]

Optimization aims to set $m_l$, $b$, and $D$ so that $E_0$ is under a target while $C_{total}$ is minimized.

- **Assignment Score**: As above, $Score(A_j, d) = w_\text{domain}\cdot Match + w_\text{perf}\cdot accuracy$ controls agent routing [2402.16713].
- **Task-robot matching**: In multi-robot planning, assignment maximizes

\[
\max \sum_{r\in R} \sum_{a \in T_r} u_a(r)
\]
subject to deadline and sequentiality, with utility $u_a(r) = \alpha q_a(r) - \beta d_a(r) - \gamma c_a(r)$ [2501.16539].

## 5. Runtime Protocols

At runtime, the orchestrator operates as a long-lived service, executing the following protocol [2402.16713, 2510.10890]:

1. **Instantiation**: Ingest user input; dynamically clarify via question–answer loop.
2. **Decomposition**: Generate the task DAG/tree, possibly interacting with the user for further disambiguation.
3. **Agent Dispatch**: Assign ready subtasks to available agent instances (with system such as LangChain or MCP).
4. **Concurrency and Monitoring**: Track task status; upon completion of dependencies, schedule downstream tasks.
5. **Checkpointing and Fault Tolerance**: Periodically record partial results to persist progress and allow recovery.
6. **Aggregation and Finalization**: Aggregate subresults via LLM prompt or symbolic function; deliver final output.

In advanced systems, “orchestra” agents holistically plan next tool invocations based on execution history and user feedback [2510.10890]. Human-in-the-loop intervention may occur at key decision points (topic scope, outline restructuring, etc.).

## 6. Empirical Results and Comparative Benchmarks

Empirical evaluation demonstrates substantive gains in accuracy, reliability, and efficiency:

- On GSM8K math (2–8 steps per task), a GPT-4 orchestrator with GPT-3.5-turbo specialists achieved a 73% solve-rate, outperforming single-agent and flat multi-agent approaches by 8–23 percentage points [2402.16713].
- In hierarchical debate for 6G network management, MCR (macro coverage rate) improved as follows for GPT-4o + GPT-4o-mini: 39.62% (baseline) → 49.75% (regular debate) → 81.19% (hierarchical debate), with similar lifts for other model combinations [2506.06519].
- In chip design, hierarchical prompting delivered >30% token and >45% runtime savings compared to flat prompting, with pass@5 rates rising from 0–10% to >90% for certain architectures [2407.18276, 2412.05393].
- In multi-robot mission planning, LLM-constructed hierarchical trees yielded tractable, near-optimal alternatives sublinear in the number of abstract tree nodes, with demonstrable flexibility across diverse mission types [2501.16539].

These results generalize to domains including programming education (DBox: +0.198 correctness, +2.33 self-efficacy) [2502.19133] and cross-task zero-shot generalization in reinforcement learning (ReflexGrad: 67% trial-0 success, zero action loops) [2511.14584].

## 7. Applications and Illustrative Examples

LLM-based hierarchical TODO decomposition frameworks are deployed in scenarios such as:

- **Travel planning**: Decomposing user requests (“Book me a return flight...”) into flight search, amenity check, booking, with agent routing and dependency management [2402.16713].
- **Robotics**: Multi-level decomposition of missions (“Reunite mom with her lost child”) into compound and primitive subroutines, capability-aware agent assignment, and utility-maximizing task allocation [2501.16539].
- **6G Management**: Hierarchical debate among LLMs for sub-task extraction (“Optimize RIS placement...”) and per-step solution refinement [2506.06519].
- **HDL/IC Generation**: Recursive submodule generation (“64-to-1 MUX” → “8 MUX8-1” → “MUX2-1”) with simulation feedback in each TODO iteration [2407.18276, 2412.05393].
- **Survey Generation and Planning**: Modular orchestration of MCP servers for search, clustering, outline generation, and content refinement [2510.10890].
- **Programming Education**: Co-decomposition of algorithmic tasks; learner-LLM step-tree alignment with dynamic hints and scaffolded code mapping [2502.19133].

Each of these exemplifies the translation of high-level, often ambiguous, natural language instructions into a structured, agent-executable workflow that supports parallelism, modular failure recovery, and extendability to new domains.

---

**Key References:**  
- “Navigating Complexity: Orchestrated Problem Solving with Multi-Agent LLMs” [2402.16713]  
- “Generalized Mission Planning for Heterogeneous Multi-Robot Teams...” [2501.16539]  
- “Hierarchical Debate-Based Large Language Model...” [2506.06519]  
- “Rome was Not Built in a Single Step: Hierarchical Prompting for LLM-based Chip Design” [2407.18276]  
- “HiVeGen -- Hierarchical LLM-based Verilog Generation...” [2412.05393]  
- “LLM$\times$MapReduce-V3: Enabling Interactive In-Depth Survey Generation...” [2510.10890]  
- “DBox: Scaffolding Algorithmic Programming Learning...” [2502.19133]  
- “ReflexGrad: Three-Way Synergistic Architecture...” [2511.14584]  
- “On the Design and Analysis of LLM-Based Algorithms” [2407.14788]

Source: https://www.emergentmind.com/topics/llm-based-hierarchical-todo-decomposition