ReAcTree: Hierarchical LLM Task Planner

• ReAcTree is a hierarchical, training-free LLM framework that decomposes long-horizon tasks into manageable, natural-language subgoals.
• It employs a dynamic agent tree with control-flow nodes and dual memory systems—episodic and working memory—for precise, contextual execution.
• Empirical evaluations on WAH-NL and ALFRED demonstrate that ReAcTree outperforms monolithic planning baselines by reducing error propagation and improving task success rates.

Searching arXiv for the primary ReAcTree paper and closely related planning baselines. ReAcTree is a hierarchical, training-free LLM agent framework for long-horizon task planning under partial observability. It replaces a monolithic trajectory with a dynamically constructed agent tree in which natural-language subgoals are delegated to LLM-powered agent nodes, while behavior tree–inspired control-flow nodes coordinate execution through sequence, fallback, and parallel semantics. The framework combines subgoal-level episodic memory with shared working memory, and was evaluated on WAH-NL/VirtualHome and ALFRED/AI2THOR, where it consistently outperformed strong planning baselines such as ReAct, Tree-Planner, and zero-shot planning across multiple LLMs (Choi et al., 4 Nov 2025).

1. Problem setting and design rationale

ReAcTree is motivated by the failure mode of monolithic LLM planning in long-horizon embodied tasks. In such settings, a single trajectory must absorb all prior decisions, observations, room transitions, tool usage, and partial observations. The reported consequence is entanglement of unrelated subproblems, which increases error propagation, hallucination, logical inconsistency, and context-window growth. The framework therefore shifts the planning unit from a single global trajectory to a hierarchy of semantically isolated subgoals (Choi et al., 4 Nov 2025).

The method targets partially observable household environments with irreversible actions. The motivating examples include tasks in which the agent must search across rooms, manipulate containers, and satisfy multi-object goals such as placing multiple items on a table. The paper explicitly contrasts this setting with action-tree search methods that assume reversible simulators and rollbacks. ReAcTree instead expands in subgoal space and executes in the actual environment state without rollback.

This design implies a particular notion of robustness. Rather than relying on self-reflection loops or trajectory-wide revision, ReAcTree localizes reasoning inside subgoal-specific contexts and uses explicit control flow to encode sequential dependence, recovery alternatives, and independent branches. This suggests that the framework treats hierarchical decomposition not as a post hoc scaffold around ReAct, but as the primary mechanism for reducing long-horizon interference.

2. Agent-tree formalism and control-flow semantics

The core object is a dynamic agent tree $T = (V, E)$, where the node set is partitioned into agent nodes $V_A$ and control-flow nodes $V_F$. Each agent node $n \in V_A$ is associated with a natural-language subgoal $g^n$ and an LLM policy $\pi^n$. Each control-flow node $n_f \in V_F$ has a type $$f \in \{\text{sequence }(\to), \text{ fallback }(?), \text{ parallel }(\Rightarrow)\}$$ and coordinates the execution of child agent nodes (Choi et al., 4 Nov 2025).

At time $t$, an agent node operates on a local context

$c^n_t = (o^n_1, a^n_1, \ldots, a^n_{t-1}, o^n_t),$

and samples an action according to

$V_A$0

Here $V_A$1 contains in-context examples retrieved from episodic memory for the current subgoal. The action space is extended beyond primitive environment skills: $V_A$2 where $V_A$3 contains executable skills, $V_A$4 contains language steps, and $V_A$5 contains expansion actions that specify a control-flow type together with a list of subgoals.

Expansion is central. If an agent emits an expansion action $V_A$6, the framework inserts an intermediate control-flow node $V_A$7 of type $V_A$8 and attaches child agent nodes for the proposed subgoals. Execution then recurses into that subtree. Agent nodes terminate with done, failure, or after exceeding a maximum decision count $V_A$9, in which case termination is failure.

The control-flow semantics are explicit. A sequence node executes children in order and succeeds iff all succeed. A fallback node executes children in order and succeeds on the first success, failing only if all fail. A parallel node executes all children, sequentially for simplicity, and aggregates status by majority voting. The fallback semantics are especially important for search under partial observability, since they encode structured alternatives such as trying multiple rooms or receptacles.

A common misconception is to treat ReAcTree as a generic tree-of-thoughts planner. The formalism is narrower and more operational. Nodes are not merely reasoning states: they are LLM agents with access to environment skills, subgoal-local context, and the ability to further expand the tree. Conversely, the tree is not a speculative search tree over reversible action prefixes; it is an executable hierarchy over subgoals in the live environment state.

3. Episodic memory and working memory

ReAcTree integrates two memory systems with distinct roles. Episodic memory stores subgoal-level experiences as tuples $V_F$0, where $V_F$1 is the text trajectory for an agent node, $V_F$2 is a Sentence-BERT embedding of the subgoal, and $V_F$3 is the termination state. Given a current subgoal $V_F$4, retrieval uses cosine similarity

$V_F$5

followed by top-$V_F$6 selection subject to a token budget, with tie-breaking that samples across success, failure, and expand outcomes (Choi et al., 4 Nov 2025).

This retrieval policy makes episodic memory subgoal-specific rather than task-global. The paper emphasizes that this differs from monolithic methods in which retrieval is tied to the overall task description even when the agent is currently performing a narrow subroutine such as locating an object or opening a receptacle. A plausible implication is that ReAcTree reduces prompt mismatch by conditioning each node on examples aligned with the immediate operational subgoal.

Working memory serves a different function. It is a shared blackboard of environment-specific observations, implemented as a Python dictionary mapping object classes to lists of observed instances and locations. It is updated automatically when movable objects are observed. ReAcTree exposes this state through a tool-like action, recall location of <object>, allowing any node to query shared knowledge rather than re-searching the environment.

The two memories are complementary rather than interchangeable. Episodic memory provides cross-task priors at the subgoal level; working memory provides within-task state sharing. The ablation study on WAH-NL with Qwen 2.5 72B illustrates the separation clearly: no memory gives $V_F$7 GSR / $V_F$8 SSR, working memory only gives $V_F$9, episodic memory only gives $n \in V_A$0, and combining episodic memory with working memory gives $n \in V_A$1. The paper also notes that WM-only can hurt small models without episodic memory, whereas larger models remain robust and benefit from it.

4. Inference procedure, prompting, and computational profile

ReAcTree is purely prompt-based and does not use fine-tuning. The inference pipeline starts from a top-level goal, initializes a root agent node, retrieves subgoal-level examples, and interleaves reasoning and acting. If the node decides that the current subgoal is too complex, it emits an expansion action and delegates execution to a newly created control-flow subtree. Status is propagated upward through the control-flow semantics until the task terminates with success, failure, or budget exhaustion (Choi et al., 4 Nov 2025).

Prompting is structured around three roles for agent nodes: think, act, and expand. The prompt also enumerates available primitive skills and control-flow types. The Guidance library is used for constrained generation of actions and control-flow choices, with temperature $n \in V_A$2 for free-form reasoning and deterministic action selection. Retrieved in-context examples are capped at $n \in V_A$3K tokens. Per-task decision caps are $n \in V_A$4 for WAH-NL and $n \in V_A$5 for ALFRED.

The framework uses different primitive action sets across simulators. In VirtualHome/WAH-NL the skills include go to, pick up, put down, open, close, and turn on. In AI2THOR/ALFRED, the action set additionally includes operations such as slice and turn off. The working-memory query action is inserted into the available action space when WM is enabled.

The computational trade-off is explicit. ReAcTree increases execution time relative to ReAct, but keeps token growth more bounded because each node reasons in a modular subgoal-local context. On shared successful WAH-NL tasks with a $n \in V_A$6B model, ReAcTree+WM takes $n \in V_A$7s versus $n \in V_A$8s for ReAct+WM, but achieves much higher GSR. Peak input tokens are also more stable: ReAcTree+WM $n \in V_A$9B reports a maximum input length of $g^n$0, compared with $g^n$1 for ReAct+WM. The paper does not present self-reflection or self-consistency loops; computational overhead comes primarily from tree construction and recursive node execution.

5. Empirical evaluation on WAH-NL and ALFRED

The empirical study covers WAH-NL in VirtualHome and ALFRED in AI2THOR, using LLaMA 3.1 $g^n$2B/$g^n$3B, Qwen 2.5 $g^n$4B/$g^n$5B, Mistral $g^n$6B, Gemma 2 $g^n$7B, and Phi-4-reasoning-plus $g^n$8B. WAH-NL contains $g^n$9 training and $\pi^n$0 test tasks across five categories, with metrics Goal Success Rate (GSR) and Subgoal Success Rate (SSR), where

$\pi^n$1

ALFRED evaluation uses valid-seen and valid-unseen splits and reports GSR (Choi et al., 4 Nov 2025).

On WAH-NL, the gains are large across models. With Qwen 2.5 $\pi^n$2B, ReAcTree+WM attains $\pi^n$3 GSR / $\pi^n$4 SSR, compared with ReAct+WM at $\pi^n$5, ReAcTree without WM at $\pi^n$6, and ReAct at $\pi^n$7. With LLaMA 3.1 $\pi^n$8B, ReAcTree+WM reaches $\pi^n$9 versus $n_f \in V_F$0 for ReAct+WM. The pattern persists for smaller models: LLaMA 3.1 $n_f \in V_F$1B obtains $n_f \in V_F$2 with ReAcTree+WM versus $n_f \in V_F$3 with ReAct+WM, and Qwen 2.5 $n_f \in V_F$4B obtains $n_f \in V_F$5 versus $n_f \in V_F$6.

Setting	ReAcTree+WM	Comparator
WAH-NL, Qwen 2.5 72B (GSR / SSR)	61.00 / 79.58	ReAct+WM: 31.00 / 54.05
WAH-NL, LLaMA 3.1 70B (GSR / SSR)	58.00 / 79.27	ReAct+WM: 33.00 / 63.15
ALFRED valid-unseen, Qwen 2.5 72B (GSR)	39.83	ReAct+WM: 39.10

The ALFRED results are smaller in magnitude but still favorable. ReAcTree+WM with LLaMA 3.1 $n_f \in V_F$7B achieves $n_f \in V_F$8 vs. $n_f \in V_F$9 on valid-seen and $$f \in \{\text{sequence }(\to), \text{ fallback }(?), \text{ parallel }(\Rightarrow)\}$$0 vs. $$f \in \{\text{sequence }(\to), \text{ fallback }(?), \text{ parallel }(\Rightarrow)\}$$1 on valid-unseen relative to ReAct+WM. With Qwen 2.5 $$f \in \{\text{sequence }(\to), \text{ fallback }(?), \text{ parallel }(\Rightarrow)\}$$2B, the corresponding values are $$f \in \{\text{sequence }(\to), \text{ fallback }(?), \text{ parallel }(\Rightarrow)\}$$3 vs. $$f \in \{\text{sequence }(\to), \text{ fallback }(?), \text{ parallel }(\Rightarrow)\}$$4 on seen and $$f \in \{\text{sequence }(\to), \text{ fallback }(?), \text{ parallel }(\Rightarrow)\}$$5 vs. $$f \in \{\text{sequence }(\to), \text{ fallback }(?), \text{ parallel }(\Rightarrow)\}$$6 on unseen. Phi-4-RP $$f \in \{\text{sequence }(\to), \text{ fallback }(?), \text{ parallel }(\Rightarrow)\}$$7B reaches $$f \in \{\text{sequence }(\to), \text{ fallback }(?), \text{ parallel }(\Rightarrow)\}$$8 vs. $$f \in \{\text{sequence }(\to), \text{ fallback }(?), \text{ parallel }(\Rightarrow)\}$$9 on seen and $t$0 vs. $t$1 on unseen.

The control-flow ablation clarifies which part of the hierarchy matters. For Qwen 2.5 $t$2B on WAH-NL, using all three control-flow types yields $t$3, using only sequence plus fallback yields $t$4, and using sequence alone drops to $t$5. This indicates that fallback carries most of the practical benefit, while parallelism is useful but not always decisive. The error analysis for Qwen 2.5 $t$6B reports $t$7 failures categorized as ambiguous ($t$8), execution ($t$9), search ($c^n_t = (o^n_1, a^n_1, \ldots, a^n_{t-1}, o^n_t),$0), and expand ($c^n_t = (o^n_1, a^n_1, \ldots, a^n_{t-1}, o^n_t),$1), with search under partial observability identified as the dominant failure mode.

6. Representative execution pattern, related methods, and limitations

A representative WAH-NL example is the goal “Make sure there is a wine and a juice on the coffee table.” The root agent expands into a parallel node with two children: move the wine onto the coffee table, and move the juice onto the coffee table. The wine branch expands into a sequence whose first component is a fallback search over rooms. Kitchen and living room attempts fail; the bedroom attempt succeeds by opening cabinet 1 and picking up wine 1. The juice branch uses working memory through recall location of juice, retrieves “juice 1 near fridge 2 in kitchen 1,” and then executes a direct pickup-and-delivery sequence. Both branches succeed, the parallel node returns success, and the root terminates with done (Choi et al., 4 Nov 2025).

This execution style clarifies ReAcTree’s relation to prior methods. Relative to ReAct, ReAcTree still interleaves reasoning and acting, but only within subgoal-local contexts; it also introduces explicit control flow and subgoal-level retrieval. Relative to tree-search methods such as Tree-Planner, the distinction is that ReAcTree does not assume reversible simulators or rollback. Relative to behavior-tree systems in robotics, the control-flow vocabulary is similar, but the tree is dynamically constructed by the LLM at inference time rather than predefined. Relative to bi-level hierarchical planners, ReAcTree generalizes to a multi-level tree with agent and control-flow nodes.

The framework’s limitations are reported directly. Computational overhead is higher than for monolithic baselines. Gains may be model-dependent, although the improvement trend is broad across the evaluated LLMs. The working-memory mechanism is deliberately simple. Search under partial observability remains the main unresolved error source, and expansion itself can fail when the model proposes poor subgoal decompositions. The paper proposes future work on learning control-flow policies, adding hallucination mitigation and self-correction, improving memory with richer object/entity structure, introducing clarification dialogues, and optimizing branching, depth, and token budgeting.

Taken together, ReAcTree defines a particular form of hierarchical embodied planning: subgoal-local LLM agency, behavior tree control flow, and dual memory systems. Its empirical contribution is not merely that a tree structure helps, but that subgoal decomposition, fallback-style recovery, and memory separation materially improve long-horizon performance under partial observability.