Task-Tree Subgoal Generation

• Task-tree style subgoal generation is a formal approach that decomposes complex tasks into hierarchical sequences of subgoals represented as tree structures with explicit dependencies.
• It utilizes algorithmic methods such as backward search, divide-and-conquer tree splitting, and LLM-based prompt decomposition to achieve interpretable and parallelizable planning.
• Applications span robotics, reinforcement learning, and script generation, yielding improvements in plan correctness, efficiency (up to O(log T) time), and hierarchical policy learning.

Task-tree style subgoal generation refers to formal, algorithmic methods for decomposing complex tasks into hierarchical sequences of subgoals, producing a branching or tree-structured plan. This paradigm underlies a wide variety of approaches in robotics, reinforcement learning (RL), script generation, and automated planning. In all such frameworks, the task tree serves as an interpretable, process-oriented, and error-resilient scaffold for achieving long-horizon objectives.

1. Formal Representations and Hierarchical Structure

Task-tree style subgoal generation encodes an entire plan as an explicit tree or acyclic graph, where each node is a subgoal, and edges represent dependency or execution order. For example, in the Functional Object-Oriented Network (FOON), the task tree is a connected directed acyclic subgraph $\tau \subset \mathcal{G}$ spanning from initial (ground) objects to a goal-node $g$ (Sakib et al., 2022, Nallu, 2023, Saini, 2022). Each node may itself correspond to a functional unit—a triple $f=(I(f),m(f),O(f))$—that explicitly specifies input object-states, an operation (motion), and output object-states.

In reinforcement learning and motion planning, subgoal trees organize an optimal or expert trajectory $\tau=s_0,\ldots,s_T$ as a binary tree. Each node represents a sub-segment $[s_i, s_j]$, recursively split at a predicted subgoal $s_m$ (Jurgenson et al., 2019, Jurgenson et al., 2020, Parascandolo et al., 2020). This enables parallel prediction or planning, as all midpoints at each tree level can be computed independently.

In language or script generation, the tree alternates between abstract subgoals and concrete steps, formalized as a tree $(g, \{(S_i, T_i)\}_{i=1}^n)$ where $g$ is the main goal, $S_i$ are level-1 subgoals, and $T_i$ are step sequences fulfilling each $S_i$ (Li et al., 2023).

2. Algorithmic Principles for Subgoal Tree Generation

Task-tree style subgoal generation typically involves two main algorithmic phases: (i) backward or divide-and-conquer subgoal search (tree expansion), and (ii) assembly into an executable plan.

a. Backward and Greedy Retrieval (Knowledge Graphs)

In object-centric domains (e.g., robotic cooking), backward search is performed from the goal node. For FOON, the “RetrieveTaskTree” algorithm initializes a subgoal queue $S=\{g\}$ and, for each unresolved subgoal $o_{\text{goal}}$, selects an eligible functional unit via similarity-based matching or heuristics. Subgoal inputs are pushed recursively until all are satisfied from ground objects (Sakib et al., 2022, Saini, 2022, Nallu, 2023).

Both exact match and approximate semantic similarity methods are supported: $$\mathsf{sim}(o_i, o_j) = \frac{\langle v(o_i), v(o_j) \rangle}{\|v(o_i)\|\|v(o_j)\|}$$ where $v(o)$ denotes an object-state embedding, allowing novel objects/states to be linked to known actions above a threshold $\theta$ (Sakib et al., 2022).

b. Divide-and-Conquer Tree Splitting (Trajectory/RL Domains)

In trajectory or RL contexts, task trees are grown by recursively predicting midpoints or subgoals that split a problem into subproblems. In the subgoal-tree dynamic programming (SGTDP) approach, for graph edge cost $c(s,s')$ and value tables $V_k(s, g)$,

$$V_k(s, g) = \min_m \left[ V_{k-1}(s, m) + V_{k-1}(m, g) \right]$$

This recursive minimization yields binary trees and admits parallel evaluation at each tree depth (Jurgenson et al., 2019, Jurgenson et al., 2020).

Stochastic or learned subgoal proposals (e.g., $p(s'|s, s_G)$) enable flexible, differentiable tree construction in DC-MCTS and similar frameworks (Parascandolo et al., 2020).

c. Language and Program Generation: Prompted Tree Decomposition

Hierarchical script generation and LLM-based planners such as STEP frame subgoal tree construction as iterative decomposition: a top-level goal is recursively split into subgoals via LLM sampling, subject to mappability and feasibility criteria (Tianxing et al., 26 Jun 2025, Li et al., 2023). Each branching expands the tree, with termination conditions at leaves grounded by environment feedback or action mapping (e.g., whether a candidate subgoal maps unambiguously to a robot primitive and is consistent with affordances).

3. Subgoal Discovery and Automatic Bottleneck Identification

Effective subgoal tree generation depends on methods for discovering subgoals that are both achievable and strategically useful. Several approaches are prominent:

• Model Switch/Free-Energy Paradigm: Subgoals correspond to locations with high unpredictability under model switches (e.g., aggregation vs. non-aggregation spaces). The count of model changes $MC(s)$ yields bottleneck candidate subgoals for hierarchical planning (Mesbah et al., 21 Dec 2024).
• Semantic Bottleneck Identification: In visual prediction or self-supervised settings, subgoals are directly optimized as intermediate images or states to minimize worst-case planning cost across subsegments, resulting in semantically meaningful waypoints aligned to task structure (Nair et al., 2019).
• Automaton-Guided Decomposition: For logic-specified domains, LTL formulae are compiled to Büchi automata, which are then traversed to extract reach-avoid subgoals; the associated subgoal tree mirrors the automaton's structure (Guo et al., 3 Aug 2025).

4. Integration with Planning, Learning, and Execution

Once constructed, task-trees serve as blueprints for fine-grained planning, learning, and execution.

• Policy Learning: Subgoal trees can drive behavioral cloning or supervised prediction, as the midpoints/subgoals at each tree node are used as training targets:

$$\hat P_\pi(\tau\mid s,g; \theta) = \prod_{(s_1,s_2)\in nodes(\tau)} \hat P_\pi(s_m | s_1, s_2; \theta)$$

(Jurgenson et al., 2019).

• Hierarchical RL: Discovered subgoals are used as landmarks/options for hierarchical policies. High-level controllers select subgoals, while low-level policies specialize in achieving each subgoal (Mesbah et al., 21 Dec 2024).
• Script Generation: In text domains, hierarchical decoding alternates between generating a set of subgoals and, for each, generating its concrete steps. Prompt-based LLMs (e.g., T5/T5-Base) are trained end-to-end to output structured scripts with explicit tree markers (e.g., <section> tokens) (Li et al., 2023).

5. Evaluation Metrics and Empirical Findings

Empirical evaluation of task-tree style subgoal generation typically quantifies plan "correctness", precision/recall of subgoal selection, and execution success.

• Correctness in FOON Retrieval: Defined as the fraction of retrieved functional units matching ground-truth task trees:

$$\mathsf{Correctness}(T_{\rm auto},T_{\rm gt}) = \frac{|\{f \in T_{\rm auto} : f \in T_{\rm gt}\}|}{|T_{\rm gt}|}$$

(Sakib et al., 2022). Thresholded semantic matching ($\theta=0.75$) yields 82% correctness versus 57% for exact match only.

• Hierarchical RL/Motion Planning: Subgoal tree frameworks realize $O(\log T)$ prediction time for behavioral cloning, and attain lower error in trajectory cost due to parallel/recursive updates (Jurgenson et al., 2019, Jurgenson et al., 2020).
• Script Generation Quality: Hierarchical pipelines improve BLEU, ROUGE, and subgoal representativeness metrics compared to flat baselines, with human annotators judging 70% of generated subgoals as valid components for achieving the goal (Li et al., 2023).
• Zero-shot LTL Generalization: One-subgoal-at-a-time policies enable $\eta_s \approx 0.95-0.99$ success and low violation rates for unseen temporal logic expressions, outperforming sequence-conditioned models (Guo et al., 3 Aug 2025).

Domain	Structure	Subgoal Selection	Evaluation Metric(s)
Cooking/FOON	DAG (FOON Graph)	Similarity + Heuristics	Correctness, Precision
RL/Trajectory	Binary Tree	Divide-and-conquer split	Planning Cost/Success
Hierarchical NLP	n-ary Tree	Unsupervised/LLM-based	BLEU, ROUGE, Human Eval
LTL Synthesis	Automaton/DAG	Automaton DFS/Büchi	Zero-shot gen. rate

6. Applications, Limitations, and Extensions

Task-tree style subgoal generation has been deployed in diverse settings: robotic cooking (Sakib et al., 2022, Nallu, 2023, Saini, 2022), visual manipulation (Nair et al., 2019), language-based robot control (Tianxing et al., 26 Jun 2025), combinatorial reasoning (Czechowski et al., 2021), and satisfaction of temporal constraints (Guo et al., 3 Aug 2025).

Limitations include:

• Knowledge base incompleteness: Exact subgoals may not be covered, necessitating semantic approximation or human-in-the-loop extension (Sakib et al., 2022).
• Heuristic dependency: Tree optimality and tractability depend on heuristic selection (e.g., success rate vs. input-count) (Saini, 2022, Nallu, 2023).
• Subgoal Discovery Sensitivity: Subpar subgoal discovery can degrade downstream plan quality and diversity, especially in high-dimensional domains (Mesbah et al., 21 Dec 2024, Li et al., 2023).

Recent directions focus on hybrid or adaptive heuristics, improved subgoal discovery via free energy or automata, and real-time closed-loop planners integrating feedback from the environment or embodiment (Tianxing et al., 26 Jun 2025).

7. Comparative Insights and Theoretical Significance

Task-tree style subgoal generation unifies and generalizes approaches across classical symbolic planning, modern RL/hierarchical reinforcement learning, and data-driven generative modeling. By representing the full plan as a structured tree, these methods enable parallelizable, robust, and interpretable policy and plan construction, outperforming sequential or flat approaches in both accuracy and efficiency.

Empirical findings across tasks demonstrate substantial gains in speed (up to $O(\log T)$), adaptability to novel objectives, and generalization to long-horizon or structurally complex requirements, confirming the centrality of task-tree structured subgoal generation in advanced task and motion planning domains (Sakib et al., 2022, Jurgenson et al., 2019, Jurgenson et al., 2020, Guo et al., 3 Aug 2025, Tianxing et al., 26 Jun 2025, Li et al., 2023, Mesbah et al., 21 Dec 2024).