Hierarchical Decomposition

• Hierarchical decomposition is a systematic method for breaking down complex tasks into manageable subtasks, facilitating efficient planning and reasoning.
• It employs a tri-level optimization framework that integrates action planning, subtask sequencing, and optimal subgoal selection to minimize costs.
• Empirical validations in navigation and puzzle tasks confirm that model-predicted bottleneck subgoals align with human planning behavior.

Hierarchical decomposition is a structured approach to representing complex tasks as nested sequences of subtasks, supporting efficient planning and reasoning by organizing choices and subgoals across multiple levels of abstraction. In resource-rational planning, hierarchical decomposition is theorized and formalized as an optimal strategy for minimizing the cognitive and computational cost of planning within an environment, where agents—biological or artificial—select hierarchical abstractions (such as subgoals) that enable more efficient solution discovery than flat, unstructured reasoning.

1. Formalization of Hierarchical Task Decomposition

In the resource-rational framework, hierarchical decomposition is cast as a representation design problem in which the agent must choose a set of subgoals, $\mathcal{Z} \subset \mathcal{S}$, to share across tasks drawn from a distribution over state–goal pairs $(s, g)$. For deterministic shortest-path tasks over a discrete state space $\mathcal{S}$ (with transitions $T \subset \mathcal{S} \times \mathcal{S}$), planning involves solving for action sequences from $s$ to $g$. Hierarchical decomposition is modeled by introducing three nested decision levels:

• Action Level: Compute shortest paths from a current state $s$ to a subgoal $z$ via a planning algorithm (e.g., Breadth-First Search, A*), minimizing path length $D(s, z)$ and incurring a planning cost $C_{\text{Alg}}(s, z)$.
• Subtask Level: Plan the best sequence of subgoals from $s$ to $g$, described by the value

$$V^g_{\mathcal{Z}}(s) = \max_{z \in \mathcal{Z}} \left\{ R(s, z) - C_{\text{Alg}}(s, z) + V^g_{\mathcal{Z}}(z) \right\}$$

where $R(s, z) = -D(s, z)$ and $V^g_{\mathcal{Z}}(z)$ applies recursively.

• Decomposition Level: Select the optimal subgoal set $\mathcal{Z}^*$ shared across tasks:

$$\mathcal{Z}^* = \arg\max_{\mathcal{Z}} \mathbb{E}_{s, g} \left[ V^g_{\mathcal{Z}}(s) \right]$$

maximizing expected rewards net of planning costs.

This formulation integrates reward and algorithm-specific planning costs, enforcing that the “best” decomposition is not one yielding globally minimal path length alone, but rather one that offers the optimal efficiency trade-off under cognitive or computational constraints.

2. Hierarchical Model Architecture and Optimization

Hierarchical decomposition is operationalized as a tri-level optimization problem, with each level consuming the output of the next lower level:

1. Action Level: For each $(s, z)$, compute plan using algorithm $\text{Alg}$ with cost $C_{\text{Alg}}(s, z)$.
2. Subtask Level: Use value iteration (with the recursive Bellman equation above) to assemble the overall strategy from available subgoals.
3. Decomposition Level: Search for $\mathcal{Z}^*$, optimizing expected planning value over the task distribution.

This architecture supports both exhaustive enumeration (small environments) and gradient-based optimization (for large-scale problems where differentiable value iteration backpropagates through the entire hierarchical computation).

3. Resource-Rationality and Subgoal Selection

By explicitly modeling computational resource usage, this framework explains why and where agents select subgoals corresponding to bottleneck or bridge states. Planning cost $C_{\text{Alg}}(s, z)$ can be estimated for algorithmic solvers (e.g., number of nodes expanded by BFS or A*), capturing practical cognitive load or CPU utilization.

Key empirical findings include:

• In environments with bottlenecks (e.g., two-room gridworlds), bottleneck states are preferentially selected as subgoals because they minimize the expected search effort.
• Subgoal selection is algorithm-dependent: for example, subgoals will differ for BFS (which benefits from state-space partitioning) vs. A* (where heuristics influence cost structure).
• Subgoals are only predicted when they genuinely deliver planning efficiency; spurious subgoals are not proposed under conditions where decompositions confer no benefit.
• Human latency and “bus stop” recognition behaviors (as in navigation or Tower of Hanoi tasks) are quantitatively mirrored by model-derived planning step counts.

4. Experimental Validation and Numerical Replication

Simulations and model comparisons quantitatively reproduce human planning and subgoal generation data. For instance, in navigation tasks and classical Tower of Hanoi experiments, predicted subgoals align with human-identified bottleneck locations and preferentially licensed transitions. The model not only accounts for accuracy of subgoal selection but also closely matches reaction-time data, supporting the hypothesis that hierarchical decomposition in humans is tuned to minimize planning costs (rather than path costs alone).

Performance and matching to empirical data are observed across multiple experiments:

Domain/Task	Model Prediction	Empirical Alignment
Two-room navigation	Bottleneck subgoals	Human “bus stop” locations
Tower of Hanoi	Subgoal “tower crossings,” altered by heuristics	Human RTs, bottleneck focus, heuristic usage
Recognition probes	Bottleneck states prioritized; faster planning	Faster human response for bottleneck queries

5. Implications for Cognitive Architecture and AI

The resource-rational hierarchical decomposition framework provides a normative and algorithmic account for the origin and content of hierarchical task representations in both natural and artificial agents. In contrast to models focusing purely on compression or statistical abstraction, this approach demonstrates that the structure of subgoals is a solution to a meta-planning optimization problem subject to computational constraints. This model is compatible with, and fundamentally underpins, theories of abstraction, bounded rationality, and adaptive problem-solving in cognitive science.

Moreover, the framework accommodates a broad family of planners: substituting in different planning algorithms or cost functions allows modeling of diverse human and machine behaviors, supporting cross-domain generalization and the paper of planning performance trade-offs under different resource regimes.

6. Implementation Considerations and Scaling

Implementing resource-rational hierarchical decomposition in practice involves:

• Value iteration at the subtask level to compute $V^g_{\mathcal{Z}}(s)$ for given subgoal sets.
• Algorithm-specific cost modeling ($C_{\text{Alg}}$) for accurate estimation of human-like or system-specific planning effort.
• Search for optimal subgoal sets via exact or gradient-based optimization, depending on state space size; efficient enumeration is practical for state spaces of modest dimension, while differentiable approaches scale to larger problems.
• Experimental and policy validation by comparing model-predicted subgoal structures and reaction times to empirical data in both human studies and artificial agent benchmarks.

Model computational requirements are governed primarily by the number of subgoals and the complexity of value iteration per task. Practical deployment in AI agents or cognitive modeling software must accommodate the cost of subgoal optimization, potentially amortized or regularized using structural priors or meta-learning.

7. Summary Table: Hierarchical Levels and Objectives

Level	Description	Formal Objective
Action-Level	Plan actions from $s \rightarrow z$	Minimize $D(s, z)$, incur cost $C_{\text{Alg}}$
Subtask-Level	Sequence subgoals to goal	Maximize $V^g_{\mathcal{Z}}(s)$ via Bellman recursion
Task Decomposition	Choose subgoal set for all tasks	$$\mathcal{Z}^* = \arg\max_{\mathcal{Z}}\mathbb{E}_{s, g}[V^g_{\mathcal{Z}}(s)]$$

This rigorous paradigm illuminates why and how hierarchical decompositions arise in task planning, providing a computational, experimentally validated, and highly generalizable foundation for both artificial and biological planning systems (Correa et al., 2020).