---
title: Resource-Rational Hierarchical Decomposition
url: https://www.emergentmind.com/topics/resource-rational-hierarchical-decomposition
type: topic
---

# Resource-Rational Hierarchical Decomposition

Resource-rational hierarchical decomposition refers to a class of principled frameworks that define how hierarchical representations or policies should be structured to optimally allocate limited computational, memory, or information-processing resources. This paradigm synthesizes ideas from hierarchical planning, bounded rationality, information theory, and reinforcement learning. Across different instantiations—spanning shortest-path planning, policy specialization, and value-function factorization—resource-rational decomposition formalizes which hierarchical abstractions or decompositions should be constructed, given the agent’s resource constraints and task demands, to best trade off task performance against computational and representational effort [2007.13862, 1907.11452, 1206.6851].

## 1. Formal Foundations and Objective Functions

Resource-rational hierarchical decomposition characterizes hierarchical task or policy representations as the solution to a constrained or regularized optimization, where optimality is defined not only in terms of task reward or utility, but also in terms of the cost of cognitive search, information-processing, or memory. The specific form of the objective depends on the domain and the nature of resource constraints:

- **Shortest-path setting:** The agent chooses a set of subgoals \(Z\) to maximize expected subtask-level value, defined as the difference between reward (negative path length) and planning cost (e.g., number of search nodes expanded). The high-level objective is
  \[
  Z^* = \arg\max_{Z\subseteq S}\, E_{(s,g)\sim p}[\,V^g_Z(s)\,],
  \]
  where \(V^g_Z(s)\) is the Bellman-optimal subtask-level value for reaching \(g\) from \(s\) via subgoals \(Z\) [2007.13862].
- **Information-theoretic decision making:** The agent partitions the state space into region-experts, subject to mutual information constraints reflecting bounded rationality. The hierarchical policy maximizes
  \[
  \mathbb{E}[\,U(s,a)\,] - \frac{1}{\beta_1} I(S;X) - \frac{1}{\beta_2} I(S;A|X),
  \]
  where \(U\) is utility, and the \(\beta\) terms control information-resource trade-offs [1907.11452].

These formulations explicitly trade off task performance with search or representation cost, leading to optimal hierarchies that are sensitive to both environment structure and computational resource limits.

## 2. Multi-level Hierarchical Optimization

Resource-rational decomposition exhibits a characteristic multi-level structure in the underlying optimization, with each level associated with a distinct aspect of the planning or policy process:

1. **Action-level optimization:** Plans concrete action sequences given a subgoal or context.
2. **Subgoal/policy-selection level:** Chooses the order or assignment of subtasks or specialized policies (experts) to maximize intermediate value, penalized by the search or information-processing cost at this level.
3. **Decomposition/design level:** Selects the decomposition itself—e.g., subgoal sets, hierarchical clusters, or expert assignments—that optimizes overall performance-cost trade-off [2007.13862, 1907.11452].

The optimization structure is typically nested, with higher levels depending recursively on value/cost calculations from lower levels.

## 3. Algorithmic Approaches and Representational Efficiency

Instantiation of resource-rational decomposition depends on the planning or learning paradigm:

- **Graph-search planning:** Planning cost is operationalized as the number of nodes expanded (BFS or A*). Hierarchical subgoal sets \(Z\) are selected to minimize average planning cost while bounding the memory/representation cost (e.g., limiting \(|Z|\)) [2007.13862].
- **Hierarchical Reinforcement Learning (HRL):** Value-function components are recursively decomposed and only those components relevant to both parent and child subroutines are represented, via factored or projected representations. Structural conditions such as "additive irrelevance" and "factored exit" enable dramatic reduction in representation and sample complexity [1206.6851].
- **Information-theoretic partitioning:** Soft, mutual-information-regularized selectors route states to expert policies, each with bounded information-processing. The online learning is achieved via actor–critic gradient updates, in which per-step objectives include both task utility and information cost [1907.11452].

These algorithmic choices enable efficient learning and tractable hierarchical abstraction even in large-scale or high-dimensional problems.

## 4. Emergence and Selection of Hierarchies

The selection of hierarchical decomposition is determined not only by environment structure, but crucially by the agent’s resource constraints—cognition, computation, or information rate. Salient findings from empirical and simulation studies include:

- **Bottleneck discovery:** In environments with bottleneck states (e.g., "doorways" in two-room gridworlds), resource-rational models predict subgoal placement at bottlenecks for both uninformed and heuristic searchers, thus aligning with observed human hierarchies [2007.13862].
- **Information-driven specialization:** Increasing the information budget (\(\beta_1, \beta_2\)) leads to crisper state partitions and sharper policy specialization. With limited resources, the system smoothly interpolates from monolithic to specialized policies, optimizing global utility under constraint [1907.11452].
- **Representation compression:** Exploiting independence or decoupling (factored exit/separator structures) enables local policies to reason only about a reduced set of variables, confining resource investment precisely to those variables that affect and are affected by local and parent modules [1206.6851].

## 5. Theoretical Properties and Efficiency Bounds

Resource-rational frameworks offer rigorous guarantees regarding optimality, suboptimality due to resource constraints, and representational/computational efficiency:

- **Optimality structure:** Exact solutions can be obtained (e.g., via Blahut–Arimoto in information-theoretic settings), with suboptimality precisely bounded by information rates (\(\frac{1}{\beta_1}I(S;X) + \frac{1}{\beta_2}I(S;A|X)\)) [1907.11452].
- **Representation cost:** The use of factored and projected representations compresses parameter complexity from the full state space to the relevant marginals, often yielding orders-of-magnitude savings (compactness) in memory and statistical sample requirements [1206.6851].
- **Computational efficiency:** Hierarchical decomposition reduces average-case planning/learning time by restricting expensive computation to critical subspaces or decision points, with ablation studies reporting substantial speed-ups and maintained hierarchical optimality [2007.13862, 1206.6851].

## 6. Applications and Generalizations

Resource-rational hierarchical decomposition has broad applicability across domains:

- **Gridworld navigation and Tower of Hanoi planning**: Principled subgoal selection aligns with intuitive task decompositions and human reaction-time data [2007.13862].
- **Classification and regression:** Mixtures of linear experts, each assigned to a partition of input space, achieve non-linear performance under information-processing constraints [1907.11452].
- **Control and gain-scheduling:** Policy switching among local controllers in dynamical systems realizes efficient piecewise-linear control regimes without manual scheduling surfaces [1907.11452].
- **Reinforcement learning in structured environments:** Recursive value-function composition with state abstraction enables hierarchically optimal control and efficient sample usage [1206.6851].

The frameworks naturally extend to deeper hierarchies and cooperative multi-agent networks, with layered selectors/experts and distributed information budgets. Under general conditions—differentiable policies, sufficient functional capacity, stable gradients—convergence to locally optimal resource-rational solutions is ensured [1907.11452].

## 7. Normative Insights and Interpretive Remarks

Resource-rational decomposition offers a unifying, normative perspective: hierarchical representations and subgoal structures are not arbitrarily imposed, but arise as principled solutions to the joint optimization of task efficiency and resource expenditure. This perspective departs from simple trajectory compression or unsupervised community detection by explicitly conditioning decomposition on the planner’s algorithms, the structure of the environment, and the agent’s cognitive constraints [2007.13862, 1206.6851]. Hierarchical structure thus constitutes a form of representational investment, economizing downstream effort by anticipating and optimizing the allocation of limited computational and memory resources.

Source: https://www.emergentmind.com/topics/resource-rational-hierarchical-decomposition