---
title: Hierarchical LLM-Based Planning Architectures
url: https://www.emergentmind.com/topics/hierarchical-llm-based-planning-architectures
type: topic
---

# Hierarchical LLM-Based Planning Architectures

A hierarchical Large Language Model (LLM)-based planning architecture is an integrated computational framework in which tasks specified in natural or formal language are decomposed into multi-level representations—typically spanning from abstract intentions to fine-grained executable actions—through the coordinated use of scene or knowledge graphs, automata, and semantic abstraction. LLMs serve crucial roles in both translating instructions to formal task specifications and providing high-level semantic guidance, while the overall planning loop incorporates rigorous guarantees via formal heuristics, multi-resolution domain decomposition, and semantic attribute grounding as seen in "Optimal Scene Graph Planning with Large Language Model Guidance" [2309.09182].

## 1. Environment Representation and Semantic Abstraction

Hierarchical LLM-based planning operates on a semantic scene representation structured as a scene graph:
\[
G = (V, E, \{A_k\}_{k=1}^K)
\]
where $V$ is the set of nodes (each representing, e.g., spatial regions or free space), $E \subseteq V \times V$ captures connectivity, and $\{A_k\}_{k=1}^K$ represents hierarchical attribute sets, each $A_k$ defining semantic classes such as objects, rooms, and floors. Each attribute $a \in A_k$ is associated with a subset $V_a \subset V$. For every node $s \in V$, atomic propositions encode semantic properties, and a labeling function $\ell: V \rightarrow 2^\mathcal{P}$ maps nodes to the propositions they fulfill.

This explicit structural hierarchy is further reduced for LLM consumption: the full graph is converted to an "attribute hierarchy" in a nested YAML format, exposing only high-level semantic entities and their inclusion relationships (e.g., rooms within floors, objects within rooms), each tagged with unique IDs to facilitate unambiguous reasoning and reduce prompt overload.

## 2. LLM-Guided Task Formalization and Semantic Heuristics

LLMs fulfill two distinct, critical functions:

**A. Natural Language to Formal Specification**  
LLMs receive the attribute hierarchy and the user's mission (e.g., "Reach the oven in the kitchen") and, through prompt engineering (with regex-based extraction and translation examples in prefix LTL notation), transduce it to a co-safe Linear Temporal Logic (LTL) formula $\varphi_\mu$. The formula undergoes syntactic and co-safety (negation in normal form, only $\bigcirc$ and $\mathcal{U}$ operators) verification and is then determinized to a finite automaton $M_{\varphi}$ reflecting task progress.

**B. Semantic Heuristic Guidance**  
During search, the LLM is further prompted with the current automaton state, scene context, and motion function signatures; it emits high-level semantic guidance—function call sequences (e.g., $\text{move}(a_i, a_j)$)—with associated user-defined costs. These sequences instantiate a heuristic $h_{\mathrm{LLM}}$, injected into the planner to bias towards transitions deemed semantically promising beyond what metric/geometric information alone could reveal.

## 3. Hierarchical Planning Domain Construction

The planning state space is structured across multiple resolution levels. For each attribute layer $k$, the planner defines:
\[
X_k = V_k \times Q, \quad \text{where}\ V_k = \bigcup_{a \in A_k} V_a
\]
and $Q$ is the automaton state set. Transitions $(x_i, x_j)$ are permitted if:
1. $s_j$ crosses from a region $V_a$ to the boundary of another $V_b$ ($a \neq b$, non-overlapping interiors),
2. The automaton transition is respected: $q_j = T(q_i, \ell(s_i))$,
3. $s_j$ is minimized according to $d(s_i, s)$, the edge/path cost.

The lowest-level "anchor" domain $X_0=V \times Q$ allows unification of all abstraction layers and connects via original scene graph transitions.

## 4. Multi-Heuristic Any-Time Search and Theoretical Guarantees

Planning exploits AMRA*: Anytime Multi-resolution Multi-heuristic A*. Two heuristics interact:

- **LTL Heuristic $h_{\mathrm{LTL}}$**:  
  This function is provably consistent (and thus ensures optimality). It computes for a planning state $(s, q)$:
\[
h_{\mathrm{LTL}}(s, q) = \min_{t \in V} [ c(s, t) + g(\ell(t), T(q, \ell(t))) ]
\]
where $g(\ell,q)$ is defined recursively via a Bellman/Dijkstra optimization over labels and automaton states, and $c(s, t)$ is the cost between nodes.

- **LLM Semantic Heuristic $h_{\mathrm{LLM}}$**:  
  The sequence of moves from the LLM, $f_i(a_j, a_k)$, each weighted by their cost, is aggregated:
\[
h_{\mathrm{LLM}}(s, q) = \sum_{i=0}^N f_i(a_j, a_k)
\]
Although potentially inadmissible, $h_{\mathrm{LLM}}$ accelerates search, while $h_{\mathrm{LTL}}$ maintains optimality guarantees.

## 5. Experimental Validation and Performance Analysis

The system is evaluated in real-world inspired virtual environments (e.g., Allensville, Benevolence, Collierville) with complex missions—e.g., "Visit bathroom and avoid the sink, then go to the dining room and sit on a chair." Configuration variants include baseline A*, full hierarchy with and without LLM guidance, and selective application of the LLM heuristic to certain levels.

Key results:
- **Computation Time**: The "ALL" LLM-heuristic configuration yields the fastest feasible path computation and fastest convergence to optimality.
- **Optimality Gap**: The cost of first-found path is closer to the optimal versus baselines.
- **Node Expansions**: Substantial reduction in nodes expanded per iteration when using LLM guidance; search efficiency is directly improved by hierarchical semantic heuristics.

These metrics demonstrate that integrating semantic LLM heuristics with hierarchical, multi-resolution planning domains yields marked improvements in both speed and optimality of complex natural language task execution.

## 6. Applications and System Implications

The described framework enables:

- **Autonomous Mobile Robots**: Robust execution of complex, compositional tasks specified in natural language, leveraging high-level scene abstractions for navigation and manipulation in semantically rich environments.
- **Symbol Grounding**: LLMs serve as a crucial link between linguistic commands and their corresponding propositional representations over the scene graph, enabling efficient symbol grounding in previously ambiguous or “free-form” settings.
- **Scalability**: Multi-resolution abstraction dramatically reduces search complexity in large-scale, multi-layer environments; LLM heuristics scale with semantic richness rather than just raw graph cardinality.
- **Extension to Dynamic/Real Scenarios**: The planner’s modular construct allows systematic extension, integration with live perceptual updates, and adaptation for safety-critical operations where formal optimality and correctness are paramount.

## 7. Summary and Theoretical Significance

This hierarchical LLM-based planning architecture exemplifies a synergistic integration of deep semantic scene abstractions, formal task logic, and both rigorous (LTL) and informed (LLM) heuristic guidance. The method achieves:

- Explicit representation of environments as hierarchical scene graphs with atomic propositions.
- Formal natural language task translation to co-safe LTL via LLMs.
- Hierarchical domain factorization reflecting geometric, semantic, and automaton structure.
- Dual-heuristic anytime planning, marrying search efficiency with guaranteed optimality.
- Empirically demonstrated gains in speed and plan quality.

As a result, it provides a technically grounded paradigm for deploying natural language-driven robotic planning in environments of significant scale and structural complexity, while maintaining guarantees required for real-world autonomy [2309.09182].

Source: https://www.emergentmind.com/topics/hierarchical-llm-based-planning-architectures