Hierarchical Task Networks (HTNs)

• Hierarchical Task Networks (HTNs) are a planning formalism that decomposes tasks into primitive actions using recursive methods.
• Probabilistic HTNs extend the concept by incorporating method preference weights and leverage EM-based grammar induction for learning from observed traces.
• Rescaling observed plan frequencies in HTNs helps correct feasibility bias, ensuring accurate derivation of user intent and improved plan selection.

Hierarchical Task Networks (HTNs) are a formalism in automated planning representing procedural knowledge as a hierarchy of tasks and decompositions. HTNs structure complex planning domains by recursively refining compound tasks into sub-tasks until a sequence of primitive actions executable in the environment is derived. This paradigm underpins classical, probabilistic, learning-based, and practical algorithmic advances fundamental to modern AI planning.

1. Formal Syntax and Semantics

An HTN domain is defined as a triple $(A, T, M)$:

• $A$: A finite set of primitive actions, each directly executable and specified with STRIPS-like preconditions and effects.
• $T$: A finite set of task symbols partitioned into primitives ($A$) and compounds, where compounds model abstract procedures requiring further breakdown.
• $M$: A finite set of methods (reduction rules), each of the form $Z \rightarrow \langle \text{body} \rangle$, where $Z$ is a compound task, and the body is either a pair of tasks $(X, Y)$ or a single primitive $a$.

HTN planning proceeds by recursively decomposing a root-level compound task $T_{root}$, at each step instantiating applicable methods to replace a task $Z$ by its body, continuing until only primitives remain. A plan $\varphi = [a_1, \dots, a_n]$ is HTN-valid iff (a) $\varphi$ is executable from the initial state and achieves the goal, and (b) there exists a decomposition tree with $T_{root}$ as the root and leaves spelling out $\varphi$ in left-to-right order (Li et al., 2010).

The search is typically depth-first, instantiating primitive actions when preconditions are satisfied, and backtracking on failure.

2. Probabilistic Extension: pHTNs and Plan Preference

Probabilistic HTNs (pHTNs) equip methods with preference weights:

• Each method $m$ refining a compound $Z$ is assigned $\theta(m)$, with $\sum_{m\in M(Z)} \theta(m) = 1$.
• The formalism is isomorphic to Probabilistic Context-Free Grammars (PCFGs): tasks as nonterminals, primitives as terminals, methods as productions with probabilities.

The probability of a plan $\varphi$ under a pHTN is

$$P(\varphi) = \sum_{d \in D(\varphi)} \prod_{m \in d} \theta(m)$$

where $D(\varphi)$ denotes all full derivations yielding $\varphi$.

pHTNs allow encoding user preferences over alternative methods within the hierarchy, capturing degrees of preference (not just feasibility) (Li et al., 2010).

3. Learning pHTNs: EM-based Grammar Induction

To learn pHTN structures and preferences from user behavior, the following approach is adapted from grammar induction:

• Given observed plan multiset $\Phi = \{\varphi_1, \dots, \varphi_N\}$, fix method schema $M$ and estimate probabilities $\theta$ by maximizing:

$L(\theta) = \sum_{i=1}^N \log P(\varphi_i|\theta)$

• Apply Expectation-Maximization (EM):
  • E-step: Compute expected counts $E[N(m|\varphi_i)]$ of each method under current $\theta$ by parsing each plan.
  • M-step: Update $$\theta(m) = E[N(m)]/ \sum_{m':head(m')=head(m)} E[N(m')]$$

A hard-EM variant assigns each plan its most probable derivation $d^*$ and aggregates method counts accordingly (Li et al., 2010).

4. Learning Under Feasibility Constraints: Rescaling

Observed plans may not reflect true user preference $U(\varphi)$ due to feasibility constraints. If plans are observed only within feasible sets $F_s$ at state $s$, the empirical frequencies underestimate $U(\varphi)$.

To correct for bias:

• Cluster plan observations by feasible set overlap.
• Infer pairwise odds $U(\varphi)/U(\psi)$ from co-occurrence frequencies.
• Rescale by assigning each plan a weight $w(\varphi)$ so that $w(\varphi)/w(\psi) \approx \text{empirical odds}$.
• Feed weighted counts to the EM learner: maximize

$$\tilde{L}(\theta) = \sum_\varphi w(\varphi) \cdot \log P(\varphi|\theta)$$

• E-step: each $\varphi$ contributes $w(\varphi)$ counts to methods in its parse.

This procedure recovers correct preferences even when many preferred plans are unobservable due to feasibility constraints (Li et al., 2010).

5. Illustrative Example: Preferences Over Travel Modes

Consider compound task $\text{Travel}$ decomposed into sequences corresponding to train and bus routes:

• $\text{Travel} \rightarrow A_1 B_2$ (train), $\text{Travel} \rightarrow A_2 B_1$ (bus)
• Assign $\theta(\text{Train})=0.8, \theta(\text{Bus})=0.2$ to encode a 4:1 train preference.
• If "train" is infeasible half the time, raw observed frequencies will bias toward bus; the rescaling recovers the true 4:1 odds for EM.

This demonstrates that pHTN can encode both plan structure and soft, graded preferences (Li et al., 2010).

6. Comparison to Classical HTN Learning

Prior HTN learning (e.g., CAMEL, HTN-Maker) generally assumes hand-crafted method structures and learns only method preconditions, focusing on modeling domain physics or search control. In contrast, the pHTN approach learns both structure (new compound tasks and decomposition rules inferred from traces) and preferences over decompositions. This enables capturing user intent and flexibility in plan generation rather than Boolean admissibility alone (Li et al., 2010).

7. Impact and Methodological Significance

Probabilistic HTNs provide a grammar-like framework for representing, learning, and deploying user-centric planning knowledge with explicit preference modeling. The EM-based grammar induction and rescaling for feasibility bias establish a methodological foundation for inferring both hierarchical structure and plan preference directly from behavioral traces. This opens applications in preference-aware planning, personalized assistive agents, and robust plan induction in domains with feasibility bottlenecks.