Belief-Delete-Relaxation Heuristic

• The paper introduces the belief-delete-relaxation heuristic to estimate goal distances in belief-space planning by aggregating relaxed distances over multiple possible worlds.
• It leverages planning graphs and Binary Decision Diagrams to represent uncertainty and extract unioned relaxed plans efficiently.
• Empirical evaluations demonstrate that this approach significantly reduces node expansions and CPU time compared to traditional heuristics in both conformant and conditional planning.

The belief-delete-relaxation heuristic, denoted $h_{DR}$, is a reachability heuristic for measuring goal distance within belief-space planning. It generalizes the classical delete-relaxation heuristic $d^+$ to the non-deterministic setting of belief states, enabling scalable conformant and conditional planning. $h_{DR}$ estimates the cost of reaching a goal belief state from a set of possible worlds (belief state) by aggregating relaxed distances over all constituent worlds, using planning graph structures and, in efficient implementations, Binary Decision Diagrams (BDDs) to represent uncertainty and perform symbolic computations (Bryce et al., 2011).

1. Formalization of Belief-State Distance

Let $B$ and $B'$ be belief states, each corresponding to a set of classical states or worlds. The set $\mathcal{M}(B) = \{S \mid S \models B\}$ enumerates the possible worlds in $B$. The single-state delete-relaxed distance $d^+(S)$ is defined as the cost (i.e., number of actions) to reach the goal from classical state $S$ in the relaxed STRIPS problem (where delete effects are ignored).

To lift this to belief-space, the belief-state distance is defined via aggregation over all worlds: $$\mathrm{dist}(B,B') = \max_{S \in \mathcal{M}(B)}\;\min_{S' \in \mathcal{M}(B')} d^+(S \to S')$$ In the common case where $d^+$0 is the goal belief state $d^+$1, the delete-relaxation heuristic is: $d^+$2 Alternative aggregation strategies exist, including sum-aggregation ($d^+$3, assuming independence), and the union-of-relaxed-plans cost ($d^+$4, handling actuation overlap).

2. Single-World Delete-Relaxation and Aggregation Methods

In classical STRIPS planning, $d^+$5 is computed from a planning graph built on state $d^+$6, extracting a relaxed plan by accumulating all actions across layers until the goal is supported: $d^+$7 Here, $d^+$8 is the set of actions at relaxed-plan layer $d^+$9. Aggregating over all $h_{DR}$0 leads to three belief heuristics:

Name	Formula	Interpretation
$h_{DR}$1	$h_{DR}$2	Admissible (positive interaction)
$h_{DR}$3	$h_{DR}$4	Optimistic independence
$h_{DR}$5	$h_{DR}$6, $h_{DR}$7	Overlap/union-of-relaxed-plans

$h_{DR}$8 is admissible, $h_{DR}$9 tends to under-guide by overestimating, and $B$0 balances overlap, empirically dominating $B$1 and $B$2 in informativeness (Bryce et al., 2011).

3. Planning-Graph Construction and BDD-Based Representation

Belief states are represented as propositional formulas over fluents with BDDs. The Labelled Uncertainty Graph (LUG) merges the planning graphs of all individual worlds while carrying a propositional label $B$3 for each node $B$4 at level $B$5. This label, a BDD formula, characterizes the set of worlds from which $B$6 is reachable within $B$7 relaxed layers.

LUG Construction Pseudocode

• Initial literal layer: Each literal $B$8’s label at layer $B$9: $B'$0.
• Action/effect layers: For action $B'$1 with precondition $B'$2, the label is obtained with a conjunction of child labels. For effect $B'$3 with antecedent $B'$4, $B'$5.
• Literal propagation: Next literal layer’s label $B'$6 is the disjunction over all effects reaching $B'$7.

All label computations operate over BDDs, allowing efficient symbolic set reasoning even with large sets of worlds.

4. Relaxed Plan Extraction from the LUG

Once the level $B'$8 is reached where $B'$9, a relaxed plan is extracted in a single bottom-up sweep. For each relaxed plan layer, the unioned action set supporting all necessary literals (worlds) is selected by minimal effective set covering (BDD-based). The process sums the size of action sets per layer: $\mathcal{M}(B) = \{S \mid S \models B\}$0 This “unioned” relaxed plan avoids double-counting actions that simultaneously advance multiple worlds. Set operations are efficiently managed on BDDs.

5. Properties of Belief-Delete-Relaxation Heuristics

$\mathcal{M}(B) = \{S \mid S \models B\}$1 is admissible if $\mathcal{M}(B) = \{S \mid S \models B\}$2 is an admissible lower bound for every world $\mathcal{M}(B) = \{S \mid S \models B\}$3. This follows because any strong conformant plan must suffice for all constituent worlds, so the heuristic lower bound is the maximum relaxed cost.

The heuristic is monotonic: along successive planning graph layers, labels only grow ($\mathcal{M}(B) = \{S \mid S \models B\}$4), so the first level supporting the goal is well-defined.

$\mathcal{M}(B) = \{S \mid S \models B\}$5 (union-of-relaxed-plans) does not over-count actions that are used in multiple worlds and does not under-count actions that are uniquely required. Empirically, $\mathcal{M}(B) = \{S \mid S \models B\}$6 is the most informative, providing guidance superior to $\mathcal{M}(B) = \{S \mid S \models B\}$7 and $\mathcal{M}(B) = \{S \mid S \models B\}$8.

6. Computational Complexity

Let $\mathcal{M}(B) = \{S \mid S \models B\}$9 be the number of fluents, $B$0 the number of actions, $B$1 the number of graph layers until fixpoint, and $B$2 the cost of a BDD operation.

• Single-world GraphPlan construction: $B$3.
• LUG construction: Per level $B$4; total $B$5.
• Relaxed plan extraction: $B$6 plus cover-set overhead from BDD operations.

Empirical findings indicate $B$7 remains moderate with BDD sizes up to thousands of worlds, and overall LUG plus plan extraction typically executes in seconds on standard benchmarks (Bryce et al., 2011).

7. Empirical Evaluation and Practical Impact

In testing with the CAltAlt conformant regression planner (A* search) and POND conditional progression planner (AO* search), belief-delete-relaxation heuristics ($B$8) dramatically outperformed basic and prior heuristics. Notable results include:

• Node Expansion: $B$9 expanded 10–100$d^+(S)$0 fewer nodes than $d^+(S)$1 or $d^+(S)$2 and solved 50–100% larger instances.
• Accuracy and Guidance: $d^+(S)$3 (union of multiple relaxed plans) is nearly as accurate as $d^+(S)$4 but 2–5$d^+(S)$5 slower.
• Simpler Heuristics: $d^+(S)$6 (single-graph) misestimates and fails on problems with $d^+(S)$710 initial states; $d^+(S)$8 is too optimistic, and $d^+(S)$9 is too pessimistic.
• Conditional Planning: $S$0 yields 30–100$S$1 fewer node expansions and 5–20$S$2 less CPU time than MBP and GPT on large benchmarks, and results in far shorter conditional branches.

Limitations include potential underestimation in domains with low world-overlap and deep serial plans, and, in principle, possible BDD explosion, although manageable in documented tests (Bryce et al., 2011).

In sum, the belief-delete-relaxation heuristic implemented via a single LUG with BDD-based labeling and unioned relaxed-plan extraction offers an advanced tradeoff between accuracy and computational efficiency, standing as a leading approach for conformant and conditional planning in belief space.