---
title: Shortcut Learning for Abstract Planning (SLAP)
url: https://www.emergentmind.com/topics/shortcut-learning-for-abstract-planning-slap
type: topic
---

# Shortcut Learning for Abstract Planning (SLAP)

Shortcut Learning for Abstract Planning (SLAP) refers to a family of frameworks and algorithmic strategies that leverage the abstraction capabilities of Task and Motion Planning (TAMP) and temporally-extended actions (options) to introduce learned "shortcut" skills into model-based or model-free long-horizon decision-making. By augmenting a fixed set of human-engineered abstract actions with automatically discovered multi-step policies, SLAP aims to minimize plan length, improve task success rate, and bridge the gap between combinatorial planning and reinforcement learning in high-dimensional, continuous, and sparse-reward robotic domains.

## 1. Formal Foundation: Abstractions and Task Decomposition

SLAP operates in the setting of deterministic or continuous-state, continuous-action Markov Decision Processes (MDPs) $\mathcal{M}=\left(\mathcal{X}, \mathcal{U}, f, R, \gamma, p_0\right)$ or their stochastic variants, with temporally extended actions—options—that provide hierarchical structure. The key ingredients are:

- **State space:** $x \in \mathcal{X} \subset \mathbb{R}^n$
- **Action space:** $u \in \mathcal{U} \subset \mathbb{R}^m$
- **Deterministic transitions:** $x_{t+1} = f(x_t, u_t)$
- **Sparse reward:** $R(x_t, u_t) = -1$ per time step; reaching a goal state $g \subset \mathcal{X}$ is non-penalized
- **Objective:** Minimize episode length (cumulative cost)

In TAMP, the state space is abstracted using predicates so that $s = \operatorname{abstract}(x)$, providing symbolic or geometrically meaningful state representations. Options are tuples $a = \langle s^{a}_{\text{init}}, \pi^a, s^{a}_{\text{term}} \rangle$ mapping initiation and termination abstract states to a low-level policy $\pi^a\!: \mathcal{X} \rightarrow \mathcal{U}$. 

The abstract planning problem builds a directed two-level graph $G = (V_{\text{low}}, V_{\text{high}}, E_{\text{low}}, E_{\text{high}})$, highlighting the compositionality and reachability of hand-engineered as well as learned skills.

## 2. The Shortcut Learning Mechanism

SLAP introduces an algorithmic process for identifying and integrating new shortcut options:

1. **Candidate Extraction:** Search the abstract graph for pairs $(s_{\text{init}}, s_{\text{term}})$ not already directly connected by an option but reachable via a multi-step plan.
2. **Pruning via Stochastic Rollouts:** For each candidate, estimate via random rollouts the empirical reachability; only proceed to learn shortcuts where naïve exploration rarely discovers the transition.
3. **Shortcut Policy Learning:** For retained candidates, instantiate a new MDP where the goal is to reach $s_{\text{term}}$ from $s_{\text{init}}$. Use model-free RL (typically PPO) to learn a policy $\pi_\theta$ under the same low-level environment dynamics and reward structure:
   $$
   J(\theta) = \mathbb{E}_{\pi_\theta} \left[ \sum_{t=0}^{T-1} \gamma^t r(x_t, u_t) \right]
   $$
   Training terminates when $\operatorname{abstract}(x) = s_{\text{term}}$.
4. **Integration:** Each learned shortcut $\hat{a} = \langle s_{\text{init}}, \pi_\theta, s_{\text{term}} \rangle$ is added to the set of available options, updating the abstract graph for planning.
   
This process is algorithmically captured as follows (pseudocode excerpt):

```text
Algorithm: SLAP Training
Input: training tasks {(x₀,g)}, transition f, options 𝒜, N_rollout, T_rollout, K_rollout
1. For each training task:
   a. Build abstract graph G using 𝒜
   b. Extract candidate state pairs (s_i, s_j) not in 𝒜 but reachable in G
   c. For each candidate, run N_rollout rollouts of length ≤T_rollout
      If K_rollout reach s_j, retain candidate.
2. For each retained candidate, create MDP and train PPO policy π_{i,j}
3. Add new shortcut to 𝒜
```

## 3. Planning with Shortcuts: Integration and Execution

At test time, SLAP uses the augmented option set (hand-crafted and learned shortcuts) to construct the planning graph. Dijkstra’s algorithm is run on the low-level graph to search for the minimal-cost trajectory (in terms of time steps). Execution proceeds by sequentially triggering policies corresponding to the selected options. If a learned shortcut fails to reach its declared $s_{\text{term}}$ within a timeout $T_{\text{eval}}$, the planner prunes the edge and replans on the fly.

The computational complexity is:

- Candidate shortcut pairs: $O(|\mathcal{S}|^2)$
- Graph construction: $O(|\text{Ops}|\cdot|\text{Objects}|^\text{arity})$
- Planning: $O(|E|+|V|\log|V|)$ per Dijkstra search
- RL training is parallelized over candidates; empirical pruning removes $\approx$99% of candidates

A property of SLAP is *completeness*: the system falls back to pure planning if no shortcuts are useful, and reduces to end-to-end RL if tasks are trivial.

## 4. Empirical Behavior: Examples and Quantitative Evaluation

SLAP is evaluated in four simulated, sparse-reward robotic environments:

- **Obstacle 2D:** Planar robot; clear target region blocked by a single obstacle; 11 learned shortcuts.
- **Obstacle Tower:** PyBullet Panda arm; stack of obstacles; 92 shortcuts.
- **Cluttered Drawer:** Manipulator in a drawer with multiple obstacles; 74 shortcuts.
- **Cleanup Table:** Manipulator, irregular toys, wiper tool; 54 shortcuts.

Learned shortcuts include "slap" (pushing stacks of objects), "wiggle" (oscillating the gripper to clear adjacent objects), and "wipe" (sweeping multiple objects in one motion), violating the STRIPS “single-object” assumption and enabling multi-object contacts.

A summary of plan length reduction is tabulated:

| Environment      | Method          | Success | Plan Length | Reduction |
|------------------|-----------------|---------|-------------|-----------|
| Obstacle 2D      | SLAP            | 100%    | 17.8±2.0    | ↓ 31%     |
|                  | Pure Planning   | 100%    | 25.8±2.2    | 0%        |
|                  | PPO             | 0%      | 100 (max)   | N/A       |
| Obstacle Tower   | SLAP            | 100%    | 73.8±4.3    | ↓ 69%     |
|                  | Pure Planning   | 100%    | 238.6±12.8  | 0%        |
| Cleanup Table    | SLAP            | 100%    | 113.7±17.0  | ↓ 66%     |
|                  | Pure Planning   | 100%    | 446.3±34.9  | 0%        |
|                  | RL baselines    | 0%      | 500         | N/A       |

SLAP consistently achieves 100% success and substantially reduced plan lengths compared to pure planning or RL baselines. Hierarchical RL using the same options does not realize the plan optimality of SLAP.

Additional findings:

- As training proceeds, more shortcuts are mastered and plan lengths monotonically decrease, up to 500K steps.
- Generalization: SLAP trained on 3-block stacks generalizes plan lengths to unseen task instances (4–6 block towers, novel object sets) due to the reuse of abstract predicates and shortcut object-substitution schemes.

## 5. Theoretical Underpinnings and Relation to Abstract Model Learning

SLAP can be linked to frameworks that learn abstract world models over options [2406.15850], where abstraction functions $\phi: S \rightarrow \tilde{S}$ are learned to preserve the option-induced transition and initiation structure. Exact dynamics-preserving abstractions enable value preservation; approximate versions guarantee bounded value loss:
$$
|Q^\pi(s, o) - Q^\pi(\phi(s), o)| \leq \frac{\sqrt{\epsilon_R} + \gamma V_\text{max}\sqrt{\epsilon_T}}{1-\gamma}
$$
This suggests that SLAP's methodology for learning and integrating shortcut options complements model-learning approaches by expanding the skill set over which abstractions can be constructed and planned.

Practical model-learning involves maximizing contrastive objectives (InfoNCE) to induce embeddings that capture option effects while compressing irrelevant state details. Planning then proceeds in the abstract MDP equipped with both hand-crafted and learned shortcut options, providing sample-efficient and generalizable policies even in continuous, high-dimensional environments.

## 6. Limitations and Future Research

Key limitations and open directions for SLAP include:

- Dependence on fixed, user-provided abstractions (predicate definitions and high-level TAMP structure). Learning or refining abstractions jointly with options is a prospective extension.
- The quadratic growth in candidate shortcut pairs; while aggressive empirical pruning alleviates this, scaling to very large abstract state spaces may require additional heuristics or learned prioritization schemes.
- Applicability to stochastic or partially observable environments is not yet fully established, though preliminary findings indicate improved robustness when shortcut options are present.
- Environments where skill effects overlap extensively may defeat abstraction, requiring more expressive or hierarchical representation mechanisms.
- Integration of automated option discovery, continual learning of abstractions, and multi-modal perception (e.g., vision and tactile sensing) are promising future directions.

SLAP demonstrates that leveraging existing TAMP abstractions to guide reinforcement learning enables the discovery of dynamic, multi-object skills that drastically shorten plans and improve long-horizon task success, moving toward integrated planning-learning systems combining generalization with physical improvisation.

Source: https://www.emergentmind.com/topics/shortcut-learning-for-abstract-planning-slap