STAP: Sequencing Task-Agnostic Policies

• STAP is a framework for long-horizon robotic manipulation that sequences independently learned skills by optimizing a product of Q-values and predicted dynamics.
• It employs product-of-Q planning and policy CEM to coordinate action dependencies and ensure geometric feasibility across sequential manipulation tasks.
• Evaluated in simulation and on real robots, STAP demonstrates competitive performance by filtering uncertainty with SCOD and integrating with TAMP systems.

STAP, short for Sequencing Task-Agnostic Policies, is a framework for long-horizon robotic manipulation that plans over a library of learned low-level skills without retraining them for every new task. Its central idea is to treat each skill as a self-contained manipulation primitive with its own policy, Q-function, and learned local dynamics model, and then optimize the parameters of an entire skill sequence jointly so that the full sequence is geometrically feasible. In this formulation, Q-functions are used as measures of skill feasibility, and planning maximizes the joint success of all skills in the sequence rather than executing each skill greedily in isolation. The framework is evaluated in simulation and on a real robot, and is also presented as a geometric reasoning module for task and motion planning (TAMP) (Agia et al., 2022).

1. Problem formulation

STAP assumes a given skill library

$\mathcal{L} = \{\psi^1, \dots, \psi^K\},$

where each skill $\psi^k$ consists of a parameterized manipulation primitive $\phi^k$ and a learned policy

$\pi^k(a^k \mid s^k)$

that predicts action parameters $a^k$ from a state $s^k$. The long-horizon setting is defined by a plan skeleton

$\tau = [\psi_1,\dots,\psi_H],$

provided by a high-level planner, together with the continuous action parameters

$\xi = [a_1,\dots,a_H]$

that STAP must choose so that the sequence is feasible and successful (Agia et al., 2022).

The framework is studied on sequential manipulation tasks such as picking an object and placing it somewhere, pulling a hook to bring a distant object into reach, pushing objects around obstacles, and TAMP-style problems in which a symbolic planner proposes a sequence of skills. These examples are used to motivate the claim that later actions depend strongly on earlier ones: a grasp choice can determine whether a later pull, place, or push is executable. This suggests that long-horizon manipulation cannot be reduced to independent optimization of single skills.

Each skill is trained as a single-step contextual bandit or one-step MDP,

$$\mathcal{M}^k = \left(\mathcal{S}^k, \mathcal{A}^k, T^k, R^k, \rho^k\right),$$

with binary reward $R^k \in \{0,1\}$. Success for a skill therefore means that its primitive executes successfully. The long-horizon environment is modeled as a sequence of such skills, with STAP operating at the level of sequencing and geometric coordination rather than end-to-end retraining.

2. Skill learning and task-agnosticity

In STAP, task-agnosticity refers to the way skills are acquired rather than to the planning problem. Each skill is trained independently on its own one-step environment using standard RL; the experiments use Soft Actor-Critic (SAC), although the framework is described as agnostic to the underlying skill-learning method. Training produces a policy $\psi^k$0, a Q-function $\psi^k$1, and a replay buffer of single-step experience $\psi^k$2 for each skill (Agia et al., 2022).

The per-skill Q-function is

$\psi^k$3

Because training is modular and independent per skill, new skills can be added later, existing skills do not need retraining, and different acquisition methods could be mixed, including RL, imitation learning, or handcrafted skills. The framework therefore separates skill acquisition from downstream long-horizon composition.

A common misconception is that “task-agnostic” means STAP ignores downstream task structure. The description is narrower: the skills are not trained on a specific long-horizon task, but only on their own local manipulation objective. The long-horizon structure re-enters at planning time, when STAP optimizes the entire sequence jointly.

3. Product-of-Q planning and action dependencies

The main observation in STAP is that a skill Q-function can be interpreted as a success probability: $\psi^k$4 where $\psi^k$5 maps the long-horizon state to the state representation used by the $\psi^k$6-th skill. On this basis, STAP defines the planning objective

$\psi^k$7

which approximates the probability that every skill in the sequence succeeds (Agia et al., 2022).

The product form is used because if any skill fails, the long-horizon plan fails. The framework explicitly contrasts this with a sum of Q-values, which can mask a catastrophic low-probability step. The motivating example is a plan with one zero-probability step versus a plan whose steps are all around $\psi^k$8: the two may have similar sums, but not similar feasibility.

STAP handles dependencies between actions by predicting future states with learned dynamics models, evaluating downstream Q-functions on those predicted states, and choosing the whole sequence jointly. Future states are rolled out as

$\psi^k$9

This is the mechanism used to reduce myopic behavior. A locally optimal action for one skill may create a bad state for the next skill; the hook-grasp example is used to illustrate precisely this failure mode.

The framework also introduces an informal geometric feasibility condition requiring overlap between the terminal-state distribution of one skill and the initial-state distribution of the next: $\phi^k$0 Since the true initial-state distributions are unknown, STAP does not compute this overlap directly. Instead, it uses learned dynamics to predict the next state and then evaluates whether predicted state-action pairs are plausible for downstream critics.

4. Planning algorithms, uncertainty, and TAMP integration

STAP uses sampling-based optimization over action sequences. The planning methods described are shooting, which samples many action plans and keeps the best, and CEM, which iteratively refines the sampling distribution toward high-scoring plans. A particularly important variant is policy shooting or policy CEM, in which samples are drawn around the learned policy mean,

$\phi^k$1

This is reported to work better than uniform random sampling in high-dimensional action spaces because the learned policy provides a better initialization (Agia et al., 2022).

The empirical ranking reported for planners is that random shooting and random CEM perform poorly, policy shooting is better, and policy CEM is strongest among the planners. This result is consistent with the framework’s dependence on good initialization in continuous action spaces.

For uncertainty quantification, STAP uses SCOD (Sketching Curvature for OOD Detection), which produces a Gaussian posterior over the Q-function,

$\phi^k$2

In the TAMP experiments, highly uncertain candidate plans are filtered out, yielding SCOD Policy CEM. The appendix also notes a lower-confidence-bound-style robust objective in which uncertainty is subtracted from each per-skill Q term.

STAP is explicitly not a symbolic task planner. In TAMP mode, a task planner such as a PDDL planner provides candidate plan skeletons; STAP grounds each skeleton by optimizing continuous action parameters, scores the geometric feasibility of each candidate, and returns the best feasible skeleton/action-plan pair. The framework is therefore a geometric reasoning module that can be plugged into integrated task and motion planning systems, and its success probabilities are noted as possible heuristics for classical planning or foundation-model-based planning.

5. Experimental setting and empirical results

The evaluated skill library contains four skills: Pick, Place, Pull, and Push. These are parameterized manipulation primitives over low-dimensional object states such as 6D poses, and actions are 4D in the experiments, for example grasp pose or push/pull direction and distance. The tasks span 9 long-horizon manipulation tasks across 3 domains—Hook Reach, Constrained Packing, and Rearrangement Push—with plan skeleton lengths ranging from 4 to 10 skills, and each task evaluated on 100 random scene instances (Agia et al., 2022).

The baselines are Oracle forward-simulation planning, Random Shooting / Random CEM, Policy Shooting / Policy CEM, Greedy execution, and DAF-style affordance planning for long-horizon manipulation. On simpler tasks where Oracle planning is tractable, Policy CEM nearly matches Oracle success. The reported evidence includes low prediction error between the product-of-Q objective and actual success, together with planning that is about four orders of magnitude faster than the Oracle simulator-based approach.

On the 9 long-horizon tasks, STAP with Policy CEM solves tasks it never saw during skill training. Compared with DAF-trained skill models, STAP is reported as competitive on several tasks and as outperforming DAF on tasks with especially complex action dependencies, especially in Rearrangement Push. The contrast is notable because DAF is trained directly on the task distribution, whereas STAP uses the same independently trained skill library for all downstream tasks.

In the TAMP setting, STAP is combined with a PDDL planner and uncertainty quantification. The reported outcomes are very high performance on a Hook Reach TAMP problem and strong improvement from SCOD-based filtering on a Constrained Packing problem with OOD distractor objects. The real-robot demonstration uses RGB-D sensing and pose estimation from a Kinect camera, and the qualitative results are used to support the claim that STAP-trained skills can be sequenced on a physical robot.

6. Significance, limitations, and nomenclature

STAP addresses a specific problem in robotic manipulation: how to reuse learned skills for new long-horizon tasks without training on every possible task sequence. Its main contribution is the replacement of greedy skill chaining with joint optimization of geometric feasibility through a product of learned Q-values, augmented by look-ahead dynamics prediction and OOD-aware search. This suggests a modular route to long-horizon manipulation in which the burden of generalization is shifted from monolithic task training to skill reuse and sequence-level planning (Agia et al., 2022).

The limitations acknowledged for the framework are equally specific. STAP assumes reasonably accurate Q-functions and dynamics models, struggles under heavy partial observability or stochastic dynamics, and is currently focused on geometrically challenging but relatively low-dimensional manipulation settings. The appendix further notes that scaling to image or point-cloud observations would require stronger representations and latent dynamics models.

The acronym STAP is not unique across technical literature. In radar, it commonly denotes space-time adaptive processing, a class of adaptive clutter-suppression methods for airborne and SAR systems (Khlebnikov et al., 2010). In approximation algorithms, STAP also denotes the Steiner Tree Augmentation Problem (Ravi et al., 2022). In mobile computing, STAP has been used for the Shuffle-Tokenized App Predictor (Fan et al., 28 May 2026). Within robotics, however, Sequencing Task-Agnostic Policies refers specifically to the long-horizon manipulation framework described above.