Predicate Action Skills (PACTS) Overview

• Predicate Action Skills (PACTS) are closed-loop visuomotor policies that jointly generate action trajectories and predicate-belief trajectories, integrating execution with symbolic outcomes.
• The approach uses generative models like DDPM or conditional flow matching with architectures such as UNet or Transformers to yield coherent action–predicate rollouts and high performance metrics.
• PACTS bridges imitation learning and symbolic planning, enabling zero-shot skill recomposition, online monitoring, and disturbance recovery in both simulated and real-world environments.

Searching arXiv for the primary PACTS paper and closely related predicate/skill abstraction work to ground the article. Predicate Action Skills (PACTS) are a class of closed-loop visuomotor policies that model a skill as a joint generative process over action trajectories and predicate-belief trajectories, so that a single sample yields a coherent action–outcome rollout. The formulation is motivated by the claim that modern Learning from Demonstration policies usually model distributions over action trajectories alone and therefore cannot expose the symbolic outcomes needed for planning, monitoring, and replanning under novel skill compositions. In PACTS, online predicate predictions serve as a symbolic interface for zero-shot composition of learned skills via planning (Quartey et al., 20 May 2026).

1. Conceptual basis

The central claim behind PACTS is that action plausibility and symbolic outcome prediction should not be separated. A standard generative skill policy models only $p(\mathbf{x}\mid \mathbf{o})$, where $\mathbf{x}$ is an action trajectory and $\mathbf{o}$ is the current observation. PACTS instead treats a skill rollout as an action–outcome object, asking the model to represent not only which action chunk to execute but also which predicates are expected to hold as execution unfolds. The motivating predicate vocabulary is explicitly symbolic: examples include “door open,” “object in goal region,” and “gripper holding object” (Quartey et al., 20 May 2026).

This changes the role of predicates from passive state labels to executable control interfaces. A predicate-belief trajectory can be used to test whether a skill’s intended effect has been achieved, whether execution has drifted, and whether replanning is required. The paper therefore positions PACTS as a bridge between generative visuomotor imitation and symbolic planning, rather than as a purely perceptual predicate classifier or a purely motor controller (Quartey et al., 20 May 2026).

Earlier work on grounding predicates through actions already showed that symbolic state variables can be learned from weak supervision derived from action schemas, using known preconditions and postconditions to train perceptual predicate detectors. That line of work, however, assumes given action schemas and learns visual predicate classifiers rather than a single joint action–predicate rollout model. PACTS can therefore be read as moving the symbolic interface from a separate perception module into the skill model itself (Migimatsu et al., 2021).

2. Formal model

For a horizon $H$, PACTS defines an action trajectory

$$\mathbf{x}=(\mathbf{a}_t,\dots,\mathbf{a}_{t+H-1})\in\mathbb{R}^{H\times d_a},$$

and a predicate-belief trajectory

$$\mathbf{z}=(\mathbf{z}_{t+1},\dots,\mathbf{z}_{t+H})\in[0,1]^{H\times J},$$

where $\mathbf{z}_{t+h,j}$ is the model’s confidence that predicate $j$ will hold at future step $t+h$. These are concatenated into

$$\mathbf{y}\triangleq[\mathbf{x}\,|\,\mathbf{z}] \in \mathbb{R}^{H\times(d_a+J)},$$

and the skill learns a single conditional generative model $\mathbf{x}$0 (Quartey et al., 20 May 2026).

The paper instantiates this joint model with either DDPM or conditional flow matching. Under DDPM, the loss is

$\mathbf{x}$1

Under conditional flow matching, with interpolation path $\mathbf{x}$2, the loss is

$\mathbf{x}$3

In both cases the training target $\mathbf{x}$4 contains both action and predicate channels, so the supervision signal is intrinsically joint rather than multi-head but factorized (Quartey et al., 20 May 2026).

Architecturally, PACTS consists of an observation encoder and a conditional trajectory generator. The generator is instantiated with either a Temporal UNet or a Transformer backbone. Predicate labels are binary in the dataset, but during generation they are treated as continuous channels, linearly normalized to $\mathbf{x}$5, generated jointly with the action channels, and then mapped back to $\mathbf{x}$6 at inference. When discrete symbolic decisions are needed, the paper thresholds predicate beliefs at $\mathbf{x}$7 (Quartey et al., 20 May 2026).

A key baseline isolates the distinction between joint sampling and ordinary multi-task learning. The baseline assumes

$\mathbf{x}$8

shares representations across action and predicate prediction, and uses

$\mathbf{x}$9

PACTS differs not merely by sharing features, but by generating action and predicate trajectories as one coupled sample (Quartey et al., 20 May 2026).

3. Supervision and training regime

The training example in PACTS is a tuple $\mathbf{o}$0 consisting of the current observation, a future action chunk, and an aligned future predicate trajectory. The paper studies both single long-horizon policy training and skill-centric training for composition, where demonstrations are segmented into skill-level sub-trajectories. In the skill-centric setting, separate PACTS models are trained for individual skills and later composed by a planner (Quartey et al., 20 May 2026).

Predicate supervision comes from two different sources. In simulation, predicates are derived from predicate oracles built from simulator state. In the real world, the paper uses a segmentation and labeling toolkit based on sparse annotation and interpolation: sparse keyframe annotation of grounded predicates defined from the PDDL domain, dense per-timestep predicate labels by forward-fill interpolation, and skill segmentation by detecting goal-achieving predicate configurations and backward-filling execution phases. Action supervision is ordinary demonstration supervision over future action trajectories (Quartey et al., 20 May 2026).

This training regime implies a specific scope. PACTS does not learn the predicate vocabulary, the PDDL domain, or operator definitions from scratch. It assumes that a symbolic domain already exists and that aligned predicate labels can be obtained, either from simulator oracles or from sparse human annotation. A plausible implication is that PACTS addresses the action–predicate coupling problem more directly than the predicate-invention problem. The paper states this limitation explicitly in its discussion of what is and is not shown (Quartey et al., 20 May 2026).

4. Planning interface and zero-shot composition

The planning interface assumes a PDDL domain $\mathbf{o}$1, a PDDL problem instance, skill-level operators with preconditions and effects over predicates, an initial predicate state, and a goal predicate condition. For a selected skill $\mathbf{o}$2, the model samples

$\mathbf{o}$3

The symbolic state update is taken from the next-step predicate beliefs:

$\mathbf{o}$4

Planning then alternates between symbolic search and receding-horizon skill execution (Quartey et al., 20 May 2026).

The execution loop is straightforward. The planner first computes a symbolic plan from the current predicate state and goal. It selects the first skill in the plan, samples a joint action–predicate rollout for that skill, executes only the first action or short prefix, observes again, thresholds the predicted next-step predicate beliefs, and repeats until the skill’s effects are satisfied or the skill terminates. If the goal is not yet achieved, planning is invoked again from the updated symbolic state (Quartey et al., 20 May 2026).

An important nuance is that $\mathbf{o}$5 is predicted before observing $\mathbf{o}$6. The method compensates by re-grounding every step through the next rollout, because the next sample is conditioned on the new observation. This yields an online monitoring loop rather than a one-shot open-loop plan (Quartey et al., 20 May 2026).

The PushBarrier experiments provide the clearest composition example. The domain requires opening a barrier and then pushing and aligning blocks into goal regions. The paper reports a replanning case in which the door is externally closed during alignment; predicate beliefs then reveal that the symbolic state no longer matches the current plan, and the planner inserts another open_door action before resuming the alignment skill. This is the paper’s canonical example of zero-shot recomposition and disturbance recovery (Quartey et al., 20 May 2026).

5. Empirical performance

The most detailed evaluation is on PushBarrier, a 2D benchmark adapted from Push-T. Across DDPM and CFM objectives and across UNet and Transformer backbones, PACTS is consistently best or near-best in reward and clearly strongest in predicate prediction. The strongest reported configuration is DDPM + UNet, where PACTS reaches Max Reward $\mathbf{o}$7, predicate F1 $\mathbf{o}$8, recall $\mathbf{o}$9, precision $H$0, and accuracy $H$1. In the same setting, the action-only + predicate-only baseline reaches Max Reward $H$2, and the shared-encoder multi-task baseline reaches $H$3 (Quartey et al., 20 May 2026).

The 3D RoboMimic results are more mixed. On Kitchen_d0 and Coffee_d0, action-only policies are already strong on reward, and PACTS is not uniformly dominant. Under DDPM + UNet, however, PACTS improves both control and predicate quality, with Max Reward $H$4, F1 $H$5, and accuracy $H$6, compared with $H$7, F1 $H$8, and accuracy $H$9 for the action-only + predicate-only baseline. In other configurations, especially DDPM + Transformer and CFM + UNet, action-only models can achieve higher reward. The paper identifies DDPM + UNet as the best overall tradeoff (Quartey et al., 20 May 2026).

A distinct evaluation targets predicate–action coherence rather than standalone accuracy. The paper asks whether predicted predicate rollouts match the symbolic outcomes actually induced by the model’s own sampled action chunks. Against constant and base-rate baselines, PACTS achieves near-perfect CrossCons and much lower CrossBrier and CrossNLL across the rollout horizon. This is presented as evidence that the generated predicate beliefs are action-conditioned symbolic futures rather than static scene priors (Quartey et al., 20 May 2026).

The real-world cube-packing demonstration is used to show compositionality beyond simulation. Monolithic demonstrations are segmented into subskills such as pack_red and pack_green, separate PACTS models are trained per subskill, and an off-the-shelf PDDL planner composes them to satisfy new requests such as packing only selected cubes. The paper presents this as real-world evidence of zero-shot composition (Quartey et al., 20 May 2026).

6. Limits, scope, and common misconceptions

PACTS does not solve the entire symbolic-skills stack. Its performance depends strongly on the chosen predicate set; if predicates are incomplete, mis-specified, or poorly aligned with task structure, planning becomes brittle and the symbolic interface loses expressivity. The method also assumes a manually defined symbolic planning domain with PDDL predicates, operators, and goals. Predicate labels are required during training, coming either from simulator oracles or from sparse human annotation plus interpolation in the real-world setting (Quartey et al., 20 May 2026).

Several broader failure modes remain outside the joint modeling contribution itself. The paper states that joint action–predicate generation does not eliminate failures due to perception noise, occlusion, state aliasing, stochasticity, or long-horizon distribution shift. The planner also reasons over a predefined finite symbolic vocabulary; open-ended semantic generalization and automatic vocabulary expansion are not demonstrated (Quartey et al., 20 May 2026).

The method is also narrower than some readers may assume. It is not a system for automatic predicate discovery, not a system for automatic operator learning, and not a general open-vocabulary symbolic world model. It shows recomposition of known skills to novel goals, online monitoring and replanning, transfer from monolithic demonstrations to skill-centric composition, and operation in both simulation and a real robot setting. It does not show automatic predicate discovery, automatic operator induction, large-scale real-world long-horizon evaluation, or guarantees under severe distribution shift (Quartey et al., 20 May 2026).

A related misconception is to equate PACTS either with ordinary multi-task learning or with classical option-learning. The paper explicitly distinguishes itself from both. Compared with multi-task learning, the novelty is not merely shared representation but joint sampling from a coupled distribution over actions and predicate beliefs. Compared with classic hierarchical RL and options, PACTS is described not as a framework centered on learned termination and value decomposition, but as a generative imitation-learned skill model with an explicit symbolic rollout (Quartey et al., 20 May 2026).

7. Position within adjacent research

PACTS sits at the intersection of predicate grounding, symbolic planning, and learned skill abstraction. One nearby line of work learns perceptual predicate detectors from weak action-centric supervision by exploiting known preconditions and postconditions of actions. That approach treats action schemas as supervision for predicate grounding, but it assumes the schemas are given and learns only the perceptual state estimator. PACTS instead treats the skill itself as a joint generator of motor behavior and predicate-belief evolution (Migimatsu et al., 2021).

Another adjacent line addresses the symbolic front-end that PACTS assumes. “Predicate Invention for Bilevel Planning” learns predicates from demonstrations by optimizing a planning-aware surrogate objective, then learns operators and samplers on top of the invented vocabulary. UniPred similarly uses LLMs to propose predicate effect distributions that supervise neural predicate learning from low-level data while learned feedback refines the symbolic hypotheses. Both are directly relevant to the problem of where a PACTS predicate vocabulary comes from, but neither is the same as PACTS’s joint action–predicate rollout model (Silver et al., 2022, Wang et al., 19 Dec 2025).

SkillWrapper is especially close in spirit. It learns, from raw RGB observations and executions of black-box skills, a predicate-based symbolic action model with typed arguments, preconditions, and effects, and then plans over those learned operators. A plausible summary is that SkillWrapper focuses on inventing plannable predicate/operator abstractions for black-box skills, whereas PACTS focuses on making each learned skill itself output an aligned symbolic forecast during execution (Yang et al., 22 Nov 2025).

Other neighboring work learns structured skill abstractions without explicit predicates. ASAP jointly learns temporally extended skills and the state/task-dependent regions in which they apply, using differentiable partitions rather than symbolic predicates. Meta-Learning Parameterized Skills learns discrete skill families with continuous latent parameters and exposes them as a learned parameterized action space. “Learning Skills from Action-Free Videos” learns a latent skill vocabulary from optical flow and plans in that learned skill space, but without explicit symbolic preconditions or effects. These systems are close to PACTS in their emphasis on reusable, compositional skills, yet they stop short of giving the planner a predicate-level state interface (Mankowitz et al., 2016, Fu et al., 2022, Fang et al., 23 Dec 2025).

The acronym should also be distinguished from several unrelated “PACT” frameworks in reactive reinforcement learning, proactive asking for task assistance, medical dialogue training, action-state communication for multi-agent systems, and controllable safety alignment; those works do not define Predicate Action Skills in the sense used here (Gavenski et al., 15 Jun 2026, He et al., 23 May 2026, Li et al., 8 Jun 2026, Huang et al., 3 Jun 2026, Si et al., 6 Feb 2026).