Atomic Action Slicing (AAS)

• Atomic Action Slicing (AAS) is a planner-aligned methodology that segments long-horizon VLA demonstrations into atomic actions with explicit types, temporal boundaries, and confidence scores.
• It employs a three-stage pipeline—discovery, schema-constrained segmentation, and validation—to ensure precise label alignment and robust temporal segmentation.
• AAS enhances robotic policy performance by bridging high-level symbolic planning with low-level control, leading to improved success rates in compositional tasks.

Atomic Action Slicing (AAS) is a planner-aligned methodology that decomposes long-horizon vision-language-action (VLA) demonstrations into short, typed segments known as atomic actions. Designed to facilitate robust policy learning and symbolic planning, AAS produces validated datasets of atomic action segments with explicit action types, temporal boundaries, and confidence scores. This approach enables compositional generalization for VLA models, bridging high-level symbolic planning and low-level control in robotic agents (Tabakov et al., 12 Dec 2025).

1. Formal Foundation and Atomic Action Definition

AAS operates on episodic data formalized as

$$\tau = (o_{1:T}, s_{1:T}, a_{1:T}, \ell, \mathcal{E}),$$

where $o_t \in \mathcal{O}$ are RGB video frames, $s_t \in \mathcal{S}$ denotes robot and object state, $a_t \in \mathcal{A}$ are low-level actions, $\ell$ is the language instruction, and $\mathcal{E}$ is the symbolic scene graph. The core output is a sequence of $K$ atomic slices,

$$\hat{\Gamma} = \big[(\hat{o}_k, t^{(k)}_s, t^{(k)}_e, c^{(k)})\big]_{k=1}^K,$$

with each slice defined by:

• $\hat{o}_k \in \Sigma$: action-type label from a fixed, typed schema $\Sigma$
• $t^{(k)}_s, t^{(k)}_e \in \{1 ... T\}$: temporal start and end frames with enforced contiguity
• $c^{(k)} \in [0,1]$: confidence score

Typical $\Sigma$ schema for LIBERO tasks includes $\mathsf{open\_drawer}$, $\mathsf{close\_drawer}$, $\mathsf{grasp}(\text{object})$, $\mathsf{release}(\text{object})$, $\mathsf{place}(\text{object, target})$, $\mathsf{move\_end\_effector}$. Each atomic action is mapped to a symbolic operator signature in STRIPS/HTN notation: $$\langle I_{\hat{o}_k}, \pi_{\hat{o}_k}, \beta_{\hat{o}_k} \rangle$$ with logical preconditions and effects.

2. Planner Alignment and Optimization Objective

The AAS objective is planner-alignment: maximizing correspondence between segmented atomic actions and the ordered symbolic plan, $P = (\bar{o}_1, ..., \bar{o}_K)$, generated by a planner (e.g., AutoGPT+P) given instruction $\ell$, scene $\mathcal{E}$, and schema $\Sigma$. The loss function is

$$\max_{\{t^{(k)}_s, t^{(k)}_e\}} L(\Gamma) = \sum_{k=1}^K \Big[\,\mathbb{I}\big(\hat{o}_k = P[k]\big) + \alpha\, \mathrm{IoU}([t^{(k)}_s, t^{(k)}_e], [t^{*(k)}_s, t^{*(k)}_e]) \Big],$$

subject to contiguity and segment duration constraints, where $[t^{*(k)}_s, t^{*(k)}_e]$ indicates an optional oracle or heuristic annotation for segment $k$, and IoU is the standard overlap metric. The optimization encourages both label agreement and tight temporal boundaries. Duration bounds $d_{\min}$ and $d_{\max}$ are class dependent.

3. Atomic Dataset Creation and Validation

AAS applies a three-stage pipeline for LIBERO demonstration decomposition:

1. Discovery: For each episode, a planner uses $(\ell, \mathcal{E}, \Sigma)$ to generate plan $P$ of cardinality $K$.
2. Schema-Constrained LLM Segmentation: Multimodal VLMs (either Gemini 2.5 Flash or Pro) are prompted with plan, keyframes, language, and scene. The VLM predicts temporally contiguous boundaries $\{t^{(k)}_s, t^{(k)}_e\}$ for each $\hat{o}_k$ in $P$, respecting plan order and schema.
3. Validation & Confidence Scoring: Predicted segments pass only if
   • cardinality matches $K$ (1_count),
   • segment labels match plan order (2_order),
   • durations fall within valid interval (3_duration).

Confidence $c^{(k)}$ for each segment is computed by blending VLM internal scores, duration slack, and segment stability under ±2-frame keyframe jitter.

Summary statistics for the GATE-VLAP atomic dataset:

Dataset	Demonstrations	Segments
LIBERO-Goal	434	758
LIBERO-Long	391	1,366
Total	825	2,124

Approximate class distribution: $\mathsf{grasp}$ 24%, $\mathsf{place}$ 18%, $\mathsf{open\_drawer}/\mathsf{close\_drawer}$ 22%, $\mathsf{release}$ 16%, $\mathsf{move\_end\_effector}$/other 20%.

4. Segmenter Models and Segmentation Metrics

AAS segmentation leverages off-the-shelf VLMs without gradient fine-tuning. The segmenter model architecture comprises:

• Vision backbone: ViT-large (patch size 14)
• Language: 2B-parameter autoregressive LLM
• Multimodal attention fusion for textual prompts and visual keyframes
• Output heads for discrete start/end frame regression per segment

Zero-shot prompt tuning is performed with multi-keyframe queries. Segmentation quality is measured via metrics:

Metric	Gemini 2.5 Flash	Gemini 2.5 Pro	Δ (Pro–Flash)
Success rate (out of 100)	74	93	+19
SeqAcc ($\mathbb{I}[\hat{o}_{1:K}=P_{1:K}]$)	0.95	0.99	+0.04
EditSim	0.96	0.99	+0.03
IoU$_\text{idx}$	0.83	0.88	+0.05
MAE$_\text{start}$ (frames)	4.3	3.1	–1.2
MAE$_\text{end}$ (frames)	4.8	3.5	–1.3
Stability@±2 frames (IoU)	0.90	0.92	+0.02

Gemini 2.5 Pro outperforms Flash, especially for difficult multi-object tasks, with notable gains in label sequence accuracy and temporal boundary precision. Both models exhibit robustness to small temporal perturbations.

5. Fine-Tuning VLA Policies with Atomic Segments

CLIP-RT+ is fine-tuned on the atomic action dataset to yield CLIP-RT+AA. Each atomic segment provides a clip $(o_{t_s:t_e}, \hat{o}, \text{context})$ carrying the segment type. The training objective combines behavior cloning (BC) loss

$$\mathcal{L}_{\mathrm{BC}} = -\sum_{t=t_s}^{t_e}\log\pi_\theta(a_t \mid o_{t_s:t},\,\ell)$$

with a cross-entropy loss for action-type prediction at each segment start:

$$\mathcal{L}_{\mathrm{type}} = -\big[\hat{o} \log p_\theta(o \mid o_{t_s:t_e})\big].$$

Fine-tuning uses 10 epochs, batch size of 16 segments, AdamW optimizer (learning rate $10^{-5}$, weight decay $10^{-4}$), and a segment-type loss weight 0.5.

Performance on LIBERO benchmarks:

Benchmark	CLIP-RT+ (Baseline)	CLIP-RT+AA	Δ (pp)
LIBERO-Goal	94.2%	95.3%	+1.1
LIBERO-Long	83.8%	88.8%	+5.0

Atomic supervision provides a measurable improvement in policy success, particularly on long-horizon compositional tasks.

6. Integration Workflow and Pipeline Structure

The AAS workflow is encapsulated as follows:

Input: Episodes {τ_i}, schema Σ, few-shot set ℱ, keyframe budget K_f Output: Validated atomic dataset 𝒟 and fine-tuned policy π_θ for each episode τ=(o_{1:T},s_{1:T},ℓ,ℰ}): P ← Planner(ℓ,ℰ,Σ) # Stage I {t_i} ← SampleKeyframes(T,K_f) # Keyframe sampling {(ŝ_k, ê_k)} ← VLM-Segment(ℓ,ℰ,Σ,P,ℱ,{t_i}) if Validate(P, ŝ, ê): compute confidences c_k add segment (o_{ŝ_k:ê_k}, ô_k=P[k], c_k) to 𝒟 else: refine prompts or drop episode Initialize π_θ ← CLIP-RT+ for epoch in 1…10: for batch in 𝒟: compute ℒ_BC + 0.5 ℒ_type θ ← θ – lr ∇_θℒ return π_θ (CLIP-RT+AA)

The pipeline processes demonstration videos into validated atomic segments by planner-guided segmentation and VLM labelling, producing datasets that directly enhance the learning of VLA policies.

7. Significance and Implications

AAS enables faithful extraction of symbolic “options” from raw demonstrations, facilitating compositional planning and robust execution in generalist robotic agents. The validated, planner-aligned atomic segments represent a data substrate that bridges high-level symbolic reasoning with low-level continuous control. Atomic action supervision has demonstrated clear empirical benefits for task completion and generalization on challenging VLA tasks (Tabakov et al., 12 Dec 2025).

A plausible implication is that AAS principles could be extended to other domains requiring tight integration of symbolic and sensorimotor representations. The released GATE-VLAP dataset and pipeline enable reproducibility and further research in atomic skill composition, action segmentation, and generalist robot policy optimization.