CycleManip: Imitation Learning for Cyclic Tasks

• The paper introduces CycleManip, which uses a cost-aware historical perception strategy to accurately execute cycle-based tasks with exact repetitions within a deadline.
• CycleManip fuses multimodal encoding of language, visual, and proprioceptive inputs with a diffusion-policy backbone and auxiliary cycle-stage prediction for robust action recognition.
• Experimental results in simulation and real-world settings demonstrate significant gains over baselines, confirming its efficiency, versatility, and improved cycle execution accuracy.

CycleManip is a framework for end-to-end imitation learning that addresses cycle-based task manipulation—tasks where a robot must repeat a given motion pattern exactly $N$ times and stop within an expected terminal time. Core contributions include a cost-aware historical perception strategy and multi-task learning to enhance historical understanding, a benchmark suite for cyclic manipulation tasks, and an automated evaluation protocol, all supporting diverse robotic platforms in simulation and real-world settings (Wei et al., 30 Nov 2025).

1. Problem Definition and Formalization

Cycle-based manipulation tasks require the agent to execute a specified action pattern (a "cycle") exactly $N$ times within a deadline $T_{\max}$. Each demonstration trajectory comprises a tuple $$\left\{\mathtt{lan}, (o_1, a_1), (o_2, a_2), \dots, (o_T,a_T)\right\}$$, where $\mathtt{lan} \in \mathcal{L}$ is a natural language instruction (e.g., "shake bottle three times"), $o_t \in \mathcal{O}$ is the observation at time $t$, and $a_t \in \mathcal{A}$ is the action.

The policy $\pi$ predicts:

$$a_t = \pi\left(\mathtt{lan}, \{o_i\}_{i=1}^t\right).$$

Let $N$0 be the cumulative cycles completed at $N$1, determined by task-specific detectors. An execution is successful if:

$N$2

where $N$3 is the episode length.

The objective is to maximize the probability that actions both accomplish exactly $N$4 cycles and terminate before the deadline. This setting exposes the failure of Markovian-short-horizon policies in distinguishing visually identical cycles, motivating CycleManip’s history-centric design.

2. Architecture and Key Methodological Components

CycleManip is an end-to-end imitation pipeline on a diffusion policy backbone, augmented to handle non-Markovian cyclic dependencies. The framework consists of the following key modules (§3):

2.1 End-to-End Imitation with Multimodal Encoding

• Language ($N$5) is encoded using a frozen CLIP text encoder ($N$6).
• High-overhead sensory data (RGB frames or point clouds) are encoded ($N$7) via CNN or point cloud encoders.
• Low-overhead proprioceptive signals (pose differences, denoted $N$8) are encoded by a Transformer into $N$9.
• Fused features ($T_{\max}$0) are formed by MLP-merging $T_{\max}$1 and $T_{\max}$2, concatenated with $T_{\max}$3 to condition the diffusion policy for action prediction ($T_{\max}$4).

Loss function (Eq. 7):

$T_{\max}$5

where $T_{\max}$6 is the expert action, $T_{\max}$7 is the discretized cycle stage bin, and $T_{\max}$8 are hyperparameters.

2.2 Effective Historical Perception: Cost-Aware Sampling (§3.2)

• Low-overhead features ($T_{\max}$9): All historical pose-deltas are encoded at each time step ($$\left\{\mathtt{lan}, (o_1, a_1), (o_2, a_2), \dots, (o_T,a_T)\right\}$$0), ensuring rich, long-horizon proprioception context.
• High-overhead features ($$\left\{\mathtt{lan}, (o_1, a_1), (o_2, a_2), \dots, (o_T,a_T)\right\}$$1): Sampling is performed under fixed budget $$\left\{\mathtt{lan}, (o_1, a_1), (o_2, a_2), \dots, (o_T,a_T)\right\}$$2. Two heuristics are employed:
  • Right-side binary: $$\left\{\mathtt{lan}, (o_1, a_1), (o_2, a_2), \dots, (o_T,a_T)\right\}$$3 evenly spread frames via interval bisection.
  • Exponential: $$\left\{\mathtt{lan}, (o_1, a_1), (o_2, a_2), \dots, (o_T,a_T)\right\}$$4 exponentially recent frames, i.e., $$\left\{\mathtt{lan}, (o_1, a_1), (o_2, a_2), \dots, (o_T,a_T)\right\}$$5 (clipped at $$\left\{\mathtt{lan}, (o_1, a_1), (o_2, a_2), \dots, (o_T,a_T)\right\}$$6), for $$\left\{\mathtt{lan}, (o_1, a_1), (o_2, a_2), \dots, (o_T,a_T)\right\}$$7.
• This split enables the policy to retain extended context without prohibitive computational cost.

2.3 Effective Historical Understanding: Multi-task Cycle-Stage Prediction (§3.3)

• Cycle progress $$\left\{\mathtt{lan}, (o_1, a_1), (o_2, a_2), \dots, (o_T,a_T)\right\}$$8, discretized into ten bins $$\left\{\mathtt{lan}, (o_1, a_1), (o_2, a_2), \dots, (o_T,a_T)\right\}$$9, is predicted from fused features via an auxiliary MLP head.
• Training uses a joint MSE (on action) and cross-entropy (on cycle-stage) loss, promoting features encoding both position-in-cycle and current action.
• This structure facilitates the network's ability to distinguish "which repetition" of the cycle is underway, improving termination accuracy.

2.4 Training Loop (High-level)

For each demonstration trajectory, at each step $\mathtt{lan} \in \mathcal{L}$0:

1. Sample $\mathtt{lan} \in \mathcal{L}$1, $\mathtt{lan} \in \mathcal{L}$2 via cost-aware strategy.
2. Encode features as above and predict $\mathtt{lan} \in \mathcal{L}$3 (action) and $\mathtt{lan} \in \mathcal{L}$4 (cycle-stage).
3. Compute total loss and backpropagate.

3. Benchmark Environments and Evaluation Protocol

CycleManip includes a dedicated cyclical manipulation benchmark (§4):

3.1 Simulated Environments

Eight tasks are built on RoboTwin 2.0:

Task	Description	Cycles Supported
Block hammering	Contact manipulation	1–8
Bottle shaking	Non-contact, cyclic	1–8
Roller rolling	Rotational cycles	1–8
Carrot cutting	Sequential slicing	1–8
Dual-knife chopping	Bimanual, cyclic	1–8
Egg beating	Stirring/rotation	1–8
Chemical mixing	Agitated mixing	1–8
Morse tapping	Rhythmic task	1–8

Each has 200 expert demonstrations annotated with target cycles $\mathtt{lan} \in \mathcal{L}$5, running count $\mathtt{lan} \in \mathcal{L}$6, and object pose.

3.2 Evaluation Metrics

Task-specific detectors count cycles:

• Contact tasks: State-machine detects collision events.
• Non-contact: Peak-detection on pose trajectories.

Metrics:

• Success Rate (Suc): Fraction of rollouts with $\mathtt{lan} \in \mathcal{L}$7.
• Cycle Count Deviation (Cyc): Mean $\mathtt{lan} \in \mathcal{L}$8.

Simulated tasks use 100 evaluation trials; real-world tasks use 16.

4. Experimental Results

4.1 Simulation (§5.3, Tab. 1)

• Compared to state-of-the-art imitation baselines (DP, DP3, RDT, $\mathtt{lan} \in \mathcal{L}$9), CycleManip achieves 86–97% success rates versus $o_t \in \mathcal{O}$035% for alternatives.
• Cycle count deviations are $o_t \in \mathcal{O}$10.81, compared to 1.5–8.3 for baselines.
• These results confirm that short-horizon policies lose track of cycle progress due to partial observability (§5, Fig. 2a).

4.2 Real-World Performance (§5.3, Tab. 2)

• On block hammering, bottle shaking, drum beating, tire pumping, knife cutting, and table cleaning across AgileX Piper grippers, BrainCO Revo2 hands, and Unitree G1 humanoid:
  • CycleManip: Success 50–100%, Cyc. deviation 0–1.5.
  • DP3 baseline: Success 0–37.5%, Cyc. deviation 0.9–3.8.
• Ablations show both cost-aware sampling and multi-task components contribute substantially to performance.

4.3 Generalization (§5.4, Tab. 3)

• On four standard (non-cyclic) tasks—place-cans, handover, bottle-picking, stamp-seal—CycleManip outperforms all baselines (e.g., 91% vs 48% on place-cans).
• This demonstrates that the history-aware feature architecture confers advantages beyond cyclic domains.

4.4 Plug-and-play with VLA Models (§5.6, Tab. 4)

• Integrating only the low-overhead history Transformer from CycleManip into $o_t \in \mathcal{O}$2 19 increases cyclic task performance from 1–27% to 41–72%.
• This modularity indicates retrofitting is possible with minimal architectural changes.

4.5 Efficiency (§5.7, Tab. 5)

• Overhead for carrot-cutting is modest: $o_t \in \mathcal{O}$3 s/training step (0.073→0.102 s) and $o_t \in \mathcal{O}$4 GB GPU, for a success increase from 38% to 86%.

5. Principles and Insights

• Non-Markovianity: Cyclic imitation tasks undermine standard Markovian/horizon-limited policies because identical observations at each cycle prevent accurate progress tracking (§5, Fig. 2a).
• Cost-aware history: Combining dense low-overhead sampling with sparse, informative high-overhead sampling allows efficient, rich temporal context (§3.2).
• Cycle-stage prediction: The cycle-stage auxiliary loss enforces learning of "repetition index," improving both performance and the ability to stop at exactly $o_t \in \mathcal{O}$5 cycles (§3.3, Fig. 2b).
• Versatility: CycleManip generalizes to both cyclic and standard manipulation, is compatible with VLA architectures, and supports diverse hardware configurations (§5.6).

6. Limitations and Future Directions

• The provided benchmark supports up to 8 cycles; expanding to longer, variable-length, or hierarchical cycles remains open (§6).
• Evaluation presently utilizes ground-truth pose or clean collision signals; extending evaluation to noisy, fully vision-based scenarios is an area for development.
• Reinforcement learning could be incorporated to adaptively modulate movement speed or force for further performance enhancement in dynamic settings.
• A plausible implication is that CycleManip’s modularity and low computational overhead make it suitable as a foundation for further research into long-horizon cyclic control and robotic skill composition.

7. Summary Table of CycleManip Components

Module	Purpose	Distinctive Mechanism
Cost-aware sampling	History perception	Dense low-overhead, sparse high-overhead
Cycle-stage auxiliary head	History understanding	Multi-task CE loss, normalized cycle index
Diffusion-policy backbone	End-to-end imitation	Multimodal fusion, language grounding

CycleManip constitutes a lightweight and effective approach to cyclic task imitation in robotics, offering precision history-awareness, adaptability to heterogeneous platforms, and practical utility across simulation and real-world deployments (Wei et al., 30 Nov 2025).