---
title: 'CycleManip: Imitation Learning for Cyclic Tasks'
url: https://www.emergentmind.com/topics/cyclemanip-framework
type: topic
---

# CycleManip: Imitation Learning for Cyclic Tasks

CycleManip is a framework for end-to-end imitation learning that addresses cycle-based task manipulation—tasks where a robot must repeat a given motion pattern exactly $N$ times and stop within an expected terminal time. Core contributions include a cost-aware historical perception strategy and multi-task learning to enhance historical understanding, a benchmark suite for cyclic manipulation tasks, and an automated evaluation protocol, all supporting diverse robotic platforms in simulation and real-world settings [2512.01022].

## 1. Problem Definition and Formalization

Cycle-based manipulation tasks require the agent to execute a specified action pattern (a "cycle") exactly $N$ times within a deadline $T_{\max}$. Each demonstration trajectory comprises a tuple $\left\{\mathtt{lan}, (o_1, a_1), (o_2, a_2), \dots, (o_T,a_T)\right\}$, where $\mathtt{lan} \in \mathcal{L}$ is a natural language instruction (e.g., "shake bottle three times"), $o_t \in \mathcal{O}$ is the observation at time $t$, and $a_t \in \mathcal{A}$ is the action.

The policy $\pi$ predicts:
$$
a_t = \pi\left(\mathtt{lan}, \{o_i\}_{i=1}^t\right).
$$

Let $C(t)$ be the cumulative cycles completed at $t$, determined by task-specific detectors. An execution is successful if:
$$
C(T) = N \quad \text{and} \quad T \leq T_{\max},
$$
where $T$ is the episode length.

The objective is to maximize the probability that actions both accomplish exactly $N$ cycles and terminate before the deadline. This setting exposes the failure of Markovian-short-horizon policies in distinguishing visually identical cycles, motivating CycleManip’s history-centric design.

## 2. Architecture and Key Methodological Components

CycleManip is an end-to-end imitation pipeline on a diffusion policy backbone, augmented to handle non-Markovian cyclic dependencies. The framework consists of the following key modules (§3):

### 2.1 End-to-End Imitation with Multimodal Encoding

- Language ($\mathtt{lan}$) is encoded using a frozen CLIP text encoder ($f_{\mathtt{lan}}$).
- High-overhead sensory data (RGB frames or point clouds) are encoded ($f_h$) via CNN or point cloud encoders.
- Low-overhead proprioceptive signals (pose differences, denoted $o^l$) are encoded by a Transformer into $f_l$.
- Fused features ($f_{lh}$) are formed by MLP-merging $f_h$ and $f_l$, concatenated with $f_{\mathtt{lan}}$ to condition the diffusion policy for action prediction ($a_t$).

Loss function (Eq. 7):
$$
\mathcal{L} = \alpha\,\mathrm{MSE}(a_t, a_t^*) + \beta\,\mathrm{CE}(y_t, y_t^*),
$$
where $a_t^*$ is the expert action, $y_t^*$ is the discretized cycle stage bin, and $\alpha=1, \beta=0.1$ are hyperparameters.

### 2.2 Effective Historical Perception: Cost-Aware Sampling (§3.2)

- **Low-overhead features ($o^l$):** All historical pose-deltas are encoded at each time step ($\mathcal{H}_l = \{o_i^l\}_{i=1}^t$), ensuring rich, long-horizon proprioception context.
- **High-overhead features ($o^h$):** Sampling is performed under fixed budget $K_{\rm high}$. Two heuristics are employed:
  - *Right-side binary*: $K_{\rm high}/2$ evenly spread frames via interval bisection.
  - *Exponential*: $K_{\rm high}/2$ exponentially recent frames, i.e., $t-2^k$ (clipped at $\geq 0$), for $k=0,\ldots,K_{\rm high}/2-1$.
- This split enables the policy to retain extended context without prohibitive computational cost.

### 2.3 Effective Historical Understanding: Multi-task Cycle-Stage Prediction (§3.3)

- Cycle progress $b_t = t/T_{\max}$, discretized into ten bins $y_t \in \{1, \ldots, 10\}$, is predicted from fused features via an auxiliary MLP head.
- Training uses a joint MSE (on action) and cross-entropy (on cycle-stage) loss, promoting features encoding both position-in-cycle and current action.
- This structure facilitates the network's ability to distinguish "which repetition" of the cycle is underway, improving termination accuracy.

### 2.4 Training Loop (High-level)

For each demonstration trajectory, at each step $t$:
1. Sample $H_h$, $H_l$ via cost-aware strategy.
2. Encode features as above and predict $a_t$ (action) and $\hat y_t$ (cycle-stage).
3. Compute total loss and backpropagate.

## 3. Benchmark Environments and Evaluation Protocol

CycleManip includes a dedicated cyclical manipulation benchmark (§4):

### 3.1 Simulated Environments

Eight tasks are built on RoboTwin 2.0:

| Task                  | Description             | Cycles Supported |
|-----------------------|------------------------|------------------|
| Block hammering       | Contact manipulation   | 1–8              |
| Bottle shaking        | Non-contact, cyclic    | 1–8              |
| Roller rolling        | Rotational cycles      | 1–8              |
| Carrot cutting        | Sequential slicing     | 1–8              |
| Dual-knife chopping   | Bimanual, cyclic       | 1–8              |
| Egg beating           | Stirring/rotation      | 1–8              |
| Chemical mixing       | Agitated mixing        | 1–8              |
| Morse tapping         | Rhythmic task          | 1–8              |

Each has 200 expert demonstrations annotated with target cycles $N$, running count $C(t)$, and object pose.

### 3.2 Evaluation Metrics

Task-specific detectors count cycles:
- Contact tasks: State-machine detects collision events.
- Non-contact: Peak-detection on pose trajectories.

Metrics:
- **Success Rate (Suc):** Fraction of rollouts with $C(T)=N$.
- **Cycle Count Deviation (Cyc):** Mean $|C(T)-N|$.

Simulated tasks use 100 evaluation trials; real-world tasks use 16.

## 4. Experimental Results

### 4.1 Simulation (§5.3, Tab. 1)

- Compared to state-of-the-art imitation baselines (DP, DP3, RDT, $\pi_0$), CycleManip achieves 86–97% success rates versus $<$35% for alternatives.
- Cycle count deviations are $<$0.81, compared to 1.5–8.3 for baselines.
- These results confirm that short-horizon policies lose track of cycle progress due to partial observability (§5, Fig. 2a).

### 4.2 Real-World Performance (§5.3, Tab. 2)

- On block hammering, bottle shaking, drum beating, tire pumping, knife cutting, and table cleaning across AgileX Piper grippers, BrainCO Revo2 hands, and Unitree G1 humanoid:
  - CycleManip: Success 50–100%, Cyc. deviation 0–1.5.
  - DP3 baseline: Success 0–37.5%, Cyc. deviation 0.9–3.8.
- Ablations show both cost-aware sampling and multi-task components contribute substantially to performance.

### 4.3 Generalization (§5.4, Tab. 3)

- On four standard (non-cyclic) tasks—place-cans, handover, bottle-picking, stamp-seal—CycleManip outperforms all baselines (e.g., 91% vs 48% on place-cans).
- This demonstrates that the history-aware feature architecture confers advantages beyond cyclic domains.

### 4.4 Plug-and-play with VLA Models (§5.6, Tab. 4)

- Integrating only the low-overhead history Transformer from CycleManip into $\pi_0$ [19] (a Vision-Language-Action model) increases cyclic task performance from 1–27% to 41–72%.
- This modularity indicates retrofitting is possible with minimal architectural changes.

### 4.5 Efficiency (§5.7, Tab. 5)

- Overhead for carrot-cutting is modest: $+0.03$ s/training step (0.073→0.102 s) and $+0.006$ GB GPU, for a success increase from 38% to 86%.

## 5. Principles and Insights

- **Non-Markovianity:** Cyclic imitation tasks undermine standard Markovian/horizon-limited policies because identical observations at each cycle prevent accurate progress tracking (§5, Fig. 2a).
- **Cost-aware history:** Combining dense low-overhead sampling with sparse, informative high-overhead sampling allows efficient, rich temporal context (§3.2).
- **Cycle-stage prediction:** The cycle-stage auxiliary loss enforces learning of "repetition index," improving both performance and the ability to stop at exactly $N$ cycles (§3.3, Fig. 2b).
- **Versatility:** CycleManip generalizes to both cyclic and standard manipulation, is compatible with VLA architectures, and supports diverse hardware configurations (§5.6).

## 6. Limitations and Future Directions

- The provided benchmark supports up to 8 cycles; expanding to longer, variable-length, or hierarchical cycles remains open (§6).
- Evaluation presently utilizes ground-truth pose or clean collision signals; extending evaluation to noisy, fully vision-based scenarios is an area for development.
- Reinforcement learning could be incorporated to adaptively modulate movement speed or force for further performance enhancement in dynamic settings.
- *A plausible implication is* that CycleManip’s modularity and low computational overhead make it suitable as a foundation for further research into long-horizon cyclic control and robotic skill composition.

## 7. Summary Table of CycleManip Components

| Module                              | Purpose              | Distinctive Mechanism               |
|--------------------------------------|----------------------|-------------------------------------|
| Cost-aware sampling                  | History perception   | Dense low-overhead, sparse high-overhead |
| Cycle-stage auxiliary head           | History understanding| Multi-task CE loss, normalized cycle index|
| Diffusion-policy backbone            | End-to-end imitation | Multimodal fusion, language grounding    |

CycleManip constitutes a lightweight and effective approach to cyclic task imitation in robotics, offering precision history-awareness, adaptability to heterogeneous platforms, and practical utility across simulation and real-world deployments [2512.01022].

Source: https://www.emergentmind.com/topics/cyclemanip-framework