Self-Evolved Imitation Learning (SEIL)

• Self-Evolved Imitation Learning (SEIL) is a few-shot imitation framework that iteratively refines weak policies using simulator rollouts to expand demonstration data.
• It employs dual-level augmentation by combining EMA-based model collaboration with environment perturbations to enhance diversity and mitigate data redundancy.
• The method integrates a lightweight trajectory selector within an iterative train–rollout–select loop, achieving state-of-the-art success rates on the LIBERO benchmark.

Self-Evolved Imitation Learning (SEIL) is a few-shot imitation learning framework for multi-task robot manipulation in which a weak initial policy is iteratively improved through simulator interaction rather than additional expert supervision. The method begins with $N_u \in \{1,2,4,8\}$ demonstrations per task, deploys the resulting policy in a simulator, records successful rollouts as new demonstrations, filters them with a lightweight selector, and retrains on the expanded dataset. Its central design combines model-level augmentation through an Exponential Moving Average (EMA) collaborator with environment-level augmentation through slight perturbations of initial object positions, and it is evaluated on the LIBERO benchmark for few-shot imitation learning (Ye et al., 23 Sep 2025).

1. Problem setting and motivation

SEIL was introduced to address few-shot imitation learning for multi-task robot manipulation, where standard imitation methods degrade sharply when only a very small number of expert demonstrations are available. The method is motivated by two linked observations: small datasets are not diverse enough, and weak few-shot policies typically produce poor rollouts when deployed. In the reported example, a train-from-scratch Diffusion Policy achieves only 0.8% success in the 1-shot setting on LIBERO-Long, compared with 50.5% in full-data training. The method therefore targets not merely low-data learning, but the problem of improving a weak few-shot policy using simulator interaction and no additional expert supervision (Ye et al., 23 Sep 2025).

The formulation isolates two difficulties in self-generated data. First, successful rollouts gathered from a weak policy may still be too homogeneous, so merely adding them back to the dataset does not necessarily enlarge the support of the training distribution in a useful way. Second, not every successful rollout is informative: some are redundant relative to the original expert data, while others may introduce inconsistency. SEIL is built around the claim that simulator interaction becomes useful only if diversity and selection are handled explicitly.

2. Iterative self-evolution procedure

The SEIL pipeline is an iterative imitation loop of the form

$$\pi_0 \rightarrow \pi_1 \rightarrow \pi_2 \rightarrow \cdots \rightarrow \pi_k \rightarrow \pi_f,$$

where $\pi_0$ is the initial few-shot policy and $\pi_f$ is the final converged policy. The initial model is trained on limited expert data and, in the reported implementation, uses BAKU as the base architecture. BAKU is described as consisting of sensory encoders, an observation trunk, and an action head (Ye et al., 23 Sep 2025).

After initialization, the policy is deployed in a simulator $S$. The simulator provides only success/failure feedback; successful rollouts are stored as Recorded Demonstrations (RDs). These trajectories are not used directly. Instead, SEIL applies a selector to score the recorded pool and retain only the most informative samples, which are then merged with the original expert demonstrations for the next round of imitation learning. The cycle is summarized in the paper as train $\rightarrow$ record $\rightarrow$ select $\rightarrow$ train $\rightarrow \cdots$, and iteration continues until performance saturates (Ye et al., 23 Sep 2025).

A notable property of this workflow is that it remains within an imitation-learning regime even though it relies on environment interaction. The simulator contributes successful trajectories and binary success signals, but the refinement step is still retraining on demonstrations rather than optimizing a shaped reward. This suggests a specific middle ground between pure behavior cloning and reward-engineered reinforcement learning.

3. Dual-level augmentation and trajectory selection

A central technical element of SEIL is dual-level augmentation, introduced to diversify the recorded demonstration pool. At the model level, SEIL maintains an EMA version of the primary policy,

$$\theta^{EMA}_t = \tau \cdot \theta^{EMA}_{t-1} + (1-\tau)\cdot \theta_t,$$

with $$\pi_0 \rightarrow \pi_1 \rightarrow \pi_2 \rightarrow \cdots \rightarrow \pi_k \rightarrow \pi_f,$$0 in the reported experiments. The EMA model is intended to provide a stable collaborator whose behavior differs from the online model without requiring a separately trained policy (Ye et al., 23 Sep 2025).

At the environment level, SEIL perturbs initial object positions before rollout. The description states that small offsets are applied to the initial $$\pi_0 \rightarrow \pi_1 \rightarrow \pi_2 \rightarrow \cdots \rightarrow \pi_k \rightarrow \pi_f,$$1 and $$\pi_0 \rightarrow \pi_1 \rightarrow \pi_2 \rightarrow \cdots \rightarrow \pi_k \rightarrow \pi_f,$$2 coordinates while preserving vertical consistency. The stated purpose is to expose the policy to varied object placements and slightly different task configurations. The paper explicitly argues that both augmentation levels are necessary. With only environment augmentation, behavioral diversity remains limited because all rollouts come from the same policy. With only EMA augmentation, the same observation can receive inconsistent action labels from the base and EMA policies, expressed in the analysis as

$$\pi_0 \rightarrow \pi_1 \rightarrow \pi_2 \rightarrow \cdots \rightarrow \pi_k \rightarrow \pi_f,$$3

which is harmful for imitation learning. In the authors’ interpretation, environment augmentation resolves this inconsistency by separating the states on which the two policies are queried (Ye et al., 23 Sep 2025).

The second mechanism is a lightweight trajectory selector. Rather than using full video, the selector consumes only the first-frame image and the action sequence. Its reported architecture uses a ResNet-18 image encoder $$\pi_0 \rightarrow \pi_1 \rightarrow \pi_2 \rightarrow \cdots \rightarrow \pi_k \rightarrow \pi_f,$$4, an MLP action encoder $$\pi_0 \rightarrow \pi_1 \rightarrow \pi_2 \rightarrow \cdots \rightarrow \pi_k \rightarrow \pi_f,$$5, concatenation of the resulting embeddings, an LSTM, and a final MLP classifier. The encoding is written as

$$\pi_0 \rightarrow \pi_1 \rightarrow \pi_2 \rightarrow \cdots \rightarrow \pi_k \rightarrow \pi_f,$$6

The selector is trained first on few-shot expert data, then frozen, and applied to recorded trajectories. Its selection rule is intentionally nonstandard: low-confidence trajectories are preferred, on the grounds that they are most distinct from the expert set and therefore more complementary (Ye et al., 23 Sep 2025).

The accompanying ablations support this design. In the selector study, Ascending selection, defined as low-confidence first, yields 72.7%, compared with 70.8% for uniform sampling, 72.0% for descending sampling, and 72.2% for mixed sampling. The representation study also reports higher selector classification accuracy when both image and sequence are used rather than sequence alone: 57.0% vs 53.0% in 1-shot, 61.0% vs 56.0% in 2-shot, 63.5% vs 54.5% in 4-shot, and 85.0% vs 74.5% in 8-shot (Ye et al., 23 Sep 2025).

4. Experimental evaluation on LIBERO

SEIL is evaluated on the LIBERO benchmark, which is divided into LIBERO-Spatial, LIBERO-Object, LIBERO-Goal, and LIBERO-Long. The paper characterizes these subsets respectively as emphasizing spatial reasoning, object recognition and manipulation, procedural generalization with varying goals, and long-horizon control. Experiments are conducted in 1-shot, 2-shot, 4-shot, and 8-shot regimes. The reported baselines are Diffusion Policy (DP), Action-Chunking Transformer (ACT), RT-1, and BAKU, with all methods trained from scratch for fairness (Ye et al., 23 Sep 2025).

Implementation details are reported explicitly. SEIL is built on top of BAKU. In each round, the base model and the EMA model each perform 25 rollouts in the simulator, and the selector chooses 15 trajectories to add to the evolving dataset. The selector uses a pretrained ResNet-18, two LSTM blocks, hidden dimension 256, the Adam optimizer, learning rate 0.001, and 80 epochs of training. Evaluation uses 50 rollouts per task, and the metric is success rate (SR) (Ye et al., 23 Sep 2025).

Across LIBERO subsets, the reported average success rates for SEIL are 49.7%, 68.1%, 80.2%, and 88.7% in the 1-, 2-, 4-, and 8-shot settings. The corresponding numbers are 35.5%, 45.4%, 64.3%, and 82.2% for BAKU; 17.2%, 25.2%, 30.7%, and 43.1% for DP; 7.3%, 18.8%, 33.8%, and 45.4% for ACT; and 0.0%, 3.2%, 3.8%, and 4.6% for RT-1. The paper reports these results as a new state of the art in few-shot imitation learning scenarios on LIBERO (Ye et al., 23 Sep 2025).

The benchmark-specific results are similarly strong. On LIBERO-Long, BAKU obtains 4.6%, 16.7%, 40.8%, and 63.4%, whereas SEIL reaches 14.6%, 40.4%, 54.3%, and 75.4% across 1-, 2-, 4-, and 8-shot settings. The paper highlights the 217.3% relative improvement at 1-shot, comparing 14.6% with 4.6%. On LIBERO-Spatial, BAKU records 42.2%, 51.8%, 75.2%, and 83.6%, while SEIL reaches 53.2%, 71.4%, 87.2%, and 87.7%; notably, SEIL with 4-shot exceeds BAKU with 8-shot. On LIBERO-Object, the respective BAKU and SEIL results are 65.0% vs 82.0%, 65.1% vs 81.8%, 69.8% vs 95.2%, and 96.8% vs 99.2%. On LIBERO-Goal, the reported comparison is 30.3% vs 49.2%, 48.0% vs 79.0%, 71.6% vs 84.2%, and 84.8% vs 92.8% (Ye et al., 23 Sep 2025).

The ablations clarify where the gains arise. In the 8-shot LIBERO-Long study, the baseline is 63.4%, and the configuration with evolution, model-level augmentation, environment-level augmentation, and the selector reaches 75.4% in the reported table. The paper further reports that 20 rollouts per round gives the best long-term behavior in the rollout-number ablation, reaching 74.2% at the fifth round. This is presented as evidence that SEIL works best when self-evolution is gradual rather than overly aggressive (Ye et al., 23 Sep 2025).

5. Relation to self-imitation learning, self-generated demonstrations, and acronym usage

SEIL belongs to a broader line of work in which agents improve by reusing their own trajectories as supervision, but it differs from earlier self-imitation formulations in both objective and domain. In reinforcement learning, "Learning Self-Imitating Diverse Policies" (Gangwani et al., 2018) treats high-return trajectories as a replay-based target distribution and derives a dense reward from Jensen-Shannon divergence, later adding Stein variational policy gradient with a JS kernel to encourage policy diversity. "Self-Imitation Learning from Demonstrations" (Pshikhachev et al., 2022) instead initializes a self-imitation replay buffer with demonstrations and relies on a positive-advantage filter so that the influence of demonstrations decreases automatically as the agent improves. Both are primarily sparse-reward RL methods rather than few-shot imitation frameworks for multi-task manipulation.

In robotics, "Self-Imitation Learning by Planning" (Luo et al., 2021) is closer in spirit to SEIL. SILP converts the robot’s own visited states into graph nodes, plans over them with A*, and relabels the resulting path as demonstrations for behavior-cloning-augmented off-policy RL. "MILES: Making Imitation Learning Easy with Self-Supervision" (Papagiannis et al., 2024) addresses an even more restrictive regime—one demonstration and one environment reset—by autonomously collecting augmentation trajectories that return to, and then follow, the demonstrated path. Relative to these methods, SEIL is distinguished by its explicit use of simulator rollouts as new demonstrations, its dual-level augmentation, and its selector that favors complementary low-confidence successful trajectories.

The acronym SEIL is itself ambiguous in the literature. "SEIL: Simulation-augmented Equivariant Imitation Learning" (Jia et al., 2022) is a distinct method that combines Transition Simulation with an $$\pi_0 \rightarrow \pi_1 \rightarrow \pi_2 \rightarrow \cdots \rightarrow \pi_k \rightarrow \pi_f,$$7-equivariant policy network for robotic manipulation from ten or fewer demonstrations. That method addresses sample efficiency through simulated nearby transitions and symmetry-aware function approximation. By contrast, Self-Evolved Imitation Learning uses iterative simulator rollouts, EMA collaboration, environment perturbation, and demonstration selection. The shared acronym should therefore not be taken to imply methodological continuity.

6. Limitations, implications, and interpretive issues

The reported limitations are direct. First, SEIL depends on a simulator to generate additional demonstrations. Second, it incurs a multi-stage training cost, because the method is iterative rather than one-shot. These constraints delimit its applicability: the framework is most natural when simulation is available and repeated train-rollout-select cycles are acceptable (Ye et al., 23 Sep 2025).

The paper also argues against a common simplification of the method. SEIL is not described as merely “collect successful rollouts and retrain.” Its effectiveness depends on three additional design decisions: dual-level augmentation to diversify the recorded pool, a selector to filter informative trajectories, and gradual evolution across rounds. The ablation results support this interpretation, since evolving without good augmentation yields only modest gains, EMA alone can be harmful because of inconsistent labels, and the selector adds a further improvement once both augmentation levels are enabled (Ye et al., 23 Sep 2025).

The broader implication drawn in the work is that large expert datasets are not strictly necessary for strong few-shot imitation performance when simulator access is available. The paper states that a weak imitation policy can bootstrap itself using simulator interactions, that carefully curated self-generated demonstrations can close much of the gap, and that imitation learning can become more data-efficient without relying on reward design. Within that framing, SEIL is best understood as a simulator-assisted demonstration expansion procedure: it retains the supervision structure of imitation learning, but relocates part of the data-generation burden from human experts to iterative self-evolution in simulation (Ye et al., 23 Sep 2025).