---
title: Self-Evolved Imitation Learning (SEIL)
url: https://www.emergentmind.com/topics/self-evolved-imitation-learning-seil
type: topic
---

# Self-Evolved Imitation Learning (SEIL)

Self-Evolved Imitation Learning (SEIL) is a few-shot imitation learning framework for multi-task robot manipulation in which a weak initial policy is iteratively improved through simulator interaction rather than additional expert supervision. The method begins with \(N_u \in \{1,2,4,8\}\) demonstrations per task, deploys the resulting policy in a simulator, records successful rollouts as new demonstrations, filters them with a lightweight selector, and retrains on the expanded dataset. Its central design combines model-level augmentation through an Exponential Moving Average (EMA) collaborator with environment-level augmentation through slight perturbations of initial object positions, and it is evaluated on the LIBERO benchmark for few-shot imitation learning [2509.19460].

## 1. Problem setting and motivation

SEIL was introduced to address few-shot imitation learning for multi-task robot manipulation, where standard imitation methods degrade sharply when only a very small number of expert demonstrations are available. The method is motivated by two linked observations: small datasets are not diverse enough, and weak few-shot policies typically produce poor rollouts when deployed. In the reported example, a train-from-scratch Diffusion Policy achieves only **0.8% success** in the **1-shot** setting on **LIBERO-Long**, compared with **50.5%** in full-data training. The method therefore targets not merely low-data learning, but the problem of improving a weak few-shot policy using simulator interaction and no additional expert supervision [2509.19460].

The formulation isolates two difficulties in self-generated data. First, successful rollouts gathered from a weak policy may still be too homogeneous, so merely adding them back to the dataset does not necessarily enlarge the support of the training distribution in a useful way. Second, not every successful rollout is informative: some are redundant relative to the original expert data, while others may introduce inconsistency. SEIL is built around the claim that simulator interaction becomes useful only if diversity and selection are handled explicitly.

## 2. Iterative self-evolution procedure

The SEIL pipeline is an iterative imitation loop of the form
\[
\pi_0 \rightarrow \pi_1 \rightarrow \pi_2 \rightarrow \cdots \rightarrow \pi_k \rightarrow \pi_f,
\]
where \(\pi_0\) is the initial few-shot policy and \(\pi_f\) is the final converged policy. The initial model is trained on limited expert data and, in the reported implementation, uses **BAKU** as the base architecture. BAKU is described as consisting of sensory encoders, an observation trunk, and an action head [2509.19460].

After initialization, the policy is deployed in a simulator \(S\). The simulator provides only **success/failure feedback**; successful rollouts are stored as **Recorded Demonstrations (RDs)**. These trajectories are not used directly. Instead, SEIL applies a selector to score the recorded pool and retain only the most informative samples, which are then merged with the original expert demonstrations for the next round of imitation learning. The cycle is summarized in the paper as **train \(\rightarrow\) record \(\rightarrow\) select \(\rightarrow\) train \(\rightarrow \cdots\)**, and iteration continues until performance saturates [2509.19460].

A notable property of this workflow is that it remains within an imitation-learning regime even though it relies on environment interaction. The simulator contributes successful trajectories and binary success signals, but the refinement step is still retraining on demonstrations rather than optimizing a shaped reward. This suggests a specific middle ground between pure behavior cloning and reward-engineered reinforcement learning.

## 3. Dual-level augmentation and trajectory selection

A central technical element of SEIL is **dual-level augmentation**, introduced to diversify the recorded demonstration pool. At the model level, SEIL maintains an EMA version of the primary policy,
\[
\theta^{EMA}_t = \tau \cdot \theta^{EMA}_{t-1} + (1-\tau)\cdot \theta_t,
\]
with \(\tau = 0.999\) in the reported experiments. The EMA model is intended to provide a stable collaborator whose behavior differs from the online model without requiring a separately trained policy [2509.19460].

At the environment level, SEIL perturbs initial object positions before rollout. The description states that small offsets are applied to the initial \(x\) and \(y\) coordinates while preserving vertical consistency. The stated purpose is to expose the policy to varied object placements and slightly different task configurations. The paper explicitly argues that both augmentation levels are necessary. With only environment augmentation, behavioral diversity remains limited because all rollouts come from the same policy. With only EMA augmentation, the same observation can receive inconsistent action labels from the base and EMA policies, expressed in the analysis as
\[
(O_t, A_t^{Base}) \neq (O_t, A_t^{EMA}),
\]
which is harmful for imitation learning. In the authors’ interpretation, environment augmentation resolves this inconsistency by separating the states on which the two policies are queried [2509.19460].

The second mechanism is a **lightweight trajectory selector**. Rather than using full video, the selector consumes only the **first-frame image** and the **action sequence**. Its reported architecture uses a **ResNet-18** image encoder \(\Phi_i\), an **MLP** action encoder \(\Phi_a\), concatenation of the resulting embeddings, an **LSTM**, and a final **MLP** classifier. The encoding is written as
\[
\mathbf{f}_A = \Phi_a(A), \quad \mathbf{f}_{\mathbf{x}} = \Phi_i(\mathbf{x}), \quad \mathbf{f}_t = [\mathbf{f}_{\mathbf{x}}; \mathbf{f}_A].
\]
The selector is trained first on few-shot expert data, then frozen, and applied to recorded trajectories. Its selection rule is intentionally nonstandard: **low-confidence trajectories are preferred**, on the grounds that they are most distinct from the expert set and therefore more complementary [2509.19460].

The accompanying ablations support this design. In the selector study, **Ascending** selection, defined as low-confidence first, yields **72.7%**, compared with **70.8%** for uniform sampling, **72.0%** for descending sampling, and **72.2%** for mixed sampling. The representation study also reports higher selector classification accuracy when both image and sequence are used rather than sequence alone: **57.0% vs 53.0%** in 1-shot, **61.0% vs 56.0%** in 2-shot, **63.5% vs 54.5%** in 4-shot, and **85.0% vs 74.5%** in 8-shot [2509.19460].

## 4. Experimental evaluation on LIBERO

SEIL is evaluated on the **LIBERO** benchmark, which is divided into **LIBERO-Spatial**, **LIBERO-Object**, **LIBERO-Goal**, and **LIBERO-Long**. The paper characterizes these subsets respectively as emphasizing spatial reasoning, object recognition and manipulation, procedural generalization with varying goals, and long-horizon control. Experiments are conducted in **1-shot**, **2-shot**, **4-shot**, and **8-shot** regimes. The reported baselines are **Diffusion Policy (DP)**, **Action-Chunking Transformer (ACT)**, **RT-1**, and **BAKU**, with all methods trained from scratch for fairness [2509.19460].

Implementation details are reported explicitly. SEIL is built on top of BAKU. In each round, the base model and the EMA model each perform **25 rollouts** in the simulator, and the selector chooses **15** trajectories to add to the evolving dataset. The selector uses a pretrained **ResNet-18**, **two LSTM blocks**, hidden dimension **256**, the **Adam** optimizer, learning rate **0.001**, and **80 epochs** of training. Evaluation uses **50 rollouts** per task, and the metric is **success rate (SR)** [2509.19460].

Across LIBERO subsets, the reported average success rates for SEIL are **49.7%**, **68.1%**, **80.2%**, and **88.7%** in the 1-, 2-, 4-, and 8-shot settings. The corresponding numbers are **35.5%**, **45.4%**, **64.3%**, and **82.2%** for BAKU; **17.2%**, **25.2%**, **30.7%**, and **43.1%** for DP; **7.3%**, **18.8%**, **33.8%**, and **45.4%** for ACT; and **0.0%**, **3.2%**, **3.8%**, and **4.6%** for RT-1. The paper reports these results as a new state of the art in few-shot imitation learning scenarios on LIBERO [2509.19460].

The benchmark-specific results are similarly strong. On **LIBERO-Long**, BAKU obtains **4.6%**, **16.7%**, **40.8%**, and **63.4%**, whereas SEIL reaches **14.6%**, **40.4%**, **54.3%**, and **75.4%** across 1-, 2-, 4-, and 8-shot settings. The paper highlights the **217.3% relative improvement** at 1-shot, comparing **14.6%** with **4.6%**. On **LIBERO-Spatial**, BAKU records **42.2%**, **51.8%**, **75.2%**, and **83.6%**, while SEIL reaches **53.2%**, **71.4%**, **87.2%**, and **87.7%**; notably, SEIL with **4-shot** exceeds BAKU with **8-shot**. On **LIBERO-Object**, the respective BAKU and SEIL results are **65.0% vs 82.0%**, **65.1% vs 81.8%**, **69.8% vs 95.2%**, and **96.8% vs 99.2%**. On **LIBERO-Goal**, the reported comparison is **30.3% vs 49.2%**, **48.0% vs 79.0%**, **71.6% vs 84.2%**, and **84.8% vs 92.8%** [2509.19460].

The ablations clarify where the gains arise. In the **8-shot LIBERO-Long** study, the baseline is **63.4%**, and the configuration with evolution, model-level augmentation, environment-level augmentation, and the selector reaches **75.4%** in the reported table. The paper further reports that **20 rollouts per round** gives the best long-term behavior in the rollout-number ablation, reaching **74.2%** at the fifth round. This is presented as evidence that SEIL works best when self-evolution is gradual rather than overly aggressive [2509.19460].

## 5. Relation to self-imitation learning, self-generated demonstrations, and acronym usage

SEIL belongs to a broader line of work in which agents improve by reusing their own trajectories as supervision, but it differs from earlier self-imitation formulations in both objective and domain. In reinforcement learning, "Learning Self-Imitating Diverse Policies" [1805.10309] treats high-return trajectories as a replay-based target distribution and derives a dense reward from Jensen-Shannon divergence, later adding Stein variational policy gradient with a JS kernel to encourage policy diversity. "Self-Imitation Learning from Demonstrations" [2203.10905] instead initializes a self-imitation replay buffer with demonstrations and relies on a positive-advantage filter so that the influence of demonstrations decreases automatically as the agent improves. Both are primarily sparse-reward RL methods rather than few-shot imitation frameworks for multi-task manipulation.

In robotics, "Self-Imitation Learning by Planning" [2103.13834] is closer in spirit to SEIL. SILP converts the robot’s own visited states into graph nodes, plans over them with **A\***, and relabels the resulting path as demonstrations for behavior-cloning-augmented off-policy RL. "MILES: Making Imitation Learning Easy with Self-Supervision" [2410.19693] addresses an even more restrictive regime—one demonstration and one environment reset—by autonomously collecting augmentation trajectories that return to, and then follow, the demonstrated path. Relative to these methods, SEIL is distinguished by its explicit use of simulator rollouts as new demonstrations, its dual-level augmentation, and its selector that favors complementary low-confidence successful trajectories.

The acronym **SEIL** is itself ambiguous in the literature. "SEIL: Simulation-augmented Equivariant Imitation Learning" [2211.00194] is a distinct method that combines **Transition Simulation** with an \(O(2)\)-equivariant policy network for robotic manipulation from ten or fewer demonstrations. That method addresses sample efficiency through simulated nearby transitions and symmetry-aware function approximation. By contrast, **Self-Evolved Imitation Learning** uses iterative simulator rollouts, EMA collaboration, environment perturbation, and demonstration selection. The shared acronym should therefore not be taken to imply methodological continuity.

## 6. Limitations, implications, and interpretive issues

The reported limitations are direct. First, SEIL depends on a **simulator** to generate additional demonstrations. Second, it incurs a **multi-stage training cost**, because the method is iterative rather than one-shot. These constraints delimit its applicability: the framework is most natural when simulation is available and repeated train-rollout-select cycles are acceptable [2509.19460].

The paper also argues against a common simplification of the method. SEIL is not described as merely “collect successful rollouts and retrain.” Its effectiveness depends on three additional design decisions: dual-level augmentation to diversify the recorded pool, a selector to filter informative trajectories, and gradual evolution across rounds. The ablation results support this interpretation, since evolving without good augmentation yields only modest gains, EMA alone can be harmful because of inconsistent labels, and the selector adds a further improvement once both augmentation levels are enabled [2509.19460].

The broader implication drawn in the work is that large expert datasets are not strictly necessary for strong few-shot imitation performance when simulator access is available. The paper states that a weak imitation policy can bootstrap itself using simulator interactions, that carefully curated self-generated demonstrations can close much of the gap, and that imitation learning can become more data-efficient without relying on reward design. Within that framing, SEIL is best understood as a simulator-assisted demonstration expansion procedure: it retains the supervision structure of imitation learning, but relocates part of the data-generation burden from human experts to iterative self-evolution in simulation [2509.19460].

Source: https://www.emergentmind.com/topics/self-evolved-imitation-learning-seil