ReGIL: Retrieval-Guided Imitation Learning from a Single Demonstration

Abstract: Learning robot manipulation policies with deep neural networks from a single demonstration remains highly challenging, as even small deviations from the demonstrated trajectory can quickly compound into failure, while collecting substantial online interaction data is costly. We propose ReGIL, a retrieval-guided imitation learning framework that treats a single demonstration as an external memory. ReGIL repeatedly queries this static memory throughout training to simultaneously guide exploration, generate the regularization buffer, and construct rewards. Specifically, it computes rewards through local temporal alignment between the current trajectory and the retrieved segment, providing step-wise and informative feedback for policy improvement. We evaluate ReGIL on robotic manipulation tasks from the LIBERO and Meta-World benchmarks under the single demonstration setting. ReGIL outperforms prior baselines in both success rate and training efficiency. In real-robot experiments, using only one demonstration and less than one hour of online training, ReGIL achieves over 75% success rate across three manipulation tasks with randomness in both initial robot pose and target position. These results demonstrate that leveraging the single demonstration as reusable memory can provide more than static supervision for efficient robot learning. More details can be found on our website: https://regil2026.github.io/

• The paper introduces ReGIL, a framework that uses a single demonstration as a persistent, queryable memory to guide exploration and reward shaping.
• ReGIL employs retrieval-guided exploration with visual similarity and local temporal alignment (S-DTW) for dynamic segmentation of expert behaviors.
• Empirical results on simulated and real-world tasks show ReGIL outperforms baselines in convergence speed and success rates, offering robust one-shot imitation learning.

Retrieval-Guided Imitation Learning from a Single Demonstration: ReGIL

Problem Formulation and Motivation

Sample-efficient policy acquisition for robotic manipulation, given only a single demonstration, poses significant algorithmic and practical challenges. Conventional offline imitation learning approaches, including behavior cloning (BC) and trajectory alignment methods, suffer from compounding distribution shift errors when an agent deviates from the limited demonstration data, especially under real-world stochasticity. Online reinforcement learning (RL), while capable of recovery from novel states, is usually hampered by exploration inefficiency and sparse rewards—issues exacerbated by the absence of effective shaping signals when only pixel observations are available. Existing retrieval-based imitation strategies either replay retrieved behaviors or offer static demonstration supervision, lacking mechanisms for robust online adaptation and generalization.

ReGIL Framework

ReGIL (Retrieval-Guided Imitation Learning) addresses these challenges by treating the single demonstration as a persistent, queryable external memory throughout the online learning process. At each timestep, the agent retrieves demonstration segments using a combination of visual similarity (via a frozen visual foundation model encoder) and local temporal alignment (using Subsequence Dynamic Time Warping, S-DTW). This architecture enables three synergistic mechanisms:

1. Retrieval-Guided Exploration: Early training phases leverage the retrieved segment to inform action replay, rapidly guiding the agent toward successful behaviors without relying on inefficient random exploration.
2. Success Buffer Expansion: Successful episodes arising from guided exploration are accumulated in an incrementally growing buffer, which serves as an adaptive regularization set and mitigates the single-demonstration data bottleneck.
3. Retrieval-Based Reward Construction: Rather than computing rewards from global, trajectory-wide alignment, ReGIL generates dense, segment-level, stepwise reward signals based on causal local temporal alignment—preserving phase and structural correspondence without non-causal information leakage.

This combination provides real-time, trajectory-aware supervision that augments both exploration and training, operationalizing the demonstration memory as a continually reused supervisory signal rather than static prior.

Technical Details

At each timestep $t$, ReGIL performs the following:

• Expert Segment Retrieval: The algorithm identifies top-$k$ visually similar states in the demonstration (embedding space distance, using DINOv3) and selects the most temporally advanced among these to avoid local minima. S-DTW aligns the agent’s most recent observation history window with the corresponding demonstration segment, providing both the relevant expert context and alignment cost.
• Exploration Strategy: During the warmup phase, the agent performs action replay according to the retrieved expert action, enriching the success buffer with diverse successful trajectories. This counteracts covariate shift and promotes robustness to initial condition variability.
• Policy Optimization: A TD3-style actor-critic is trained with dense retrieval-based rewards (normalized S-DTW alignment cost), plus a decaying behavior cloning regularizer over the success buffer. The regularization weight decreases over time, moving policy optimization from supervised imitation toward reinforcement learning, thus facilitating both stability and surpassing the demonstrated strategy.

Empirical Evaluation

Simulated Benchmarks

Experiments on the Meta-World and LIBERO benchmarks demonstrate that ReGIL markedly outperforms established baselines—including BC, transformer-based imitation (BAKU), and various trajectory-alignment inverse RL approaches (ROT, TOT)—in both convergence speed and asymptotic success rate. The improvements are particularly prominent in long-horizon and high-diversity tasks where standard methods are brittle due to limited demonstration coverage.

Ablation studies establish the critical roles of (1) retrieval-guided exploration, (2) behavior cloning regularization on the success buffer, and (3) the use of segment-level reward proxies. Notably, removing retrieval or using only state-level similarity rewards (as opposed to the proposed local trajectory alignment) substantially degrades learning efficiency and final policy performance.

Real-World Robotic Control

ReGIL was validated on a Franka Panda platform across three manipulation tasks: Reach, Insert, and Open. Results show consistent task acquisition from a single RGB-only demonstration and under one hour of online interaction, achieving over 75% success rate (randomized target/pose) and highlighting the framework’s real-time applicability (processing frequency $\approx$ 31.5 Hz). Notably, ReGIL is robust to significant variation in initial robot configuration and task parameters, outperforming baselines under both fixed and randomized settings.

Theoretical Implications

ReGIL operationalizes the notion that a single demonstration can be leveraged as dynamic, context-dependent supervision, transcending static behavior cloning. This reframes the single-demonstration regime: rather than static distribution matching or replay, the demonstration acts as an evolving source of temporal structure and reward signal through continual, locality-aware retrieval. The success buffer expansion further reinterprets online adaptation as incrementally extending the empirical support of the agent’s behavior space, mitigating distribution shift and aliasing.

ReGIL’s reward formulation is causally aligned and computationally lightweight compared to global optimal transport approaches; segment-level alignment yields rich, temporally consistent shaping signals without future information dependence, enabling real-time closed-loop control.

Limitations and Future Directions

While ReGIL offers strong sample efficiency and robustness, several limitations remain. Retrieval quality governs performance, with sensitivity to visual ambiguity (notably under occlusion or lighting perturbations) and brittle visual representations in the Insert and Open tasks. Non-rigid object manipulation highlights the limitation of vision-only input. Additionally, the diversity and size of the success buffer limit performance in highly complex settings. Extensions to multimodal input (e.g., tactile sensing), adaptive retrieval mechanisms, or leveraging action-free (e.g., human video) demonstrations constitute promising avenues. Integration with prior pre-trained policies and more expressive representations could further address the curse of high-dimensional observation and action spaces.

Conclusion

ReGIL provides a principled, empirically validated architecture for one-shot imitation learning in robotics by treating demonstrations as reusable aligned memory, enabling exploration guidance, continual reward shaping, and adaptive regularization. The framework achieves substantial advances in sample efficiency, real-world robustness, and policy generalization, setting a new direction for data-efficient policy learning from minimal supervision (2606.09381). Continued research along these lines may yield scalable, deployable robotic systems with minimal task programming and substantial autonomy in novel, unstructured environments.