Good in Bad (GiB): Sifting Through End-user Demonstrations for Learning a Better Policy

Abstract: Imitation learning offers a promising framework for enabling robots to acquire diverse skills from human users. However, most imitation learning algorithms assume access to high-quality demonstrations an unrealistic expectation when collecting data from non-expert users, whose demonstrations often contain inadvertent errors. Naively learning from such demonstrations can result in unsafe policy behavior, while discarding entire demonstrations due to occasional mistakes wastes valuable data, especially in low-data settings. In this work, we introduce GiB (Good-in-Bad), an algorithm that automatically identifies and discards erroneous subtasks within demonstrations while preserving high-quality subtasks. The filtered data can then be used by any policy learning algorithm to train more robust policies. GiB first trains a self-supervised model to learn latent features and assigns binary weights to label each demonstration as good or bad. It then models the latent feature distribution of high-quality segments and uses the Mahalanobis distance to detect and evaluate poor-quality subtasks. We validate GiB on the Franka robot in both simulated and real-world multi-step tasks, demonstrating improved policy performance when learning from mixed-quality human demonstrations.

• The paper introduces a novel GiB algorithm that filters demonstration data at the subtask level, preserving useful segments from non-expert inputs.
• It details a two-stage pipeline combining self-supervised learning and Mahalanobis distance scoring to identify and excise corrupted subtasks.
• The method shows significant improvements on both simulated and real-world robotic tasks, outperforming traditional full-trajectory filtering approaches.

Good-in-Bad (GiB): Segment-Level Filtering of End-User Demonstrations for Robust Policy Learning

Motivation and Problem Formulation

Real-world deployment of imitation learning (IL) for robotics frequently relies on demonstrations supplied by non-experts, which often contain significant errors and inconsistencies. Existing IL pipelines typically presume either expert or simulation-generated data, thus sidestepping the challenge of unreliable human demonstrations. Discarding entire trajectories upon detection of a single error is inefficient, especially when demonstration collection is costly and such errors are local to specific subtasks.

The "Good-in-Bad" (GiB) algorithm addresses this core problem with a segment-level approach: autonomously identifying and excising only erroneous subtasks, thus preserving usable information from partially flawed demonstrations. GiB is designed for sequential visual manipulation tasks and leverages advances in self-supervised representation learning, Mahalanobis-based deviation scoring, and object-centric subtask segmentation.

GiB Algorithm Overview

GiB operates in a two-stage pipeline: initial self-supervised representation learning and trajectory assessment, followed by subtask-level evaluation and pruning.

Figure 1: GiB subtask evaluation pipeline: (A) Self-supervised learning estimates latent embeddings and assigns a binary quality label to each trajectory; (B) Trajectories are segmented into subtasks and high-quality latent feature distributions are estimated per subtask; (C) Mahalanobis distance scores identify the most divergent (bad) subtasks for removal.

Stage 1: Self-Supervised Identification of Good and Bad Demonstrations

GiB employs a self-supervised policy inspired by the BED framework, optimizing an inconsistency-based loss to produce a trajectory quality weight $w_i \in \{0,1\}$. This loss incorporates action, state, and goal inconsistency terms, utilizing trajectory-wise latent representations and terminal embeddings. Trajectories with elevated inconsistency are weighted as bad ($w_i \approx 0$), those conforming to dominant behaviors as good ($w_i \approx 1$).

Stage 2: Subtask-Level Deviation Analysis

Each trajectory is heuristically or programmatically partitioned into object-centric subtasks. GiB models the latent space distribution of high-quality (i.e., $w_i = 1$) subtasks via multivariate Gaussian estimators. For every subtask in each demonstration, Mahalanobis distance is computed between its latent trajectory and the corresponding nominal subtask distribution, providing a deviation score. The hyperparameter $\rho$ governs the number of subtasks with the highest deviation to be labeled as corrupted and excluded from the training set. Unlike prior full-trajectory filtering approaches, this affords fine-grained data curation without wholesale dismissal of informative behaviors.

Experimental Protocols

GiB was evaluated on both simulated and real-world multi-step robotic manipulation tasks. The experiments considered:

• Simulation tasks: Square, Coffee, Mug Cleaning, and Kitchen tasks from the MimicGen environment, encompassing varying sequence lengths and object manipulations.
• Real-world setup: A Franka Emika Panda robot performing a Mug Cleaning analog involving sequential drawer interactions and object placements.

  Figure 2: Left: Simulated Kitchen environment; Right: Real Franka robot performing sequential pick-place tasks.

Demonstration datasets were constructed with a mix of expert and non-expert human contributors using teleoperation, yielding significant between-trajectory quality variation.

Baseline comparison encompassed trajectory- and segment-based filtering approaches such as BED, DemInf, S2I, LOF, and Oracle masking. Downstream policies were trained using the Diffusion Policy framework with identical hyperparameters across conditions.

Results and Quantitative Analysis

Simulation Results:

GiB consistently outperformed or matched all baselines across a spectrum of tasks. For complex, long-horizon or data-limited settings (notably the Square and Kitchen tasks), GiB-masked data led to the highest policy success rates. For instance, in the Square task, GiB achieved an average success rate of 0.49, surpassing BED, DemInf, and LOF filtering. In the Kitchen task, GiB matched the top baseline at 0.87.

Real-World Results:

On the Franka "Drawer" task, GiB-masked data significantly outperformed all alternatives at both the subtask and full-task granularity. It achieved a substantial improvement in end-to-end task completion (0.70 full-task success), compared to only 0.15 with naive "all-data" or oracle trajectories.

Figure 3: Visualization of Mahalanobis distance trends over time in a two-step coffee-making task, highlighting one subtask's deviation and another's conformity to nominal behavior.

Qualitative Analysis and Insights:

GiB surfaces error localization and latent feature deviations across timesteps. In Figure 3, high Mahalanobis distances reveal precisely where demonstration execution departs from nominal, while receding deviations indicate subsequent behavioral recovery—providing both interpretability and the potential for future automated correction modeling.

Multimodal Generalization:

GiB (building on the BED loss) further enables learning robust representations in environments where multiple valid execution modes exist. Comparative t-SNE visualizations (not shown here) indicate that GiB learns compact latent clusters encompassing all successful strategies while preserving distinction from error modes—a property not shared by baseline filters that artificially split valid behaviors.

Figure 4: GiB's latent representation learns to group multimodal valid executions while isolating only poor demonstrations.

Effect of Pruning Hyperparameter ($\rho$):

Systematic ablation demonstrates an optimal range for $\rho$ that maximizes policy performance—excessive pruning removes informative data, while insufficient filtering retains corrupt demonstrations, reducing task success rates.

Figure 5: Success rate as a function of $\rho$ (corrupted subtasks filtered), highlighting the optimal pruning regime for Square and Mug tasks.

Theoretical Implications and Future Developments

GiB reasserts the critical role of data curation granularity in imitation learning. Rather than treating demonstration quality as a trajectory-level binary, segment-wise evaluation maximizes informative data retention, particularly crucial for field deployment with non-experts. GiB’s approach can be extended to incorporate recovery behavior attribution, potentially providing self-correcting signals for both human and autonomous agents. Future research may explore learning-based, rather than heuristic, segmentation algorithms to improve consistency and generalization across more chaotic or unstructured demonstrations.

From the theoretical standpoint, this work re-contextualizes the anomaly detection literature within the sequential, high-dimensional setting of visual imitation learning, proposing a pipeline compatible with arbitrary downstream policy optimizers and independent of external labeling or curated gold standard subsets.

Conclusion

GiB advances imitation learning under realistic, non-expert data conditions by introducing a robust, policy-agnostic procedure for discarding only low-quality segments from demonstration trajectories. Empirical results demonstrate strong improvements in policy performance, both in simulation and on physical platforms, relative to existing data filtering baselines. This underscores that value can be extracted even from corrupted demonstrations when segment-level quality assessment and curation are applied. Extending GiB to model and exploit corrective subtrajectories represents a promising direction for achieving resilience and generalization in real-world robot learning.