DiffClone: Enhanced Behaviour Cloning in Robotics with Diffusion-Driven Policy Learning
Abstract: Robot learning tasks are extremely compute-intensive and hardware-specific. Thus the avenues of tackling these challenges, using a diverse dataset of offline demonstrations that can be used to train robot manipulation agents, is very appealing. The Train-Offline-Test-Online (TOTO) Benchmark provides a well-curated open-source dataset for offline training comprised mostly of expert data and also benchmark scores of the common offline-RL and behaviour cloning agents. In this paper, we introduce DiffClone, an offline algorithm of enhanced behaviour cloning agent with diffusion-based policy learning, and measured the efficacy of our method on real online physical robots at test time. This is also our official submission to the Train-Offline-Test-Online (TOTO) Benchmark Challenge organized at NeurIPS 2023. We experimented with both pre-trained visual representation and agent policies. In our experiments, we find that MOCO finetuned ResNet50 performs the best in comparison to other finetuned representations. Goal state conditioning and mapping to transitions resulted in a minute increase in the success rate and mean-reward. As for the agent policy, we developed DiffClone, a behaviour cloning agent improved using conditional diffusion.
- Offline reinforcement learning with implicit q-learning, 2021.
- Decision transformer: Reinforcement learning via sequence modeling, 2021.
- A minimalist approach to offline reinforcement learning, 2021.
- Imagenet large scale visual recognition challenge, 2015.
- Glue: A multi-task benchmark and analysis platform for natural language understanding, 2019.
- Rb2: Robotic manipulation benchmarking with a twist, 2022.
- Train offline, test online: A real robot learning benchmark, 2023.
- U-net: Convolutional networks for biomedical image segmentation, 2015.
- Diffusion policy: Visuomotor policy learning via action diffusion, 2023.
- When should we prefer offline reinforcement learning over behavioral cloning?, 2022.
- Bootstrap your own latent: A new approach to self-supervised learning, 2020.
- The edge of orthogonality: A simple view of what makes byol tick, 2023.
- Momentum contrast for unsupervised visual representation learning, 2020.
- Representation learning with contrastive predictive coding, 2019.
- Dean A. Pomerleau. Alvinn: An autonomous land vehicle in a neural network. In D. Touretzky, editor, Advances in Neural Information Processing Systems, volume 1. Morgan-Kaufmann, 1988.
- The surprising effectiveness of representation learning for visual imitation, 2021.
- Deep residual learning for image recognition, 2015.
- Film: Visual reasoning with a general conditioning layer, 2017.
- Attention is all you need, 2023.
- Denoising diffusion probabilistic models, 2020.
- Improved denoising diffusion probabilistic models, 2021.
- Human motion diffusion model, 2022.
- Diffusers: State-of-the-art diffusion models, 2022.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
