Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks (1812.07252v3)

Abstract: Real world data, especially in the domain of robotics, is notoriously costly to collect. One way to circumvent this can be to leverage the power of simulation to produce large amounts of labelled data. However, training models on simulated images does not readily transfer to real-world ones. Using domain adaptation methods to cross this "reality gap" requires a large amount of unlabelled real-world data, whilst domain randomization alone can waste modeling power. In this paper, we present Randomized-to-Canonical Adaptation Networks (RCANs), a novel approach to crossing the visual reality gap that uses no real-world data. Our method learns to translate randomized rendered images into their equivalent non-randomized, canonical versions. This in turn allows for real images to also be translated into canonical sim images. We demonstrate the effectiveness of this sim-to-real approach by training a vision-based closed-loop grasping reinforcement learning agent in simulation, and then transferring it to the real world to attain 70% zero-shot grasp success on unseen objects, a result that almost doubles the success of learning the same task directly on domain randomization alone. Additionally, by joint finetuning in the real-world with only 5,000 real-world grasps, our method achieves 91%, attaining comparable performance to a state-of-the-art system trained with 580,000 real-world grasps, resulting in a reduction of real-world data by more than 99%.

An Overview of "Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks"

The paper "Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks" by Stephen James et al. presents a novel approach to bridging the visual reality gap for robotic grasping tasks, which are traditionally limited by the high cost of real-world data collection. The authors introduce Randomized-to-Canonical Adaptation Networks (RCANs), a method that translates highly randomized simulated images to non-randomized canonical versions, allowing for zero-shot sim-to-real transfer without using real-world data during training.

Methodology

RCAN uses an image-conditioned generative adversarial network (cGAN) that maps images from a randomized simulation domain to a canonical simulation domain. By doing so, RCAN is capable of transforming real-world images to match the canonical simulated output, effectively enabling sim-to-real transfer for robotic grasping. The training setup involves collecting paired observations from both the randomized and canonical simulation environments to create a dataset that facilitates supervised learning of the adaptation function.

The generator within the RCAN framework is trained using a composite loss function that includes an equality loss ensuring pixel, semantic, and depth similarity and a generative adversarial loss that preserves high-frequency details. This design allows the RCAN to produce canonical images with semantic and geometric fidelity critical for downstream robotic tasks.

Experimental Evaluation

The efficacy of RCAN is validated through its integration with the Q-function Targets via Optimization (\qtopt{}), a reinforcement learning algorithm. The experimental results indicate that an agent trained entirely in the canonical simulated environment achieves a striking 70\% grasp success rate on unseen objects in the real world during zero-shot transfer, which is nearly double the performance of an agent trained directly on domain-randomized inputs.

The paper also evaluates the potential benefits of joint fine-tuning in real-world scenarios, showing that with only 5,000 additional real-world grasps, RCAN achieves a 91% grasp success rate — outperforming a baseline model trained with 580,000 real-world grasps by a significant margin. This reduction in necessary real-world data by over 99% underscores the practicality of RCAN in data-limited settings.

Implications and Future Directions

RCAN represents a substantive advancement in data-efficient robotic learning, providing a path to reduce dependence on costly real-world data by leveraging simulation with intelligent domain adaptation. The successful adaptation from simulation to real demonstrated here has broad implications for robotics and other computer vision applications requiring domain transfer.

The method's interpretability and compatibility with various reinforcement learning algorithms make it versatile for different robotic contexts. Future work could explore the integration of unlabelled real-world data to further refine the reality-to-canonical transformation and improve accuracy in dynamic and complex real-world environments. Additionally, adopting similar frameworks for tasks beyond grasping, such as navigation and manipulation in robotic systems, holds promise for expanding the impact and applicability of sim-to-real transfer approaches in robotics.