Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics (1703.09312v3)

Abstract: To reduce data collection time for deep learning of robust robotic grasp plans, we explore training from a synthetic dataset of 6.7 million point clouds, grasps, and analytic grasp metrics generated from thousands of 3D models from Dex-Net 1.0 in randomized poses on a table. We use the resulting dataset, Dex-Net 2.0, to train a Grasp Quality Convolutional Neural Network (GQ-CNN) model that rapidly predicts the probability of success of grasps from depth images, where grasps are specified as the planar position, angle, and depth of a gripper relative to an RGB-D sensor. Experiments with over 1,000 trials on an ABB YuMi comparing grasp planning methods on singulated objects suggest that a GQ-CNN trained with only synthetic data from Dex-Net 2.0 can be used to plan grasps in 0.8sec with a success rate of 93% on eight known objects with adversarial geometry and is 3x faster than registering point clouds to a precomputed dataset of objects and indexing grasps. The Dex-Net 2.0 grasp planner also has the highest success rate on a dataset of 10 novel rigid objects and achieves 99% precision (one false positive out of 69 grasps classified as robust) on a dataset of 40 novel household objects, some of which are articulated or deformable. Code, datasets, videos, and supplementary material are available at http://berkeleyautomation.github.io/dex-net .

Overview of Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics

Dex-Net 2.0 presents an innovative approach to robotic grasp planning by leveraging deep learning techniques trained on a synthetic dataset of 6.7 million point clouds, grasps, and associated analytic grasp metrics. The central contribution of this research is the development of the Grasp Quality Convolutional Neural Network (GQ-CNN) which predicts the robustness of grasps derived from depth images. The grasps are defined by their planar position, angle, and depth relative to the gripper.

Key Contributions

The paper makes several notable contributions to the field of robotic grasping:

1. Dex-Net 2.0 Dataset: A comprehensive dataset consisting of 6.7 million point clouds and parallel-jaw grasps associated with robust analytic grasp metrics. This dataset spans 1,500 3D models, establishing a substantial foundation of data for training machine learning models.
2. GQ-CNN Model: The introduction of the GQ-CNN, which classifies the robustness of grasps in depth images using expected epsilon quality as supervision. This network is trained offline and is capable of rapid inference, making it suitable for real-time applications.
3. Grasp Planning Method: A robust grasp planning method that samples antipodal grasp candidates and ranks them using the trained GQ-CNN. This method operates with high precision and efficiency, reducing the computational load compared to traditional registration-based approaches.

Numerical Results and Performance

The Dex-Net 2.0 grasp planner exhibited impressive empirical results across a series of benchmarks:

• Known Objects: The GQ-CNN trained solely on synthetic data demonstrated a 93% success rate on eight objects with adversarial geometries.
• Novel Objects: The planner achieved the highest success rate amongst various methods when evaluated on a dataset of 10 novel rigid objects, maintaining a 99% precision rate (one false positive out of 69 robustly classified grasps).
• Planning Efficiency: The GQ-CNN-based grasp planner operates three times faster than methods utilizing point cloud registration.

Practical and Theoretical Implications

The implications of this research are profound for both the practical deployment of robotic systems and the theoretical foundations of grasp planning. Practically, the ability to rapidly and reliably predict the success of grasps directly from depth images allows for more efficient and adaptable robotic systems in dynamic environments. This capability is critical for applications such as automated assembly lines, warehouse management, and service robotics where real-time adaptability is essential.

Theoretically, the approach highlights the potential of synthetic datasets to train machine learning models that generalize well to real-world tasks. Moreover, the usage of robust analytic grasp metrics as a training target consolidates the connection between physics-based grasp analysis and data-driven methods, offering a hybrid methodology that leverages the strengths of both paradigms.

Future Developments

Future research could explore several areas extending the foundation laid by Dex-Net 2.0:

1. Active Learning for Grasp Refinement: Utilizing adaptive policies initialized with the GQ-CNN to iteratively improve grasp success rates through active learning algorithms.
2. Multi-View Grasp Planning: Extending the GQ-CNN to incorporate point clouds from multiple viewpoints, enhancing the ability to handle occlusions and complex object geometries.
3. Cluttered Environments: Developing robust policies for grasping in cluttered scenes, potentially incorporating pushing or reorienting actions to isolate target objects.
4. Material and Pose Estimation: Integrating estimations of object material properties and dynamic pose adjustments to further refine the predicted robustness of grasps.

Conclusion

Dex-Net 2.0 represents a substantial advancement in robust robotic grasp planning through the integration of synthetic datasets with advanced deep learning architectures. Its contributions enhance the viability of reliable and efficient robotic manipulation in practical settings, laying a pathway for future work that could lead to near-perfect grasping performance across diverse and complex environments.