GraspNet-1Billion Benchmark Overview

• GraspNet-1Billion Benchmark is a large-scale, RGB-D dataset with dense 6-DoF annotations enabling advanced robotic grasping research.
• The evaluation system employs a force-closure metric and pose-NMS to quantify grasp success through analytic computation.
• Experiments using the benchmark yield high grasp success rates up to 96%, validating its real-world application in robotic manipulation.

The GraspNet-1Billion Benchmark is a large-scale dataset and evaluation platform designed to facilitate research and development in robotic object grasping, particularly in cluttered and multi-object scenes. This benchmark addresses limitations in existing datasets by providing a vast collection of real-world sensor data accompanied by dense annotations and a unified evaluation system, making it a critical resource in the field of robotic manipulation.

1. Dataset Composition

GraspNet-1Billion consists of 87,040 RGB-D images derived from 170 distinct scenes. Each scene is recorded from 512 different viewpoints using two types of RGB-D cameras: the Intel RealSense 435 and the Kinect Azure, which together contribute to the dataset's richness in data diversity and view perspectives. The annotations provided include:

• Over 370 million 6-DoF (degrees of freedom) grasp poses.
• Accurate 6D poses for objects in the scenes, based on manual annotation of the first frame and automatic propagation using known camera movements.
• Additional information including rectangle-based grasp poses, object instance masks, and bounding boxes.

This substantial density and variety of annotations enable the training and evaluation of models that require substantial real-world data.

2. Evaluation System

The GraspNet evaluation system is pioneering for its use of analytic computation to assess grasp success, as opposed to relying on exhaustive manual annotations. This allows for a flexible evaluation of grasp poses, irrespective of their representation. The evaluation follows these steps:

• Uses a "force-closure" metric to determine if a predicted grasp is successful, based on incremental increases in friction coefficient $\mu$.
• Computes a grasp score $s = 1.1 - \mu$, ensuring that lower friction coefficients, which contribute to more robust grasps, yield higher scores.
• Evaluates "Precision@k", an average across different friction levels from $\mu = 0.1$ to $0.5$ to gauge model performance more effectively.
• Employs pose-based Non-Maximum Suppression (pose-NMS) to remove redundant predictions, assessing translation and rotation distances between grasps to retain only top-rated ones.

This system supports a unified approach to evaluating grasp strategies, ensuring consistency and comparability across different works.

3. Experiments and Results

The benchmark has been utilized to conduct extensive experiments, including real-world robotic testing. These experiments have established a strong correlation between high-grasp scores and successful grasp execution. For example, robotic trials with objects marked via ArUco for pose verification showed nearly a 96% success rate for high-score grasps while significantly lower for less robust grasps. This consistency reinforces the validity of the force-closure scoring metric.

4. Technical Details

GraspNet takes advantage of sophisticated annotations and propagation techniques to ensure precision in data labeling:

• For annotating object poses, a method utilizes recorded camera movements to extrapolate poses across frames, facilitating more reliable continuous tracking.
• Grasp poses are generated using analytical computation across sampled points and grid configurations in the camera's alignment, adapting them into world coordinates for scene-level integration.
• Collision checks ensure that only feasible grasp candidates are retained, promoting realistic predictions.

5. Public Availability

The full GraspNet dataset, including its source code and models, has been made open-access for the research community. This poses significant benefits for replication, transparency, and further advancements in the field of grasp research, setting a standard for open collaboration.

6. Implications for Robotic Grasping Research

GraspNet-1Billion significantly propels robotic grasping research forward by combining extensive, densely annotated data with a strong computational evaluation backend. This framework not only encourages benchmark-driven innovation but also facilitates a broad application potential, from laboratory research to practical deployments in automated environments. With the dataset and evaluation mechanisms openly available, the benchmark serves as a cornerstone for future exploration and breakthrough developments in handling real-world object manipulation.