GraspGen: Diffusion-Based 6-DOF Grasping

• GraspGen is a diffusion-based framework for object-centric 6-DOF grasping that models grasp generation as conditional denoising over rigid poses.
• It couples a conditional diffusion generator with an on-generator-trained discriminator, reducing errors and boosting recall through efficient scoring.
• The framework leverages comprehensive simulated datasets and object-centric point clouds to deliver diverse, high-success grasps across multiple gripper embodiments.

GraspGen is a diffusion-based framework for object-centric 6-DOF grasping that models grasp generation as conditional denoising over rigid grasp poses and couples that generator to a learned grasp evaluator trained on the generator’s own outputs. Introduced in "GraspGen: A Diffusion-based Framework for 6-DOF Grasping with On-Generator Training" (Murali et al., 17 Jul 2025), it takes a segmented target object point cloud as input and returns a set of scored grasp poses $\{(g_i,s_i)\}_{i=1}^K$, where each $g_i \in SE(3)$ and each $s_i \in [0,1]$. The framework is designed for diverse successful grasps, multiple grippers, and deployment under noisy visual observations, and it is organized around three elements: a conditional diffusion generator, an efficient discriminator, and an on-generator training recipe for aligning the discriminator with the failure modes actually produced by the generator.

1. Problem formulation and object-centric scope

GraspGen addresses object-centric 6-DOF grasp generation rather than scene-centric grasp prediction. Its input is an object-centric point cloud $\mathcal{X}$, obtained either from a full mesh or from partial observation, and its output is a set of scored grasp poses suitable for downstream filtering by inverse kinematics, collision checking, and motion planning (Murali et al., 17 Jul 2025). This framing is motivated by the observation that grasp distributions are continuous and highly multimodal, and that modern instance segmentation systems make an object-centric pipeline practical.

A grasp is represented as a rigid pose in $SE(3)$, decomposed into translation and orientation. The framework therefore targets the full spatial grasping problem required by arbitrary object poses, cluttered settings, and non-tabletop manipulation, rather than planar or contact-point-only formulations. To compare poses and compute coverage, the paper uses the distance

$$d(g_i, g_j) = \| t_i - t_j \| + \left\| \mathrm{LogMap}(\mathbf{R}_i^{-1} \mathbf{R}_j) \right\|,$$

where $t_i,t_j$ are grasp translations and $\mathbf{R}_i,\mathbf{R}_j$ are grasp rotations. This metric is used in the Earth Mover’s Distance analysis and in coverage matching.

The object-centric formulation also defines GraspGen’s limits. It assumes a segmented target object point cloud rather than jointly reasoning over scene-wide clutter geometry during generation itself. A plausible implication is that GraspGen is strongest when object isolation is reliable and when downstream planning can exploit multiple candidate grasps rather than requiring a single end-to-end scene policy.

2. Generator architecture and grasp representation

The generator is a DiffusionTransformer conditioned on object geometry. Object shape is encoded with PointTransformerV3 (PTv3) rather than PointNet++, and the grasp pose is represented by translation in $\mathbb{R}^3$ together with rotation in the Lie algebra of $SO(3)$. The paper reports that replacing PointNet++ with PTv3 reduces translation error by 5.3 mm and increases recall/coverage by 4% (Murali et al., 17 Jul 2025).

Before generation, point clouds are mean-centered and grasps are expressed in the same centered frame. Translations are additionally normalized by a dataset-level scalar

$g_i \in SE(3)$0

where $g_i \in SE(3)$1 denotes the translation component of all positive grasps for object $g_i \in SE(3)$2. For the Franka dataset, the paper reports $g_i \in SE(3)$3. This normalization is important because rotation already lives in a bounded domain, whereas translation scale varies with object size and grasp spread.

Training follows a DDPM-style noise-prediction objective. Given a positive grasp $g_i \in SE(3)$4, timestep $g_i \in SE(3)$5, and noisy grasp $g_i \in SE(3)$6, the network $g_i \in SE(3)$7 predicts the applied noise from $g_i \in SE(3)$8 using

$g_i \in SE(3)$9

At inference, GraspGen samples a batch of random grasp noise vectors and performs reverse diffusion for $s_i \in [0,1]$0 steps. The paper attributes this small step count to the low dimensionality of 6-DOF grasps relative to image generation.

A notable empirical design choice is to use two separate denoising processes with their own dedicated schedulers, one for translation and one for rotation, rather than a single joint process. The paper reports that this performs better than a single DDPM over both components. It also finds that 6D rotation, Euler angles, and Lie algebra perform similarly when properly normalized, though Lie algebra is used in the main experiments.

3. Discriminator design and on-generator training

GraspGen’s second learned component is a grasp discriminator that scores generated poses. Rather than converting each candidate grasp into a transformed gripper point cloud and processing that together with the object, the discriminator reuses the generator’s frozen object encoder, concatenates the resulting object embedding $s_i \in [0,1]$1 with the explicit grasp pose $s_i \in [0,1]$2, and feeds that representation to an MLP that outputs a sigmoid success score (Murali et al., 17 Jul 2025). This design is both simpler and cheaper than prior point-cloud-based evaluators; the paper reports it is 21× less memory than the discriminator in 6-DOF GraspNet, while improving by +6.7% AUC and +5.87% mAP.

The framework’s distinctive training contribution is on-generator training. The paper argues that offline negatives differ from the false positives actually emitted by a trained grasp generator, so a discriminator trained only on offline data is mismatched to deployment. GraspGen therefore trains the discriminator on grasps sampled from the generator itself. Its training recipe is:

1. train the diffusion generator on positive grasps only;
2. run the trained generator on training objects to sample on-generator grasps;
3. relabel those grasps in simulation as positive or negative;
4. train the discriminator on this on-generator distribution.

Formally, the procedure computes $s_i \in [0,1]$3, trains $s_i \in [0,1]$4, samples $s_i \in [0,1]$5, simulates $s_i \in [0,1]$6, and trains $s_i \in [0,1]$7 on those relabeled outputs. The paper estimates the distribution shift between offline and on-generator grasps with Earth Mover’s Distance and reports that the shift is non-zero for positives and larger for negatives. The empirical consequence is direct: a discriminator trained exclusively on on-generator data performs best, while an offline-only discriminator is 6.5% lower AUC.

This training strategy is central to GraspGen’s identity. It does not change the diffusion formulation itself, but it changes how the system ranks and filters grasps, and much of the framework’s empirical advantage is tied to this evaluator alignment.

4. Dataset construction and empirical performance

To scale training across objects and grippers, GraspGen introduces a new simulated dataset built from Objaverse. Starting from 36,366 meshes overlapping with the 1,156 LVIS categories and licensed under CC BY, the paper selects 8,515 object meshes as an ACRONYM-scale subset. For each object, it samples 2K grasp transformations. For the Franka Panda gripper and Robotiq-2F-140, grasp labels are generated in Isaac simulator / Isaac Gym with a shaking-based stability test; for the 30 mm suction cup, labels are generated analytically in the style of Dex-Net 3.0. Each gripper contributes about 17 million grasps, for a total of 53.1 million grasps across the three embodiments (Murali et al., 17 Jul 2025).

In isolated-object simulation on the Franka-ACRONYM benchmark, evaluation uses 815 objects, 2K grasps/object, and about 1.6 million grasp executions. The main metric is the precision-coverage curve, where precision is grasp success rate in simulation and coverage is the fraction of the ground-truth positive set matched by predictions under a 1 cm threshold. The paper reports that GraspGen outperforms baselines by over 48% AUC on Franka-ACRONYM.

On FetchBench, GraspGen is evaluated on 100 scenes and 60 tasks per scene, for 6,000 grasp executions, using a Franka Panda and cuRobo. The paper reports an oracle upper bound of 65% grasp success and 49.2% task success when ground-truth grasps are supplied to the planner, underscoring that even perfect grasp proposals do not eliminate motion-planning and accessibility failures. Within this benchmark, GraspGen surpasses Contact-GraspNet by 16.9% and M2T2 by 7.8%.

Real-robot experiments use a UR10, an extrinsically calibrated RealSense D435, SAM2 for segmentation, FoundationStereo for depth, NVBlox for collision mapping, and cuRobo for planning. The deployed model is trained on the Robotiq-2F-140 dataset. Reported grasp success rates are 90.5% for isolated objects, 83.3% for table clutter, 80.0% for baskets, 71.4% for shelves, and 81.3% overall. In the same setup, M2T2 achieves 52.6% overall and AnyGrasp achieves 63.7% overall. The largest gaps appear in harder non-tabletop clutter settings such as baskets and shelves.

The paper also studies observability. A model trained only on complete point clouds performs poorly on partial point clouds; a model trained only on partial clouds performs poorly on complete clouds; training on a 50/50 mix generalizes well to both. This result aligns with the framework’s deployment emphasis: the generator is object-centric, but the observation regime still strongly shapes learned robustness.

5. Extensions and later developments

Subsequent work has extended GraspGen in two main directions: cross-embodiment conditioning and embodiment-specific deployment adaptation. "GraspGen-X: Cross-Embodiment 6-DOF Diffusion-based Grasping" (Han et al., 31 May 2026) preserves the generator-plus-discriminator structure of GraspGen but conditions both modules on an explicit gripper representation derived from a 12-dimensional swept-volume descriptor measured at fully open and half-open states. In zero-shot evaluation on 10 held-out real test grippers and novel objects, it reports average mAUC of 0.506 for GraspGen-X, compared with 0.398 for retargeting and 0.126 for direct transfer. In real-world experiments on a UR10 with a novel Robotiq-2F140 gripper, it reports 79.0% overall success, versus 65.2% for GraspGen-RTG and 61.4% for AnyGrasp.

A different line of work modifies deployment rather than training. "MVB-Grasp: Minimum-Volume-Box Filtering of Diffusion-based Grasps for Frontal Manipulation" (Akada et al., 29 Jun 2026) does not alter GraspGen’s diffusion model. Instead, it fits a PCA-based oriented bounding box to the target object, selects accessible faces, filters candidate grasps whose approach axes do not align with those faces, and re-scores survivors using

$s_i \in [0,1]$8

with $s_i \in [0,1]$9. In 81 MuJoCo episodes on a Unitree Z1, this yields 59.3% success versus 24.7% for vanilla GraspGen, a 2.4× improvement, while adding only 6.78 ms of selection overhead.

GraspGen has also been used as a modular grasp proposal component in broader manipulation systems. "NovaFlow: Zero-Shot Manipulation via Actionable Flow from Generated Videos" (Li et al., 9 Oct 2025) uses GraspGen to produce object-specific grasp proposals and then attaches the end effector to an inferred object trajectory through

$\mathcal{X}$0

This suggests that GraspGen’s object-centric 6-DOF grasps can function not only as terminal pick poses but also as attachment transforms inside larger zero-shot manipulation pipelines.

6. Limitations and significance

GraspGen’s limitations follow directly from its design choices. The paper explicitly notes dependence on depth quality and instance segmentation quality, difficulty with cuboid-like objects, and a computational footprint of about 3K GPU hours on NVIDIA V100 8-GPU nodes for data generation and training (Murali et al., 17 Jul 2025). Because the method is object-centric, it also depends on a separate upstream segmentation module and does not itself solve joint scene reasoning or clutter interaction planning.

The benchmark analysis clarifies another limit: grasp generation is only one part of a manipulation stack. On FetchBench, even an oracle planner supplied with ground-truth grasps reaches only 65% grasp success and 49.2% task success, so accessibility, collision-free retraction, and motion-planning feasibility remain bottlenecks outside the generator. GraspGen improves the proposal and ranking stages, but it does not eliminate downstream failures.

Its significance lies in how it combines three ideas that had often been treated separately: a stronger diffusion-based pose sampler over $\mathcal{X}$1, a lightweight discriminator that scores explicit grasp poses efficiently, and an on-generator evaluator-training strategy that aligns the ranking model with the generator’s actual error distribution. In that sense, GraspGen is not only a grasp sampler but also a statement about how 6-DOF grasp generation systems should be trained and deployed: generation and evaluation should be learned together at the distribution actually induced by the generator, and object-centric pipelines remain viable when paired with robust filtering and planning.