DiPGrasp: Differentiable Grasp Planner

• DiPGrasp is a differentiable grasp planner that uses surface matching and force-closure metrics for rapid, optimized, and collision-free grasp synthesis.
• It integrates gradient-based optimization with differentiable collision handling to support dataset generation, instance-conditioned planning, and pose refinement for high-DOF hands.
• GPU-accelerated parallel local search yields ~600× speed improvements over traditional methods, enabling robust grasping in diverse robotic applications.

DiPGrasp is a differentiable grasp planner for robotic manipulation that leverages geometric surface-matching and force-closure metrics to enable fast, parallelizable, and optimizable grasp planning for robot hands of arbitrary degrees of freedom. As a standalone planner, DiPGrasp integrates gradient-based optimization with collision handling, supporting applications in grasp dataset generation, instance-conditioned grasping, and pose refinement for high-DOF grippers. The approach is characterized by modularity, algorithmic differentiability, and direct compatibility with GPU acceleration, yielding significant throughput improvements and enabling effective downstream integration with perception and learning pipelines (Xu et al., 2024).

1. Grasp Quality Metric and Geometric Optimization

At the core of DiPGrasp is a surface-matching energy augmented by a force-closure regularizer. The robot hand is described by a wrist pose $(R, t) \in SO(3) \times \mathbb{R}^3$ and finger joints $q \in \mathbb{R}^k$, constrained by joint limits. The transformed finger link surfaces are matched to their nearest neighbors on the object model, yielding a set of contact pairs $(x_{j_i}, y_{j_i})$ with associated normals.

The objective function for grasp pose optimization comprises:

• Point-to-plane error:

$$E_p(R, t, q) = \sum_{i=1}^k \sum_{j=1}^{m} \bigl[(x_{j_i} - y_{j_i})^T n^y_{j_i}\bigr]^2$$

• Normal alignment:

$$E_n(R) = \sum_{i=1}^k \sum_{j=1}^{m} \bigl[(R n^x_{j_i})^T n^y_{j_i} + 1\bigr]^2$$

• Force-closure regularizer:

Contact points $x_i$ (sampled via FPS on the palm) yield a wrench matrix $G$; with contact normals $c \in \mathbb{R}^{3n}$:

$E_{fp}(R, t, q) = E_p(R, t, q) + \|G c\|_2$

Thus, the full differentiable energy is:

$E(R, t, q) = E_{fp}(R, t, q) + E_n(R)$

This metric captures both kinematic alignment and stability-critical grasp properties (Xu et al., 2024).

2. Differentiable Collision Handling

To ensure physically realizable and collision-free grasps, DiPGrasp employs differentiable barrier terms within the optimization:

• Object collision barrier: Each pair-wise distance $q \in \mathbb{R}^k$0 is checked against a threshold $q \in \mathbb{R}^k$1, and a smooth penalty is applied for near-collisions:

$q \in \mathbb{R}^k$2

• Joint-limit barrier:

$q \in \mathbb{R}^k$3

with $q \in \mathbb{R}^k$4 a smooth logarithmic barrier function.

All terms are composed to form the final objective:

$q \in \mathbb{R}^k$5

This structure enables end-to-end backpropagation of gradients for accelerated optimization and supports collision-aware local refinement (Xu et al., 2024).

3. Parallel Gradient-Based Local Search

A distinguishing aspect of DiPGrasp is its use of massively parallel local searches enabled by GPU acceleration. The method samples $q \in \mathbb{R}^k$6 initialization points (using Farthest Point Sampling) on the object's point cloud. For each sample:

• The palm is positioned at a set offset with the normal aligned to the local surface.
• Initial open-hand joint configurations are set.
• Poses in collision are filtered out with bounding-box tests.

Batched optimization proceeds by:

1. Computing the total energy $q \in \mathbb{R}^k$7 for all $q \in \mathbb{R}^k$8 poses in a single forward pass.
2. Calculating gradients with respect to $q \in \mathbb{R}^k$9.
3. Updating each pose in parallel, subject to joint and collision constraints.
4. Final collision-check and culling, with collision-free high-quality grasps retained for execution or dataset inclusion.

This approach yields a significant speedup—quantitatively, grasp generation runs $(x_{j_i}, y_{j_i})$0600$(x_{j_i}, y_{j_i})$1 faster than EigenGrasp, with valid proportions and quality exceeding prior analytical planners (e.g., ISF, DexGraspNet) (Xu et al., 2024).

4. Applications: Dataset Generation, Mask-Conditioned Planning, Pose Refinement

DiPGrasp is architected for versatility in multiple manipulation-centric pipelines:

• Grasp dataset construction: DiPGrasp generates diverse, high-quality grasp datasets for various robot hands (e.g., Barrett, Schunk SVH) with minimal compute time and high validity. For example, Barrett grasps achieve 67.7% valid rate at 0.03 s per grasp, outperforming ISF and DexGraspNet (Xu et al., 2024).
• Mask-conditioned planning: Integrated with Mask3D instance segmentation, DiPGrasp restricts sampling to segmented object regions, enabling instance-specific, collision-aware grasp execution. Inclusion of the force-closure metric adds ~8% absolute improvement in success rates.
• Pose refinement: Neural networks (e.g., SimpleGrasp) provide coarse pose predictions that DiPGrasp locally optimizes, dramatically reducing error metrics (BSM) with a small number of gradient steps (from 863.6 → 60.3 for Barrett after 13 steps).

5. Experimental Validation and Quantitative Performance

Extensive simulation and real-world experiments demonstrate DiPGrasp’s efficacy:

• Real-hardware trials: On 50 segmented objects, success rates reach 60% for Barrett and 48% for Schunk, outperforming ISF (40%) and DexGraspNet (22%) (Xu et al., 2024).
• Noise robustness: Gaussian noise on point clouds $(x_{j_i}, y_{j_i})$2 results in graceful degradation (Barrett: 67.7%$(x_{j_i}, y_{j_i})$337.5% valid; Schunk: 26.5%$(x_{j_i}, y_{j_i})$414.7% valid).
• Speed: Parallel GPU batch optimization enables throughput unachievable by existing planners.

Ablation studies confirm the necessity of the force-closure term for successful physical execution, and highlight collision-handling as critical for practical deployment.

6. Limitations and Prospective Directions

DiPGrasp’s energy landscape is highly non-convex; therefore, final grasp success depends on the diversity and quality of initializations. Current GPU memory limits batch size (∼80 poses/8 GB), and the method struggles with extremely smooth or thin objects.

Future directions include:

• Learnable sample policies to bias toward high-reward regions of the object surface.
• Adaptive hand–object weighting for tuned pinch/wrap functionals.
• Integration into fully differentiable manipulation pipelines, bridging perception, planning, and control in an end-to-end framework.
• In-loop physics-based simulation and contact modeling (Xu et al., 2024).

7. Relation to Other Differentiable Grasp Planners

DiPGrasp builds on the principles established in differentiable grasp planning for high-DOF hands (2002.01530), extending the approach with efficient parallel local search, direct collision barriers, and adaptivity to variable perception inputs (e.g., instance masks). Compared to earlier approaches based on global image-to-grasp supervised learning (2002.01530), DiPGrasp emphasizes direct geometric reasoning with explicit surface and force-closure metrics, trading off some global search breadth for executional speed and differentiability throughout the grasp-planning stack.

The DiPGrasp framework does not target end-to-end interaction prediction (push+grasp) in clutter, as in DIPN+GN (Huang et al., 2020), but provides a general, methodologically unified basis for high-throughput grasp synthesis across diverse hand kinematics and object sets.