UltraDexGrasp: Universal Bimanual Grasping

• UltraDexGrasp is a framework that defines universal dexterous grasping by enabling dual-arm, multi-strategy grasp selection for varied objects.
• It integrates optimization-based grasp synthesis with planning-based demonstration generation and a synthetic 20M-frame dataset.
• The framework’s point-cloud policy, trained on extensive synthetic data, achieves over 80% success in both simulation and real-world trials.

UltraDexGrasp is a framework for universal dexterous grasping with bimanual robots that couples optimization-based grasp synthesis, planning-based demonstration generation, and point-cloud policy learning for a dual-arm, dual-hand platform. It is formulated around two 6-DoF UR5e arms and two 12-DoF XHand dexterous hands, and targets robust grasp acquisition across objects with varied shapes, sizes, and weights by selecting among two-finger pinch, three-finger tripod, whole-hand, and bimanual grasp strategies. The framework produces UltraDexGrasp-20M, a synthetic dataset comprising 20 million frames across 1,000 objects, and trains a point-cloud policy with unidirectional attention that is reported to achieve 84.0% average success in simulation and 81.2% average success in real-world universal dexterous grasping under zero-shot sim-to-real transfer (Yang et al., 5 Mar 2026).

1. Scope, embodiment, and grasp formulation

UltraDexGrasp defines “universal dexterous grasping” as a bimanual setting in which a robot autonomously chooses and executes different grasp strategies according to object geometry and physical requirements. The intended strategy set is explicitly multi-modal: two-finger pinch is used for tiny objects, three-finger tripod for small or medium objects, whole-hand grasp for medium objects, and bimanual grasp for objects that are large or heavy. The framework is presented for arbitrary rigid objects and emphasizes grasps that are physically plausible, geometrically conforming, and able to resist external wrenches (Yang et al., 5 Mar 2026).

For the bimanual system, a grasp pose is written as

$$g = \left\{ \left( \boldsymbol{t}_h, \boldsymbol{R}_h, \boldsymbol{q}_h \right) \mid h = 0, 1 \right\},$$

where $h$ indexes the two hands, $\boldsymbol{t}_h \in \mathbb{R}^3$ is hand translation, $\boldsymbol{R}_h \in SO(3)$ is hand rotation, and $\boldsymbol{q}_h$ denotes hand joint positions. Contact is modeled with a hard-finger, point-on-plane formulation. The friction cone is

$$\mathcal{F} = \left\{ \boldsymbol{f} \;\middle|\; \|\boldsymbol{f}_{\mathrm{tan}}\| \leq \mu \|\boldsymbol{f}_{\mathrm{n}}\|,\;\; f_z \geq 0 \right\},$$

and the contact wrench and grasp map are defined through

$$\boldsymbol{w}_i = \begin{bmatrix} \boldsymbol{f}_i \ \alpha (\boldsymbol{d}_i \times \boldsymbol{f}_i) \end{bmatrix}, \qquad \boldsymbol{G}_i = \begin{bmatrix} \mathbf{I}_{3 \times 3} \ (\boldsymbol{p}_i - \boldsymbol{m})_{\times} \end{bmatrix} \boldsymbol{O}_i.$$

The corresponding grasp wrench space is

$$\mathcal{W} := \left\{ \boldsymbol{w} \;\middle|\; \boldsymbol{w} = \sum_{i=1}^k \boldsymbol{G}_i \boldsymbol{f}_i,~ \boldsymbol{f}_i \in \mathcal{F}_i,\;\; i=1,\ldots,k \right\}.$$

In this formulation, grasp quality is tied directly to whether the selected contacts can realize target force and torque patterns while remaining compatible with friction, kinematics, and collision constraints (Yang et al., 5 Mar 2026).

2. Optimization-based grasp synthesis

The grasp synthesis stage operates on an object mesh $\boldsymbol{M}$ with center of mass $\boldsymbol{m}$, candidate contact sets for both hands, and a strategy-specific active contact subset. The decision variables are the bimanual grasp pose

$h$0

and the contact forces $h$1. The optimization objective is

$h$2

subject to joint limits, friction-cone constraints, and $h$3. The first term matches realizable wrenches to target wrenches, the second pushes contacts toward the object surface through signed distance, and the remaining terms penalize hand-object and hand-hand collision (Yang et al., 5 Mar 2026).

The framework treats this optimization as a bilevel problem, following the same broad decomposition used in BODex, where a lower-level quadratic program solves for contact forces and an upper-level process optimizes hand pose (Chen et al., 2024). In UltraDexGrasp, strategy specialization is encoded by the active contact set $h$4: two-finger pinch uses a small fingertip subset, tripod uses three fingertip contacts, whole-hand uses more distributed contacts, and bimanual grasps distribute contacts across both hands (Yang et al., 5 Mar 2026).

Candidate generation is geometry-driven. For unimanual strategies, a point is sampled on the object convex hull and the hand is initialized with palm facing the surface; for bimanual strategies, two points are sampled on roughly opposite sides of the object. For each object, the system generates about 500 candidate grasps through this optimization process, filters them by physical validation and inverse-kinematics feasibility, and ranks the remaining grasps by an $h$5 distance to the current end-effector poses:

$h$6

with

$h$7

The closest feasible grasp is used as the preferred grasp for demonstration generation (Yang et al., 5 Mar 2026).

3. Demonstration generation and UltraDexGrasp-20M

UltraDexGrasp does not stop at static grasps. Each preferred grasp is converted into a full demonstration trajectory with four phases: pregrasp, grasp, squeeze, and lift. The pregrasp target is offset by 0.1 m along the negative palm normal to create a collision-avoiding approach pose; the lift stage raises the object by 0.2 m. Bimanual motion planning is then used to synthesize collision-free coordinated trajectories for both arms, and adjacent steps with negligible motion are merged to reduce hesitation in the demonstrations. Execution in simulation uses PD control (Yang et al., 5 Mar 2026).

Trajectory admission is explicitly physical. A trajectory is marked successful only if the object is lifted at least 0.17 m above its initial pose and remains lifted for at least 1 second without dropping or large unexpected motion. Successful trajectories are stored together with joint trajectories, sensor data, and rendered point clouds. A notable design choice is that the depth point cloud is augmented with an imaged robot point cloud generated from joint states in both simulation and real deployment, so the robot geometry is presented in a consistent form across domains (Yang et al., 5 Mar 2026).

The resulting dataset, UltraDexGrasp-20M, comprises 20 million frames across 1,000 objects from DexGraspNet and is presented as the first large-scale multi-strategy dexterous grasp dataset for bimanual robots. The data-generation pipeline itself attains an average success rate of 68.5%, which becomes an important reference point for policy scaling experiments. The coverage of the dataset is tied to the four grasp strategies and to objects with broad variation in size and mass, and this suggests that the dataset is intended as a foundation for closed-loop grasp acquisition rather than a repository of isolated grasp poses (Yang et al., 5 Mar 2026).

4. Policy architecture and training from synthetic data

The learned policy takes a scene point cloud as input and predicts low-level control commands. The point cloud is sampled to 2,048 points by farthest point sampling and encoded by PointNet++ with two set abstraction layers. The first layer keeps 2,048 points and aggregates local neighborhoods with $h$8 nearest neighbors using 1×1 convolutions, batch normalization, ReLU, and max pooling. The second layer downsamples to 256 points and produces the point features consumed by the transformer backbone (Yang et al., 5 Mar 2026).

The transformer is decoder-only and uses unidirectional attention. The architecture distinguishes between point tokens, which represent the encoded scene geometry, and learnable action query tokens, which aggregate scene information to support action prediction. The critical asymmetry is that action queries attend to point tokens without perturbing the point representation. The policy head predicts a bounded Gaussian distribution over actions using a truncated normal parameterization, and training minimizes the negative log-likelihood of the demonstration action under that distribution:

$h$9

This choice makes the policy probabilistic rather than purely regressive (Yang et al., 5 Mar 2026).

Training is performed exclusively on synthetic data from UltraDexGrasp-20M. The paper reports that performance increases monotonically with data scale and that, once the number of training frames exceeds 1M, the learned policy surpasses the average success rate of the demonstration generator itself. With the full dataset, the same policy is used across the different grasp strategies, and no explicit strategy label is required at inference time; strategy selection emerges from geometry and demonstration coverage. This suggests that the principal inductive bias lies in the multi-strategy data distribution and in the attention mechanism rather than in explicit modularization (Yang et al., 5 Mar 2026).

5. Empirical performance in simulation and on hardware

Evaluation in simulation uses 600 objects spanning seen and unseen categories, with object weights from 5 g to 1,000 g. Object scale varies from cases with longest bounding-box edge below 0.03 m to cases with shortest edge above 0.5 m. Each object is tested over 10 trials. Compared with DP3 and DexGraspNet, UltraDexGrasp reports substantially higher success across small, medium, and large objects, with DexGraspNet not applicable to large objects because it does not handle bimanual grasps (Yang et al., 5 Mar 2026).

Setting	Baseline(s)	UltraDexGrasp
Simulation average	DP3 46.7%; DexGraspNet 58.8%	84.0%
Real-world average	DP3 46.7%; DexGraspNet 62.3%	81.2%

The detailed simulation results show 78.8% on seen small objects, 84.3% on seen medium objects, and 90.4% on seen large objects; on unseen objects the corresponding numbers are 76.9%, 85.8%, and 87.5%. The overall average is 84.0%, versus 46.7% for DP3 and 58.8% for DexGraspNet on the subset where DexGraspNet applies. These results indicate that the policy generalizes across object categories without a large seen-to-unseen collapse (Yang et al., 5 Mar 2026).

Real-world evaluation is performed zero-shot on 25 objects using two Azure Kinect DK cameras and the same dual-UR5e, dual-XHand platform. The reported success rates are 72.0% for small objects, 82.2% for medium objects, and 89.3% for large objects, yielding an average of 81.2%. The tested objects range from 18 cm$\boldsymbol{t}_h \in \mathbb{R}^3$0 to 26,400 cm$\boldsymbol{t}_h \in \mathbb{R}^3$1 in volume and from 3.6 g to 1,095 g in weight. Qualitatively, the policy is described as switching between tripod or pinch for small objects, whole-hand grasp for medium objects, and bimanual grasp for large or heavy items (Yang et al., 5 Mar 2026).

Two ablations isolate the main architectural decisions. Removing distribution prediction and replacing it with direct regression reduces success from 84.0% to 73.5%, while removing unidirectional attention reduces success to 68.2%. Together with the data-scaling curve, these ablations make clear that the reported performance depends jointly on demonstration scale, probabilistic action modeling, and the directional attention structure (Yang et al., 5 Mar 2026).

6. Position within recent dexterous grasping research

UltraDexGrasp sits at the intersection of large synthetic grasp corpora, analytic grasp generation, and universal policy learning. CrossDex studies one vision-based policy across four training dexterous hands and two unseen embodiments by using human-hand eigengrasps as a universal action space and fingertip-plus-palm geometry as a unified observation space, reporting 0.800 vision-based success on training hands and 0.352 on unseen hands (Yuan et al., 2024). $\boldsymbol{t}_h \in \mathbb{R}^3$2 Grasp instead models a dense hand-object interaction representation and reports an average simulation success rate of 87.53% in less than one second across Barrett, Allegro, and ShadowHand, with 89% average success on LeapHand in real-world experiments (Wei et al., 2024). UniGraspTransformer emphasizes offline distillation: 3.2M successful trajectories from 3,200 dedicated Shadow Hand teachers are distilled into a universal Transformer, reaching 88.9% vision-based success on seen objects and 86.8% on unseen categories (Wang et al., 2024). BODex provides the scalable bilevel optimization precedent for dexterous grasp synthesis, synthesizing over 49 grasps per second on a single 3090 GPU and producing datasets that improve learning-based success from around 40% to 80% in simulation (Chen et al., 2024).

Against this background, UltraDexGrasp is distinguished by three properties that appear together less often in earlier work: explicit bimanual embodiment, multi-strategy grasp selection within a single policy, and a data engine that integrates optimization-based grasp synthesis with planning-based demonstration generation (Yang et al., 5 Mar 2026). It is not framed as a cross-embodiment method in the sense of CrossDex or $\boldsymbol{t}_h \in \mathbb{R}^3$3 Grasp, and it is not centered on single-hand policy distillation in the style of UniGraspTransformer. Instead, it uses large-scale synthetic trajectories to learn closed-loop bimanual grasp acquisition from point clouds.

The framework is centered on rigid DexGraspNet objects and on trajectories composed of pregrasp, grasp, squeeze, and lift. This suggests a primary emphasis on grasp acquisition rather than post-grasp dexterous manipulation, even though the bimanual, multi-strategy structure makes it a plausible substrate for later extensions toward broader manipulation tasks.