HUGS: Guiding Unified Dexterous Grasp Synthesis Across Modes and Scales via Learned Human Priors

Abstract: Dexterous grasping across diverse object scales requires contact modes ranging from two-finger pinches to bimanual grasps. Existing dexterous grasp synthesis methods reduce the high-dimensional optimization space with manually designed expected contacts and initialization heuristics, which struggle to balance synthesis success rate and diversity. We present HUGS (Human-prior-guided Unified Dexterous Grasp Synthesis), a human-prior-guided framework for unified dexterous grasp synthesis across modes and scales. Instead of directly retargeting human demonstrations, HUGS learns an object-conditioned human prior that captures human grasp preferences and guides downstream force-closure-aware optimization. The prior is trained on a compact self-collected human grasp dataset with 1.8K grasps over 304 objects, providing broad coverage of object scales and contact modes. During synthesis, HUGS adaptively proposes contact modes and wrist initializations, substantially improving the balance between contact-mode coverage and synthesis success rate over heuristic-based methods. With HUGS, we synthesize 3.2M robotic grasps over 157K scenes, spanning object half-diagonal lengths from 2 cm to 30 cm and modes from two-finger to bimanual grasps. Models trained on the synthesized dataset autonomously select appropriate contact modes in the real world, enabling grasping from screws to large boxes.

• The paper introduces a unified framework that combines learned human priors with force-closure optimization to generate diverse and stable robotic grasps.
• The approach utilizes a comprehensive human grasp dataset to capture multiple contact modes and wrist poses, enhancing scalability across object sizes.
• The study validates the method through extensive simulations and real-world experiments, achieving improved precision in grasp mode prediction and synthesis success.

HUGS: Human-Prior-Guided Unified Dexterous Grasp Synthesis Across Modes and Scales

Introduction and Motivation

Dexterous robotic grasping requires robust handling across variable object scales and contact modes, encompassing fine-grained two-finger pinches to complex bimanual manipulations. Previous methods have been constrained either by heuristic, manually crafted rules that limit generalizability and diversity, or by the direct retargeting of human demonstrations, which limits scalability and adaptation to robot-specific physical constraints. The HUGS (Human-prior-Guided Unified Grasp Synthesis) framework proposes a paradigm shift: rather than direct demonstration retargeting, it leverages a learned object-conditioned human prior to guide robot grasp synthesis, factoring in both human-preferred contact modes and wrist poses for diverse and scalable data generation.

Figure 1: HUGS leverages an object-conditioned human prior to predict both specialized contact modes and wrist initializations, guiding force-closure-aware optimization for diverse and stable grasps from two-finger to bimanual strategies.

Compact Human Grasp Dataset and Prior Learning

The HUGS-Human dataset is central to this framework, comprising approximately 1.8K manually collected human grasps over 304 canonical objects, systematically annotated for contact mode and wrist pose. Unlike previous datasets narrowly focused on single-hand postures or lacking in cross-scale diversity, HUGS-Human emphasizes coverage along both contact mode (from sparse to full/bimanual contacts) and object scaling axes. The dataset supports four contact modes: Single-Two, Single-Three, Single-Full, and Both-Full (bimanual), with strong evidence that individual objects frequently permit more than one viable strategy.

Figure 2: HUGS-Human dataset analysis: broad object-scale coverage compared to prior datasets, diverse contact modes, and multiple unique grasps per object.

The human prior is trained as a generative model, predicting a distribution over both contact modes and wrist initializations for a given object. Object geometry is encoded using sparse 3D convolutional backbones, and supervision is provided by empirical distributions from the dataset. The wrist-pose head implements diffusion-based generation, conditioned on both object features and the mode embedding.

Figure 3: HUGS learns from geometry, wrist poses, and contact modes; during synthesis, the prior proposes contact-mode/wrist-pose distributions, which are transferred and optimized for robotic execution.

Human-Prior-Guided Grasp Synthesis Pipeline

The synthesis process begins by rescaling the object point cloud to account for embodiment differences between human and robotic hands. The learned prior predicts contact mode probabilities and samples wrist initializations for each candidate mode, establishing a “synthesis budget” that concentrates optimization on human-plausible modes.

Subsequent optimization combines both global bimanual and per-hand force-closure objectives, supporting stable synthesis across modes and object scales. The combinatorial search space of kinematic and contact state is thus made tractable by the hierarchical application of the human prior (for coarse guidance) and robot-specific force closure optimization (for physical feasibility and adaptation).

Figure 4: Object-specific, cross-scale allocation of synthesis effort using human priors leads to both higher grasp synthesis success rates and improved diversity, compared to heuristic or scale-only allocation.

Empirical Results and Numerical Analysis

Extensive simulation experiments on the Shadow Hand and LEAP Hand evaluate contact-mode allocation, synthesis success, and pose diversity. Quantitative comparison shows that the human prior significantly outperforms scalar-scale or heuristic rules in predicting valid mode distributions and wrist initialization regions, with lower KL divergence and higher precision/recall on held-out objects.

Practically, over 3.2M robot grasps are synthesized across 157K scenes, spanning object half-diagonals from 2 cm to 30 cm. Synthesis success rates are consistently higher than all heuristic baselines, with notably stronger performance in challenging object-scale bins and when mode diversity is required for the same object.

Figure 5: Grasp generation networks distilled from HUGS-synthesized data demonstrate higher contact-mode prediction accuracy and grasp robustness compared to heuristic-based supervision.

Furthermore, grasp generators trained on HUGS-generated datasets (rather than heuristic synthetic data) yield superior precision-recall curves for predicting contact-mode availability (F1=0.934). Grasp generation success rates stay above 70% for the majority of object scales, although performance lags on very small and large objects, indicating persistent representational challenges.

Qualitative and real-world robotic experiments on LEAP Hands validate cross-scale and multi-mode deployment. The system dynamically and correctly adapts contact modes and approach styles in unconstrained as well as environment-constrained settings, from executing top-down single-hand pinches on small screws to coordinated bimanual grasps for large, heavy objects.

Figure 6: Real-world demonstrations on LEAP Hands: cross-scale grasps and adaptive mode selection under challenging deployment constraints.

Failure Analysis and Limitations

Despite the advantages conferred by the learned human prior, several failure modes are observed:

• Functional awareness gaps: The current optimization lacks explicit modeling of task functionality, leading to geometrically plausible but weak grasps.
• Mode misprediction: The prior occasionally assigns modes infeasible for extreme object scales.
• Physical overlap and penetration: Particularly for bimanual grasps, prior-predicted wrist poses may produce overlap or penetration between hands or with objects, partially mitigated via downstream optimization.

Figure 7: Failure cases of the pipeline include functionally weak grasps, infeasible mode assignment, and hand/object overlap. Robot-specific optimization mitigates but does not eliminate these issues.

In real-world settings, perceptual noise in calibration, segmentation, and point cloud fusion is the predominant bottleneck, especially for small or irregularly shaped objects.

Theoretical and Practical Implications

The HUGS framework empirically validates the hypothesis that high-level, object-conditioned human priors can serve as scalable guides for synthesis of robotic grasp datasets, eliminating the need for exhaustive demonstration retargeting. Adaptivity (geometry-aware mode/wrist-pose proposal), scalability (generalization to unseen object geometries), and robot-specific physical plausibility (force-closure-aware optimization) are effectively decoupled.

Practically, HUGS provides a path toward rapidly generating high-diversity, physically robust grasp datasets suited to a wide array of robot hands, including those with embodiment and scale differences from humans. The resulting datasets can be used to train online grasp generators that generalize across modes and scales and allow for adaptive decision-making under deployment constraints.

Theoretically, this work lays the groundwork for future research to expand both the contact mode taxonomy (to cover more subtle human strategies) and the scale/embodiment range, potentially leveraging larger, more diverse egocentric datasets. Integration with functionally- and affordance-aware optimization and end-to-end closed-loop grasp execution remains a promising research direction.

Conclusion

The HUGS framework represents a comprehensive system for scalable, unified, and adaptive dexterous grasp synthesis, leveraging object-conditioned human priors for guiding optimization across modes and scales. The synthesis of a large, diverse dataset and demonstration of cross-scale, multi-modal grasping both in simulation and on real robots confirms the effectiveness of the approach. Persistent challenges, notably in perception, functional grasp awareness, and generator robustness at the extremes of scale and shape, set a rich agenda for future work.