UniFucGrasp: Unified Functional Grasp Annotation

• UniFucGrasp is a human-hand-inspired framework that enables functional, task-aligned grasp synthesis across diverse robotic hand morphologies.
• It integrates RGB-D human hand capture with biomimetic pose reconstruction and unified human-to-robot mapping to generate stable and human-like grasps.
• The framework is validated on a UFG dataset with over 100K annotations and a unified CVAE synthesis model that supports both fully actuated and underactuated hands.

UniFucGrasp is a human-hand-inspired framework for functional dexterous grasp annotation and synthesis across heterogeneous robotic hands. It combines RGB-D-based human hand capture, biomimetic hand-pose reconstruction, a unified human-to-robot mapping, coupling-aware actuation conversion, and geometry-based force-closure validation to produce functional, stable, and human-like grasps for multiple hand morphologies. The accompanying UFG dataset contains over 100K functional grasp annotations for 1,108 objects from 21 categories across ShadowHand, InspireHand, and HnuHand, and the paper further provides a unified CVAE-based synthesis model as a dataset-validation baseline (Lin et al., 5 Aug 2025).

1. Functional grasping and the multi-hand problem

UniFucGrasp is formulated around the distinction between stable grasping and functional grasping. In the paper’s framing, a stable grasp concerns whether an object can be held securely, whereas a functional grasp must additionally place the hand in a task-appropriate configuration, such as grasping a hammer by the handle for use, positioning fingers on a drill trigger, or grasping a mug by the handle (Lin et al., 5 Aug 2025). This emphasis places UniFucGrasp in the task-oriented branch of dexterous grasping rather than in purely stability-oriented grasp synthesis.

The method is motivated by two limitations in earlier dexterous-grasp pipelines. First, many existing datasets emphasize grasp stability rather than downstream-use alignment. Second, large-scale functional-grasp resources have generally been tied to ShadowHand, which the paper characterizes as fully actuated, high-DOF, expensive, and relatively difficult to control (Lin et al., 5 Aug 2025). This produces both an annotation-cost bottleneck and a generalization bottleneck for lower-cost underactuated hands.

UniFucGrasp addresses these issues by using the human hand as an intermediate representation. Natural human hand motion is captured, converted into a human joint representation, mapped into robot-specific joint space, converted into actuator commands according to the target hand’s coupling structure, and finally validated through force closure and simulation-based checks (Lin et al., 5 Aug 2025). The paper’s unification claim is therefore morphological rather than purely architectural: the same annotation strategy is intended to support fully actuated and underactuated hands within a single framework.

This positions UniFucGrasp differently from universal dexterous control systems such as UniDexGrasp and UniGraspTransformer, which focus on low-level universal grasp execution across many objects, and differently from multi-hand functional pipelines such as DexGrasp-Diffusion, which focus on functional grasp synthesis conditioned on affordance instructions (Xu et al., 2023, Wang et al., 2024, Zhang et al., 2024). UniFucGrasp is centered on annotation strategy and dataset construction, with an overview model serving primarily to validate the utility of the resulting data (Lin et al., 5 Aug 2025).

2. Human-hand-inspired annotation pipeline

The annotation pipeline has four stages: human hand keypoint capture, human hand kinematic reconstruction, human-to-robot mapping, and robot actuation conversion with force-closure validation (Lin et al., 5 Aug 2025). The required inputs are an object mesh, a robot hand URDF, and 21 3D human hand keypoints from an RGB-D camera. The output is a robotic grasp specified by hand pose and articulation, written in the paper using rotation $R$, translation $T$, and joint configuration $J$ or $Q$ depending on context (Lin et al., 5 Aug 2025).

Human hand keypoints are detected with MediaPipe as 21 2D points and projected into 3D using depth and camera intrinsics. These 3D points are then registered to a biomimetic hand model. A specific design choice is to replace the wrist keypoint with the palm center to stabilize palm-normal estimation (Lin et al., 5 Aug 2025). The palm normal is computed as

$$\mathbf{n}_{\text{Palm}} = (\mathbf{p}_{\text{Ring}}-\mathbf{p}_{\text{Palm}}) \times (\mathbf{p}_{\text{Index}}-\mathbf{p}_{\text{Palm}}).$$

For finger $i$, the vector from joint $n$ to joint $n+1$ is denoted $\mathbf{q}_{i,n}^{\mathrm{HH}}$. The MCP vector is projected onto the palm plane to form $\mathbf{n}_{i,1}^{\mathrm{HH}}$. The abduction-adduction angle is then

$T$0

and the flexion-extension angle is

$T$1

These computations produce a human joint-angle vector $T$2 (Lin et al., 5 Aug 2025).

The paper also references a K2J module for keypoint-to-joint-angle conversion. It is functionally part of the pipeline, but the architecture is not specified in detail in the provided material (Lin et al., 5 Aug 2025). A plausible implication is that UniFucGrasp treats human-pose reconstruction as a practical intermediate step rather than as the principal learning contribution.

3. Unified mapping across robotic hand morphologies

The core mathematical abstraction of UniFucGrasp is a two-stage mapping: from human joint space to robot joint space via a mapping matrix $T$3, and from robot joint space to actuator space via a coupling matrix $T$4 (Lin et al., 5 Aug 2025).

For each finger/joint, the human-to-robot relation is modeled as

$T$5

where $T$6 is hand-specific and $T$7 is an error term. The paper defines the dimension of $T$8 according to human and robot DoF: $T$9 This allows square mappings for equal-DoF hands and rectangular mappings for compression or expansion when morphologies differ (Lin et al., 5 Aug 2025).

For ShadowHand, the paper states $J$0, so $J$1 is a diagonal matrix mainly used to scale joint angles. For InspireHand, $J$2 and $J$3, hence

$J$4

The full matrix is decomposed fingerwise as

$J$5

For the InspireHand index finger, the mapping is explicitly written as

$J$6

The reported calibrated coefficients are

$J$7

$J$8

These were estimated from 60 sets of finger-joint data collected from 6 volunteers in a human–robot finger-alignment setup (Lin et al., 5 Aug 2025).

The second stage maps desired robot joint angles to actuator commands: $J$9 For InspireHand, $Q$0 and is sparse, with nonzero structure determined by the mechanical coupling of the hand (Lin et al., 5 Aug 2025). This separation of $Q$1 and $Q$2 is what allows UniFucGrasp to claim support for both fully actuated and underactuated hands. A plausible implication is that the method is unified at the representation level, but still requires hand-specific engineering in the form of calibrated $Q$3 and measured $Q$4.

The paper defines the angle prediction error as

$Q$5

The formula is given, but no numerical value for this error is reported in the provided text (Lin et al., 5 Aug 2025).

4. Force-closure validation and grasp quality constraints

UniFucGrasp does not treat retargeted human poses as sufficient by themselves. Mechanical validity is enforced through geometry-based force closure, along with force feedback and collision detection in simulation during dataset construction (Lin et al., 5 Aug 2025).

For a contact point $Q$6 with surface normal $Q$7 and friction coefficient $Q$8, the friction-cone half-angle is

$Q$9

Choosing a vector $$\mathbf{n}_{\text{Palm}} = (\mathbf{p}_{\text{Ring}}-\mathbf{p}_{\text{Palm}}) \times (\mathbf{p}_{\text{Index}}-\mathbf{p}_{\text{Palm}}).$$0 not parallel to $$\mathbf{n}_{\text{Palm}} = (\mathbf{p}_{\text{Ring}}-\mathbf{p}_{\text{Palm}}) \times (\mathbf{p}_{\text{Index}}-\mathbf{p}_{\text{Palm}}).$$1, the tangent basis is

$$\mathbf{n}_{\text{Palm}} = (\mathbf{p}_{\text{Ring}}-\mathbf{p}_{\text{Palm}}) \times (\mathbf{p}_{\text{Index}}-\mathbf{p}_{\text{Palm}}).$$2

The $$\mathbf{n}_{\text{Palm}} = (\mathbf{p}_{\text{Ring}}-\mathbf{p}_{\text{Palm}}) \times (\mathbf{p}_{\text{Index}}-\mathbf{p}_{\text{Palm}}).$$3-th discretized friction-cone direction is

$$\mathbf{n}_{\text{Palm}} = (\mathbf{p}_{\text{Ring}}-\mathbf{p}_{\text{Palm}}) \times (\mathbf{p}_{\text{Index}}-\mathbf{p}_{\text{Palm}}).$$4

where

$$\mathbf{n}_{\text{Palm}} = (\mathbf{p}_{\text{Ring}}-\mathbf{p}_{\text{Palm}}) \times (\mathbf{p}_{\text{Index}}-\mathbf{p}_{\text{Palm}}).$$5

Thus each contact contributes six force directions (Lin et al., 5 Aug 2025).

The corresponding wrench is

$$\mathbf{n}_{\text{Palm}} = (\mathbf{p}_{\text{Ring}}-\mathbf{p}_{\text{Palm}}) \times (\mathbf{p}_{\text{Index}}-\mathbf{p}_{\text{Palm}}).$$6

and for $$\mathbf{n}_{\text{Palm}} = (\mathbf{p}_{\text{Ring}}-\mathbf{p}_{\text{Palm}}) \times (\mathbf{p}_{\text{Index}}-\mathbf{p}_{\text{Palm}}).$$7 contact points the grasp matrix is

$$\mathbf{n}_{\text{Palm}} = (\mathbf{p}_{\text{Ring}}-\mathbf{p}_{\text{Palm}}) \times (\mathbf{p}_{\text{Index}}-\mathbf{p}_{\text{Palm}}).$$8

The force-closure criterion is then whether the origin lies inside the convex hull of the wrench vectors (Lin et al., 5 Aug 2025).

This validation stage is significant because UniFucGrasp’s functional emphasis could otherwise bias annotations toward semantically plausible but mechanically weak grasps. The pipeline instead combines three constraints: human-likeness from human capture, functionality from naturally task-aligned human grasps, and stability from force closure and simulation checks (Lin et al., 5 Aug 2025). The paper does not define a single scalar objective combining these factors, so the balance is procedural rather than end-to-end optimized.

5. UFG dataset

The UFG dataset is presented as the first multi-hand functional grasp dataset. It contains 1,108 objects from 21 daily-use categories and over 100K functional grasp annotations, with over 70 validated functional grasp demonstrations per object. The supported hands are ShadowHand, InspireHand, and HnuHand (Lin et al., 5 Aug 2025).

Component	Value	Notes
Objects	1,108	Daily-use objects
Categories	21	Daily-use categories
Annotations	100K+	Functional grasps
Hands	3	ShadowHand, InspireHand, HnuHand
Split	8.5:1.5	Train:test

The dataset is constructed in MuJoCo using robotic-hand control driven by tracked natural hand motions. In the comparison table described in the paper, UniFucGrasp is identified as supporting stable and functional grasps, RGBD observations, real-sim collection context, human hand mapping, and generalization across diverse hands (Lin et al., 5 Aug 2025). The annotations include functional grasp poses represented by hand rotation, hand translation, and hand articulation, together with category labels. The paper does not specify an explicit symbolic task vocabulary beyond the functional grasp framing itself.

The practical significance of UFG lies in its cross-hand scope. Earlier large-scale functional dexterous datasets such as DexVLG / DexGraspNet 3.0 and DexFuncGrasp are described as ShadowHand-specific (Lin et al., 5 Aug 2025). UniFucGrasp therefore shifts the data question from “How can one collect functional grasps for one high-DOF hand?” to “How can one annotate functional grasp structure once and map it across multiple hand embodiments?” This suggests a dataset philosophy closer to canonical retargeting than to hand-specific grasp optimization.

6. Unified synthesis model, experiments, and limitations

To validate the dataset, the paper introduces an end-to-end functional gesture generation model trained jointly across multiple hands. The model takes a hand point cloud of size $$\mathbf{n}_{\text{Palm}} = (\mathbf{p}_{\text{Ring}}-\mathbf{p}_{\text{Palm}}) \times (\mathbf{p}_{\text{Index}}-\mathbf{p}_{\text{Palm}}).$$9 and an object point cloud of size $i$0 as input, encodes them separately with DGCNN, fuses them with a lightweight Transformer inspired by DCP using 4 heads and feedforward hidden dimension 128, and feeds the fused representation into a CVAE encoder (Lin et al., 5 Aug 2025). A latent vector $i$1 is sampled and concatenated with max-pooled hand and object features to form a 260-dimensional joint representation, from which the network predicts hand rotation $i$2, translation $i$3, and joint angles $i$4 (Lin et al., 5 Aug 2025).

The training loss is

$i$5

with

$i$6

$i$7

and

$i$8

The model is trained with Adam at learning rate $i$9 for 15 epochs on 2 NVIDIA RTX 3090 GPUs (Lin et al., 5 Aug 2025).

The main baseline is DexFuncGrasp. In terms of efficiency, DexFuncGrasp uses separate models per hand, totaling 11.67M parameters and 11.9 h training time, whereas the unified UniFucGrasp model uses 11.2M parameters and 7.82 h training time for three hands jointly (Lin et al., 5 Aug 2025). In IsaacSim, predicted grasps are considered successful if the object is grasped and lifted by 10 cm. The reported success rates are:

Category	UniFucGrasp	DexFuncGrasp
Bottle	72.72%	68.62%
Drill	62.50%	55.00%
Spraybottle	64.28%	58.07%
Flashlight	77.77%	91.03%
Mug	57.14%	54.62%
Total	68.74%	66.86%

On these five categories, the total simulation success rate improves from 66.86% to 68.74% (Lin et al., 5 Aug 2025). In real-world evaluation on a UR5 with InspireHand, RealSense, FoundationPose, a calibration board, ArUco codes, and a FreeScan X3 scanner, the reported results are 29/50 successes for UniFucGrasp versus 26/50 for DexFuncGrasp, corresponding to 58% versus 52% overall (Lin et al., 5 Aug 2025). The paper emphasizes gains on drill and spray bottle, where precise functional finger placement is particularly important.

Several limitations are explicit or implicit. The framework is unified, but each new hand still requires a calibrated $n$0 and a defined or measured $n$1. The real-world evaluation is limited to five categories with ten trials each. The paper does not provide formal ablations isolating the effects of force closure, human-inspired mapping, or multi-hand joint training. It also does not report a zero-shot hand-transfer benchmark in the strict sense, even though it argues for efficient cross-hand generalization (Lin et al., 5 Aug 2025). This suggests that UniFucGrasp’s strongest contribution is not a universally transferable controller, but a cross-morphology annotation and data-construction framework that makes multi-hand functional grasp learning more tractable.

In the broader literature, this makes UniFucGrasp complementary to universal dexterous control methods and to affordance-conditioned grasp generators. A plausible interpretation is that UniFucGrasp supplies the missing data layer for multi-hand functional grasping: human-derived, force-closure-validated, morphology-aware annotations that are not confined to a single expensive dexterous hand (Lin et al., 5 Aug 2025).