HUG-BENCH Robotic Grasping Benchmark

• HUG-BENCH is a standardized benchmarking framework for dexterous, multi-fingered robotic grasping using 90 real-world 3D object meshes across diverse geometric categories.
• It employs a fixed pre-grasp, grasp, and lift protocol in both simulation and real-robot experiments, assessing performance via success rate and fingertip contact error.
• The benchmark facilitates sim-to-real transfer and reproducibility by providing detailed object taxonomy, collision definitions, and integration guidelines with various robotic systems.

HUG-BENCH is a standardized benchmarking framework and dataset for evaluating dexterous, multi-fingered robotic grasping methods, with a particular focus on generalization to real-world, diverse household objects. It consists of 90 metric-scale 3D mesh reconstructions of everyday items, organized by geometric category and size. The benchmark protocol encompasses both simulated and real-robot environments, enabling zero-shot and sim-to-real evaluation of data-driven robotic grasping methods on previously unseen objects using physical robot hands and a variety of camera modalities (Wu et al., 15 Jun 2026).

1. Dataset Composition and Object Taxonomy

HUG-BENCH comprises 90 everyday objects, each reconstructed as watertight triangle meshes at real-world metric scale. Objects span five geometric categories—cylindrical (e.g., glue stick), spheroidal (e.g., strawberry), prismatic (e.g., eraser), appendaged (e.g., nail clipper), and amorphous (e.g., rubber duck)—and are further partitioned across three size bins (small, medium, large). In the test split, each category contains two objects per size bin.

Key object attributes include measured mass and volume for each mesh (e.g., rubber duck: 10.8 g, 41.6 cm³; storage bin: 237.2 g, 2632 cm³). Objects are treated as rigid bodies, with no explicit modeling of material properties.

Meshes are provided as watertight triangle meshes (OBJ or PLY formats) for rendering and as convex decomposition parts for collision (URDF and MuJoCo MJCF entries), all in meter units and with gravity-aligned coordinate frames (z-up), resting on a table at $z=0$.

Category	Example	Test Split Objects (per size)
Cylindrical	Glue stick, bottle	2 small, 2 medium, 2 large
Spheroidal	Strawberry, hacky sack	2 small, 2 medium, 2 large
Prismatic	Eraser, storage bin	2 small, 2 medium, 2 large
Appendaged	Nail clipper, picnic basket	2 small, 2 medium, 2 large
Amorphous	Rubber duck, tape dispenser	2 small, 2 medium, 2 large

For evaluation, the 90 objects are partitioned into 60 validation and 30 test set objects, stratified such that each category-size combination is represented in both splits. There is no separate training set of objects; training uses the 1M-HUGs dataset of human grasp demonstrations.

2. Simulation Protocol and Robotic Embodiments

Simulations are conducted in MuJoCo using a timestep of approximately 2 ms. Scenes involve a single object resting on a table under gravity (9.81 m/s²), with no other contacts present. The default hand embodiment is a position-controlled MANO hand model (6 DoF free wrist, 15 finger joints), with forward-skin capsules representing phalanges and a convex palm.

The evaluation rollout follows a fixed "pre-grasp → grasp → lift" sequence:

• Pre-grasp: The wrist is offset by +3 cm along the object normal axes, with fingers fully open.
• Grasp: The system linearly interpolates the wrist to the predicted target pose and closes each joint to the predicted MANO angle, adding a small extra flexion.
• Lift: The object is lifted straight up by 0.5 m under gravity.

This protocol is matched in both simulation and real-world experiments for consistent comparison.

3. Success Metrics and Evaluation Criteria

Performance is assessed via two primary metrics:

• Success Rate (SR):

$$SR = \frac{1}{N} \sum_{i=1}^N \mathbf{1}[\text{object is held off the table after lift}]$$

where $N$ is the number of trials per object. SR quantifies the frequency with which the robot successfully lifts an object.

• Fingertip Contact Error (FC):

$$FC = \frac{1}{2} \left( \lvert d_\text{thumb} \rvert + \min_{f\in F} | d_f | \right)$$

where $d_\text{thumb}$ and $d_f$ are signed distances from thumb/non-thumb fingertip surfaces to the object surface, measured in millimeters. Lower FC indicates closer alignment of the predicted grasp's fingertip positions to the object's surface.

Each test and validation object is evaluated with 10 random seeds (i.e., different initial poses and conditions).

4. Real-Robot and Sim-to-Real Evaluation

Evaluation is conducted both in simulation and on physical robots. The 30-object test set is used for both modalities. In real-world trials:

• Tabletop: Uses a 7-DoF xArm, 6-DoF Ability Hand, and ZED stereo camera, with controlled lighting/background. A single camera position is used, and the object is placed on a clear table with minor random jitter.
• In-the-wild: Utilizes a YOR mobile manipulator, 20-DoF WUJI Hand, and Aria Gen 2 stereo smart glasses. Objects are scattered across home environments (kitchen, bedroom, living room, office), with one unique view per trial in natural or indoor lighting and clutter.

Simulation SR serves as an upper bound, reflecting both model quality and mesh fidelity. Sim-to-real transfer is measured by comparing the simulation SR to the real-world SR on identical objects.

5. Usage, Integration, and Customization

HUG-BENCH assets (mesh files and supporting code) are downloadable via https://grasping.io. The directory structure separates test and validation objects and includes all mesh formats, collision definitions (URDF, MJCF), and per-object metadata in JSON (mass, volume, category, size bin).

To load assets in simulation, the provided MuJoCo XML files can be instantiated via the MuJoCo Python API:

from mujoco_py import MjSim, load_model_from_path model = load_model_from_path("hug-bench/test/storage_bin/mesh.xml") sim = MjSim(model)

Standardized execution steps for a grasp model:

1. Read RGB, depth, and camera intrinsics from the JSON.
2. Convert depth to point cloud; compute 3D query point via back-projection.
3. Run HUG model prediction: grasp_state = [t_x, t_y, t_z, R6D_wrist(6), R6D_fingers(15×6)].
4. Retarget grasp to robot-specific joint angles (Ability Hand via AnyTeleop retargeting; WUJI Hand via WujiHand retargeting).
5. Execute open-loop grasp and lift in MuJoCo or on hardware.
6. Determine success by checking whether the object is lifted off the table.

To add new objects, recordings from Aria Gen 2 can be reconstructed with the aria2mesh tool, producing mesh and metadata in the required format. New cameras can be incorporated by supplying updated intrinsics/extrinsics; model retraining is not required.

6. Baseline Performance and Comparative Results

Baseline results are reported on the 30-object test split using tabletop (xArm+Ability+ZED) and in-the-wild (YOR+WUJI+Aria Gen 2) setups:

• Dex1B (1B simulation-trained demos): SR = 43.7%
• Contact-Anchored Policies (CAP): SR = 32.7%
• HUG (zero-shot from human data): SR = 66.7% (tabletop); 62.0% (in-the-wild)

The HUG model exceeds Dex1B and CAP baselines by +23 and +34 percentage points, respectively, on tabletop trials. In simulation with the MANO hand, HUG achieves 73.0% SR, while a human-grasp oracle attains 94.0%. Ablation studies reveal substantial drops in SR without the 3D loss (32.7%) and with unimodal inputs (PC-only: 70.7%; RGB-only: 29.7%). Fingertip Contact Error for HUG is 14.6 mm; the oracle exhibits 7.4 mm.

Method	Tabletop SR	In-the-wild SR	Sim SR
HUG	66.7%	62.0%	73.0%
Dex1B	43.7%	N/A	N/A
CAP	32.7%	N/A	N/A
Human Oracle (sim)	N/A	N/A	94.0%

This suggests robust sim-to-real transfer and data-driven generalization across unseen objects and environments (Wu et al., 15 Jun 2026).

7. Broader Significance and Applications

HUG-BENCH provides a standardized infrastructure for benchmarking dexterous, multi-fingered grasping on real-world objects, enabling meaningful comparison across methodological advances. Its coverage of geometric diversity, stratified size bins, and integration with real-robot and simulation modalities addresses key limitations in prior evaluation protocols. The protocol enables systematic assessment of model robustness, sim-to-real transfer, and generalization for both academic research and practical system development. The benchmark’s design—metric-scale meshes, paired simulation/robot assets, detailed per-object metadata, and extensible architecture—facilitates reproducibility and adaptation to new domains or hardware platforms.