DemoGrasp: Demonstration-Based Robotic Grasping

• DemoGrasp is a family of robotic grasping methods that use one or few demonstrations as an inductive bias instead of large annotated datasets.
• It spans various formulations—from single-demonstration visual grasp detection and few-shot RGB-D reconstruction to one-step dexterous trajectory editing.
• The methods show high efficiency in controlled settings with improvements via augmentation, reinforcement learning, and semantic conditioning, while noting limitations in open-world generalization.

DemoGrasp is a demonstration-centered family of robotic grasping methods in which grasp policies, grasp poses, or grasp-edit operators are derived from one or a few demonstrations rather than from large annotated datasets or long-horizon trial-and-error. In the cited literature, the term refers to multiple distinct formulations: a single-demonstration visual grasp detector trained from one centered image and deployed through closed-loop alignment (Molle et al., 2018), a few-shot RGB-D pipeline that reconstructs human–object interaction and transfers the demonstrated hand–object relation to a robot gripper (Wang et al., 2021), and a universal dexterous grasping framework that edits a single demonstration trajectory in a one-step reinforcement-learning formulation (Yuan et al., 26 Sep 2025). Closely related extensions include functional grasping with explicit style–affordance conditioning (Mao et al., 15 Dec 2025) and zero-shot functional grasping from generated human demonstrations (Tang et al., 8 Apr 2026). Earlier work on sampling grasps from demonstrations established a complementary model-free lineage in which a handful of demonstrated poses define the modes of a learned grasp affordance density (Zech et al., 2016).

1. Scope, terminology, and research lineage

The literature does not use “DemoGrasp” for one canonical algorithm. Instead, it denotes a broader methodological commitment: demonstrations provide the inductive bias that substitutes for exhaustive grasp labels, dense reward engineering, or object-specific geometric modeling. A common misconception is that all DemoGrasp systems perform the same task under the same assumptions. In fact, the cited works differ substantially in sensory input, action representation, embodiment, and deployment regime. Some are per-object few-shot systems tied to a demonstration workspace, while others use one demonstration only as a seed and then optimize a universal policy across hundreds or thousands of objects (Molle et al., 2018, Wang et al., 2021, Yuan et al., 26 Sep 2025).

Earlier demonstration-driven grasp learning already exhibited two properties that reappear in later DemoGrasp variants: multimodal grasp representations and direct use of demonstration structure rather than object geometry. “Grasp Learning by Sampling from Demonstration” models grasps as 6D gripper poses in $SE(3)$, represents the target affordance density as $\pi(g)\propto \mu_{\mathrm{GWS}}(g)$, and combines MCMC Kameleon with Generalized Darting Monte Carlo so that demonstrated grasps act as mode centers for exploration (Zech et al., 2016). That work requires only the object pose, not the object mesh, and therefore occupies an important antecedent position in the DemoGrasp lineage.

Formulation	Core representation	Main output
“Learning to Grasp from a Single Demonstration” (Molle et al., 2018)	Binary classifier over RGB crops, applied fully convolutionally	2D activation map for closed-loop planar grasp alignment
“DemoGrasp: Few-Shot Learning for Robotic Grasping with Human Demonstration” (Wang et al., 2021)	RGB-D hand–object reconstruction, object completion, relative pose transfer	Robot gripper goal pose on Toyota HSR
“DemoGrasp: Universal Dexterous Grasping from a Single Demonstration” (Yuan et al., 26 Sep 2025)	One-step MDP over wrist and hand-joint edits to a demonstration trajectory	Universal dexterous grasping policy
“Universal Dexterous Functional Grasping via Demonstration-Editing Reinforcement Learning” (Mao et al., 15 Dec 2025)	Style–affordance-conditioned one-step editor	Functional grasp satisfying semantic condition
“Grasp as You Dream” (Tang et al., 8 Apr 2026)	Generated human demonstrations plus embodiment-specific retargeting	Zero-shot functional grasp plan

This progression suggests a shift from demonstration as a localized supervision signal, to demonstration as a reusable geometric template, and finally to demonstration as a structured prior for universal policy optimization.

2. Single-demonstration visual grasp localization

“Learning to Grasp from a Single Demonstration” formulates grasping as binary classification over image crops rather than explicit regression of a grasp rectangle or 6D pose (Molle et al., 2018). A single 640×480 demonstration frame is recorded with the target object centered, and 128×128 crops are extracted for training. Positive samples are center crops with small random rotation uniformly in $[-3^\circ,+3^\circ]$; negative samples are any other crop and rotation. Brightness and contrast are perturbed uniformly in $[0.5,1.5]$. In the “multi” setting, positive crops from other demonstrations are reused as negatives for the current target.

The network, GraspNet, is intentionally compact: four convolutional layers with filter sizes $5\times5$ and stride $2$, channel progression $8,8,16,16$, average pooling after the last convolutional layer, and two “fully connected” layers implemented as $1\times1$ convolutions with 16 hidden units. Hidden layers use ReLU activations, and the final sigmoid neuron outputs a grasp-success probability for the input crop. When evaluated fully convolutionally on a full image, the network produces a 2D activation map whose argmax indicates the most likely target-object location.

Orientation is not regressed by the network. Instead, the image is evaluated at a batch of discrete rotations; the rotation with the highest maximum activation is selected, and the robot rotates until the unrotated image again yields the highest activation. The controller computes a direction vector from the image center to the activation-map argmax and uses that vector to drive planar motion. Once the argmax is centered, the arm moves down and closes the gripper. The system assumes the same workspace and object type as the demonstration, a grasp perpendicular to the workspace plane, and object positional offsets up to approximately 8 cm from the demo position.

Training uses Adam with learning rate 0.001, batch size 64, and 512 iterations for both “single” and “multi” settings. Reptile meta-learning, following Nichol and Schulman’s algorithm, is used to learn initialization parameters $\phi$ over demonstrations, with update

$\phi \leftarrow \phi + \epsilon (W-\phi).$

The initialization is trained on nine demonstrations for 250 outer iterations, with outer step size 0.6 linearly annealed to 0, meta-batches over four tasks, and 10 inner iterations per task. Fine-tuning on a new demonstration uses 200 iterations, batch size 64, and Adam at 0.001.

The reported evaluation highlights the constrained but effective nature of the approach. Testing used 5,000 positives and 5,000 negatives per block. “Multi” generally improves or matches “single,” and “Reptile” converges faster and sometimes attains the best final accuracy. Many blocks achieve at least 0.99 accuracy with “multi,” and Reptile reaches $\pi(g)\propto \mu_{\mathrm{GWS}}(g)$0 on Block 10. On the Franka Panda platform, closed-loop grasping is robust without rotation handling. With batched rotations, orientation detection works, but approximately 700 ms processing latency on an Intel NUC hinders precise closed-loop control. The method therefore demonstrates that a single centered frame, aggressive augmentation, and a lightweight CNN can suffice under strong geometric and workspace constraints, but it does not establish broad object-class or open-world generalization.

3. Few-shot RGB-D reconstruction and human-to-robot pose transfer

“DemoGrasp: Few-Shot Learning for Robotic Grasping with Human Demonstration” addresses a different problem: converting a short human RGB-D interaction sequence into a reusable grasp skill for a service robot (Wang et al., 2021). The input is a 10–60 s RGB-D sequence, typically 100–400 frames, captured from Toyota’s Human Support Robot head camera while a person manipulates the object from multiple viewpoints. The method seeks to avoid both object CAD requirements and large annotated grasp datasets by reconstructing the demonstrated hand–object relation directly.

The pipeline first segments hand and object with Mask R-CNN trained on ObMan synthetic data. Class-specific depth pixels are back-projected to 3D and fused into a shared TSDF volume with KinectFusion, while camera tracking is maintained via frame-to-model ICP. Because the hand contributes geometric structure, the shared TSDF stabilizes tracking for simple household objects. After fusion, hand and object volumes are separated, yielding two TSDFs.

The object TSDF is then completed with a 3D U-Net operating on a $\pi(g)\propto \mu_{\mathrm{GWS}}(g)$1 TSDF grid. The network encodes geometry and concatenated RGB features into a 512-d latent representation and decodes an occupancy grid, from which a watertight mesh is extracted by marching cubes. Occupancy prediction uses class-averaged focal loss with $\pi(g)\propto \mu_{\mathrm{GWS}}(g)$2:

$\pi(g)\propto \mu_{\mathrm{GWS}}(g)$3

Hand reconstruction is obtained from the hand branch of Hasson et al. (2019), which predicts MANO parameters from RGB. The resulting MANO mesh is refined by ICP against the fused hand point cloud via

$\pi(g)\propto \mu_{\mathrm{GWS}}(g)$4

The central transferable quantity is the relative hand–object transform from the demonstration,

$\pi(g)\propto \mu_{\mathrm{GWS}}(g)$5

At inference, the current scene object pose $\pi(g)\propto \mu_{\mathrm{GWS}}(g)$6 is estimated by matching PPF-FoldNet descriptors between the completed demonstration object mesh and the current scene point cloud, followed by RANSAC and ICP refinement. The gripper goal in the robot base frame is then composed as

$\pi(g)\propto \mu_{\mathrm{GWS}}(g)$7

The hand-to-gripper mapping uses the transformed thumb tip, index fingertip, and wrist. The grasp center is $\pi(g)\propto \mu_{\mathrm{GWS}}(g)$8, the approach direction is $\pi(g)\propto \mu_{\mathrm{GWS}}(g)$9, and the gripper frame is constructed from $[-3^\circ,+3^\circ]$0 and the thumb–index spread vector.

The reported performance underscores the value of interaction-based transfer. In simulation, on drill, hole punch, cookie box, and shampoo bottle, the method achieves mean success of 91.7% over 15 trials per object. Removing shape completion reduces success to 85.0%, with a larger effect on small or heavily occluded objects. Against 6DoF-GraspNet, DemoGrasp attains higher mean success, 91.7% versus 76.7%, while planning one pose per trial rather than evaluating many candidates. On real HSR experiments with cup, can, bottle, and remote control, 72 runs yield 94.4% success for side grasps, 58.3% for top grasps, and 76.4% overall. Using only 50% of demonstration frames lowers average success to 68.3%, and using 30% lowers it to 46.7%, indicating that demo completeness is critical for reliable reconstruction and alignment. Transparent objects, symmetric top views, and poor depth remain major failure modes.

4. Universal dexterous grasping as one-step demonstration editing

“DemoGrasp: Universal Dexterous Grasping from a Single Demonstration” reformulates dexterous grasp learning around a single successful demonstration trajectory and learns how to edit that trajectory for new objects and poses (Yuan et al., 26 Sep 2025). The key change is conceptual: instead of optimizing long-horizon arm–hand control directly, the policy makes a single decision about where and how to modify the demonstration. The state-based simulation observation is

$[-3^\circ,+3^\circ]$1

where $[-3^\circ,+3^\circ]$2 is the initial end-effector pose, $[-3^\circ,+3^\circ]$3 is the initial object pose, and $[-3^\circ,+3^\circ]$4 is a 512-point object point cloud. The action is

$[-3^\circ,+3^\circ]$5

with $[-3^\circ,+3^\circ]$6 bounded in $[-3^\circ,+3^\circ]$7 m, $[-3^\circ,+3^\circ]$8 bounded in $[-3^\circ,+3^\circ]$9 rad, and $[0.5,1.5]$0 bounded in $[0.5,1.5]$1 rad per joint.

The environment is single-step: it edits the demonstration trajectory once, replays the edited motion, and returns a terminal reward

$[0.5,1.5]$2

Success means the object is lifted at least 10 cm and the average distance between object center and hand keypoints is less than 12 cm at the end. Robot–table collision is assessed by penetration of hand keypoints into the table. To enable thin-object grasps, collisions are randomly disabled in half of the parallel environments, so the expected reward becomes 1 for collision-free success, 0.5 for success with hand–table contact in allowed environments, and 0 for failure. The objective is

$[0.5,1.5]$3

Trajectory editing separates wrist placement from hand shaping. Before lift, the edited end-effector pose in object coordinates is

$[0.5,1.5]$4

followed by a fixed vertical offset at lift. Hand joints are modified by interpolating from the demonstrated open hand toward a grasp configuration shifted by $[0.5,1.5]$5:

$[0.5,1.5]$6

This preserves the temporal structure of the demonstration while allowing object-specific adjustment of closure and contact configuration.

The state-based policy encodes the point cloud with PointNet into a 128-d feature and concatenates it with end-effector and object pose information. Actor and critic are MLPs with hidden sizes $[0.5,1.5]$7, ELU activations, and tanh outputs rescaled to the allowed edit ranges. PPO is trained in 7,000 IsaacGym environments, with learning rate $[0.5,1.5]$8, clip parameter 0.2, episode length 1, and about 24 hours of training on a single NVIDIA RTX 4090.

Empirically, the formulation is designed for universality rather than per-object adaptation. On DexGraspNet with the Shadow Hand, state-based success is 95.2% on train objects, 95.5% on seen-category unseen shapes, and 94.4% on unseen-category objects. The corresponding vision-based policy reaches 92.2%, 92.3%, and 90.1%. Compared with UniDexGrasp, UniDexGrasp++, and UniGraspTransformer, the paper reports approximately 4–16% absolute gains depending on modality. In cross-embodiment experiments trained on only 175 objects, the average success rate on six unseen object datasets is 84.6%. Real-world deployment with FR3+Inspire and two RealSense D435i cameras grasps 110 unseen objects with 86.5% overall success, including 95.3% on normal-sized objects, 68.3% on flat/thin objects under 1.5 cm, and 76.7% on small objects under 3.5 cm. Ablations show that editing wrist rotation is especially important: naive demo replay achieves 75.29%/73.43% train/test, while the full action space $[0.5,1.5]$9 reaches 96.24%/82.74%.

5. Functional, language-guided, and generated-demonstration extensions

DemoGrasp’s 2025 formulation is extended in “Universal Dexterous Functional Grasping via Demonstration-Editing Reinforcement Learning,” which introduces explicit semantic conditioning on grasp style and affordance (Mao et al., 15 Dec 2025). The condition space is factorized as $5\times5$0, where $5\times5$1 denotes grasping style and $5\times5$2 denotes affordance. Style is represented by a one-hot label, a canonical joint target $5\times5$3, and a style-specific contact mask; affordance is represented by a 3D point $5\times5$4 on the object surface, or by its 2D image projection for RGB policies. The one-step action becomes $5\times5$5, where $5\times5$6 edits the wrist trajectory, $5\times5$7 is a residual joint offset, and $5\times5$8 is a scalar style scaling factor. Style adaptation uses

$5\times5$9

The terminal reward balances success, affordance alignment, and style fidelity:

$2$0

The components are defined as

$2$1

$2$2

and

$2$3

Evaluation adds functional metrics absent from basic grasp-success reporting: Success Affordance Distance (SAD), Style Accuracy (SA), Style Diversity (SD), and Grasp Success Rate (GSR). On 1,000 simulated environments, mean SAD improves from 6.29 cm for DemoGrasp to 3.03 cm for the style–affordance-conditioned method, with seen categories improving from 6.27 to 3.02 and unseen categories from 6.20 to 3.21. In state-based ablations, the full model attains 77.04% GSR, 3.02 cm SAD, and 94.74% SA. Real-world vision-based evaluation reports approximately 71% success with human-provided affordances and 64.4% with VLM-provided affordances. The same work integrates a VLM planner, Embodied-R1, to ground language instructions into affordance points.

A different extension appears in “Grasp as You Dream,” which uses visual generative models to synthesize human functional-grasp demonstrations and then retargets them to a robot embodiment (Tang et al., 8 Apr 2026). Inputs are an RGB-D observation $2$4, a language instruction $2$5, and a URDF description $2$6; the output is a grasp plan $2$7 with $2$8. The pipeline has three stages: human demonstration generation by a VGM conditioned on a task–object–part decomposition, hand motion estimation and optimization using HaMeR, MANO, ICP-based alignment, and rendering consistency, and human-to-robot retargeting using VLM-predicted grasp taxonomy and finger mapping. Taxonomy-aware retargeting minimizes

$2$9

with temporal smoothness and contact-refinement terms added during optimization.

The reported results position generated demonstrations as an alternative source of DemoGrasp priors rather than a replacement for grasp execution. On TaskGrasp, object generalization yields 78.6% Success, 86.1% Part Identification Accuracy, and 3.6 Intent; task generalization yields 79.5%, 85.2%, and 3.8. On a DexGraspNet subset, Shadow Hand achieves 80.2/85.7/3.4 on kitchenware and 83.1/87.2/3.6 on tool tasks, while Allegro attains 78.5/84.3/3.3 and 82.6/85.4/3.5. Real-world evaluation reports 76/100 successful grasps for a Robotiq 2F-85 gripper and 73/100 for an Allegro Hand. This extension makes the role of demonstration more abstract: the demonstration need not be recorded from the robot at all, but can be synthesized from internet-scale human priors and later retargeted.

6. Common assumptions, limitations, and adjacent directions

Across these formulations, demonstrations do not eliminate inductive assumptions; they relocate them. The 2018 visual system assumes the same workspace and object type as the demonstration, perpendicular grasps, and small positional offsets. The 2021 RGB-D transfer system assumes that the object can be reconstructed well enough for TSDF fusion, shape completion, and PPF-based registration; transparent bottles, short demos, and symmetric top views reduce reliability. The 2025 dexterous system assumes known object pose and point cloud in state-based RL, and its one-step editor is open-loop during the RL stage, with finer closed-loop behavior appearing only after vision-based imitation. DemoFunGrasp adds semantic conditioning, but centimeter-scale affordance precision, VLM grounding errors, and lack of tactile sensing remain limiting factors. GraspDreamer depends on VGM and VLM priors, and its failure modes include hallucinated contacts, residual scale ambiguity, morphology mismatches, and degraded long-horizon consistency.

Another common misconception is that demonstration-driven grasping is inherently synonymous with functional grasping. The record is more nuanced. Early methods often optimize pickup success or activation quality rather than semantic suitability for downstream use. Functional grasping becomes explicit only when the condition space or the generated demonstration encodes where and how the object should be grasped for a task, as in style–affordance factorization or task–object–part prompting (Mao et al., 15 Dec 2025, Tang et al., 8 Apr 2026). Stable lifting and functional intent therefore remain distinct objectives.

Adjacent work on deformable-object grasping highlights a further boundary of current DemoGrasp formulations. DefGraspSim models 3D deformable objects with a GPU-based corotational FEM, predicts deformation, stress, strain energy, and stability, and identifies pre-grasp features such as pure_dist, perp_dist, num_contacts, edge_dist, squeeze_dist, gripper_sep, and grav_align that correlate with grasp performance (Huang et al., 2021). This suggests a possible offline evaluation layer for DemoGrasp-like methods when the target domain includes fruits, soft packaging, hollow containers, or other objects for which damage proxies such as max von Mises stress or max displacement matter alongside pickup success.

Taken together, the DemoGrasp literature marks a shift from grasp synthesis as a purely geometric or large-scale data problem toward grasp synthesis as structured reuse of demonstrated interaction. The exact structure varies—from binary crop classification, to explicit hand–object transform transfer, to one-step trajectory editing, to semantic factorization and VGM-generated demonstrations—but the unifying principle is consistent: demonstrations define a low-entropy grasp prior that can be amplified by augmentation, geometry reconstruction, reinforcement learning, or retargeting rather than replaced by exhaustive supervision.