FineGrasp: Reliable 6-DoF Grasping
- FineGrasp is a semantic 6-DoF grasping method designed to reliably detect grasps for small, delicate regions even in cluttered scenes.
- It employs three key modifications—instance-normalized graspness labels, multi-range attention, and surface-normal priors—to improve pose regression and targeting.
- The approach integrates a simulated dataset, SimGraspNet, with mixed sim-to-real training to counter sensor noise and enhance real-world performance.
Searching arXiv for FineGrasp and closely related grasping papers to ground the article in current literature. Using the arXiv search tool now. FineGrasp is a semantic 6-DoF grasping method designed to make robot grasp detection substantially more reliable on small, delicate, and narrow grasp regions, particularly in cluttered scenes where standard semantic grasp pipelines often fail even when semantic localization is correct. It targets 6-DoF grasp detection for parallel-jaw grippers from a single-view RGB-D input, and it is presented as an enhancement of the EconomicGrasp supervision paradigm through three coordinated modifications: instance-normalized graspness labels, Multi-range Attention, and surface-normal priors, together with a new simulated dataset, SimGraspNet, and mixed sim-to-real training (Du et al., 8 Jul 2025).
1. Problem formulation and motivating failure mode
FineGrasp is situated in the common real-world setup in which a vision-LLM or segmentation system first identifies the target object or object part, and then a point-cloud grasp detector predicts feasible grasps only inside that region. The method retains this semantic grasping objective but focuses on the cases in which existing pipelines break down: small objects, thin parts, narrow handles, and other delicate regions. In those settings, the grasp detector often fails to produce any good grasp pose for the semantically correct region, so the entire pipeline can fail despite correct semantic grounding (Du et al., 8 Jul 2025).
A grasp is represented as
where is the grasp center, is the gripper width, and are Euler angles describing orientation. The paper attributes the failure of prior methods such as GSNet, Scale Balanced Grasp, and EconomicGrasp on small-scale grasps to scale imbalance in grasp supervision and point sampling. In particular, the standard graspness normalization used in GSNet applies a global scene-level scaling, so small-object grasp points can be crushed toward low scores; when only a small number of seed points are sampled for later grasp regression, the tiny object may not survive the proposal stage. This is especially damaging in semantic grasping because the semantic mask can be correct while no usable grasp candidate remains inside it.
A common misconception is that semantic localization is the main difficulty in semantic grasping. FineGrasp argues for a different bottleneck: the point-cloud grasp detector itself can be systematically biased toward larger and easier grasp regions, which turns semantic correctness into execution failure.
2. Pipeline structure and system-level design
FineGrasp keeps the common two-stage grasping structure but modifies the entire training and inference pipeline to better preserve fine-grained grasp opportunities. Its operational pipeline is described as three steps: semantic localization, in which a VLM is used to ground the target object or part; grasp detection, in which FineGrasp generates grasp poses for the entire scene using a point-cloud-based detector; and mask filtering, in which only grasp candidates lying inside the semantic mask are retained (Du et al., 8 Jul 2025).
Compared with prior semantic grasping systems, FineGrasp differs in four specific ways. First, it does not rely on a segmentation network during grasp inference to balance object sampling. Second, it introduces instance-normalized graspness labels so small objects are not suppressed during seed selection. Third, it adds architectural changes—Multi-range Attention and surface-normal priors—to improve pose regression for delicate regions. Fourth, it introduces a simulated grasp dataset and mixed sim-to-real training to compensate for missing or noisy depth on small objects.
The system is therefore not a new end-to-end language-grounded segmentation model. Its intervention point is the grasp detector and its supervision. This suggests that FineGrasp is best understood as a reliability-oriented refinement of semantic grasp execution rather than a replacement for the semantic grounding front end.
3. Core technical components
The first and most important modification is instance-normalized graspness. FineGrasp argues that scene-level normalization causes small-object graspness to be globally suppressed, and therefore performs intra-instance normalization first and then scene-level normalization. Conceptually, the first normalization makes graspable regions within each object category visible to the learner, while the second keeps the final score distribution compatible with scene-level point sampling. The paper presents this as removing the need for an auxiliary object segmentation network at inference, unlike Scale Balanced Grasp (Du et al., 8 Jul 2025).
The second modification is Multi-range Attention. After the backbone and seed sampling, the point cloud features are arranged as . For radius ranges, cylinder grouping yields
This multi-range tensor is processed by a Transformer encoder,
and adaptive fusion combines the range-specific features,
The stated purpose is to capture dependencies across receptive-field scales and to learn which scale is most useful for each point. Because delicate objects are highly sensitive to local geometry and object size, the paper emphasizes that the “right” receptive field depends strongly on object size and local geometry.
The third modification is the use of surface normals as an approaching prior. The paper observes that good grasps are not uniformly distributed over views and that, in GraspNet-1Billion, high-quality grasps often align with the surface normal. It therefore computes a normal vector for each point from neighboring points,
and defines the point feature as 0. The intended effect is a more accurate estimate of grasp orientation in narrow regions where approach direction matters disproportionately.
Taken together, these modifications address three different error sources: label imbalance at the supervision level, scale-sensitive feature aggregation at the representation level, and orientation ambiguity at the pose-regression level.
4. SimGraspNet and mixed sim-to-real training
FineGrasp introduces SimGraspNet, a simulated grasp dataset built in Isaac Sim. The simulation uses a virtual RGB-D camera mounted on the robot wrist and follows a trajectory of 256 viewpoints on a quarter-sphere, mirroring the GraspNet-1Billion capture setup. For each viewpoint, the simulation records RGB-D images, segmentation masks, camera poses, intrinsics, and metadata. The dataset is built from the same 40 objects used in the GraspNet-1Billion training set, and scenes are generated by randomly selecting a subset of objects, dropping them from varying heights and randomized initial poses so they collide naturally with the table and with one another, producing about 400 cluttered scenes (Du et al., 8 Jul 2025).
Ground-truth grasp labels are generated using the same methodology as GraspNet-1Billion. The paper frames the value of SimGraspNet in two ways: it provides additional cluttered scenes, and it exposes the model to object arrangements that may be underrepresented in real data, especially small objects that are otherwise affected by depth sensor noise or missed by the real capture pipeline. For sim-to-real correction, FineGrasp uses Gaussian shift on simulated depth to mimic real sensor noise, plus an off-the-shelf depth restoration method at test time.
The mixed training regime combines real GraspNet-1Billion data with SimGraspNet. The implementation details reported are PyTorch on 8 Nvidia 4090 GPUs, Adam optimizer, batch size 4 per GPU, initial learning rate 0.001, cosine decay with linear warm-up, and 10 epochs to convergence, with training taking about 2 hours. The paper does not specify a complex domain adaptation loss; instead, it treats simulation as a useful augmentation of the real dataset rather than a replacement.
This training design addresses another common misconception: that simulation alone is sufficient for delicate grasping. FineGrasp reports that simulation-only training gives reasonable performance, but mixed SimGraspNet and GraspNet-1B training improves results further.
5. Benchmarks, ablations, and reported performance
FineGrasp is evaluated primarily on GraspNet-1Billion, which the paper describes as containing 190 real cluttered scenes, with 100 scenes for training and the official test split divided into Seen, Similar, and Novel. The RealSense split is used for better depth quality. The official metric is average precision on the top 50 grasp poses after non-maximum suppression. Compared baselines include GraspNet-Baseline, Scale Balanced Grasp, HGGD, GSNet, AnyGrasp, and EconomicGrasp (Du et al., 8 Jul 2025).
On the GraspNet-1Billion RealSense split, FineGrasp outperforms all compared baselines. The headline numbers are summarized below.
| Setting | Reported result | Comparator |
|---|---|---|
| FineGrasp | Seen 71.67, Similar 62.83, Novel 27.40, Average 53.97 | EconomicGrasp average 51.63 |
| FineGrasp + Collision Detection | Seen 73.71, Similar 64.56, Novel 28.14, Average 55.47 | FineGrasp average 53.97 |
| Real-world delicate-object grasping | 91% success, 100% completion | GSNet 69% success, 20% completion; EconomicGrasp 78% success, 40% completion |
| Object grounding | 86% success | EconomicGrasp 73% |
| Part grounding | 84% success | GSNet 56% |
The ablations isolate the contribution of each component on small objects. The baseline EconomicGrasp-like setup gives Small 11.47. Adding Instance-Norm Graspness yields Small 14.74; MRA only gives Small 14.33; Normal Prior only gives Small 13.01; ING + MRA gives Small 14.63; ING + NP gives Small 15.40; ING + MRA + NP gives Small 16.23; and all three plus SimGraspNet gives Small 18.87. The paper’s interpretation is explicit: ING most directly fixes the label imbalance and gives the largest isolated boost on small objects; MRA improves scale-sensitive feature aggregation across object sizes; Normal Prior improves orientation prediction, with particularly noticeable gains on Novel objects; and SimGraspNet gives a further substantial gain on small objects by compensating for real depth sensor noise and undercoverage of tiny instances.
A separate MRA ablation compares the proposed fusion with the Scale Balanced Grasp fusion strategy: MsCG yields Small 15.87, Medium 50.52, Large 55.51, Average 40.63, while MRA yields Small 16.23, Medium 50.67, Large 56.92, Average 41.27. In the simulation dataset ablation, using EconomicGrasp as the fixed model, training on GraspNet-1B gives Seen 68.13, Similar 59.94, Novel 25.32; training on SimGraspNet gives Seen 59.35, Similar 56.74, Novel 23.68; SimGraspNet + Sim2real gives Seen 62.47, Similar 58.41, Novel 24.16; and SimGraspNet + GraspNet-1B gives Seen 71.22, Similar 64.24, Novel 26.12.
The paper emphasizes that the gains are especially meaningful because the method is designed for delicate objects but still improves general benchmark performance. In the real robot experiment, the setup consists of a 7-DoF Franka FR3, a parallel-jaw gripper, a RealSense D435i, and 18 small objects arranged into 5 scenes. The reported completion-rate improvement is central: FineGrasp clears all five scenes, whereas the baselines often get stuck after repeated failures.
6. Scope, limitations, and relation to adjacent research
FineGrasp’s stated limitation is that it still depends on accurate semantic grounding from a VLM in the first stage; if the mask is wrong, the grasp filtering step can still fail. The authors also state that future work should extend simulation to more small and articulated objects to improve generalization. In addition, although the method removes the need for segmentation during grasp inference, it still relies on a strong point-cloud grasp detector and benefits from collision detection at test time (Du et al., 8 Jul 2025).
Within the broader grasping literature, FineGrasp occupies a specific niche. It is a semantic 6-DoF grasp detector for parallel-jaw grippers in cluttered scenes, optimized for delicate grasp regions. By contrast, “DenseTact-Mini” addresses grasping multi-scale objects from flat surfaces through a compact optical tactile sensor with a soft, rounded, smooth gel surface, an attachable synthetic fingernail, and three distinct grasping strategies—tap grasping, fingernail grasping, and fingertip grasping—with a global lifting success rate exceeding 90.2% (Do et al., 2023). “GrainGrasp” addresses dexterous grasp generation from object point clouds only by predicting five separate fingertip contact maps and optimizing a hand pose with DCoG, emphasizing fingertip-specific coordination rather than parallel-jaw grasp detection (Zhao et al., 2024). “TacRefineNet” addresses the grasp execution stage rather than grasp proposal generation, using tactile-only feedback for arbitrary in-hand pose refinement and reporting millimeter-level grasp accuracy using only tactile input (Wang et al., 30 Sep 2025). “SemGrasp” addresses static human grasp generation aligned with linguistic intent through a discrete representation with orientation, manner, and refinement tokens and a grasp-text-aligned dataset, CapGrasp (Li et al., 2024).
A related supervision-level contrast appears in “Hybrid Physical Metric For 6-DoF Grasp Pose Detection,” which argues that force closure alone produces coarse confidence scores and introduces a hybrid score based on force closure, flatness, gravity-center relation, and collision perturbation for FGC-GraspNet (Lu et al., 2022). FineGrasp’s intervention is different: it refines graspness normalization and seed preservation for small semantic targets rather than redefining the physical confidence score itself.
The practical significance of FineGrasp lies in the failure mode it isolates. The method argues that delicate grasping is not merely a matter of better regression; it requires better supervision balance, scale-aware feature aggregation, orientation priors, and training data that includes the failure modes of real sensors. In that sense, FineGrasp is less a general theory of grasping than a targeted answer to a recurrent systems problem in semantic manipulation: the correct object or part is identified, but the grasp detector discards the only grasps that matter.