A Cross-view Fusion Framework for Robust 6-DoF Grasp Pose Estimation

Abstract: In this paper, we propose a cross-view fusion framework that enhances the robustness of 6-DoF grasp pose estimation in corner views. Our framework alleviates occlusion by incorporating an auxiliary view and avoids the time-consuming, task-agnostic multi-view reconstruction through a post-fusion strategy. To enhance cross-view fusion, we propose a self-supervised contrastive learning strategy that leverages cross-view associations to regularize point cloud features. In brief, a cross-view point pair is considered a match if the two points correspond to the same 3D location, and a non-match if they represent distinct grasp directions. The learning strategy significantly enhances the spatial consistency and direction distinctiveness of point features, thereby facilitating cross-view fusion and improving estimation robustness. Furthermore, we propose a cross-view-aligned cylinder integration module to fuse grasp-relevant geometry into a comprehensive representation. Specifically, the module first aligns the cross-view points and features according to their similarity to enhance the robustness against noise. Subsequently, these points are registered into the cylindrical coordinate frame, emphasizing the rotation-symmetric geometry which is important for grasping. Finally, local self-attention and seed cross-attention layers are alternately employed, respectively enabling interactions within single views and across views, which supports fine-grained representation of grasp-relevant geometry. Our framework achieves strong performance on the GraspNet-1Billion benchmark and in real-world applications. Code is available at https://github.com/KJZhuAutomatic/Cross-view-Grasp.

• The paper introduces a targeted post-fusion strategy that enhances spatial fidelity and robustness in occluded grasp scenarios.
• It employs self-supervised contrastive learning to align cross-view features and improve directional discriminability for grasp planning.
• Empirical results show significant AP gains and a 96% real-world success rate, underscoring its practical advantages over prior methods.

Cross-View Fusion for Robust 6-DoF Grasp Pose Estimation

Introduction

This work presents a cross-view fusion framework designed to address the persistent problems of partial observation and occlusion in 6-DoF robotic grasp pose estimation, especially under challenging "corner-view" scenarios. Existing single-view and pre-fusion multi-view systems for grasp pose estimation suffer from insufficient scene geometry and computational inefficiency, respectively. The proposed architecture adopts a post-fusion strategy, selectively incorporating an auxiliary viewpoint to enhance the completeness and spatial fidelity of grasp-relevant regions, which is critical for robust grasp planning in practical applications.

Figure 1: The grasp pipeline: a wrist-mounted camera observes the workspace from multiple configurations; the framework fuses observations to enhance robustness with minimal time overhead.

Framework Overview

The core contribution consists of three synergistic components:

1. Cross-view Fusion Pipeline: Eschewing full-scene pre-reconstruction, the method introduces a post-fusion approach wherein only grasp-relevant regions are fused, thereby preserving high-resolution geometric details and significantly boosting efficiency.
2. Self-supervised Contrastive Learning: To bridge feature mismatches and spatial inconsistency across views, a contrastive regularization is employed. This loss promotes both spatial alignment (feature similarity for spatial matches) and direction discriminability (feature separation for distinct approach vectors on the same object), directly targeting the grasp-relevant context regularization bottleneck encountered in baseline architectures.
3. Cross-view-aligned Cylinder Integration: Taking inspiration from geometric registration methods, the framework aligns cross-view features in a localized cylindrical frame, exploiting rotation symmetry inherent to gripper kinematics. Alternating self-attention and seed cross-attention modules are deployed for efficient fusion and intra/inter-view context integration, all within the cylindrical coordinate embedding.

   Figure 2: Processing pipeline: reference/auxiliary point clouds are encoded, grasp seeds are predicted and grouped, and local geometric context is fused via similarity-based alignment and cylindrical embedding, with attention layers for cross-view context enrichment.

Methodology

Problem Formulation

Given a reference and auxiliary point cloud aligned via robot kinematics, the algorithm identifies grasp seeds from the reference, predicts potential approaching directions, and augments their descriptors using the auxiliary view. Grasp pose parameters—including approach vector, in-plane rotation, approach depth, and gripper width—are then regressed.

Cross-View Cylinder Fusion

Within a cylindrical region (parameterized by the grasp seed and approach direction), feature alignment is achieved by combining spatial proximity and semantic feature similarity, mitigating sensor noise and localization error. Features of corresponding points are averaged, and triple source sets (reference, auxiliary, overlap) are transformed into a cylindrical coordinate frame where in-plane rotation, grasp width, and depth are explicit, reducing learning burden.

Sequences of self-attention (intra-view) and cross-attention (inter-view) layers enable contextually rich and discriminative feature formation at the seed level.

Self-Supervised Contrastive Loss

Contrastive regularization comprises two components:

• Spatial Consistency: Enforces feature closeness between cross-view matches, counteracting drift in unobserved/corner regions.
• Direction Discriminability: Features of grasp candidates with similar spatial locations but distinct approach directions are separated by an adaptive margin, ensuring orientation-resolved feature encoding.

  Figure 3: Contrastive learning: left—features of corner regions are supervised by clear-region correspondences to improve consistency; right—features of different grasp approaches on the same object are separated to ensure discriminability.

Empirical Results

Benchmark Evaluation

Extensive experiments on GraspNet-1Billion demonstrate absolute AP gains of up to 3.55 (Seen), 0.23 (Similar), and 0.81 (Novel) over the best prior art (ZeroGrasp) on RealSense sequences; on Kinect, the margins are 1.61, 1.68, and 1.84 over EconomicGrasp, underscoring robust generalization across objects and sensor domains.

Figure 4: Qualitative comparison to GSNet. The proposed method predicts denser and higher-scoring 6-DoF grasps, demonstrating the improved robustness and accuracy from cross-view fusion.

Ablation and Analysis

Contrastive losses alone (without auxiliary view input) yield AP increases up to 5.56 (Seen split) over the baseline, confirming the crucial role of view association regularization. Adding the cross-view-aligned fusion module delivers further improvements (up to 4.72 AP). Notably, direct pre-fusion (full-scene merge) provides only marginal (<0.5 AP) benefit, highlighting the necessity of targeted, post-fusion design.

Feature map visualizations (t-SNE) reveal that contrastive learning not only fosters cross-view correspondence but also sharpens class boundaries for approach directions.

Figure 5: t-SNE visualization: (a) cross-view point clouds; (b) baseline features lack cross-view consistency; (c) spatially consistent, but not yet direction-discriminative; (d) full contrastive loss yields features simultaneously consistent and discriminative.

For the most challenging "corner views" (single-view AP < 40), cross-view fusion achieves gains averaging 28.11 AP, substantially exceeding the overall test-set average improvement.

Figure 6: AP improvements in challenging corner-view scenarios: average AP gain exceeds 28.11, confirming resilience to self-occlusion and partial observations.

Real-World Deployment

In physical one-shot grasp experiments (Dobot CR5, 12 unseen objects), the method achieves 96% success rate, outperforming leading baselines by 14–19%, while maintaining competitive runtime (4.6s per grasp, including cross-view acquisition and fusion). Grasp execution leverages a main-overhead and a lateral auxiliary view.

Figure 7: Real-world setup: articulated robot, RGB-D camera, and cluttered object configurations.

Figure 8: Execution pipeline: auxiliary/primary observation and grasp activation, with relevant point clouds boxed in red.

Practical and Theoretical Implications

The practical impact is clear—plug-and-play robustness against occlusion can be realized with minimal hardware cost (single auxiliary view) and without the latency and degradation of volumetric scene reconstruction. The approach is fully compatible with conventional pipeline modules (seed sampling, attention modules), facilitating integration into existing systems.

Theoretically, the explicit decoupling of grasp pose estimation into cross-view association and context fusion bridges geometric registration and robotic perception. Contrastive inter-view and inter-direction regularization provides a new avenue for robust scene understanding under partial observability, with potential applications in other perceptual tasks involving uncertainty and occlusion.

Future Directions

Potential directions include adapting the framework to variable and uncalibrated multi-view observation setups, leveraging active view planning for optimal auxiliary acquisition, extending to general manipulation primitives beyond grasping, and integrating with diffusion-based or language-driven policy architectures for open-world embodied AI.

Conclusion

This framework provides a robust, efficient solution for 6-DoF grasp pose estimation under severe self-occlusion and observation sparsity. By coupling targeted post-fusion of grasp-relevant regions with contrastive cross-view feature regularization, it advances the state of the art in both simulated and real-world benchmarks, setting a foundation for future research in multi-view robotic perception and manipulation (2606.06878).