---
title: 'BiGraspFormer: Transformer Bimanual Grasping'
url: https://www.emergentmind.com/topics/bigraspformer
type: topic
---

# BiGraspFormer: Transformer Bimanual Grasping

BiGraspFormer is an end-to-end transformer-based framework for coordinated bimanual grasp generation directly from object point clouds. It predicts two 6-DoF grasp poses, one for each arm, together with a grasp quality score, thereby producing a 12-DoF bimanual solution. The framework was introduced to address a characteristic weakness of prior approaches: existing methods either focus solely on single-arm grasping or employ separate grasp generation and bimanual evaluation stages, which can produce coordination problems including collision risks and unbalanced force distribution. Its central mechanism is the Single-Guided Bimanual (SGB) strategy, which first generates diverse single grasp candidates and then conditions joint bimanual prediction on their learned features, with the stated aim of reducing the complexity of the 12-DoF search space while preserving coordinated manipulation [2509.19142].

## 1. Problem formulation and motivation

Bimanual grasping is essential for robots to handle large and complex objects. In the formulation used by BiGraspFormer, the target output is a coordinated pair of 6-DoF grasps plus a scalar quality estimate, so the prediction problem is intrinsically higher-dimensional than conventional single-arm grasp generation. The paper identifies the principal technical challenge as efficient search and evaluation in a high-dimensional 12-DoF action space, with the additional requirement that the resulting grasps be mutually compatible rather than merely individually valid [2509.19142].

The motivation is explicitly tied to coordination. A pipeline that generates candidate grasps independently for each arm and evaluates them only afterward can fail in several ways even if the single-arm predictions are locally plausible. The paper names collision risks and unbalanced force distribution as representative failure modes. This suggests that the problem is not only one of grasp feasibility at individual contact regions but also one of inter-arm coupling, in which geometry, force closure, torque balance, and mutual accessibility interact.

The design of BiGraspFormer therefore departs from exhaustive or modular joint search. Instead, it treats coordinated bimanual grasping as a structured prediction problem in which single-arm grasp features can serve as intermediate anchors for learning pairwise compatibility. A plausible implication is that the method exploits the combinatorial structure of the task without requiring explicit post-hoc enumeration at inference.

## 2. Single-Guided Bimanual strategy

The Single-Guided Bimanual strategy is the paper’s main conceptual advance. It is organized as a three-step procedure. First, the model generates diverse, force-closure stable single-arm grasp candidates using transformer-based decoding. Second, it systematically pairs the resulting single-arm grasps and filters out combinations with collisions or lacking torque or force balance. Third, conditioning on the learned features from the single grasps, it jointly refines and predicts the final bimanual 12-DoF grasp together with its quality score [2509.19142].

The stated benefit of this decomposition is that it reduces the 12-DoF search to a tractable, staged process. In the authors’ account, the strategy leverages the structure of the problem and allows the model to learn the interdependence between arms explicitly via attention. Rather than predicting two unrelated grasps and asking a downstream module to reconcile them, the framework conditions bimanual generation on a feature space already shaped by stable single-arm contacts.

This staged construction should not be confused with a conventional modular pipeline. The paper presents the framework as unified and end-to-end, and the conditioning mechanism is learned inside the transformer architecture rather than imposed through a separate external evaluator. A common misunderstanding would be to treat SGB as simple candidate pairing. In the paper’s formulation, pairing is only one part of the training procedure; the central mechanism is feature reuse from single-grasp generation into joint bimanual decoding.

## 3. Architecture and processing pipeline

BiGraspFormer is organized into four main modules: an Object Encoder, a Single Grasp Proposer (SGP), a Bimanual Pair Matcher (BPM), and a Bimanual Grasp Generator (BGG). Together they define the end-to-end pipeline from point cloud input to ranked bimanual grasp output.

The Object Encoder extracts both local and global geometric features from the input point cloud. Its design combines PointNet++ with a transformer encoder. Specifically, it uses two Set Abstraction layers for local, multi-scale features, followed by six transformer encoder blocks performing self-attention over local PointNet++ features to capture spatial relationships and global structure. The module outputs global object features denoted \(\mathcal{F}'_{g} \in \mathbb{R}^{N' \times C}\).

The Single Grasp Proposer generates a diverse set of stable 6-DoF single grasps. It is implemented as a DETR-style transformer decoder that receives \(K'\) learnable grasp queries. Each query attends through cross-attention to object features so as to focus on different regions or poses. An MLP prediction head outputs pose parameters for each query, yielding a set of single-arm grasp candidates \(\hat{\mathcal{G}} = \{\hat{\mathbf{g}}_i\}_{i=1}^{K'}\).

The Bimanual Pair Matcher is used during training to create reliable ground-truth bimanual pairs for supervision. Its procedure is explicit: it enumerates all pairs among SGP single-arm outputs, scores them using bimanual quality criteria including force closure, torque balance, and dexterity as used in the DA2 dataset, removes pairs with collisions using CAD models, and then, for each anchor grasp, keeps the highest-quality collision-free pair. The paper emphasizes that BPM is used only during training and is unavailable at inference because object mesh is not provided.

The Bimanual Grasp Generator jointly refines and predicts final bimanual grasps and associated quality scores using both single-grasp features and object features. It is also implemented as a DETR-style transformer decoder, now with \(M'\) learnable bimanual queries. The decoder incorporates the SGB Attention Layer, then performs further cross-attention with object global features and SGP features, and finally uses a 7-layer MLP prediction head to output two 6-DoF poses and quality \(q\). The resulting set is denoted \(\hat{\mathcal{B}} = \{(\hat{\mathbf{g}}^1_i, \hat{\mathbf{g}}^2_i, \hat{q}_i)\}_{i=1}^{M'}\) [2509.19142].

## 4. Attention mechanisms, representation, and optimization

The architectural key for coordination is the SGB Attention Layer. In the notation reported in the paper, it is written as
\[
\text{SGB-Attention}(\mathbf{Q}^b, \mathbf{F}_{sgp}) =
\text{softmax}\left(\frac{\mathbf{Q}^b (\mathbf{F}_{sgp})^{T}}{\sqrt{d}}\right)\mathbf{F}_{sgp}.
\]
Here, \(\mathbf{Q}^b\) denotes the set of bimanual grasp queries used by the decoder, \(\mathbf{F}_{sgp}\) denotes learned features from the Single Grasp Proposer, and \(d\) is the embedding dimension.

The functional interpretation given in the paper is that each bimanual query can selectively aggregate relevant information from the set of possible single-arm grasp features. The intended consequences are coordination of the two grasps, exploitation of local stable contact regions, use of the object’s global geometry, and propagation of information about possible grasp synergy, collision avoidance, and force or torque balance. SGB attention is then followed by further cross-attentions between bimanual queries and global object features, so that both local context from SGP and global context from the encoded point cloud remain available during decoding.

The paper also specifies the training losses. For single-arm grasp regression, it uses
\[
\mathcal{L}_{\text{single}}(\mathcal{G}_i, \hat{\mathcal{G}}_j) =
\left\| \mathbf{v}_i - \hat{\mathbf{v}}_j \right\|_2,
\]
where \(\mathbf{v}\) encodes grasp representation points, with Contact-GraspNet cited in the details as the reference representation. For bimanual grasp prediction, the reported loss is
\[
\begin{split}
\mathcal{L}_{\text{bimanual}}(\mathcal{B}_i, \hat{\mathcal{B}}_j) ={}&
\mathcal{L}_{\text{dist}}(\mathbf{g}^1_i, \hat{\mathbf{g}}^1_j) +
\mathcal{L}_{\text{dist}}(\mathbf{g}^2_i, \hat{\mathbf{g}}^2_j) \\
&\quad + \left|q_i - \hat{q}_j\right|.
\end{split}
\]
Training employs bipartite matching with the Hungarian algorithm to match predictions to ground truths before loss computation, in the standard set-prediction style associated with DETR-type frameworks [2509.19142].

## 5. Search-space reduction and coordination mechanism

A central claim of BiGraspFormer is that the framework reduces the complexity of bimanual grasp search by exploiting a structured decomposition. The baseline difficulty is clear: independently searching over all joint combinations for two arms in a 12-DoF action space is combinatorially challenging. BiGraspFormer addresses this by first generating likely stable grasps for each arm and then conditioning bimanual generation on relevant single-arm features rather than searching the full joint space directly.

The paper characterizes this as a reduction of the effective action space explored. Because single-arm candidates already encode local contact stability and object-specific geometric affordances, the bimanual decoder can focus on compatibility, balance, and coordination. This suggests that the model shifts computational effort from brute-force joint search toward learned relational inference between candidate grasp features.

This mechanism also clarifies the distinction between training-time supervision and inference-time prediction. The Bimanual Pair Matcher uses explicit pair enumeration, scoring, and collision filtering during training because CAD models and quality criteria are available there. At inference, however, the framework does not rely on mesh-based post-pairing. A common misconception would be to assume that BiGraspFormer requires exhaustive combinatorial pairing at deployment. The paper states the opposite: the framework maintains efficient inference speed because it avoids the slow modular pipelines used by baselines [2509.19142].

## 6. Empirical validation

The paper reports comprehensive simulation experiments and real-world validation. In simulation, evaluation was performed in Isaac Gym on the DA2 dataset, using 100 held-out objects and physics-based simulation with bimanual Panda grippers. For success rate under normal force using the top-1% of predictions, BiGraspFormer achieved 89.67%, compared with 71.61% for CGDF+DPN-GPD. The details further state that success rates remain high even under disturbance, described as a weighted cube dropped onto the object.

The paper also reports diversity in terms of the area of the object surface covered by successful grasps. BiGraspFormer obtained 36.99%–46.47%, which is described as much higher than the state-of-the-art baselines. Inference time is reported as less than 0.05 s per object for GPU forward pass with no post-pairing, while state-of-the-art baselines are stated to run from 4 to 18 seconds because of slow modular pipelines. Qualitatively, the model is described as producing more diverse and robust grasps, with far fewer collision or balance failures [2509.19142].

Ablation studies are summarized in two principal findings. Removing SGB attention or BPM, that is, not using SGB decomposition, substantially decreases both success and diversity. In addition, performance scales with the number of bimanual queries but outperforms baselines even with fewer candidates. These results are consistent with the paper’s claim that both decomposition and feature-sharing are structurally important rather than incidental engineering choices.

Real-world experiments used dual-UR5e arms with Robotiq 2F-140 grippers and Azure Kinect RGB-D cameras on large, complex household objects in varied poses. The reported success rate was 7–10 out of 10 trials for each object. Failures were attributed mainly to extreme object weight or torque imbalance rather than grasp pose quality. For execution, curobo was used for trajectory planning, with BiGraspFormer providing the ranked set of viable bimanual grasps. This is important for system interpretation: BiGraspFormer is a grasp generation framework, not a complete motion-planning stack.

## 7. Positioning, scope, and limitations

The paper attributes BiGraspFormer’s empirical advantage to four factors: unified end-to-end generation, explicit modeling of grasp coordination, efficient search via structured decomposition, and the context-modeling capacity of transformer self-attention and cross-attention. In that account, the framework differs from approaches that depend on modular post-hoc pairing or external quality scoring, because coordination is learned jointly rather than retrofitted after independent single-arm prediction [2509.19142].

The scope of the method is also delimited in the reported experiments and system design. Input is an object point cloud; output is a ranked set of viable bimanual grasps; downstream trajectory generation is handled by a separate planner. The Bimanual Pair Matcher is a training-only component because object mesh is not provided at inference. This suggests that the learned bimanual decoder is expected to internalize compatibility structure that is explicitly available only during training.

The reported failure modes are also specific. In real-world trials, failures were mainly due to extreme object weight or torque imbalance, not grasp pose quality. That distinction matters because it indicates that some unsuccessful executions arose from physical loading conditions rather than geometric misprediction. A plausible implication is that future work might integrate richer dynamic or load-aware criteria into the ranking stage, but the paper itself frames the current contribution around coordinated grasp generation from point clouds rather than full manipulation dynamics.

Overall, BiGraspFormer is defined by its combination of transformer-based perception, DETR-style set prediction, and the Single-Guided Bimanual strategy for feature-conditioned joint grasp synthesis. Within the reported setting, it is presented as a framework for generating diverse, stable, and coordinated bimanual grasps directly from object point clouds with efficient inference and demonstrated transfer from simulation to real robots.

Source: https://www.emergentmind.com/topics/bigraspformer