6-DoF Bimanual Pose Acquisition

• 6-DoF bimanual pose acquisition is a method that accurately determines three translations and three rotations for dual manipulators, essential for precise robotic manipulation.
• The process employs robust hand–eye calibration, frame harmonization, and sensor fusion to minimize drift and ensure high-fidelity performance.
• Benchmarking shows that systems like ViTaMIn-B achieve millimeter-level drift control and superior demo validity compared to traditional vision-only SLAM approaches.

A 6-DoF bimanual pose acquisition process refers to any method that yields the full six degrees-of-freedom—three translations and three rotations—of two independent manipulators (“hands,” cameras, or grippers), with respect to each other or a fixed world frame. Acquisition of precise, drift-minimized 6-DoF trajectories for both “hands” is foundational for bimanual robotic manipulation, human demonstration capture, and dual-robot coordination. Core requirements include accurate frame definitions, robust sensor fusion, precise calibration, and resilience to dynamic or contact-rich scenarios.

1. Fundamental Coordinate Frames and Calibration Protocols

In high-fidelity bimanual pose acquisition, clear and compatible frame definitions are essential. For handheld manipulator solutions such as ViTaMIn-B (Li et al., 8 Nov 2025), the following frames are defined:

• World frame (W): Left-handed, as returned by the Meta Quest system, for all controller pose outputs.
• Controller frame (Q): The native pose origin and axes of the Quest controller, at its geometric center.
• End-effector/tool frame (EE): Right-handed, attached to the gripper jaws—the “tip” where manipulation occurs.
• Robot base frame (B): Used during calibration only, fixed to a reference robot arm.

To harmonize Quest’s internal left-handed outputs with right-handed robot conventions, the Z axis of all Quest-reported matrices is systematically inverted—this sign-flipping operation is critical to maintain correct handedness across all transforms.

Hand–Eye Calibration:

Calibration finds the constant rigid transform ${}^{Q}T_{EE}$ from controller to tool frame. The process:

1. Rigidly mount the controller on an industrial robot's flange, with accessible ${}^{B}T_{EE_k}$.
2. For $n=10$ diverse poses, record controller pose in world frame (${ }^{W}T_{Q_k}$) and the robot EE in base frame (${ }^{B}T_{EE_k}$).
3. The relationship at each pose is

$${}^{W}T_{Q_k} \cdot {}^{Q}T_{EE} = {}^{W}T_{B} \cdot {}^{B}T_{EE_k}$$

Solve using established hand–eye methods (Tsai–Lenz or Horaud–Dornaika)—yielding a single, fixed ${}^{Q}T_{EE}$ for all subsequent data acquisition.

2. Real-Time Pipeline for Dual 6-DoF Trajectory Generation

During operation, both left and right hand controllers supply 6-DoF pose streams (${ }^{W}T_{Q_i}$) at 72 Hz. The unified acquisition pipeline in ViTaMIn-B entails:

• Handedness correction: Apply Z-column/row sign flips to every Quest pose for right-handed compatibility.
• Pose chaining: Compute tool pose in the unified frame per

${}^{W}T_{EE_i} = {}^{W}T_{Q_i} \cdot {}^{Q}T_{EE}$

• Temporal alignment: To synchronize with lower-rate vision/tactile cameras (e.g., 30 Hz), carry out SE(3) interpolation—linear for positions, SLERP for orientations.
• Incremental motion derivation: For playback or motion analysis,

$${}^{EE_{i+1}}T_{EE_i} = ({}^{W}T_{EE_{i+1}})^{-1} \cdot {}^{W}T_{EE_i}$$

The core mathematical object is the homogeneous transform

$${}^{A}T_{B} = \begin{bmatrix} R_{AB} & t_{AB} \ 0^{\mathrm{T}} & 1 \end{bmatrix}$$

where $R$ is a $3 \times 3$ rotation, $t$ a $3 \times 1$ translation.

3. Sensor Fusion, Error Models, and Multi-Modal Synchronization

Quest controller pose estimation leverages its on-board visual-inertial SLAM (VIO) fusion engine—integrating IMU (gyroscope/accelerometer), depth, and visual features with in-headset loop closure for drift suppression. No custom EKF or additional external SLAM backend is required; all raw controller pose outputs are a product of Quest’s internal VIO.

For systems integrating multiple sensors (e.g. vision cameras, tactile arrays), proper temporal synchronization is achieved via empirical latency measurement (e.g., Quest ≈ 10 ms, visual ≈ 140 ms, tactile ≈ 80 ms) and per-stream timestamp correction.

In robust bimanual manipulation planning, explicit error modeling is critical (Sinha et al., 2020):

• Joint error: Modeled as zero-mean Gaussian in concatenated dual-arm joint space, $\delta\Theta \sim \mathcal{N}(0, \sigma^2 I)$.
• Task-space error ellipsoids: Linearized mapping via Jacobians, yielding probabilistic bounds on both relative position and orientation errors in $SE(3)$.

4. Pose Acquisition via Mutual Observation and Relative Placement

Alternative 6-DoF bimanual pose acquisition frameworks utilize reciprocal fiducial observation (e.g., Mutual Localization (Dhiman et al., 2013)). Here, each “arm” or robot is equipped with uniquely identified markers; each camera observes the other's markers, eliminating reliance on egomotion or world landmarks.

The mutual localization problem is solved as follows:

• Extract unit bearing vectors from each camera to each observed marker, given known intrinsics.
• Set up four reciprocal equations for the marker correspondences, introducing unknown scale factors for depth.
• Use inter-point distance invariants to eliminate rotation and translation, yielding a single 8th-degree polynomial in the depth of one marker (solved numerically).
• Recover $R, t$ via Procrustes/SVD or quaternion methods.
• The constraint structure ensures that only correct, non-spurious hypotheses are accepted, contingent on disambiguation with a 4th marker.

This approach achieves median translation errors of ≈1.6 cm and rotation errors of ≈0.33° at 2 m separation, outperforming one-sided fiducial tracking (ARToolKit) by over an order of magnitude in translation accuracy.

5. Benchmarking, Drift Performance, and Task Success Metrics

Drift and temporal stability sharply distinguish bimanual pose acquisition systems. In ViTaMIn-B (Li et al., 8 Nov 2025):

• Update rate: Controller pose at 72 Hz, downsampled/interpolated to 30 Hz for cross-modality alignment.
• Latency: End-to-end controller-to-tool pose latency ≈10 ms, with temporal jitter <1 ms post-correction.
• Drift: Over 5 m operational trajectories (cabinet opening, object relocation), total drift was <5 mm. In contrast, vision-only SLAM (e.g., ORB-SLAM3) drifted by 50–100 mm, with frequent tracking loss under dynamic scene changes.
• Demo validity: ViTaMIn-B with VR controllers yielded a 100% validity rate in demonstration collection on the Weight Placement task, compared to 16% for SLAM-based methods.
• Data efficiency: Imitation policies trained from VR-controller trajectories reached a 0.7 task success rate, versus 0.1 when trained on SLAM-collected data.

In robust-IK driven bimanual assembly (Sinha et al., 2020), statistical certification is possible by computing the maximal worst-case error metric $\mathcal{M}^*(\Theta)$ (which combines position and orientation error contributions weighted by $\gamma$). This enables the system to self-certify feasibility with explicit probability guarantees:

$$P[\| \Delta x \| \leq \epsilon ] \geq 1 - \delta, \quad \epsilon = \epsilon_p + \gamma \epsilon_o$$

where $\epsilon_p, \epsilon_o$ are derived from the largest Jacobian eigenvalues and user-specified confidence.

6. Limitations, Generalizations, and Failure Modes

Standard limitations in current 6-DoF bimanual pose processes include:

• Frame-handling ambiguities: Mismatched handedness or improperly calibrated transforms yield systematic errors.
• Assumptions of Gaussian, uncorrelated error: Real robot joints may exhibit non-Gaussian or correlated disturbances, affecting the validity of analytical ellipsoidal bounds (Sinha et al., 2020).
• Linearization constraints: The robust-IK method requires “small” joint errors; significant error magnitudes would necessitate higher-order propagation.
• Occlusion and visibility: Mutual localization fails if fewer than three non-collinear markers are visible in both frames (Dhiman et al., 2013).
• Modality-specific limitations: Pure SLAM approaches collapse under fast-moving, contact-rich manipulation, while VR-controller-based VIO remains drift-resilient under extended operation (Li et al., 8 Nov 2025).

Possible generalizations include:

• Extending to force-guided or compliant bimanual tasks with dynamic/stiffness modeling.
• Monte Carlo empirical validation of theoretical confidence bounds.
• Augmenting with online correction via vision or external markers, particularly in multi-robot or large workspace settings.

---

In summary, modern 6-DoF bimanual pose acquisition integrates rigorous frame definitions, hand–eye calibration, visual-inertial sensor fusion, and careful error modeling. High-performance systems such as ViTaMIn-B deliver millimeter-level drift, robust demo validity, and efficient policy learning, substantiating the centrality of precise pose management to advanced bimanual manipulation. Reciprocal-fiducial and robust-IK frameworks extend the methodological spectrum for specialized cooperative or uncertainty-aware settings.