VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tunin (2510.14930v1)

Abstract: Humans excel at bimanual assembly tasks by adapting to rich tactile feedback -- a capability that remains difficult to replicate in robots through behavioral cloning alone, due to the suboptimality and limited diversity of human demonstrations. In this work, we present VT-Refine, a visuo-tactile policy learning framework that combines real-world demonstrations, high-fidelity tactile simulation, and reinforcement learning to tackle precise, contact-rich bimanual assembly. We begin by training a diffusion policy on a small set of demonstrations using synchronized visual and tactile inputs. This policy is then transferred to a simulated digital twin equipped with simulated tactile sensors and further refined via large-scale reinforcement learning to enhance robustness and generalization. To enable accurate sim-to-real transfer, we leverage high-resolution piezoresistive tactile sensors that provide normal force signals and can be realistically modeled in parallel using GPU-accelerated simulation. Experimental results show that VT-Refine improves assembly performance in both simulation and the real world by increasing data diversity and enabling more effective policy fine-tuning. Our project page is available at https://binghao-huang.github.io/vt_refine/.

• The paper presents a real-to-sim-to-real framework that integrates visuo-tactile sensing and reinforcement learning for precise bimanual assembly.
• It employs diffusion policy pre-training on limited demonstrations followed by simulation fine-tuning using high-fidelity tactile simulation.
• Quantitative results show up to a 40% improvement over vision-only policies, demonstrating robust sim-to-real transfer.

VT-Refine: Simulation-Fine-Tuned Visuo-Tactile Policy Learning for Bimanual Assembly

Introduction and Motivation

VT-Refine introduces a real-to-sim-to-real framework for learning precise, contact-rich bimanual assembly policies by integrating visuo-tactile feedback and simulation-based reinforcement learning. The method addresses two major bottlenecks in bimanual robotic assembly: the scarcity and suboptimality of human demonstrations, and the difficulty of simulating tactile feedback for robust sim-to-real transfer. The approach leverages a small set of real-world demonstrations to pre-train a diffusion policy, then refines this policy in simulation using high-fidelity tactile sensing and large-scale RL, before deploying the improved policy back to the real robot.

Figure 1: VT-Refine pipeline: real-world demonstration collection, visuo-tactile diffusion policy pre-training, simulation-based RL fine-tuning, and sim-to-real transfer for bimanual assembly.

Visuo-Tactile Sensing and Simulation

The hardware setup employs FlexiTac, a piezoresistive tactile sensor array with 12×32 units per pad, mounted on soft grippers. This design prioritizes normal force sensing, which is both easier to simulate and more robust for sim-to-real transfer compared to vision-based tactile sensors. The simulation environment uses TacSL, a GPU-accelerated tactile simulation library, to model the sensor's response via a Kelvin-Voigt spring-damper model. Each tactile point computes contact normal force as $f_n = -(k_n d + k_d \dot{d}) n$, where $d$ is penetration depth and $\dot{d}$ is normal velocity.

Figure 2: Real-world tactile sensor hardware and its simulation counterpart, with spring-damper modeling for realistic tactile signal generation.

Figure 3: Tactile simulation pipeline: finger model, taxel grid sampling, and penalty-based spring-damper contact modeling for normal force and depth signals.

Calibration aligns simulated tactile readings with real sensor outputs by fitting the spring-damper parameters to empirical force-response curves, minimizing the sim-to-real gap. The system supports efficient, scalable simulation for RL rollouts, enabling large-scale policy refinement.

Two-Stage Policy Optimization

The policy learning pipeline consists of two stages:

1. Pre-training: A diffusion policy is trained via behavioral cloning on a small set of human demonstrations (∼30 episodes), using synchronized visual and tactile inputs. The observation space is a unified point cloud representation, merging visual and tactile points, processed by a PointNet encoder and concatenated with proprioceptive features.
2. Simulation Fine-Tuning: The pre-trained policy is further optimized in simulation using Diffusion Policy Policy Optimization (DPPO), a PPO-based RL algorithm adapted for diffusion models. The actor network is initialized from pre-trained weights, while the critic receives low-dimensional state representations. Sparse rewards (success/failure) are used, leveraging the strong prior from demonstrations to guide exploration.

   Figure 4: Two-stage training: real-world demonstration-based diffusion policy pre-training, followed by simulation-based RL fine-tuning.

Experimental Results and Analysis

Quantitative Performance

Experiments on five plug-and-socket bimanual assembly tasks (clearance ≈2mm) demonstrate that RL fine-tuning in simulation substantially improves policy performance in both simulated and real environments. The visuo-tactile policy consistently outperforms the vision-only baseline, both in initial success rates and final performance after fine-tuning.

• Real-world success rates: RL fine-tuning yields a ∼20% improvement for vision-only and ∼40% for visuo-tactile policies.
• Simulation results: Fine-tuned visuo-tactile policies achieve near-perfect success rates, while vision-only policies often fail to close the final gap due to lack of contact feedback.
• Ablation on demonstration quantity: Increasing demonstrations from 10 to 30 yields significant gains, but further increases to 50 provide diminishing returns; RL fine-tuning is critical for achieving high precision.

  Figure 5: Policy rollout: successful visuo-tactile policy execution with iterative re-orientation and tactile feedback, contrasted with failure modes due to misalignment or poor contact.

  

  Figure 6: Effect of tactile calibration: calibrated simulation enables monotonic improvement during RL fine-tuning, while uncalibrated simulation causes initial performance degradation.

Representation Transfer

The unified point cloud representation enables robust transfer between real and simulated domains. Sim-to-real transfer incurs a minor performance drop (5–10%), which is outweighed by the gains from simulation-based fine-tuning. Domain alignment is achieved via normalization and noise injection in simulation to mimic real sensor characteristics.

Trade-offs and Limitations

Sensor Modality

• Resolution: Vision-based tactile sensors offer higher spatial resolution but are difficult to simulate, resulting in larger sim-to-real gaps. FlexiTac's normal force sensing is lower resolution but more transferable.
• Shear Force: FlexiTac does not directly sense shear, but temporal policy history can infer shear effects.
• Integration: FlexiTac is thin, flexible, and easily integrated into compliant grippers, unlike bulky vision-based sensors.

Methodological Constraints

• Calibration: Manual alignment of visual and tactile domains is required.
• Simulator Capabilities: Current pipeline does not support deformable objects or shear force sensing.
• CAD Dependency: Training requires object CAD models; a CAD-free pipeline would broaden applicability.

Implementation Details

• Sensor fabrication: Flexible PCB-based pads, cost-effective and scalable, with open-source design files.
• Tactile simulation: GPU-parallelized, fully vectorized over environments, with automatic taxel sampling and normalization.
• Policy architecture: PointNet encoder for point cloud inputs, MLP for proprioception, U-Net backbone for diffusion policy, asymmetric actor-critic for RL.
• DPPO hyperparameters: Conservative learning rates for fine-grained tasks, KL penalty for decoder regularization, sparse reward structure.

Implications and Future Directions

VT-Refine demonstrates that simulation-based RL fine-tuning, combined with transfer-friendly tactile sensing, can substantially improve the precision and robustness of bimanual assembly policies. The approach enables scalable policy improvement with minimal real-world data, and the unified visuo-tactile representation facilitates robust sim-to-real transfer. Future work should address deformable object simulation, shear force sensing, and CAD-free training pipelines to extend applicability to broader manipulation domains.

Conclusion

VT-Refine establishes a practical framework for learning precise bimanual assembly policies by integrating visuo-tactile feedback, high-fidelity tactile simulation, and RL fine-tuning. The method achieves strong alignment between simulation and real-world performance, with significant gains over vision-only baselines. The approach is modular, scalable, and well-suited for contact-rich manipulation tasks, with clear pathways for future enhancement in sensor design and simulation fidelity.