OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies

Abstract: Visual policies learned from human videos, teleoperation, and robot demonstrations offer scalable motion priors, but often fail in contact-rich manipulation, where success significantly depends on local force and contact geometry. Tactile sensing provides these complementary signals, yet tactile data remain costly to collect and hard to generalize across sensors, robots, and tasks. We introduce OmniTacTune, a policy-agnostic real-world RL pipeline that adapts tactile feedback to pretrained visual policies through residual correction. OmniTacTune uses a two-stage design: it first bootstraps tactile-aware learning from autonomous base-policy rollouts, then learns a lightweight tactile residual policy through online interaction. Extensive experiments show that OmniTacTune generalizes across diverse contact-rich tasks, visual base policies, and tactile representations. Across four real-world contact-rich tasks, it improves visual base policies from 5-40% success to 85-100% within 40-80 minutes, demonstrating an efficient path for adapting tactile feedback to scalable visual robot policies. Project page: https://colinyu1.github.io/omnitactune-site/

• The paper introduces a two-stage, policy-agnostic RL framework that uses a warm-start phase and an online tactile residual actor to adapt visual policies.
• It achieves significant performance gains by boosting manipulation success rates from as low as 5% to between 85% and 100% across various contact-rich tasks.
• The approach integrates dense visuo-tactile reward shaping with diverse tactile encoders to robustly adapt without relying on extensive paired visuo-tactile demonstrations.

OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies

Motivation and Problem Setting

Recent advances in visual imitation learning have enabled scalable acquisition of robot manipulation policies from large amounts of human demonstrations, teleoperated data, and in-the-wild videos. However, policies learned using vision alone struggle in contact-rich manipulation tasks where precise local interaction is needed, as vision lacks direct measurement of force and contact geometry. Incorporating tactile sensing can supply this missing information, but data-driven tactile learning remains fundamentally constrained by the low scalability of tactile data collection, hardware specificity, and poor generalization of tactile representations across robots and tasks.

The core challenge addressed by this work is to achieve policy adaptation by fusing the generality of scalable visual policies with the local precision enabled by tactile feedback, without incurring the excessive cost and rigidity of collecting large-scale paired visuo-tactile demonstrations. The proposed framework, OmniTacTune, realizes this goal by treating tactile adaptation as a residual correction problem atop a frozen visual policy, learning the tactile correction exclusively via efficient real-world reinforcement learning (RL) in a policy-agnostic way.

System Overview

OmniTacTune is structured as a two-stage RL pipeline that operates directly in the real world and supports arbitrary visual base policies, whether trained on human videos or robot demonstration data.

Figure 1: OmniTacTune adapts tactile feedback to diverse visual base policies trained from human videos or robot data (left) through a two-stage real-world tactile residual RL pipeline (middle), enabling efficient tactile adaptation across challenging contact-rich manipulation tasks (right).

The system first collects visual demonstrations, either via human video retargeting or teleoperation, to train a base visual policy. In the first RL stage, termed warm-start, the robot autonomously executes the frozen base policy, collecting tactile data that is used both to bootstrap a task-specific tactile encoder and to initialize a flow-tactile critic. This stage enables adaptation to task-specific tactile signal distributions and stabilizes subsequent learning. In the second stage, the system learns a lightweight residual policy conditioned on tactile feedback, which corrects the potentially suboptimal actions produced by the visual base policy during contact-rich interaction.

A dense multi-sensory reward, combining object-centric visual subgoal shaping with tactile contact and safety signals, is used to improve learning efficiency and exploration in real-world trials.

Figure 2: System overview of OmniTacTune: collection of visual demos (left), two-stage tactile adaptation (top right), and dense visuo-tactile reward shaping (bottom).

Task Domains and Data Acquisition

OmniTacTune is evaluated on four real-world contact-rich manipulation tasks executed with an xArm7 robot equipped with a gripper and high-resolution GelSight tactile sensor. The tasks are:

• Peg-in-Hole: Grasp and insert a peg requiring millimeter-level alignment and spatial generalization.
• Charger Insertion: Insert a charger into a power strip with minimal clearance.
• Cap Opening: Use a tool to open a bottle cap, demanding dynamic contact reasoning.
• Box Opening: Open a box with precise edge alignment and dynamic contact.

Figure 3: Real-world contact-rich manipulation tasks for evaluation: peg-in-hole, charger insertion, cap opening, and box opening.

Human demonstrations are collected with a third-person RGB camera and a Meta Quest headset for pose tracking, while teleoperation data uses VR interface mapping for direct robot control. These two sources are complementary, as teleoperation data quality degenerates in contact-rich scenes due to lack of haptic feedback available to the human operator.

Method: Tactile Residual Reinforcement Learning

Two-Stage RL Pipeline

1. Warm-Start Phase: The base policy is rolled out in the real world to collect initial tactile experiences under the policy's own distribution. This data is used both to adapt a pretrained tactile encoder (e.g., AnyTouch2, Sparsh, T3) and to bootstrap the Q-function (critic) for RL. ControlTac-based trajectory-level augmentation synthesizes further diverse contact-rich tactile transitions to enrich the initialization buffer.
2. Online Residual RL: A policy-agnostic tactile residual actor is trained via Soft Actor-Critic (SAC), taking as input (a) the robot proprioception, (b) flow-based object motion encoding, (c) the adapted tactile encoding, and (d) the base policy's action chunk. The actor outputs a corrective action which is summed (with scheduled scaling) with the base policy to produce the final action. The architecture is explicitly designed to be agnostic to the base policy's underlying structure.

Visuo-Tactile Reward Shaping

A dense reward $r_t$ is synthesized using:

• An object-centric reaching and flow reward, computed from the motion of tracked keypoints between current and generated/target flows.
• A grasp/tactile reward, indicating stable contact based on tactile sensing statistics.
• A safety penalty, triggered by excessive tactile marker displacement (overforce contacts).

This reward structure ensures smooth shaping towards the goal, leveraging visual structure for long-horizon guidance and tactile grounding for precise, safe operation.

Empirical Results

Sample Efficiency and Performance Gains

On all four contact-rich real-world tasks, the OmniTacTune framework achieves substantial improvements in final manipulation success rate compared to both vision-only and other visuo-tactile RL or imitation methods:

• Base visual policies alone achieve between 5–40% success.
• OmniTacTune consistently raises this to 85–100% within 40–80 min of real-world adaptation, outperforming adapted baselines (PLD*, ViTAL) by over 40 percentage points.

Numerically, the method achieves:

• Peg-in-Hole: 100% (from 40%)
• Charger Insertion: 100% (from 10%)
• Cap Opening: 90% (from 5%)
• Box Opening: 85% (from 5%)

  Figure 4: OmniTacTune consistently improves diverse base visual policies.

Broad Policy Agnosticism

OmniTacTune is validated with diverse frozen visual base policies—including flow policies from human or teleop demonstrations, ACT, Diffusion Policy, and $\pi_{0.5}$ vision-language-action models—demonstrating consistent improvement to >75% success regardless of policy structure or demonstration source.

Tactile Representation Generality

The tactile adaptation pipeline is robust across multiple tactile encoding strategies, including high-capacity pretrained neural image encoders (AnyTouch2, Sparsh, T3) and low-dimensional marker-based encodings.

Figure 5: Adapting different tactile representations via OmniTacTune, demonstrating compatibility across encoders.

Baseline Comparisons

Compared to recent imitation-based and slow-fast multi-modal visuo-tactile policies (e.g., RDP), and simple tactile feature concatenation or supervised fine-tuning approaches, OmniTacTune achieves 20–30% higher success rates and greater data efficiency, with no need for per-task paired visuo-tactile demonstrations.

Analysis and Ablations

A series of ablation experiments support the critical design choices:

• Removing multi-sensory reward shaping, warm-start optimization, or trajectory-level tactile augmentation consistently destabilizes early learning and reduces final performance.
• The combination of trajectory-level keypoint guidance and contact-aware gating outperforms both per-step keypoint conditioning and vanilla visuo-tactile policy fusion, cementing the utility of object-centric priors and gated this-gaited tactile features.
• Analysis of demonstration smoothness via SPARC and LDLJ metrics quantitatively confirms the superior temporal consistency of human demonstration-based flow policies over teleoperation-based ones, especially in contact-rich phases.

Limitations and Future Directions

Current limitations include the need for manual environment resets and susceptibility to hardware degradation due to repeated contact-rich interactions, especially with sensitive tactile sensors. The framework presumes the availability of robust visual motion priors and still requires real-world RL trial cycles for tactile adaptation.

Future research avenues include:

• Automation of resets and augmentation of real-world data efficiency, potentially via the integration of predictive world models for simulated pretraining and planning.
• Generalization to additional sensor/robot embodiments, leveraging the policy-agnostic structure.
• Development of more resilient tactile sensors and adaptive safety-aware learning mechanisms.

Conclusion

OmniTacTune demonstrates that efficient, policy-agnostic real-world tactile adaptation of visual manipulation policies is feasible without task-specific tactile demonstrations or heavy simulation. By decomposing contact-rich manipulation into scalable vision-derived priors and local trial-and-error tactile correction, OmniTacTune enables robots to reliably perform demanding manipulation tasks in real-world settings with sample efficiency, policy diversity, and tactile encoder flexibility, establishing a practical and extensible architecture for integrating touch into modern visual robot learning pipelines.

---

Reference: "OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies" (2607.03723)