VLA-Touch: Tactile-Enhanced VLA Models

• VLA-Touch is a dual-level framework that integrates tactile sensing into VLA models, enabling both semantic feedback for task planning and precise low-level control for contact-rich tasks.
• It separates tactile input into a high-level tactile-language module for replanning and a low-level interpolant-based controller for action refinement.
• Empirical evaluations on cup, wipe, and peel tasks show improved planning efficiency and manipulation success, highlighting significant performance gains over vision-only approaches.

Searching arXiv for the specified VLA-Touch paper and closely related tactile-VLA work to ground the article. Searching for "VLA-Touch tactile VLA" on arXiv. VLA-Touch is an approach for enhancing Vision-Language-Action (VLA) models with tactile sensing through a dual-level design that separates high-level task planning from low-level action refinement. Introduced as a wrapper around an off-the-shelf VLA backbone—specifically Robot Diffusion Transformer (RDT-1B)—it augments the base policy without fine-tuning the underlying VLA, using a pretrained tactile-LLM for semantic tactile feedback and an interpolant-based diffusion controller for contact-rich execution (Bi et al., 23 Jul 2025). In the reported formulation, tactile information is used both as language-like feedback for re-planning and as a low-dimensional control signal for refining action chunks, with the stated objective of improving task planning efficiency and execution precision in contact-rich manipulation.

1. Conceptual basis and scope

VLA-Touch addresses a specific limitation of contemporary VLA models: although such models can leverage vision, language, and action priors effectively, they generally lack the ability to interpret and use tactile signals. The paper frames this as especially consequential for contact-rich tasks, where the relevant state often cannot be inferred reliably from vision alone. It also identifies a practical obstacle to tactile integration: the absence of large multi-modal datasets pairing vision, language, action, and touch at scale (Bi et al., 23 Jul 2025).

The method’s central claim is that tactile feedback should enter the control stack at two distinct levels. At the planning level, tactile observations are converted into compact linguistic descriptions that can be appended to a Vision-LLM prompt. At the control level, a tactile-conditioned diffusion controller refines VLA-generated actions for short-horizon execution. This decomposition reflects the paper’s distinction between semantic tactile feedback for high-level task planning and direct tactile signals for contact-rich manipulation (Bi et al., 23 Jul 2025).

A plausible implication is that VLA-Touch treats tactile sensing less as an additional visual stream and more as a modality with heterogeneous roles across the hierarchy of robot control. In this respect, it differs from later tactile-augmented VLA systems that emphasized simulator RL (Ma et al., 10 Jun 2026), tactile-force alignment (Huang et al., 28 Jan 2026), online adaptation (Zheng et al., 8 Jun 2026), offline tactile distillation (Gubernatorov et al., 16 Mar 2026), tactile world models (Ye et al., 29 Dec 2025), or high-frequency history compression (Wang et al., 2 Mar 2026).

2. System architecture

VLA-Touch is organized around two loosely coupled modules layered on top of a base VLA policy. The first is a Task Planner (GPT-4o). Given a language goal $g$ and scene image $s_t$, it produces a single primitive manipulation instruction $I_k$. The planner is explicitly designed with the assumption that tactile feedback can be queried after each primitive. The second is a Tactile-LLM (Octopi), which receives a recent window of raw GelSight frames $o^m_{t-n:t}$ and outputs a compact linguistic description $L_t^m$, exemplified in the paper by phrases such as “hardness 4.3, roughness 5.7.” That text is then appended to the planner prompt so that the next instruction can condition jointly on vision and touch (Bi et al., 23 Jul 2025).

The execution layer consists of the base VLA policy and a tactile controller. The base VLA is written as

$\pi(a_t \mid s_t, I_k),$

where the instruction $I_k$ and the current visual/proprioceptive state $s_t$ are consumed by RDT to produce an action chunk $a_{t:t+T_a}$. This chunk is not executed directly. Instead, each shorter sub-chunk of length $T_r$ is refined by an Interpolant-based Diffusion Controller

$s_t$0

conditioned on both the VLA’s proposed actions and the tactile vector $s_t$1. In the paper, $s_t$2 is the 3D force estimate computed from the GelSight’s $s_t$3 marker array (Bi et al., 23 Jul 2025).

The architecture is presented as plug-and-play because the VLA model itself is never fine-tuned on tactile data. That detail is central to the identity of VLA-Touch. Tactile competence is introduced through the external tactile-language and diffusion components rather than by altering the pretrained VLA backbone (Bi et al., 23 Jul 2025).

3. Planning and control mechanisms

At the controller level, VLA-Touch adopts an interpolant diffusion formulation. Letting $s_t$4, the forward dynamics are given as

$s_t$5

with boundary conditions $s_t$6 and $s_t$7. Here $s_t$8 is the RDT action chunk, $s_t$9 is the expert action, and $I_k$0 is the conditioning state $I_k$1 (Bi et al., 23 Jul 2025).

The training objective for the interpolant controller is a mean-squared-error regression to expert actions: $I_k$2 This makes the controller a corrective mechanism rather than a full policy replacement: it learns to improve source actions generated by the base VLA (Bi et al., 23 Jul 2025).

At the planning level, the high-level feedback loop is written as

$I_k$3

The operational loop alternates between generating and executing refined actions, collecting new tactile history, converting that history into text with Octopi, and re-prompting GPT-4o for the next primitive. The paper’s condensed algorithm initializes $I_k$4 from goal $I_k$5 and scene $I_k$6, observes $I_k$7, generates an action chunk from $I_k$8, refines each sub-chunk with $I_k$9, breaks early if execution fails, then updates the instruction through GPT-4o after Octopi converts tactile data into linguistic feedback (Bi et al., 23 Jul 2025).

This division between tactile-conditioned replanning and tactile-conditioned local refinement is the core structural feature of VLA-Touch. A plausible implication is that the method treats planning errors and contact-execution errors as distinct failure modes requiring distinct interfaces to touch.

4. Hardware, data, and task suite

All data in the reported experiments were recorded on a Franka Panda arm with a Robotiq 2F-140 gripper and a GelSight Mini on one finger. Two Intel RealSense RGBD cameras provided overhead and wrist views. Data were recorded at 10 Hz, including RGB images from scene and wrist cameras, GelSight frames, and robot proprioception in the form of end-effector pose and gripper state (Bi et al., 23 Jul 2025).

The demonstrations were collected through kinesthetic teaching. The dataset consisted of three contact-rich tasks, each decomposed into labeled episodes with a single linguistic instruction for RDT training. For the Cup task, the data comprised Pick demos: 40 episodes (≈ 8 min) and Place demos: 60 episodes. For the Wipe task, the data comprised Pick demos: 40 and Wipe demos: 60 (≈ 15 min). For the Peel task, the data comprised Pick: 60 and Peel: 120 (≈ 30 min). The VLA model was then fine-tuned for 20 k steps each task, after which it was used to generate source actions for the interpolant-controller dataset $o^m_{t-n:t}$0 (Bi et al., 23 Jul 2025).

The three evaluation tasks were defined as follows. In Cup, the robot must determine whether a cup is empty or full via tactile sensing, then pick and place it without spilling. In Wipe, it must compare the roughness of two sponges, pick the smoother one, and wipe ink off a plate. In Peel, it must compare the hardness of two mangoes, select the softer one, and peel it with a hand tool (Bi et al., 23 Jul 2025).

The planning baselines were GPT-4o (vision only), GPT-4o + raw tactile images, and GPT-4o + Octopi linguistic feedback, each evaluated over 20 trials/task. Manipulation baselines were evaluated over 20 trials/task as counts of full task success and included RDT, + Residual, and + Interpolant (Bi et al., 23 Jul 2025).

5. Empirical results and ablations

For high-level tactile reasoning, the paper reports that Octopi-mediated tactile language improved planning outcomes relative to vision-only prompting. On force inference, the success rates were approximately GPT-4o only ≈ 50%, raw images ≈ 50%, and Octopi ≈ 90%. On roughness, the results were GPT-4o only ≈ 0%, raw images 100%, and Octopi 100%. On hardness, the results were GPT-4o only ≈ 0%, raw images 60%, and Octopi 75% (Bi et al., 23 Jul 2025).

For manipulation, the low-level controller comparison showed that the Interpolant variant outperformed both the base VLA and the residual baseline. The reported full-task success counts were Cup: RDT 7/20, + Residual 6/20, + Interpolant 10/20; Wipe: RDT 5/20, + Residual 6/20, + Interpolant 12/20; and Peel: RDT 6/20, + Residual 7/20, + Interpolant 10/20 (Bi et al., 23 Jul 2025).

The dual-level ablation isolated the contribution of planning-touch and control-touch. The reported results were Cup: w/o Planning 5/20, w/o Control 6/20, VLA-Touch 9/20; Wipe: w/o Planning 5/20, w/o Control 5/20, VLA-Touch 12/20; and Peel: w/o Planning 6/20, w/o Control 4/20, VLA-Touch 7/20. The summary findings stated that high-level tactile (linguistic) raised planning success by up to 40%, low-level tactile with interpolant diffusion boosted manipulation success by 35%–140%, and combined dual-level feedback outperformed single-level variants by up to 35% (Bi et al., 23 Jul 2025).

The ablation narrative further reported that removing tactile conditioning from the interpolant controller—labeled “w/o Touch”—reduced placement, wiping, and peeling success by roughly 40%–50%, while ablating vision—“w/o Vision”—also degraded positioning, but to a lesser extent. The paper therefore attributes the strongest performance to a complementarity between vision-guided spatial positioning and tactile-guided local correction (Bi et al., 23 Jul 2025).

6. Limitations, terminology, and subsequent developments

The paper notes three explicit limitations. First, a Gripper–Octopi domain mismatch can degrade hardness estimates. Second, because RDT was originally bimanual, the reported single-arm tasks limit cross-task generalization. Third, the interpolant runs at 8 Hz, whereas true tactile control may require $o^m_{t-n:t}$1 Hz. The listed future directions include event-triggered inference, active grasping with dynamic visual-tactile loops, and broader, task-agnostic policy refinement (Bi et al., 23 Jul 2025).

The terminology surrounding “VLA-Touch” broadened after the 2025 paper. Later works used closely related tactile-augmented VLA formulations, sometimes explicitly describing them as visuo-tactile extensions or as realizing a “VLA-Touch.” TacCoRL introduced a tactile-conditioned extension of a pretrained VLA policy with sim-real co-training and simulator PPO, reporting 72.5% average real-world success versus a 50.0% vision-only baseline across four bimanual contact-rich tasks (Ma et al., 10 Jun 2026). TaF-VLA shifted from tactile-vision alignment to tactile-force alignment using a TaF-Dataset of >10 million aligned samples, and reported 64.8% average success over seven tasks versus 37.1% for vision only (Huang et al., 28 Jan 2026). TORL-VLA added tactile-guided online reinforcement learning and an intervention-censored critic, reaching 28/30 full-task success and 165.5 s average full-task time in its latch-box benchmark (Zheng et al., 8 Jun 2026).

Other systems explored different trade-offs. HapticVLA learned tactile-aware behavior offline and deployed without inference-time tactile sensing, reporting a mean success rate of 86.7% and explicitly outperforming baseline VLAs, including versions with direct tactile feedback during inference (Gubernatorov et al., 16 Mar 2026). DreamTacVLA combined Hierarchical Spatial Alignment with a tactile world model and reported up to 95.0% ± 0.2% success on Peg-in-Hole (Ye et al., 29 Dec 2025). TacMamba focused on the spatiotemporal mismatch between fast tactile loops and slow VLA reasoning, using a Mamba-based tactile history compressor with 0.45 ± 0.02 ms latency and reporting 100% success on Sequential Button Pressing at convergence (Wang et al., 2 Mar 2026).

This broader trajectory suggests that VLA-Touch occupies an early and specific position within a rapidly diversifying line of research: it established a dual-level formulation in which tactile feedback can inform both semantic planning and action-level correction without modifying the base VLA itself, while later systems expanded the design space toward force-grounded pretraining, simulator post-training, online adaptation, distillation, forecasting, and real-time tactile memory (Bi et al., 23 Jul 2025).