---
title: 'VLA-Touch: Tactile-Enhanced VLA Models'
url: https://www.emergentmind.com/topics/vla-touch
type: topic
---

# VLA-Touch: Tactile-Enhanced VLA Models

Searching arXiv for the specified VLA-Touch paper and closely related tactile-VLA work to ground the article.
Searching for "VLA-Touch tactile VLA" on arXiv.
VLA-Touch is an approach for enhancing Vision-Language-Action (VLA) models with tactile sensing through a dual-level design that separates high-level task planning from low-level action refinement. Introduced as a wrapper around an off-the-shelf VLA backbone—specifically Robot Diffusion Transformer (RDT-1B)—it augments the base policy without fine-tuning the underlying VLA, using a pretrained tactile-language model for semantic tactile feedback and an interpolant-based diffusion controller for contact-rich execution [2507.17294]. In the reported formulation, tactile information is used both as language-like feedback for re-planning and as a low-dimensional control signal for refining action chunks, with the stated objective of improving task planning efficiency and execution precision in contact-rich manipulation.

## 1. Conceptual basis and scope

VLA-Touch addresses a specific limitation of contemporary VLA models: although such models can leverage vision, language, and action priors effectively, they generally lack the ability to interpret and use tactile signals. The paper frames this as especially consequential for contact-rich tasks, where the relevant state often cannot be inferred reliably from vision alone. It also identifies a practical obstacle to tactile integration: the absence of large multi-modal datasets pairing vision, language, action, and touch at scale [2507.17294].

The method’s central claim is that tactile feedback should enter the control stack at two distinct levels. At the planning level, tactile observations are converted into compact linguistic descriptions that can be appended to a Vision-Language Model prompt. At the control level, a tactile-conditioned diffusion controller refines VLA-generated actions for short-horizon execution. This decomposition reflects the paper’s distinction between semantic tactile feedback for high-level task planning and direct tactile signals for contact-rich manipulation [2507.17294].

A plausible implication is that VLA-Touch treats tactile sensing less as an additional visual stream and more as a modality with heterogeneous roles across the hierarchy of robot control. In this respect, it differs from later tactile-augmented VLA systems that emphasized simulator RL [2606.11743], tactile-force alignment [2601.20321], online adaptation [2606.09337], offline tactile distillation [2603.15257], tactile world models [2512.23864], or high-frequency history compression [2603.01700].

## 2. System architecture

VLA-Touch is organized around two loosely coupled modules layered on top of a base VLA policy. The first is a **Task Planner (GPT-4o)**. Given a language goal $g$ and scene image $s_t$, it produces a single primitive manipulation instruction $I_k$. The planner is explicitly designed with the assumption that tactile feedback can be queried after each primitive. The second is a **Tactile-Language Model (Octopi)**, which receives a recent window of raw GelSight frames $o^m_{t-n:t}$ and outputs a compact linguistic description $L_t^m$, exemplified in the paper by phrases such as “hardness 4.3, roughness 5.7.” That text is then appended to the planner prompt so that the next instruction can condition jointly on vision and touch [2507.17294].

The execution layer consists of the base VLA policy and a tactile controller. The base VLA is written as
\[
\pi(a_t \mid s_t, I_k),
\]
where the instruction $I_k$ and the current visual/proprioceptive state $s_t$ are consumed by RDT to produce an action chunk $a_{t:t+T_a}$. This chunk is not executed directly. Instead, each shorter sub-chunk of length $T_r$ is refined by an **Interpolant-based Diffusion Controller**
\[
\pi_I(\hat a_{t:t+T_r}\mid a_{t:t+T_r}, s_t, m_t),
\]
conditioned on both the VLA’s proposed actions and the tactile vector $m_t$. In the paper, $m_t$ is the 3D force estimate computed from the GelSight’s $7\times 9$ marker array [2507.17294].

The architecture is presented as plug-and-play because the VLA model itself is never fine-tuned on tactile data. That detail is central to the identity of VLA-Touch. Tactile competence is introduced through the external tactile-language and diffusion components rather than by altering the pretrained VLA backbone [2507.17294].

## 3. Planning and control mechanisms

At the controller level, VLA-Touch adopts an interpolant diffusion formulation. Letting $z\sim\mathcal N(0,I)$, the forward dynamics are given as
\[
a_t = I(t,a_0,a_1,x) + \gamma(t)z,\qquad
d a_t = b_F(t,a_t,x)\,dt + \sqrt{2\,\epsilon(t)}\,dW_t,
\]
with boundary conditions $I(0,a_0,a_1,x)=a_0$ and $I(1,a_0,a_1,x)=a_1$. Here $a_0$ is the RDT action chunk, $a_1$ is the expert action, and $x$ is the conditioning state $(s_t,m_t)$ [2507.17294].

The training objective for the interpolant controller is a mean-squared-error regression to expert actions:
\[
\min_\theta\;
\frac{1}{|D_{\mathrm{VLA}}|}\sum_{(a,s,m,a^{\rm Exp})\in D}
\left\|a^{\mathrm{Exp}}_{t:t+T_a}-\pi_I(\hat a_{t:t+T_a}\mid a_{t:t+T_a},\,s_t,\,m_t;\theta)\right\|^2.
\]
This makes the controller a corrective mechanism rather than a full policy replacement: it learns to improve source actions generated by the base VLA [2507.17294].

At the planning level, the high-level feedback loop is written as
\[
I_{k+1}=\mathrm{Plan}(g,s_t,I_k,L_t^m), \qquad
L_t^m=\mathrm{Octopi}(o^m_{t-n:t}).
\]
The operational loop alternates between generating and executing refined actions, collecting new tactile history, converting that history into text with Octopi, and re-prompting GPT-4o for the next primitive. The paper’s condensed algorithm initializes $I_0$ from goal $g$ and scene $o_0$, observes $(s_t,m_t)$, generates an action chunk from $\pi_{\rm VLA}$, refines each sub-chunk with $\pi_I$, breaks early if execution fails, then updates the instruction through GPT-4o after Octopi converts tactile data into linguistic feedback [2507.17294].

This division between tactile-conditioned replanning and tactile-conditioned local refinement is the core structural feature of VLA-Touch. A plausible implication is that the method treats planning errors and contact-execution errors as distinct failure modes requiring distinct interfaces to touch.

## 4. Hardware, data, and task suite

All data in the reported experiments were recorded on a **Franka Panda arm** with a **Robotiq 2F-140 gripper** and a **GelSight Mini on one finger**. Two **Intel RealSense RGBD cameras** provided overhead and wrist views. Data were recorded at **10 Hz**, including RGB images from scene and wrist cameras, GelSight frames, and robot proprioception in the form of end-effector pose and gripper state [2507.17294].

The demonstrations were collected through **kinesthetic teaching**. The dataset consisted of three contact-rich tasks, each decomposed into labeled episodes with a single linguistic instruction for RDT training. For the **Cup task**, the data comprised **Pick demos: 40 episodes (≈ 8 min)** and **Place demos: 60 episodes**. For the **Wipe task**, the data comprised **Pick demos: 40** and **Wipe demos: 60 (≈ 15 min)**. For the **Peel task**, the data comprised **Pick: 60** and **Peel: 120 (≈ 30 min)**. The VLA model was then fine-tuned for **20 k steps each task**, after which it was used to generate source actions for the interpolant-controller dataset $D_{\mathrm{VLA}}$ [2507.17294].

The three evaluation tasks were defined as follows. In **Cup**, the robot must determine whether a cup is empty or full via tactile sensing, then pick and place it without spilling. In **Wipe**, it must compare the roughness of two sponges, pick the smoother one, and wipe ink off a plate. In **Peel**, it must compare the hardness of two mangoes, select the softer one, and peel it with a hand tool [2507.17294].

The planning baselines were **GPT-4o (vision only)**, **GPT-4o + raw tactile images**, and **GPT-4o + Octopi linguistic feedback**, each evaluated over **20 trials/task**. Manipulation baselines were evaluated over **20 trials/task** as counts of full task success and included **RDT**, **+ Residual**, and **+ Interpolant** [2507.17294].

## 5. Empirical results and ablations

For high-level tactile reasoning, the paper reports that Octopi-mediated tactile language improved planning outcomes relative to vision-only prompting. On **force inference**, the success rates were approximately **GPT-4o only ≈ 50%**, **raw images ≈ 50%**, and **Octopi ≈ 90%**. On **roughness**, the results were **GPT-4o only ≈ 0%**, **raw images 100%**, and **Octopi 100%**. On **hardness**, the results were **GPT-4o only ≈ 0%**, **raw images 60%**, and **Octopi 75%** [2507.17294].

For manipulation, the low-level controller comparison showed that the **Interpolant** variant outperformed both the base VLA and the residual baseline. The reported full-task success counts were **Cup: RDT 7/20, + Residual 6/20, + Interpolant 10/20**; **Wipe: RDT 5/20, + Residual 6/20, + Interpolant 12/20**; and **Peel: RDT 6/20, + Residual 7/20, + Interpolant 10/20** [2507.17294].

The dual-level ablation isolated the contribution of planning-touch and control-touch. The reported results were **Cup: w/o Planning 5/20, w/o Control 6/20, VLA-Touch 9/20**; **Wipe: w/o Planning 5/20, w/o Control 5/20, VLA-Touch 12/20**; and **Peel: w/o Planning 6/20, w/o Control 4/20, VLA-Touch 7/20**. The summary findings stated that **high-level tactile (linguistic) raised planning success by up to 40%**, **low-level tactile with interpolant diffusion boosted manipulation success by 35%–140%**, and **combined dual-level feedback outperformed single-level variants by up to 35%** [2507.17294].

The ablation narrative further reported that removing tactile conditioning from the interpolant controller—labeled **“w/o Touch”**—reduced placement, wiping, and peeling success by roughly **40%–50%**, while ablating vision—**“w/o Vision”**—also degraded positioning, but to a lesser extent. The paper therefore attributes the strongest performance to a complementarity between vision-guided spatial positioning and tactile-guided local correction [2507.17294].

## 6. Limitations, terminology, and subsequent developments

The paper notes three explicit limitations. First, a **Gripper–Octopi domain mismatch** can degrade hardness estimates. Second, because **RDT was originally bimanual**, the reported **single-arm tasks limit cross-task generalization**. Third, the **interpolant runs at 8 Hz**, whereas true tactile control may require **$\ge 25$ Hz**. The listed future directions include **event-triggered inference**, **active grasping with dynamic visual-tactile loops**, and **broader, task-agnostic policy refinement** [2507.17294].

The terminology surrounding “VLA-Touch” broadened after the 2025 paper. Later works used closely related tactile-augmented VLA formulations, sometimes explicitly describing them as visuo-tactile extensions or as realizing a “VLA-Touch.” TacCoRL introduced a tactile-conditioned extension of a pretrained VLA policy with sim-real co-training and simulator PPO, reporting **72.5%** average real-world success versus a **50.0%** vision-only baseline across four bimanual contact-rich tasks [2606.11743]. TaF-VLA shifted from tactile-vision alignment to tactile-force alignment using a **TaF-Dataset** of **>10 million aligned samples**, and reported **64.8%** average success over seven tasks versus **37.1%** for vision only [2601.20321]. TORL-VLA added tactile-guided online reinforcement learning and an intervention-censored critic, reaching **28/30** full-task success and **165.5 s** average full-task time in its latch-box benchmark [2606.09337].

Other systems explored different trade-offs. HapticVLA learned tactile-aware behavior offline and deployed without inference-time tactile sensing, reporting a **mean success rate of 86.7%** and explicitly outperforming baseline VLAs, including versions with direct tactile feedback during inference [2603.15257]. DreamTacVLA combined **Hierarchical Spatial Alignment** with a tactile world model and reported up to **95.0% ± 0.2%** success on Peg-in-Hole [2512.23864]. TacMamba focused on the spatiotemporal mismatch between fast tactile loops and slow VLA reasoning, using a Mamba-based tactile history compressor with **0.45 ± 0.02 ms** latency and reporting **100%** success on Sequential Button Pressing at convergence [2603.01700].

This broader trajectory suggests that VLA-Touch occupies an early and specific position within a rapidly diversifying line of research: it established a dual-level formulation in which tactile feedback can inform both semantic planning and action-level correction without modifying the base VLA itself, while later systems expanded the design space toward force-grounded pretraining, simulator post-training, online adaptation, distillation, forecasting, and real-time tactile memory [2507.17294].

Source: https://www.emergentmind.com/topics/vla-touch