TopoLiDM: Dual-Level Tactile VLA Framework

• TopoLiDM is a dual-level tactile augmentation framework that separates high-level semantic planning from low-level tactile-conditioned action refinement in VLA models.
• It integrates GPT-4o based task planning with a diffusion-based interpolant controller to leverage both linguistic tactile feedback and numeric force estimates.
• Empirical results show that dual-level tactile feedback can boost planning success by up to 40% and manipulation performance by 35–140% in contact-rich scenarios.

VLA-Touch is a visuo-tactile augmentation framework for Vision-Language-Action (VLA) models that introduces tactile feedback at two distinct levels of control: semantic tactile feedback for high-level task planning and tactile-conditioned action refinement for low-level execution. In its original formulation, VLA-Touch enhances an off-the-shelf VLA backbone—instantiated with Robot Diffusion Transformer (RDT-1B)—without fine-tuning the base VLA itself, instead wrapping it with a GPT-4o task planner, an Octopi tactile-language module, and an interpolant-based diffusion controller that refines VLA-generated action chunks using tactile signals (Bi et al., 23 Jul 2025).

1. Conceptual role within tactile VLA research

VLA models supply strong priors over visual perception, language grounding, and action generation, but they are typically limited in contact-rich regimes because they do not natively interpret tactile signals. The VLA-Touch formulation addresses this gap by separating the tactile problem into two control layers. At the planning layer, tactile observations are converted into language so that a large vision-LLM can re-plan using semantically interpretable touch feedback. At the control layer, a diffusion-based policy corrects the base VLA’s proposed motions using low-dimensional tactile state.

This design differs from several later visuo-tactile VLA directions. TacCoRL injects tactile feedback directly into a pretrained VLA backbone, warm-starts the resulting policy with mixed simulated and real demonstrations, and then refines it with reinforcement learning in a real-aligned simulator (Ma et al., 10 Jun 2026). TaF-VLA replaces tactile-vision alignment with tactile-force alignment and grounds tactile embeddings in true force measurements through a vector-quantized, contrastively trained TaF-Adapter (Huang et al., 28 Jan 2026). TORL-VLA couples tactile-aware reference prediction with online actor-critic refinement and an intervention-censored critic for long-horizon adaptation under contact shift (Zheng et al., 8 Jun 2026). HapticVLA, by contrast, learns tactile-aware manipulation offline and deploys without direct haptic sensing at inference through tactile distillation (Gubernatorov et al., 16 Mar 2026). DreamTacVLA introduces a tactile world model that predicts future tactile latents (Ye et al., 29 Dec 2025), while TacMamba targets the spatiotemporal mismatch between high-frequency touch and low-frequency VLA reasoning through a Mamba-based tactile history compressor (Wang et al., 2 Mar 2026).

Taken together, these works situate VLA-Touch as an early dual-level architecture in a broader family of tactile VLA systems. A plausible implication is that the field has diversified along three axes: semantic versus physically grounded tactile representations, offline versus online adaptation, and direct tactile inference versus tactile distillation.

2. Dual-level architecture

VLA-Touch is organized around two loosely coupled tactile modules. The first is a high-level planner based on GPT-4o. Given a language goal $g$ and scene image $s_t$, it outputs a single primitive manipulation instruction $I_k$. The second is a tactile-LLM, Octopi, which processes the most recent window of raw GelSight frames $o^m_{t-n:t}$ and returns a compact linguistic description $L_t^m$, such as “hardness 4.3, roughness 5.7.” This description is appended to the VLM prompt so that the next primitive can be generated using both vision and touch (Bi et al., 23 Jul 2025).

The planning update is written as

$$I_{k+1}=\mathrm{Plan}(g,s_t,I_k,L_t^m), \qquad L_t^m=\mathrm{Octopi}(o^m_{t-n:t}).$$

At the execution layer, the base VLA policy is the RDT backbone $\pi(a_t\mid s_t,I_k)$, which consumes the current visual and proprioceptive state together with the primitive instruction and produces an action chunk $a_{t:t+T_a}$. VLA-Touch does not modify or fine-tune this base VLA. Instead, it adds an interpolant-based diffusion controller,

$\pi_I(\hat a_{t:t+T_r}\mid a_{t:t+T_r}, s_t, m_t),$

which refines shorter sub-chunks of the VLA proposal using the current tactile signal $m_t$. Here $s_t$0 is a low-dimensional tactile vector: the 3D force estimate computed from the GelSight’s $s_t$1 marker array.

This division of labor is central. High-level tactile input is represented linguistically and used for primitive selection; low-level tactile input is represented numerically and used for action correction. The architecture therefore does not require the base VLA to learn tactile representations internally. Instead, the tactile pathway is externalized into planner-facing language and controller-facing force estimates.

3. Control dynamics and execution loop

The interpolant controller is formulated as a stochastic process between a source action and an expert action distribution. With $s_t$2, the forward dynamics are

$s_t$3

with boundary conditions $s_t$4 and $s_t$5 (Bi et al., 23 Jul 2025).

In this setting, $s_t$6 is the RDT-generated action chunk, $s_t$7 is the expert action, and $s_t$8 is the conditioning state $s_t$9. The controller is trained to map VLA proposals toward expert behavior under tactile conditioning via a mean-squared-error objective. The role of the interpolant is therefore not to replace the VLA prior, but to reshape it locally in contact-sensitive phases.

The inference loop alternates planning and tactile refinement. The system initializes a primitive instruction from the goal and scene, observes $I_k$0, generates an action chunk with the VLA, refines each sub-chunk with the interpolant controller, executes the refined action, updates the state, and, after chunk completion, converts the latest tactile history into language through Octopi before re-prompting GPT-4o. If execution fails during a sub-chunk, the controller breaks early and returns to re-planning.

This loop makes VLA-Touch “plug-and-play” in a specific sense: tactile intelligence is layered around the base VLA rather than fused into its weights. That property is explicit in the original formulation. It also marks a substantive contrast with systems such as TacCoRL and TaF-VLA, where tactile conditioning is integrated into the policy backbone and the policy itself is fine-tuned (Ma et al., 10 Jun 2026).

4. Data regime, sensing stack, and task suite

The VLA-Touch experiments are conducted on a Franka Panda arm with a Robotiq 2F-140 gripper and a GelSight Mini mounted on one finger. Two Intel RealSense RGBD cameras provide overhead and wrist views. Data are recorded at 10 Hz and include RGB images from the scene and wrist cameras, GelSight frames, and robot proprioception consisting of end-effector pose and gripper state (Bi et al., 23 Jul 2025).

Training data are collected through kinesthetic teaching. The task-specific demonstration counts are as follows: for the Cup task, 40 pick demonstrations and 60 place demonstrations; for the Wipe task, 40 pick demonstrations and 60 wipe demonstrations; for the Peel task, 60 pick demonstrations and 120 peel demonstrations. Each episode is labeled with a single linguistic instruction to train RDT. The VLA model is then fine-tuned for 20 k steps on each task and subsequently used to generate source actions for the interpolant controller dataset $I_k$1.

The three evaluation tasks are explicitly contact-rich and include both tactile inference and manipulation:

• Cup: determine if a cup is empty vs. full via tactile feedback, then pick and place without spilling.
• Wipe: compare the roughness of two sponges, pick the smoother one, and wipe ink off a plate.
• Peel: compare the hardness of two mangoes, pick the softer or riper one, and peel it with a hand tool.

These tasks are structured so that tactile input matters at both decision scales. Hardness and roughness discrimination affect primitive selection, while grasping, placement, wiping, and peeling require contact-sensitive corrective control. This suggests that VLA-Touch was designed not merely to append touch as another observation modality, but to test whether tactile information changes the decomposition of planning and execution.

5. Empirical results and ablations

The original experiments separate planning performance from manipulation performance. For planning, three baselines are evaluated over 20 trials per task: GPT-4o with vision only, GPT-4o with raw tactile images, and GPT-4o with Octopi linguistic feedback. On force inference, GPT-4o only is approximately 50%, raw images are approximately 50%, and Octopi is approximately 90%. On roughness, GPT-4o only is approximately 0%, raw images are 100%, and Octopi is 100%. On hardness, GPT-4o only is approximately 0%, raw images are 60%, and Octopi is 75% (Bi et al., 23 Jul 2025).

For manipulation, full-task success is reported over 20 trials per task. The baseline RDT achieves 7/20 on Cup, 5/20 on Wipe, and 6/20 on Peel. Adding a residual controller yields 6/20, 6/20, and 7/20. Replacing the residual with the interpolant controller yields 10/20, 12/20, and 10/20. The paper summarizes this by stating that low-level tactile with interpolant diffusion boosted manipulation success by 35%–140%.

A second ablation isolates the dual-level structure. Removing planning-touch yields 5/20 on Cup, 5/20 on Wipe, and 6/20 on Peel. Removing control-touch yields 6/20, 5/20, and 4/20. The full VLA-Touch system yields 9/20, 12/20, and 7/20. The paper summarizes the combined effect by stating that high-level tactile raised planning success by up to 40%, combined dual-level feedback outperformed single-level variants by up to 35%, and ablating tactile from the interpolant controller cut placement, wiping, and peeling success by roughly 40%–50%.

These results support two distinct claims. First, tactile language is useful for semantic discrimination tasks such as force, roughness, and hardness assessment. Second, tactile-conditioned diffusion control improves execution precision beyond the action prior supplied by the VLA. The system’s gains are therefore not reducible to either better prompting or better servoing alone.

6. Limitations, interpretation, and subsequent developments

The original VLA-Touch paper identifies three limitations. First, the Gripper–Octopi domain mismatch can degrade hardness estimates. Second, RDT was originally bimanual, whereas the reported experiments use single-arm tasks, which limits cross-task generalization. Third, the interpolant controller runs at 8 Hz, while true tactile control might require at least 25 Hz (Bi et al., 23 Jul 2025).

These limitations anticipate several later design shifts in tactile VLA research. TacMamba explicitly addresses control-frequency mismatch by running a tactile reflex loop at 100 Hz and reporting 0.45 $I_k$2 0.02 ms inference latency for its tactile encoder (Wang et al., 2 Mar 2026). TacCoRL addresses rare near-failure contact states by using a calibrated simulator, sparse terminal rewards, mixed sim-real co-training, and post-training with PPO plus a real-data anchor; across four bimanual contact-rich tasks, its final visuo-tactile policy achieves an average success rate of 72.5%, compared to 50.0% for a vision-only baseline (Ma et al., 10 Jun 2026). TaF-VLA addresses representational semantics by aligning tactile observations with force measurements rather than treating tactile images as auxiliary visual textures, and reports 64.8% average success over 7 tasks versus 37.1% for a vision-only $I_k$3 policy (Huang et al., 28 Jan 2026). TORL-VLA addresses distribution shift during deployment by adding online RL and intervention-censored value learning, reaching 28/30 full-task success against 12/30 for a vision-only baseline in its latch-box benchmark (Zheng et al., 8 Jun 2026). HapticVLA explores the opposite deployment trade-off by retaining tactile-aware behavior without tactile sensors at inference and reports a mean success rate of 86.7% on fragile-object tasks (Gubernatorov et al., 16 Mar 2026). DreamTacVLA extends tactile reasoning into prediction by learning a tactile world model and reports up to 95.0% $I_k$4 0.2% success on Peg-in-Hole (Ye et al., 29 Dec 2025).

In that broader trajectory, VLA-Touch can be understood as a modular dual-level baseline: tactile semantics are translated into language for planning, and tactile contact is translated into local force-aware refinement for control. This suggests that its lasting significance lies less in end-to-end tactile policy learning than in establishing a decomposition that later work either integrated more tightly, grounded more physically, accelerated to higher control rates, or replaced with distillation and online adaptation.