Text-Conditioned Tactile Reward Strategies

• Text-conditioned tactile reward is a framework that evaluates touch-based interactions using external semantic cues rather than solely relying on local, hand-authored metrics.
• Various methodologies integrate analytic tactile metrics, multimodal reinforcement learning, and LLM-designed reward functions to refine grasping and manipulation tasks.
• Empirical findings indicate that such reward designs can boost task success rates while allowing reduced tactile sensing during deployment, highlighting practical benefits in robotic control.

Searching arXiv for the cited works to ground the article in current papers. Text-conditioned tactile reward denotes a family of reward-construction strategies in which tactile interaction is evaluated under an external semantic condition rather than by a purely hand-authored, modality-local scalar objective. In the cited literature, this idea appears in several non-identical forms: analytic tactile/contact metrics used as reinforcement-learning rewards for grasp refinement; tactile-grounded policy optimization objectives that reward reasoning only when authentic tactile input matters; externally conditioned rewards in which a non-tactile modality defines task progress while tactile sensing drives control; and LLM-authored reward functions that explicitly expose tactile state variables to natural-language reward synthesis (Koenig et al., 2021, Lai et al., 26 May 2026, Guzey et al., 2023, Field et al., 9 Sep 2025).

1. Conceptual scope and problem setting

The literature does not present a single standardized formalism for text-conditioned tactile reward. Instead, it spans at least four reward-design regimes. In grasp refinement, tactile contact information enters the reward through analytic grasp-stability terms rather than through language (Koenig et al., 2021). In tactile reasoning MLLMs, the reward is composite and includes an input-side tactile grounding objective that tests whether predictions change when touch is perturbed (Lai et al., 26 May 2026). In dexterous manipulation with visual incentives, the reward is computed from vision while tactile observations remain central to the policy state, making the setup explicitly analogous to a language-conditioned system in which text would define the target semantics (Guzey et al., 2023). In LLM-driven reward synthesis, the textual condition is the task description plus environment context, and the resulting reward function directly references tactile/contact variables (Field et al., 9 Sep 2025).

A common thread is that touch is treated as physically informative but semantically ambiguous when used alone. The cited works repeatedly separate two roles that are often conflated: tactile sensing as a control input and tactile sensing as a reward substrate. This suggests that the main technical question is not whether touch should be used, but where it should appear in the learning pipeline and under what conditioning structure.

2. Analytic tactile reward shaping for grasp refinement

"The Role of Tactile Sensing in Learning and Deploying Grasp Refinement Algorithms" studies grasp refinement in multi-fingered robotic hands with reinforcement learning and simulated tactile signals. The reward ablation is organized around three tactile/reward-relevant signal types—contact positions, contact normals, and contact forces—and four reward frameworks: $\epsilon+\delta$, $\epsilon$ only, $\delta$ only, and a sparse binary baseline $\beta$. Here $\epsilon$ is a geometry-based grasp stability metric built from contact positions and normals; $\delta$ is a force-based stability metric built from actual contact forces; and $\beta$ is $1$ if the object remains in the hand after holding and $0$ otherwise (Koenig et al., 2021).

The geometry-based term $\epsilon$ is derived from classic grasp analysis and combines force and torque resistance. The resistible force set is

$\epsilon$0

with

$\epsilon$1

The torque-space analog uses

$\epsilon$2

and

$\epsilon$3

The paper states that $\epsilon$4 is a weighted combination of $\epsilon$5 and $\epsilon$6.

The force-based term has two stage-dependent forms. During the lift and hold stages, the reward uses the current-state force stability metric

$\epsilon$7

where contacts are weighted by force magnitude. During the refinement stage, before lifting, the paper uses a task-oriented lower bound over expected task wrenches: $\epsilon$8

$\epsilon$9

Rewards are normalized because SAC is sensitive to reward scale.

The empirical conclusion is that combining contact position, normal, and force information in the reward yields the best test performance across cuboid, cylinder, and sphere grasps under translation errors from $\delta$0 to $\delta$1 cm and rotation errors from $\delta$2 to $\delta$3. The combined tactile reward $\delta$4 achieves $\delta$5 average success across all objects, with $\delta$6 for cuboids, $\delta$7 for cylinders, and $\delta$8 for spheres, and outperforms the sparse baseline $\delta$9 by $\beta$0. One-sided paired t-tests report $\beta$1, $\beta$2, and $\beta$3. The paper further shows that when training uses the tactile-enabled reward $\beta$4, tactile information in the policy state can be heavily reduced, with at most a $\beta$5 performance decrease for no tactile sensing in the state. This establishes a central design pattern: reward richness can be concentrated during training, while deployment can use a smaller sensor suite.

3. Tactile-grounded reinforcement learning for multimodal reasoning

"Touch-R1: Reinforcing Touch Reasoning in MLLMs" addresses a different regime: tactile reasoning in multimodal LLMs. The work introduces TouchReason-1M, described as a large-scale multimodal dataset comprising over 1M synchronized tactile pairs across four distinct sensors, and TouchReason-Bench for evaluating tactile perception and visual-tactile conflict resolution. The model, Touch-R1, is based on Qwen2.5-VL-7B and extends GRPO with a tactile-specific objective because answer correctness alone can reward predictions that follow visual priors or object-category shortcuts rather than touch (Lai et al., 26 May 2026).

The final objective is

$\beta$6

where the composite output reward $\beta$7 contains ordinal-aware accuracy, cross-sensor physical consistency, and structured-format control, while $\beta$8 is an input-side tactile grounding objective. The ordinal-aware accuracy reward is

$\beta$9

with $\epsilon$0. The cross-sensor consistency reward is

$\epsilon$1

and the format reward is

$\epsilon$2

These are combined as

$\epsilon$3

The tactile grounding term is not a direct task reward but a KL-based sensitivity term: $\epsilon$4 Its role is to assign tactile-use credit only when authentic tactile inputs outperform counterfactual controls where the tactile stream is removed, shuffled, or noise-masked; the method specifically describes $\epsilon$5 as replacing one randomly selected tactile stream with shape-matched Gaussian noise. This design operationalizes the statement that reward should verify tactile dependence rather than merely outcome correctness.

The reported results show that Touch-R1-7B outperforms Octopi-13B by $\epsilon$6 and GPT-4o by $\epsilon$7 on average on TouchReason-Bench. The main benchmark table reports an average score of $\epsilon$8, compared with $\epsilon$9 for SToLa and $\delta$0 for Gemini-2.5-Pro. The same table reports $\delta$1 versus $\delta$2 for SToLa, $\delta$3 versus $\delta$4, and $\delta$5 versus $\delta$6. The ablation sequence is also explicit: zero-shot Qwen2.5-VL-7B yields $\delta$7, $\delta$8 cold-start SFT yields $\delta$9, $\beta$0 GRPO $\beta$1 yields $\beta$2, $\beta$3 ordinal reward yields $\beta$4, $\beta$5 consistency reward yields $\beta$6, $\beta$7 format reward yields $\beta$8, and full Touch-R1 yields $\beta$9. The paper further states that ordinal reward lowers OMAE from $1$0 to $1$1 and consistency reward raises CSC from $1$2 to $1$3.

4. External conditioning when tactile reward is semantically weak

"See to Touch: Learning Tactile Dexterity through Visual Incentives" addresses the case in which tactile signals are useful for control but unreliable for defining task progress. The method, Tactile Adaptation from Visual Incentives (TAVI), explicitly replaces hard-to-design tactile rewards with an externally shaped reward derived from vision. The policy is optimized online to maximize a visual reward while executing with tactile-based observations, and the paper states that this arrangement is directly analogous to a text-conditioned tactile-reward setting in which language would specify the goal or desired state and guide reward construction (Guzey et al., 2023).

TAVI proceeds in three stages. First, it learns visual representations from task demonstrations using InfoNCE with a robot-state-change prediction loss: $1$4

$1$5

$1$6

The paper uses $1$7. Second, it defines an optimal-transport reward from one human demonstration. Given expert and robot visual trajectories encoded into latent sequences, it solves OT and uses

$1$8

Third, it trains the tactile policy online with DrQv2, which the appendix states uses DDPG to maximize the reward function.

A crucial design decision is that TAVI does not compute reward from tactile features. The paper reports that including tactile features in the OT reward leads to degenerate solutions, such as pinching fingers together to trigger touch sensors. To improve robustness, TAVI also departs from full-trajectory FISH-style matching and instead matches only the last 10 frames of the robot trajectory to the last frame of the expert trajectory, because OT over all frames can produce bad time-invariant matches and reward failed rollouts through cross-temporal aliasing.

The system is implemented on a 6-DOF Jaco arm with a 16-DOF AllegroHand, 15 XELA uSkin tactile sensors with $1$9 tri-axial force readings, and an RGB camera. On six tasks—peg insertion, sponge flipping, eraser turning, bowl unstacking, plier picking, and mint opening—TAVI achieves $0$0 average success, with per-task success of $0$1, $0$2, $0$3, $0$4, $0$5, and $0$6, respectively. The reported averages for baselines are $0$7 for T-Dex, $0$8 for BC-BeT, $0$9 for Tactile Only, $\epsilon$0 for Image + Tactile Reward, and $\epsilon$1 for AVI. The paper states that TAVI is $\epsilon$2 higher than policies using tactile and vision-based rewards and $\epsilon$3 higher than policies without tactile observational input. The same work reports limitations: imperfect generalization to new objects, strong dependence on camera viewpoint, no historical context in the observation representation, and manual selection of residual action dimensions.

5. LLM-designed tactile rewards for dexterous in-hand manipulation

"Text2Touch: Tactile In-Hand Manipulation with LLM-Designed Reward Functions" makes the textual condition explicit. The system uses a natural-language task description together with environment code and context to prompt an LLM to write a TorchScript reward function for gravity-invariant, multi-axis in-hand object rotation in palm-up and palm-down configurations. Tactile sensing is vision-based TacTip sensing on a fully actuated four-fingered Allegro Hand, and the reward-design context includes rich tactile variables such as net_tip_contact_forces, net_tip_contact_force_mags, tip_object_contacts, n_tip_contacts, n_non_tip_contacts, per-finger contact booleans, tip_contact_force_pose, tip_contact_force_pose_low_dim, and tip_contact_force_pose_bins (Field et al., 9 Sep 2025).

The prompting pipeline is Eureka-style and iterative. Iteration 0 contains a system prompt telling the LLM it is writing a TorchScript reward function, the environment code as context, and a carefully worded task description emphasizing repositioning, reorientation, regrasping, and finger gaiting. Later iterations provide conversation history including the original prompt, the best previous reward code, policy feedback from training, and code feedback telling the model what to change. This reflection loop is repeated for five iterations, with multiple reward candidates per iteration.

A major technical issue is that the environment exposes more than 70 variables. To make code generation reliable, the paper replaces the generic reward-function signature with an explicit, fully enumerated signature containing all available environment variables and their types, including hand state, fingertip state, tactile contact state, object/goal state, keypoint-based features, task-axis information, and termination/success shaping variables such as success_bonus and early_reset_penalty_value. The expected output is a compute_reward(...) -> Tuple[Tensor, Dict[str, Tensor]] function that returns both total reward and a dictionary of reward components.

The paper emphasizes that the “Bonus/Penalty + Modified signature” strategy is the one that consistently works. Table 1 shows that with this strategy, rotation performance reaches around $\epsilon$4–$\epsilon$5 rotations per episode and high solve rates for some models, whereas removing the scalable bonus/penalty context or the modified signature causes performance to collapse to near-zero rotations per episode and $\epsilon$6 solve rate. The best LLM-generated rewards are reported to use roughly one-tenth as many environment variables, about one-quarter as many lines of code, and about one-eighth the Halstead volume of the human-designed baseline.

A representative reward decomposes the task into keypoint alignment, success shaping, contact quality, and motion regularization. The appendix code includes $\epsilon$20 progressive success shaping, $\epsilon$21 contact shaping, $\epsilon$22 and penalties, $\epsilon$23 combined as $\epsilon$24 The paper notes that targeting only two fingertip contacts reflects the observation that the task benefits from controlled regrasping or finger gaiting rather than maximal grasp closure.

Training uses a teacher-student distillation pipeline. A privileged teacher is trained in simulation with the best LLM-generated reward and can observe object pose, object velocity, and other full-state variables. A student with the same architecture is then distilled by minimizing mean-squared error between student and teacher actions; the student observes only proprioception and tactile inputs. The real robot runs at $\epsilon$7 Hz and is evaluated on ten household objects across three axes and palm-up/palm-down orientations. In simulation, the human baseline reaches about $\epsilon$8 rotations per episode, while Gemini-1.5-Flash reaches $\epsilon$9, GPT-4o $\epsilon$00, o3-mini $\epsilon$01, Llama3.1-405B $\epsilon$02, and DeepSeek-R1-671B $\epsilon$03. In distillation tests on heavier objects and novel shapes, the baseline obtains $\epsilon$04 and $\epsilon$05 rotations per episode, while LLM-derived students achieve roughly $\epsilon$06–$\epsilon$07 and $\epsilon$08–$\epsilon$09, respectively. In real-world evaluation, the baseline averages $\epsilon$10 rotations and $\epsilon$11 s time-to-terminate, compared with $\epsilon$12 and $\epsilon$13 s for GPT-4o, $\epsilon$14 and $\epsilon$15 s for Gemini-1.5-Flash, and $\epsilon$16 and $\epsilon$17 s for DeepSeek-R1-671B. The paper highlights that DeepSeek achieves about a $\epsilon$18 increase in rotations and a $\epsilon$19 increase in episode duration compared with the baseline.

6. Design principles, misconceptions, and limitations

A common misconception is that text-conditioned tactile reward necessarily means feeding tactile data through language and optimizing a single end-to-end scalar objective. The cited literature shows a broader design space. In grasp refinement, the reward is tactile-informed but analytic and non-linguistic (Koenig et al., 2021). In Touch-R1, the reward is partly output-side and partly an input-side tactile grounding objective that verifies sensitivity to touch (Lai et al., 26 May 2026). In TAVI, the reward is not tactile at all, even though the policy is tactile-based; text-conditioned reward is presented there as the natural analogue of vision-based incentives (Guzey et al., 2023). In Text2Touch, language specifies the task and environment context used to synthesize reward code, but deployment proceeds through a tactile student policy rather than through language-conditioned inference (Field et al., 9 Sep 2025).

A second misconception is that more tactile sensing in the policy state is always required if tactile sensing was important during training. The grasp-refinement study shows the opposite pattern: richly tactile rewards during training can support later deployment with drastically reduced tactile state, including a no-tactile-state policy with only a modest performance decrease. This suggests a general separation between training-time evaluative privilege and test-time observation budget.

A third misconception is that adding tactile terms to reward is automatically beneficial. TAVI reports that tactile features in the OT reward can be gamed by contact-activation shortcuts such as pinching fingers together. Touch-R1 addresses an analogous shortcut problem by assigning tactile-use credit only when intact tactile input outperforms counterfactual perturbations. Text2Touch shows a different failure mode: text-conditioning alone is insufficient unless the prompt exposes success and failure scalars and explicitly structures a large tactile state space through a modified signature. Across these cases, reward design is less about maximizing the presence of touch than about making tactile information causally relevant to the correct behavior.

The limitations reported in the cited works are likewise heterogeneous. Shape dependence remains a challenge in grasp refinement, with spheres hardest and likely to roll when contacted. Touch-R1 is motivated by ordinal attributes and cross-sensor distribution shifts inherent in optical tactile hardware. TAVI is strongly sensitive to camera viewpoint and lacks historical context in the observation representation. Text2Touch depends on prompt engineering choices, privileged teacher training in simulation, and a structured variable interface. Taken together, these results indicate that text-conditioned tactile reward is best understood not as a single algorithm, but as a design paradigm for aligning tactile interaction with task semantics through analytic structure, counterfactual grounding, cross-modal incentives, or language-driven reward synthesis.