Text2Touch: Automated Tactile Reward Design

• Text2Touch is a framework for automated tactile reward design that uses LLMs to synthesize concise, interpretable reinforcement-learning reward functions from detailed sensory and environment data.
• It employs an iterative loop combining LLM-generated Python code, vision-based TacTip tactile sensing, and sim-to-real distillation to optimize policies for multi-axis in-hand object rotation.
• Empirical evaluations show that LLM-crafted reward functions outperform human-engineered baselines with improved stability, reduced code complexity, and enhanced task performance.

Text2Touch is a framework for automated reward design in tactile robotic manipulation that uses LLMs to generate reinforcement-learning reward functions for real-world, multi-axis in-hand object rotation. The system combines LLM-crafted reward code, vision-based tactile sensing, and sim-to-real distillation on a tactile-enabled, fully actuated four-fingered dexterous robot hand. Its central claim is that concise, interpretable reward functions synthesized from language prompts and environment context can outperform a carefully tuned human-engineered baseline on challenging palm-up and palm-down rotation tasks, while reducing reward-engineering complexity and supporting faster translation from task specification to deployable tactile policies (Field et al., 9 Sep 2025).

1. Scope and problem setting

Text2Touch addresses a specific problem in dexterous manipulation: automated reward design for tactile in-hand rotation, rather than manual reward engineering for reinforcement learning. The work is motivated by the observation that LLMs are beginning to automate reward design for dexterous manipulation, but that prior work had not considered tactile sensing, despite touch being critical for human-like dexterity (Field et al., 9 Sep 2025).

The task domain is gravity-invariant, multi-axis object rotation. Policies are required to rotate objects about arbitrary axes $x$, $y$, and $z$, in both palm-up and palm-down configurations, while holding the object aloft. This formulation forces simultaneous handling of orientation control, stable grasp maintenance, and contact adaptation under changing gravitational load. The environment uses a rolling curriculum in which the target rotation is incremented each time a sub-goal is reached, enabling continuous evaluation over multiple full rotations in a single episode. Performance is measured by completed $360^\circ$ rotations per episode, episode longevity, and stability, including whether the object is dropped and whether touch is maintained (Field et al., 9 Sep 2025).

A common misunderstanding would be to read the title as referring to textual rendering into tactile output. In this framework, language serves as the specification modality for reward synthesis, while touch serves as the sensing modality for robotic control. The operational output is not a tactile display or text interface, but a learned manipulation policy.

2. Robotic embodiment and tactile state representation

The platform is a fully actuated Allegro Hand equipped with vision-based TacTip tactile sensors on each fingertip. These TacTip sensors provide high-resolution images that are processed into signals of contact, shear, force, and slip, which the system treats as essential for dexterous in-hand manipulation (Field et al., 9 Sep 2025).

The reinforcement-learning environment exposes over 70 state variables. These variables cover proprioceptive state, object pose, contact location and quality, tactile force vectors, and related quantities. The same high-dimensional environment context is provided both to the policy-learning process and to the LLM during reward-code generation. This is a defining architectural choice: the reward model is not generated from a short textual task description alone, but from a detailed interface to the environment’s available signals (Field et al., 9 Sep 2025).

The tactile objectives induced by these state variables are explicitly manipulation-centric. Reward functions are expected to guide the robot toward maintaining stable and adaptable grip, detecting and correcting slip, and coordinating regrasp or finger gaiting during multi-axis rotation. This makes Text2Touch distinct from work where touch is treated only as a contact detector or auxiliary signal. Here, tactile sensing is part of the reward-design substrate itself.

3. Iterative LLM reward synthesis

Text2Touch adopts an iterative reward-design loop inspired by Eureka-style reward generation. An LLM is prompted with a natural language task description and detailed environment context, and produces Python reward-function code. Candidate rewards are then used to train reinforcement-learning agents for a set number of steps. Their performance, including sub-goal successes and task metrics, is fed back to the LLM for subsequent refinement (Field et al., 9 Sep 2025).

The paper summarizes the loop as

$$\text{Initialization:}~\{R^{(0)}\} \sim \text{LLM}(l, M)$$

and for each iteration $k \ge 0$,

$$\begin{align*} & \text{Train policies } \{\pi^{(k)}\} \text{ with rewards } \{R^{(k)}\} \ & \text{Evaluate: } s^{(k)} = F(\{\pi^{(k)}\}) \ & R^* = \text{SelectBest}(\{R^{(k)}, s^{(k)}\}) \ & \{R^{(k+1)}\} \sim \text{LLM}(l, M, \text{Feedback}(s^*, R^*)) \end{align*}$$

The LLMs used in this process include GPT-4o, Gemini-1.5-Flash, Llama3.1-405B, Deepseek-R1-671B, and o3-mini. A core requirement is that the generated reward code be valid, effective, and compatible with torch.jit.script. The resulting reward functions are described as interpretable and succinct, typically consisting of terms for object position and orientation, tactile contact, and sparse success bonuses (Field et al., 9 Sep 2025).

One reported example, generated by Gemini-1.5-Flash, has the form

$$\text{total\_reward} = 2 \cdot \text{pos\_reward} + 3 \cdot \text{orn\_reward} + \text{contact\_reward} + \text{sparse\_reward} - \text{early\_reset\_penalty}$$

with exponential position and orientation terms, a contact-quality term using $\tanh$, and a sparse bonus gated by a positional threshold. The broader pattern is a small set of well-scaled reward components tied directly to pose error, contact quality, and success (Field et al., 9 Sep 2025).

4. Prompt engineering for high-dimensional tactile environments

A major technical issue in Text2Touch is scale. The paper states that LLMs previously struggled to produce valid reward code when prompted with environments containing a large number of state variables. The solution is a prompt-engineering strategy designed to scale to over 70 environment variables (Field et al., 9 Sep 2025).

The prompt structure includes explicit reward-function signatures containing the full function header, all input variables, and their types. The LLM is also scaffolded with a system prompt that constrains code style and validity, with environment code, task description, and, in later iterations, optimization feedback. This makes the LLM operate less like a free-form code generator and more like a reward-synthesis component attached to a formalized environment interface.

Another design change concerns success bonuses and penalties. Instead of using fixed bonus or penalty values, Text2Touch exposes these as scalar inputs in the prompt so that the LLM can choose appropriate scaling in the final reward code. The paper contrasts an original form

$R_\text{total} = R_\text{LLM}(o) + B$

with the final Text2Touch setup

$y$0

which gives the model more flexibility to balance reward magnitudes across heterogeneous terms (Field et al., 9 Sep 2025).

The framework also uses reward reflection: policy feedback, including success rates and component statistics, is returned to the LLM so that it can analyze and revise subsequent reward code. This suggests a tightly coupled optimization loop in which language generation and policy evaluation are mutually conditioning stages rather than isolated steps.

5. Distillation, transfer, and empirical performance

Text2Touch bridges simulation and hardware through a teacher-student distillation pipeline. The teacher policy is trained in simulation with privileged observations, including full object pose and velocities, using the best LLM-generated reward function. The student policy is then trained by distillation to mimic the teacher’s action distribution using mean-squared error on actions, but it receives only proprioceptive and tactile inputs. The student is directly deployed on the real Allegro Hand with TacTip sensors, and no further fine-tuning is needed for hardware operation (Field et al., 9 Sep 2025).

The empirical results reported in the paper indicate consistent gains over the human-engineered baseline in simulation, on out-of-distribution objects, and on real hardware. In simulation, the best LLM-designed reward reaches 5.48 rotations per episode versus 4.92 for the best human baseline. On tactile-only student policies evaluated on out-of-distribution heavy objects, the LLM-distilled student reaches 3.38 average rotations per episode versus 2.94 for the baseline. On real hardware, all three tested LLM-based policies outperform the human baseline in both average full rotations per episode and average time to termination (Field et al., 9 Sep 2025).

Policy source	Sim rotations/episode	Real hardware: avg full rotations / avg time
Human baseline	4.92	0.99 / 20.0 s
Gemini-1.5-Flash	5.48	1.28 / 23.8 s
Deepseek-R1-671B	5.26	1.37 / 25.1 s
GPT-4o	5.46	1.18 / 20.7 s

The reward functions are also materially simpler than the human baseline. The paper states that they use roughly one-tenth the variables, one-quarter the lines of code, and one-eighth the code complexity as measured by Halstead Volume. At the same time, they attain longer episodes and better stability. The authors further report that the LLM-based policies are more stable in-hand in the real world, with richer tactile sensing compensating for the fast policy dynamics learned in simulation (Field et al., 9 Sep 2025).

6. Position within adjacent touch and language research

Text2Touch belongs to a broader research landscape in which touch, language, and generative or inferential models are increasingly coupled, but it occupies a distinct niche. In mobile HCI, touch gestures have been used to control LLM-mediated text editing directly on the text itself through mappings such as spread-to-generate and pinch-to-shorten, with “word bubbles” providing a closed-loop visual feedback mechanism (Zindulka et al., 11 Feb 2025). That line of work concerns end-user interaction with LLMs on touchscreens, not reward synthesis for robotic control.

In visuo-tactile generation, latent diffusion has been used to synthesize images from tactile signals and vice versa, including tactile-driven stylization, handless image generation, and tactile-driven shading estimation (Yang et al., 2023). Here, touch is a conditioning modality for generative scene synthesis. By contrast, Text2Touch uses language as a conditioning modality to synthesize reward code, and touch as the sensor substrate for downstream manipulation policy learning.

Other neighboring areas emphasize sensing or input rather than tactile reward design. The 2DR system demonstrates fine-grained two-dimensional touch localization from a single custom RFID tag via phase-difference features and statistical learning (Zhu et al., 2018). Finger Based Techniques for non-visual touchscreen text entry map virtual keys to finger-holding positions for blind users and identify 11 accessible regions on smartphones (Fakrudeen et al., 2017). TapType decodes ten-finger text entry on passive surfaces from wrist-worn IMUs by combining Bayesian finger classification with n-gram LLMs (Streli et al., 2024). These works show the breadth of touch research across accessibility, ubiquitous sensing, and mobile text input, but none targets the automated design of tactile reinforcement-learning rewards for dexterous manipulation.

Within that landscape, Text2Touch’s specific significance is methodological. It treats natural language not as a command channel to a deployed policy, but as a medium for constructing the reward functions that shape policy learning. This suggests a broader role for LLMs in multimodal robotics: not merely generating text, plans, or code fragments, but producing compact optimization objectives that can exploit rich tactile state in high-dimensional environments (Field et al., 9 Sep 2025).