TouchThinker-Bench: Tactile Commonsense Benchmark

• TouchThinker-Bench is an evaluation benchmark that tests transferable tactile semantics through held-out objects and sensors in open-world settings.
• It assesses both classification-style and generation-based tactile reasoning by leveraging dynamic tactile interaction cues and action-aware modeling.
• Benchmark results demonstrate improved sensor invariance and cross-object generalization using methods like question-guided token fusion and Gaussian Temporal MoE.

TouchThinker-Bench is the dedicated evaluation benchmark introduced with "TouchThinker: Scaling Tactile Commonsense Reasoning to the Open World with Large-scale Data and Action-aware Representation" for open-world tactile commonsense reasoning (Lyu et al., 10 Jun 2026). Its function is complementary to the training corpus TouchThinker-1M: the larger corpus provides scale and diversity for tactile-language learning, whereas TouchThinker-Bench provides held-out evaluation designed to test whether a model has learned transferable tactile semantics rather than overfitting to narrow attribute labels, templated question answering, or sensor-specific appearance statistics. In the paper’s usage, the benchmark’s “open-world” character is made concrete through objects unseen during training, sensors unseen during training, and more realistic, open-ended task formats beyond fixed templates.

1. Purpose and conceptual scope

TouchThinker-Bench is motivated by the claim that prior tactile reasoning evaluation is limited by small datasets, template-heavy formats, and narrow sensor coverage, which makes those settings weak tests of genuine physical reasoning from touch (Lyu et al., 10 Jun 2026). The benchmark is therefore designed to broaden evaluation along three axes: task diversity, cross-object generalization, and cross-sensor generalization. This design choice is central to the benchmark’s role in the paper. It is not presented as a generic multimodal benchmark, nor as a training set, but as the evaluation counterpart to a larger tactile-language data pipeline.

The benchmark’s target capability is broader than static tactile attribute recognition. Across the paper, tactile interactions are associated with action types such as pressing, sliding, rotation, and twisting-like motions, because different actions reveal different physical properties: pressing reveals hardness, sliding friction, and rotation texture. As a result, TouchThinker-Bench evaluates whether a model can infer object properties and tactile commonsense from tactile interaction dynamics, not merely from isolated tactile frames.

A related clarification concerns what the benchmark is intended not to reward. The authors explicitly position it against evaluation regimes dominated by predefined attribute spaces or fixed question templates, since those can reward shallow answer-space matching rather than grounded tactile understanding. This suggests that TouchThinker-Bench is best understood as a transfer benchmark for tactile-LLMs under distribution shift in both object space and sensor space.

2. Construction protocol and source composition

TouchThinker-Bench is built partly from TouchThinker-1M and partly from new or external held-out data (Lyu et al., 10 Jun 2026). TouchThinker-1M itself is described at a high level as containing 1,001,344 tactile frames, spanning 9 source datasets, 7 tactile sensing platforms, 4 major acquisition actions, and over 415 deduplicated objects; in the abstract it is summarized as covering 415 objects, 8 scenarios, and 7 sensor types. The benchmark is formed first by selecting samples from TouchThinker-1M using an object-level 6:1 train-test split.

The appendix describes TouchThinker-Bench as having two components: a cross-object component built from held-out TouchThinker-1M test objects, and a cross-sensor component built from unseen-sensor sources. The appendix states that the benchmark covers 10 tactile sensors and 82 test objects. Elsewhere, however, the main benchmark section says that manual verification yields test samples across 200 object categories, with the category distribution shown in Figure 1(d). The paper gives both statistics and does not explicitly clarify the discrepancy; the safest reading provided in the paper is that 82 test objects refers to benchmark evaluation components, whereas 200 object categories may refer to a broader taxonomy used in the semantic distribution figure.

The benchmark’s source composition is detailed in the appendix.

Component	Source	Held-out detail
Cross-object	PhysiCLEAR	5 daily objects, GelSight var.1
Cross-object	VTV-150K	5 common objects, GelSight Mini, DIGIT, Tac3D
Cross-object	Touch and Go	18 material surfaces, GelSight17 var.1
Cross-object	YCB-Slide	3 YCB objects, DIGIT
Cross-object	Touch-Slide	3 toy-kitchen objects, DIGIT
Cross-object	HaTT	9 material textures, GelSight var.2
Cross-object	FeelSight-Real	1 real object, four DIGIT sensors on an Allegro hand
Cross-object	ObjectFolder-Real	9 real household objects, GelSight17 var.2
Cross-sensor	TacQuad / AnyTouch	DuraGel, 15 test objects
Cross-sensor	T3 / VisGel	GelSight Var. 3, 4 objects
Cross-sensor	Self-collected	10 categories of everyday objects, UR5 robotic arm, commercial GelSight Mini sensor

The three unseen-sensor sources are emphasized because these sensors differ in imaging mechanism, color distribution, and gel deformation characteristics, making them useful for measuring domain shift. The benchmark’s construction therefore operationalizes cross-sensor generalization as a distinct held-out regime, rather than treating sensor variation as incidental nuisance variation.

3. Task structure, input-output format, and open-ended QA generation

TouchThinker-Bench contains three core task categories (Lyu et al., 10 Jun 2026). The first is Basic Tactile Property Understanding. The main text lists the properties as hardness, roughness, elasticity, and friction, while the unified schema and several benchmark discussions use hardness, protrusion, elasticity, and friction; the result tables use Hardness, Protrusion, Elasticity, Friction, which is the operative evaluation decomposition. The second category is Basic Tactile Reasoning, consisting of Surface Feature Distinction (SFD), Surface Optimality Identification (SOI), Object Sensation Correlation (OSC), and Tactile Scenario Analysis (TSA). The third category is Open-ended Tactile Commonsense Reasoning, grouped into Touch Attribute Understanding (TAU), Touch Interaction Understanding (TIU), and Touch Knowledge Reasoning (TKU).

For the basic property and reasoning tasks, the input is a tactile video plus a textual question or prompt, and the output is a discrete final conclusion. For the open-ended tasks, the input is likewise tactile observation plus text, but the output is free-form language generation. The paper characterizes the benchmark accordingly as classification-style QA for the basic subtasks and generation-based QA for the open-ended subtasks.

The appendix describes the construction of the open-ended QA portion in some detail. The authors “construct open-ended tactile question-answering samples from TouchThinker-Bench using tactile images and their category labels,” apply a unified prompt template, use object category labels and tactile attribute cues as conditions to DeepSeek-V4 to generate candidate QA pairs, and then apply manual verification. The prompt begins:

“You are an AI tactile assistant interacting with a single touched object. You will understand the touched object from the text prompt, where the object class is <object class>. Imagine that you are physically touching this object.”

It also instructs:

“Design conversations between you and a person asking about this object. The answers should be written in a tone indicating that you are touching the object and answering based on tactile perception.”

and

“All answers must be based on the confirmed tactile attribute tendencies and the object class, and should not introduce details that cannot be supported by the given information.”

The paper gives “a microfiber cloth” as a concrete object-class example. For TAU, the prompt requests separate conversations for tactile attributes such as hardness, protrusion, elasticity, and friction. For TIU, it asks about graspability, pressure feedback, surface discriminability, and manipulation stability, selecting three and generating one question for each. For TKU, it requests more complex tactile-perspective commonsense questions involving background knowledge, usage patterns, and interaction scenarios, with more detailed answers and possibly multiple paragraphs. The authors further note that each question is queried independently to avoid unstable formatting and answer-segmentation errors before outputs are parsed, deduplicated, and formatted into single-turn tactile QA samples.

4. Evaluation protocol and reported results

The benchmark uses different evaluation protocols for structured and open-ended tasks (Lyu et al., 10 Jun 2026). For Basic Tactile Property Understanding and Basic Tactile Reasoning, the metric is subtask accuracy, computed by exact match of the final conclusion. The paper states: “For basic tactile property understanding and reasoning, we report subtask-level accuracy by exactly matching the final conclusions, without considering reasoning processes or semantic equivalence.”

For Open-ended Tactile Commonsense Reasoning, the reported metrics are METEOR, plus two LLM-judge scores from GPT-5 and DeepSeek-V4. The LLM judges score each answer on semantic correctness, tactile consistency, commonsense and reasoning plausibility, information completeness, and language quality, with final score

$S = \frac{1}{5}\sum_{i=1}^{5} s_i.$

The open-ended comparison includes Octopi-7B, Octopi-13B, VTV-LLM-7B, and TouchThinker-7B. On TAU, TouchThinker-7B reaches METEOR 34.06, GPT-5 8.17, and DeepSeek-V4 8.33. On TIU, it reaches 28.71, 7.49, and 7.21. On TKU, it reaches 27.43, 7.87, and 7.81. The paper states that TouchThinker-7B is best on all three TAU metrics, best on TIU METEOR and GPT-5, and best on all three TKU metrics. A notable nuance is that on TIU DeepSeek-V4, Octopi-13B scores 7.79, above TouchThinker-7B’s 7.21, so TouchThinker is second-best on that metric.

For the exact-match unseen-object and unseen-sensor evaluation, the same baseline set is used. TouchThinker-7B reports Hardness 68.3, Protrusion 69.7, Elasticity 70.6, Friction 51.7, Combined 37.2, SFD 68.2, SOI 52.1, OSC 47.2, TSA 62.0, and Average 58.6. The benchmark tables show it as best on every reported metric. Relative to VTV-LLM-7B, the gains include +16.6 on SFD, +9.3 on Average, and +8.8 on the difficult Combined property prediction. The paper interprets this pattern as evidence of more sensor-invariant tactile semantics, rather than sensor-specific appearance bias.

The paper emphasizes three benchmark-level findings. First, open-ended tactile reasoning is substantially harder than template-based QA. Second, generalization gaps under unseen sensors and unseen objects are real and large. Third, large-scale multi-source data together with action-aware modeling improve robustness and reduce “shallow semantic matching and spurious associations,” yielding answers that are “better grounded in tactile evidence and more logically consistent.”

5. Coupling to action-aware representation learning

Although TouchThinker-Bench is an evaluation benchmark, it is tightly coupled to the paper’s action-aware representation method (Lyu et al., 10 Jun 2026). The authors’ premise is that tactile signals are both redundant and action-specific, and that benchmark conditions involving unseen objects and unseen sensors make naive framewise memorization especially brittle. The benchmark is thus used to motivate and test question-guided token fusion and an action-aware Gaussian Temporal MoE.

Given a tactile video $V=\{I_t\}_{t=1}^{T}$, the frozen tactile encoder extracts framewise features $F=[f_1,\ldots,f_T]\in\mathbb{R}^{T\times d}$, and a frozen text encoder produces word-level and sentence-level question features projected into tactile space. The question-aware tactile representation is defined as

$$F_{\mathrm{qa}} = \mathrm{SelfAttn}\left(\mathrm{CrossAttn}\left(F,\tilde{Q}_w,\tilde{Q}_w\right)\right).$$

The temporal mixture then computes

$$f_{\mathrm{moe}} = \sum_{k=1}^{K}\pi_k \sum_{t=1}^{T} \alpha_{k,t} E_k(f_{\mathrm{qa},t}),$$

with Gaussian temporal weights

$$\alpha_{k,t} = \mathrm{Softmax}_{t}\left(-\frac{(\hat{\tau}_t-\mu_k)^2}{2\sigma_k^2}\right),$$

where $\hat{\tau}_t \in [0,1]$ is normalized time. The paper’s illustrative example is that when the question concerns protrusion, the model can dynamically localize pressing-related segments.

This coupling matters because TouchThinker-Bench is intended to expose failures of models that rely on average frame appearance or sensor texture rather than question-relevant contact dynamics. The ablation evidence reported elsewhere in the paper is consistent with this design. On VTV-150K, removing action-aware modeling lowers average basic reasoning performance from 67.0 to 61.6; removing Stage I tactile-text alignment lowers it to 58.3; removing Stage II supervised instruction tuning lowers it to 53.3. A more targeted ablation shows that Question-Guided Token Fusion + Gaussian Temporal MoE outperforms either component alone.

6. Limitations, ambiguities, and benchmark context

TouchThinker-Bench inherits several limitations explicitly acknowledged for the broader TouchThinker setup (Lyu et al., 10 Jun 2026). The tactile attribute schema remains limited to hardness, protrusion, elasticity, friction, and combinations, while omitting properties such as malleability and prickliness. The underlying data mostly involve short-term tactile interactions, since TouchThinker-1M primarily contains 6–7 second contact clips, which likely constrains the benchmark as a testbed for long-horizon tactile manipulation reasoning. The use of 7B and 14B LLM backbones also raises computational cost concerns. In the broader ethics discussion, the authors further caution that the benchmark’s sensor types, object categories, and attribute space cannot fully capture real-world variability, which could produce generalization biases for low-resource sensors, rare materials, or safety-critical scenarios.

A second limitation concerns documentation ambiguity. The benchmark is described as covering 10 tactile sensors and 82 test objects in the appendix, but the main text also refers to 200 object categories after manual verification, and the paper does not reconcile the two counts. There is also a minor inconsistency in the property vocabulary, where one part of the paper lists roughness and other parts, including the evaluation tables, use protrusion.

Within the broader literature on reasoning benchmarks, TouchThinker-Bench occupies a distinct position. THiNK is a think-aloud-inspired, feedback-driven framework for iterative revision of mathematical word problems rather than a static tactile benchmark (Yu et al., 26 May 2025). VTBench, introduced with V-Thinker, targets perception, instruction-guided interaction, and interactive reasoning in image-centric settings (Qiao et al., 6 Nov 2025). TIR-Bench broadens agentic “thinking-with-images” to 13 tasks requiring tool-based visual manipulation, such as cropping, rotating, and drawing auxiliary structure (Li et al., 3 Nov 2025). BenchBench shifts the focus again, evaluating automated benchmark generation through domain cards, quota-controlled suite construction, and designer–answerer matrices (Zheng et al., 21 Mar 2026). Against that background, TouchThinker-Bench is notable for making open-world transfer concrete in the tactile domain through held-out objects, held-out sensors, and a mixture of exact-match and free-form evaluation centered on tactile commonsense rather than purely visual or text-only reasoning.