TacSL Benchmark: Visuotactile Simulation

• TacSL benchmark is a visuotactile sensor simulation library that standardizes contact-rich robotic tasks like peg insertion and bolt alignment with calibrated sensor models.
• It leverages GPU-accelerated simulation to generate high-speed tactile RGB images and force fields, ensuring robust randomization and scalable learning.
• TacSL supports multiple policy-learning setups including behavior cloning and asymmetric actor-critic methods, demonstrating effective sim-to-real transfer in contact-intensive tasks.

Searching arXiv for TacSL benchmark and closely related papers to ground the article. TacSL is a GPU-based visuotactile sensor simulation and learning library that also functions as a benchmark suite for contact-rich robotic manipulation, particularly peg placement, peg insertion, and bolt-on-nut alignment tasks in Isaac Gym (Akinola et al., 2024). It standardizes sensor models, contact-intensive environments, learning setups, and evaluation procedures for studying tactile image policies, multimodal visuotactile policies, and sim-to-real transfer. In later work, TacSL is also used explicitly as a standard insertion benchmark for evaluating visuo-tactile fusion strategies under randomized, contact-rich conditions, with the same insertion task setup and success criteria adopted from the original framework (Lee et al., 14 Feb 2026).

1. Definition and benchmark scope

TacSL, pronounced “taxel,” is presented as a library for visuotactile sensor simulation and learning built on top of NVIDIA’s Isaac Gym (Akinola et al., 2024). Its benchmark is not limited to a single task or metric. Rather, it consists of a standardized combination of sensor models, contact-rich manipulation tasks, learning algorithms, and evaluation procedures. The stated goals are to make tactile simulation fast enough for large-scale on-policy RL and online distillation, to provide a standardized suite of tasks and sensors for comparing tactile learning methods, and to enable end-to-end tactile image policies for contact-rich manipulation to transfer from simulation to the real world.

The benchmark comprises four tightly coupled elements. First, it includes visuotactile sensor models, specifically GelSight-like sensors, with both tactile RGB images and normal/shear force fields. Second, it includes manipulation tasks that require fine contact reasoning: Peg Placement, Peg Insertion, and Bolt-on-Nut Alignment for Screwing. Third, it provides multiple learning setups, including behavior cloning, DAgger-style online distillation, asymmetric actor-critic RL, and Asymmetric Actor-Critic Distillation. Fourth, it defines an evaluation protocol centered on success rates in simulation, real-world success rates under zero-shot sim-to-real transfer, simulation speed, and modality and algorithm ablations (Akinola et al., 2024).

A later paper frames TacSL more narrowly as a standard insertion benchmark for contact-rich manipulation with vision and touch, emphasizing its role as a source of environment configuration, sensor setup, evaluation criteria, and a baseline reference for fusion strategies (Lee et al., 14 Feb 2026). In that usage, the benchmark is explicitly associated with a plug or peg insertion task, randomized poses and contact parameters, and comparative evaluation of wrist vision, tactile sensing, contact-force signals, and visuo-tactile fusion methods.

2. Task suite and randomized problem structure

TacSL’s core benchmark tasks are contact-rich assembly-style manipulations performed with a parallel-jaw gripper equipped with visuotactile sensing on each finger (Akinola et al., 2024). The main tasks are Peg Placement and Peg Insertion, with Bolt-on-Nut Alignment positioned as a challenge task.

Peg Placement requires the robot to place a cylindrical rod upright on a flat support plate so that the peg remains stable after release. Difficulty is induced by peg misalignment in the gripper, randomization of peg position, and randomization of the initial end-effector pose. The policy does not receive direct access to object pose in the gripper and must infer peg orientation and pose from contact geometry.

Peg Insertion requires a cylindrical peg to be inserted into a cylindrical socket fixed to a table. The initial peg pose in the gripper is randomized in position and orientation; the socket position is randomized in front of the robot; a noisy estimate of socket pose is provided; and physics parameters such as compliant stiffness, compliant damping, and robot joint damping are randomized. This produces a task in which visual localization is imprecise, occlusion occurs near contact, and tactile sensing becomes important for precise alignment and smooth insertion.

The Bolt-on-Nut Alignment task requires a bolt held in the gripper to be aligned with a threaded hole in a nut so that an automatic screwing primitive succeeds. The paper identifies it as more sensitive to rotational misalignment than Peg Insertion, and it is treated as a challenge task rather than the main quantitative focus (Akinola et al., 2024).

For the insertion setting reused in later work, the environment is characterized as a robustified insertion benchmark rather than a single easy peg-in-hole configuration (Lee et al., 14 Feb 2026). The reported randomization ranges include end-effector position with $X \in [0.4, 0.6]$ m, $Y \in [-0.1, 0.1]$ m, and $Z \in [0.1, 0.2]$ m; end-effector Euler orientation with $X \in [3.04, 3.24]$ rad, $Y \in [-0.1, 0.1]$ rad, and $Z \in [-1.0, 1.0]$ rad; socket position with $X \in [0.4, 0.6]$ m, $Y \in [-0.1, 0.1]$ m, and $Z \in [0.0, 0.02]$ m; peg-in-gripper $Z$ offset in $Y \in [-0.1, 0.1]$0 m; peg rotation about the $Y \in [-0.1, 0.1]$1-axis in $Y \in [-0.1, 0.1]$2 rad; socket XYZ perceived-location noise in $Y \in [-0.1, 0.1]$3 m on each coordinate; compliance stiffness in $Y \in [-0.1, 0.1]$4 N/m; compliance damping in $Y \in [-0.1, 0.1]$5 N/(m/s); and joint damping noise in $Y \in [-0.1, 0.1]$6 N/(m/s).

These randomization schemes define TacSL’s benchmark difficulty. They enforce generalization across initial conditions, contact parameters, and observation uncertainty rather than permitting memorization of a fixed insertion geometry. A plausible implication is that TacSL measures not merely control quality but also robustness to uncertainty at the interface of vision, touch, and contact dynamics.

3. Sensor models, observations, and simulation machinery

TacSL’s benchmark is organized around visuotactile sensing rather than purely kinematic state estimation (Akinola et al., 2024). It supports GelSight R1.5 and GelSight Mini models, each specified by a visual mesh, a collision mesh, soft contact parameters, camera pose and intrinsics, and a calibrated lookup table mapping depth to RGB. The simulator outputs both tactile RGB images and tactile force fields, including normal and shear components at sampled surface points.

The underlying simulation is implemented inside Isaac Gym with almost all computation on the GPU. Physics uses a GPU-accelerated soft contact model; tactile images are generated via depth rendering followed by a GPU-side LUT; and tactile force fields are computed with GPU kernels using precomputed SDFs. TacSL’s soft contact model is based on a Kelvin-Voigt formulation, with normal force

$Y \in [-0.1, 0.1]$7

and an implicit discrete-time formulation used for stability with stiff springs. Tactile images are produced by rendering a depth image and applying a calibrated mapping

$Y \in [-0.1, 0.1]$8

Tactile force fields are computed from interpenetration and tangential motion using penalty-based expressions for normal and shear forces.

In the original benchmark, high-dimensional observation channels include tactile RGB images of size $Y \in [-0.1, 0.1]$9 per finger, tactile force fields of size $Z \in [0.1, 0.2]$0 per finger, and a wrist RGB camera of size $Z \in [0.1, 0.2]$1, alongside reduced-state and privileged-state features (Akinola et al., 2024). In the later insertion-fusion study, the TacSL instantiation uses a wrist-mounted RGB camera at $Z \in [0.1, 0.2]$2, tactile arrays of size $Z \in [0.1, 0.2]$3 per finger with channels $Z \in [0.1, 0.2]$4, and an optional privileged $Z \in [0.1, 0.2]$5 contact force vector (Lee et al., 14 Feb 2026). The later paper also describes the visual encoder as a 3-layer CNN followed by Spatial SoftArgMax and describes the tactile preprocessing as residual-force computation relative to calibrated references or zero for symmetric objects.

TacSL’s simulation speed is a central benchmark attribute. For tactile image generation, TacSL reports 1631 FPS at 512 parallel environments versus 7.28 FPS for Taxim on the stated comparison hardware, corresponding to over $Z \in [0.1, 0.2]$6 faster performance (Akinola et al., 2024). For tactile force fields, reported throughput reaches 1,541,043 FPS at 32768 parallel environments for a $Z \in [0.1, 0.2]$7 taxel grid, with a speedup of approximately $Z \in [0.1, 0.2]$8 relative to the cited CPU baseline, and 103,493 FPS at 4096 environments for a $Z \in [0.1, 0.2]$9 grid, with a speedup of approximately $X \in [3.04, 3.24]$0. These figures define TacSL not only as a benchmark specification but also as an infrastructure benchmark for scalable tactile RL.

4. Learning formulations and policy training

TacSL’s benchmark is explicitly designed to compare multiple policy-learning paradigms under a shared task and sensor setup (Akinola et al., 2024). The principal methods are policy distillation, asymmetric actor-critic, and Asymmetric Actor-Critic Distillation.

In policy distillation, a teacher policy $X \in [3.04, 3.24]$1 is trained on privileged low-dimensional state using PPO, and a student policy $X \in [3.04, 3.24]$2 is trained from realistic observations using either offline behavior cloning or online DAgger-style aggregation. In asymmetric actor-critic, the actor consumes realistic or partial observations while the critic consumes privileged simulator state, providing value estimates that are more informative than those obtainable from partial observations alone. In AACD, training is staged: a low-dimensional actor-critic expert is first trained on privileged state; then a new high-dimensional actor is initialized for realistic observations while reusing the learned critic as a pretrained critic, either frozen or fine-tuned.

The benchmark’s action space is a 6-D relative end-effector pose command with maximum position displacement per axis of 0.01 m and maximum orientation displacement per axis of 0.05 rad, executed via a task-space impedance controller at 60 Hz (Akinola et al., 2024). PPO is used with standard hyperparameters, including clip ratio 0.2, $X \in [3.04, 3.24]$3, and $X \in [3.04, 3.24]$4. In the later TacSL insertion study, the policy is likewise a Gaussian policy with $X \in [3.04, 3.24]$5 action outputs, trained with PPO using learning rate $X \in [3.04, 3.24]$6, rollout length 512, batch size 512, PPO epochs 4, $X \in [3.04, 3.24]$7, $X \in [3.04, 3.24]$8, clip ratio 0.2, entropy coefficient 0, and value loss coefficient 2 (Lee et al., 14 Feb 2026).

The original benchmark also defines a task reward for peg tasks as

$X \in [3.04, 3.24]$9

where the terms respectively penalize keypoint misalignment, action magnitude, and large contact forces (Akinola et al., 2024). By contrast, the later insertion-fusion study states that TacSL’s insertion success is modeled with a sparse reward,

$Y \in [-0.1, 0.1]$0

and that the policy maximizes the expected discounted sum of this reward while using TacSL’s success criteria (Lee et al., 14 Feb 2026). This suggests that TacSL supports more than one reward formulation across task instantiations and research uses, while retaining a stable benchmark identity through task definitions, sensing assumptions, and success-based evaluation.

5. Evaluation protocol and empirical performance

TacSL’s primary benchmark metric is success rate (Akinola et al., 2024). In simulation, success rate is computed over 1024 randomized episodes per policy per condition; in real-world evaluation, success is reported over 81 episodes. For insertion in the later fusion study, success rate is defined as

$Y \in [-0.1, 0.1]$1

and is reported as mean $Y \in [-0.1, 0.1]$2 standard deviation over three random seeds (Lee et al., 14 Feb 2026). The later study also reports steps to success as an auxiliary efficiency metric.

The benchmark includes evaluation across sensing modalities and learning algorithms. On Peg Insertion in simulation, TacSL reports that privileged state with DAgger or AACD reaches approximately 95–97% success, reduced state alone is poor at approximately 5–9%, reduced state plus tactile images reaches approximately 82–91%, reduced state plus wrist reaches approximately 93–97%, and reduced state plus tactile and wrist reaches up to approximately 94% (Akinola et al., 2024). The paper interprets this as evidence that both touch and vision improve performance over reduced state, and that combined sensing tends to yield the best performance and sample efficiency.

Real-world zero-shot transfer is another core benchmark dimension. For Peg Placement, TacSL reports 27.2% success for a Vanilla Color tactile-image policy trained without augmentation, 87.7% for ColorAug, 91.4% for Diff+ColorAug, and 77.9% for Concat+ColorAug, all over 81 trials (Akinola et al., 2024). For Peg Insertion, a ColorAug policy trained in simulation achieves 82.7% success, corresponding to 67/81 successful trials. These results ground TacSL’s claim to be a sim-to-real benchmark rather than a simulation-only benchmark.

The later visuo-tactile fusion study uses TacSL’s insertion setup to compare observation configurations more finely (Lee et al., 14 Feb 2026). The reported success rates are summarized below.

Configuration	Success rate	Notes
Privileged	96.74 ± 1.63 %	Compact state only
Privileged + contact forces	98.96 ± 0.83 %	Highest reported baseline
Tactile only	91.41 ± 5.51 %	Reduced sensing
Wrist only	93.23 ± 2.00 %	Vision only
Wrist + contact forces	96.09 ± 1.41 %	Privileged visuo-force
Naïve Fusion	92.97 ± 1.41 %	Simple concatenation
Gated Fusion	94.53 ± 2.73 %	No symmetry regularization
CMT Fusion	96.22 ± 0.98 %	No symmetry regularization
Gated Fusion + symmetry regularization	95.05 ± 1.76 %	$Y \in [-0.1, 0.1]$3
CMT + symmetry regularization	96.59 ± 2.11 %	Main reported model

These results are used to support two conclusions about TacSL as a benchmark. First, tactile sensing supplies information that vision alone cannot recover reliably near contact, especially under occlusion and micro-misalignment. Second, the benchmark is sensitive to fusion design: naïve visuo-tactile concatenation yields negligible or negative gains, while structured self- and cross-attention with symmetry-based regularization approaches privileged performance (Lee et al., 14 Feb 2026). A plausible implication is that TacSL is diagnostically useful not only for ranking policies but also for distinguishing between competing theories of multimodal fusion in contact-rich control.

6. Benchmark role, later extensions, and limitations

TacSL is positioned against prior tactile simulators such as Taxim, TACTO, and the force-field framework of Xu et al. (2023), as well as against broader visuomotor benchmarks such as RoboNet and RLBench (Akinola et al., 2024). Its distinguishing features are GPU-scale visuotactile simulation, support for arbitrary triangle meshes through SDF-based force computation, explicit integration of tactile images and force fields, contact-intensive assembly-style tasks, and demonstration of zero-shot sim-to-real transfer. The later visuo-tactile fusion paper further positions TacSL as the canonical testbed for insertion with vision and touch, especially for studying the gap between realistic sensing and privileged sensing (Lee et al., 14 Feb 2026).

The benchmark is also notable for the way later work inherits its evaluation framework. The fusion study states that it follows the evaluation framework established in TacSL and adopts the same insertion task setup and success criteria, while using the TacSL insertion task configuration in IsaacGymEnvs. In that study, TacSL is not redefined from scratch; instead, it serves as an established benchmark environment whose sensor modalities, randomization protocol, and success-based evaluation are already standardized. This continuity is a strong indicator of benchmark maturity.

Several limitations are explicitly stated in the original TacSL work (Akinola et al., 2024). The contact model is linear Kelvin-Voigt rather than a nonlinear alternative such as Hunt-Crossley. The depth-to-RGB mapping is a hand-calibrated polynomial LUT, with future work suggested around learned generative mappings. Sim-to-real experiments focus on tactile image modality rather than transferring force-field modality. The policy architectures are CNN+LSTM+MLP rather than more expressive architectures such as transformers or diffusion-based policies. The task set is contact-rich but limited primarily to peg and bolt assembly tasks, with future directions including dexterous in-hand manipulation, deformable object manipulation, and more complex industrial assembly sequences.

These limitations delimit the scope of the TacSL benchmark rather than undermining it. Within its intended domain, TacSL standardizes a reproducible and computationally scalable setting for evaluating visuotactile sensing, multimodal fusion, contact-rich policy learning, and sim-to-real transfer. Later results on symmetric plug insertion further suggest that TacSL is especially effective as a stress test for whether tactile information is used in a principled way, rather than merely appended to visual features (Lee et al., 14 Feb 2026).