UltraTac: Ultrasound-Enhanced Tactile Sensor

• UltraTac is an integrated ultrasound-augmented visuotactile sensor that combines optical touch with acoustic interrogation to enable material and content recognition in robotics.
• It employs a coaxial optoacoustic architecture aligning a micro-camera with an annular ultrasound transducer to achieve true multimodal fusion and spatial co-registration.
• Through dynamic switching of sensing modes, UltraTac delivers high accuracy in proximity detection (R² up to 0.99), material classification (99.20%), and dual-mode recognition (92.11%).

UltraTac is an integrated ultrasound-augmented visuotactile sensor for robotic perception that combines visuotactile imaging with ultrasound sensing through a coaxial optoacoustic architecture. It is designed to address a specific limitation of visuotactile sensors: although they provide high-resolution tactile information, they are incapable of perceiving the material features of objects. The system therefore couples optical touch sensing with acoustic interrogation, shares structural components across the two modalities, and maintains a consistent sensing region for both. It further incorporates acoustic matching into the traditional visuotactile sensor structure and uses tactile feedback to dynamically adjust the operating state of the ultrasound module for flexible functional coordination (Gong et al., 28 Aug 2025).

1. Functional objective and sensing rationale

UltraTac targets three perceptual functions within a single sensor stack: pre-contact proximity sensing, contact-based material classification, and joint texture-material recognition. In the reported robotic setting, these functions are further extended to concurrent detection of container surface patterns and internal content. The underlying motivation is that visuotactile sensing alone can recover local contact information, but not material properties or hidden internal-state cues; ultrasound is introduced to supply those missing channels (Gong et al., 28 Aug 2025).

The design rationale is explicitly multimodal rather than merely additive. Optical and acoustic pathways are arranged so that both modalities sense the same physical area, which reduces spatial offset between what is imaged tactually and what is acoustically interrogated. This co-location is central to the claim of “true multi-modal fusion” in the reported experiments. A plausible implication is that the sensor is intended not only to improve classification performance, but also to simplify cross-modal registration during manipulation.

UltraTac also uses state-dependent coordination. Before contact, the ultrasound pathway operates in a proximity-sensing mode based on time-of-flight in air. After contact is detected from touch image frames, the system switches the ultrasound role to material or content sensing while the visuotactile channel supports texture and pattern analysis. This dual-pathway switching is part of the system design rather than a downstream application-layer heuristic.

2. Coaxial optoacoustic architecture and material stack

The defining structural feature of UltraTac is a coaxial optoacoustic architecture in which a narrow micro-camera is aligned along the central axis of an annular piezoelectric ultrasound transducer. The transducer is an annular PZT ring resonant at 1.05 MHz, while the micro-camera and illumination ring are integrated inside the ring. This arrangement produces spatial alignment between optical and acoustic sensing and allows the imaging region, approximately 15 mm, to match the ultrasound scan area (Gong et al., 28 Aug 2025).

The layered stack is specified in detail. The top layer is a translucent, ultra-thin PDMS elastomer with a 30:1 base-to-curing-agent mix. The conventional opaque membrane used in visuotactile sensors is replaced by PDMS mixed with hollow glass microspheres at a 1:1 volume ratio. This composite serves simultaneously as a light-blocking layer for optical imaging and as an acoustic matching layer that reduces impedance mismatch at the PDMS-air boundary. Beneath this is a 0.7 mm acrylic support layer selected for quarter-wavelength acoustic matching. The back layer is tungsten-loaded epoxy with a 3:2 volume ratio, used as backing to absorb stray acoustic waves, avoid echo artifacts, and fulfill impedance matching for the PZT (Gong et al., 28 Aug 2025).

The matching design is described with the standard relations

$Z_m = \sqrt{Z_1 \cdot Z_2}$

for single-layer acoustic impedance matching and

$d = \frac{c_m}{4f}$

for quarter-wavelength thickness selection. The reported acoustic impedances include PDMS at 1.1 MRayl, acrylic at 3.2, PZT at 25–35, and tungsten at 100. These values are presented as material-selection guidance for the integrated acoustic-optical stack rather than as abstract background formulas.

The electronics are compact and mode-aware. A 4 × 4 cm PCB integrates a 144 MHz MCU and an analog front end. The ultrasound subsystem can dynamically adjust excitation up to 40 V pulses, receiver gain up to 55.5 dB, and switching behavior through touch-based feedback. The reported acquisition rate is approximately 50 Hz. The visuotactile module uses a 6 mm diameter micro-camera with 1.88 mm focal length and surrounding LED illumination arranged to avoid shadowing the ultrasound pathway.

3. Sensing modes and computational pipeline

UltraTac’s processing pipeline is organized into sensor, preprocessing, and processing levels. At the sensor level, the ultrasound path consists of the analog front end and MCU, while the optical path consists of the camera and LED illumination. During preprocessing, ultrasound signals are filtered with a Kalman filter, visual data undergo augmentation, and the two streams are synchronized at 50 Hz for ultrasound and 30 Hz for imaging. At the processing level, a touch trigger determines whether the system runs a pre-contact proximity pathway or a post-contact recognition pathway (Gong et al., 28 Aug 2025).

In proximity mode, the system uses ultrasound time-of-flight in air. Distance is calculated from the time to peak in the echo after removing an environmental reference baseline. This is explicitly a non-contact measurement mode, and it is the mechanism that supports approach and grasp alignment before the visuotactile channel enters a deformation-driven regime.

In material-classification mode, the system analyzes reflected ultrasound echoes in the frequency domain. The reported features include spectral contrast, kurtosis, skewness, and entropy, and the classifier is XGBoost. In texture analysis, the visuotactile image is processed by a ResNet18 convolutional neural network. In the dual-mode recognition setting, ultrasound-derived material information and visuotactile image-derived surface-pattern information are combined to generate a joint prediction.

This division of labor across modalities is important. UltraTac does not treat ultrasound as a redundant secondary signal for contact geometry; it assigns ultrasound to proximity, material, and internal-content inference, while visuotactile imaging supports surface pattern and fine tactile appearance. That allocation is consistent with the paper’s motivating distinction between material features and image-based touch features.

4. Reported quantitative performance

The paper reports systematic evaluation of UltraTac on proximity sensing, material classification, texture-material dual-mode recognition, and robotic sorting tasks. The core results are summarized below (Gong et al., 28 Aug 2025).

Capability	Method	Reported result
Proximity sensing	Ultrasound ToF in air	3–8 cm range; abstract reports $R^2 = 0.90$
Proximity sensing	Ultrasound ToF in air	Detailed summary reports $R^2 = 0.99$ and typical error $< \pm 0.5$ cm
Material classification	Fourier features + XGBoost	99.20% average accuracy
Texture-material dual-mode recognition	Ultrasound + visuotactile fusion	92.11% accuracy on a 15-class task
Robotic sorting	Dual-modal sensing on grasped objects	All containers correctly identified and placed; summary table reports 100% correct placement

The proximity experiments were conducted across five tested materials—acrylic, iron, nylon, resin, and wood—and are described as demonstrating material-independence. The detailed summary reports a dynamic range of 3–8 cm, linearity $R^2 = 0.99$, and typical error below $\pm 0.5$ cm; the abstract separately reports proximity sensing in the 3–8 cm range with $R^2 = 0.90$. Both values appear in the provided record and therefore represent two reported characterizations of the same sensing mode.

For material classification, the contact-based ultrasound task uses five classes: acrylic, iron, nylon, rubber, and wood. The reported average accuracy is 99.20%. PCA is described as showing clear class clustering, and the confusion matrix reportedly indicates 100% accuracy for iron, rubber, and wood, with minor confusion between acrylic and nylon.

For texture-material dual-mode recognition, the task contains 15 classes formed by three materials and five patterns: circle, rectangle, hexagon, triangle, and stripe. The reported average accuracy is 92.11%, with errors mainly between similar shapes within the same material. The reported interpretation is that the system provides robust dual-modal discrimination.

An ablation statement in the summary indicates that all UltraTac components contribute to performance: removing gating or adaptation of $t$ or $\pi$ reduces both accuracy and calibration, and removing guards harms stability in long streams. However, that statement belongs to the separate UL-TTA paper rather than UltraTac and is not part of UltraTac’s results; it should not be conflated with the sensor’s own reported ablations.

5. Robotic manipulation and content-aware interaction

UltraTac is evaluated in a robotic manipulation system using two sensors mounted on a robot gripper attached to a robot arm. The task involves nine containers with varied surface patterns and internal contents—air, water, and oil—that must be recognized and sorted based on both surface pattern and internal content. This setup is intended to show that the sensor can support perception of external and internal object attributes within a single manipulation cycle (Gong et al., 28 Aug 2025).

The reported workflow is staged. During approach and grasp, ultrasound time-of-flight provides pre-contact proximity information for safe and accurate approach. Once contact is detected, the touch trigger initiates tactile image processing for texture and pattern recognition, together with ultrasound spectral analysis for internal-content inference. The robot then transports the object to a location determined jointly by pattern and content type.

The reported outcome is that all containers are correctly identified and placed, and that confusion between containers with the same pattern but different content, or vice versa, is avoided. The summary explicitly states that only dual-modal fusion—ultrasound plus visuotactile sensing—achieves robust recognition in this scenario. In this sense, the application is not merely a demonstration of multimodal sensing in isolation; it is a demonstration of task-level disambiguation where surface appearance and internal state must both be inferred.

The paper further identifies application potential in quality control, non-destructive testing, dexterous in-hand manipulation, safe handling of delicate or unseen items, warehouse and industrial automation, assistive robotics, advanced human-machine interfaces, and precise robotic manipulation. These are presented as prospective deployment domains rather than as experimentally validated benchmarks.

6. Position within multimodal tactile sensing research

UltraTac occupies a specific position within recent multimodal tactile sensing research: it augments a visuotactile sensor with ultrasound while preserving a shared sensing region through coaxial integration. Its emphasis is on acoustic-optical co-location, acoustic matching within the elastomeric stack, and dynamic switching between pre-contact and contact-based functions (Gong et al., 28 Aug 2025).

A useful comparison is TransTac, which addresses a different limitation in the same broader area. TransTac integrates visual observation and marker-based tactile reconstruction through a transparent ultraviolet-encoded elastomer and binocular RGB/UV imaging, with strong emphasis on visual transparency, marker localization, and prior-guided Delaunay stereo matching (Yang et al., 3 Jun 2026). UltraTac and TransTac therefore represent two distinct integration strategies: acoustic augmentation in one case, and transparent visuo-tactile modality transition in the other.

This suggests that current sensor development is exploring at least two complementary routes to richer robotic perception. One route, exemplified by UltraTac, adds an acoustic channel to recover proximity, material, and internal-content information that visuotactile imaging alone cannot provide. The other, exemplified by TransTac, re-engineers the tactile medium itself to preserve visual transparency and improve cross-modal alignment. The comparison does not imply equivalence of tasks or benchmarks; rather, it clarifies that UltraTac’s contribution is specifically tied to ultrasound-augmented perception and content-aware manipulation.

Within that landscape, UltraTac’s main significance lies in the claim that mechanical and signal-level co-location enables real multi-modal fusion across pre-contact sensing, contact-based material recognition, and surface-pattern analysis. Its reported performance on proximity sensing, 99.20% material classification, 92.11% texture-material dual-mode recognition, and successful robotic sorting positions it as a concrete example of ultrasound-enhanced visuotactile sensing for robotic perception and manipulation (Gong et al., 28 Aug 2025).