Sound of Touch: Acoustic Sensing in Tactile Interfaces

• Sound of touch is a family of methods that use acoustic and vibro-acoustic signals to analyze, localize, and render tactile interactions.
• Techniques employing passive and active sensing achieve high-resolution contact localization and force estimation in robotic manipulation and human-computer interaction.
• Emerging designs integrate physical modeling with learning-based inference, advancing applications in material recognition, security, and affective communication.

In current usage across this literature, the “sound of touch” denotes a family of methods that treat touch as an acoustic or vibro-acoustic phenomenon: contact can be localized, tracked, classified, or rendered by analyzing vibrations generated by impact, friction, or structural resonance, and tactile experiences can conversely be synthesized through acoustically driven structures. The topic spans robotic manipulation, large-area touch interaction, soft tactile skins, affective human-robot interaction, multimodal representation learning, musical haptics, and security. Across these domains, the central technical premise is consistent: touch perturbs a mechanical system, that perturbation leaves an informative spectral-temporal signature, and inference or actuation can be built around that signature (Amri et al., 28 Jan 2026, Yi et al., 18 Feb 2026).

1. Physical basis and signal models

A common formulation models touch as a force input propagating through a structure to one or more sensors. In Vibro-Sense, a point poke at location $\mathbf{p}$ generates a contact force $f(t)$ and the measured signal at microphone $i$ is

$y_i(t) = \bigl(f * h_i\bigr)(t) + n_i(t),$

where $h(\mathbf{m}_i,\mathbf{p},t)$ is the impulse response and $n_i$ is additive noise. After STFT, the localization problem is cast as learning a mapping from a stacked multi-channel spectrogram $\mathbf{X}$ to $\mathbf{p}=(x,y,z)$ (Amri et al., 28 Jan 2026).

In ALTo, the governing variable is the time difference of arrival. If a tap reaches two sensors at times $t_i$ and $t_j$, then

$f(t)$0

and, after calibration, $f(t)$1. The touch location lies on a hyperbola defined by the difference in distances to the two sensors. ALTo uses independent intra-device TDOA for the left/right and top/bottom stereo pairs, then solves the corresponding hyperbolic equations to obtain $f(t)$2 on a rigid surface (Seshan, 2021).

Active acoustic systems replace passive listening with continuously excited resonant structures. In Sound of Touch, the sensing element is a tensioned steel guitar string governed by the one-dimensional wave equation

$f(t)$3

with natural modes

$f(t)$4

Contact at position $f(t)$5 with force $f(t)$6 changes vibration modes through local clamping or reaction force and through an increase in effective tension $f(t)$7, producing measurable spectral shifts at contact microphones (Yi et al., 18 Feb 2026).

AcousTac encodes touch through acoustic resonance in pneumatic chambers. Its basic resonance relations are $f(t)$8 for a pipe open at one end and “closed” at the other, and $f(t)$9 when a compliant silicone cap is compressed by $i$0. Combined with the experimentally fit shell relation $i$1, this yields a force-to-frequency mapping (Li et al., 2023).

These formulations also clarify that the sound of touch is not restricted to airborne audio. The smartphone side-channel attack “Hearing your touch” explicitly states that a tap generates a sound wave that propagates on the screen surface and in the air, and that built-in microphones can recover location-dependent distortions of that wave (Shumailov et al., 2019). A plausible implication is that many apparently different systems are variations on the same inverse problem: infer contact state from structure-borne and airborne propagation signatures.

2. Passive acoustic localization and trajectory tracking

Passive systems infer contact from touch-generated signals without continuous excitation. The best-documented results span tables, plates, mobile devices, and robotic hands.

Vibro-Sense equips the RH8D robotic hand with seven piezoelectric contact microphones, downsamples raw $i$2 kHz signals to $i$3 kHz, trims a $i$4 ms contact-centered window, subtracts pre-trigger steady-state noise, computes an STFT with $i$5 and hop size $i$6, stacks the seven spectrograms, and feeds them to an Audio Spectrogram Transformer with $i$7 transformer blocks and token dimension $i$8. In leave-one-material-out evaluation over $i$9 seeds, static impulse localization achieved $y_i(t) = \bigl(f * h_i\bigr)(t) + n_i(t),$0 mm on metal, $y_i(t) = \bigl(f * h_i\bigr)(t) + n_i(t),$1 mm on soft plastic, $y_i(t) = \bigl(f * h_i\bigr)(t) + n_i(t),$2 mm on hard plastic, and $y_i(t) = \bigl(f * h_i\bigr)(t) + n_i(t),$3 mm on wood. In dynamic trajectory tracking, wood was best at $y_i(t) = \bigl(f * h_i\bigr)(t) + n_i(t),$4 mm for the fixed-hand scenario, $y_i(t) = \bigl(f * h_i\bigr)(t) + n_i(t),$5 mm for random movement, and $y_i(t) = \bigl(f * h_i\bigr)(t) + n_i(t),$6 mm for the combined setting. The system maintained effective tracking during active robot motion, and the authors released raw audio, preprocessed spectrograms, splits, CAD mounts, checkpoints, and scripts under an MIT license (Amri et al., 28 Jan 2026).

ALTo instruments a $y_i(t) = \bigl(f * h_i\bigr)(t) + n_i(t),$7 cm $y_i(t) = \bigl(f * h_i\bigr)(t) + n_i(t),$8 $y_i(t) = \bigl(f * h_i\bigr)(t) + n_i(t),$9 cm wooden board with four piezoelectric disks placed near the midpoints of the four edges, records all four channels through two stereo inputs at up to $h(\mathbf{m}_i,\mathbf{p},t)$0 kHz, detects the first threshold crossing in each channel, computes intra-device TDOA for left/right and top/bottom pairs, and solves the multilateration problem after calibration of $h(\mathbf{m}_i,\mathbf{p},t)$1 and $h(\mathbf{m}_i,\mathbf{p},t)$2. Its prototype achieved mean absolute errors of $h(\mathbf{m}_i,\mathbf{p},t)$3 cm in the $h(\mathbf{m}_i,\mathbf{p},t)$4-direction and $h(\mathbf{m}_i,\mathbf{p},t)$5 cm in the $h(\mathbf{m}_i,\mathbf{p},t)$6-direction, with one-dimensional calibration showing $h(\mathbf{m}_i,\mathbf{p},t)$7 at $h(\mathbf{m}_i,\mathbf{p},t)$8 kHz (Seshan, 2021).

The Lamb-wave tactile plate uses a thin copper plate, four PZT patches, and a $h(\mathbf{m}_i,\mathbf{p},t)$9-tone excitation from $n_i$0 kHz to $n_i$1 kHz. A touch changes the FFT-amplitude signature through local viscoelastic loading, and localization is performed by nearest-neighbor matching against calibrated reference spectra with a Manhattan distance. The reported spatial resolution is down to $n_i$2 mm for one-finger localization, with total latency under $n_i$3 ms, and a double-validation check improved single-point localization from approximately $n_i$4 to approximately $n_i$5–$n_i$6 over $n_i$7 trials (Liu et al., 2009).

The smartphone acoustic side channel is not a tactile interface in the usual sense, but it is a localization system in adversarial form. Using only the device microphones at $n_i$8 kHz, it achieved $n_i$9 recovery of $\mathbf{X}$0 $\mathbf{X}$1-digit PIN-codes within $\mathbf{X}$2 attempts on a tablet and recovered $\mathbf{X}$3 words of size $\mathbf{X}$4–$\mathbf{X}$5 letters with $\mathbf{X}$6 attempts on a smartphone (Shumailov et al., 2019).

System	Mechanism	Reported result
Vibro-Sense	Seven piezo microphones + AST	Under $\mathbf{X}$7 mm average static error
ALTo	Four piezo disks + intra-device TDOA	$\mathbf{X}$8 cm in $\mathbf{X}$9, $\mathbf{p}=(x,y,z)$0 cm in $\mathbf{p}=(x,y,z)$1
Lamb-wave plate	Lamb-wave absorption + pattern matching	$\mathbf{p}=(x,y,z)$2 mm resolution, under $\mathbf{p}=(x,y,z)$3 ms
Smartphone side channel	Built-in mics + tap acoustics	$\mathbf{p}=(x,y,z)$4 PIN recovery within $\mathbf{p}=(x,y,z)$5 attempts

A recurrent empirical finding is that material properties shape observability. Vibro-Sense reports that stiff materials such as metal produce sharp, high-bandwidth impact pulses that are advantageous for impulse localization, whereas textured materials such as wood generate frictional micro-impacts that are more informative for trajectory tracking (Amri et al., 28 Jan 2026). This suggests that acoustic touch perception is not governed by geometry alone; it is jointly determined by substrate transfer functions, contact mechanics, and the time scale of the task.

3. Active acoustic tactile sensing and resonant skins

Active acoustic tactile sensing uses driven resonators whose state changes under contact. Sound of Touch implements this with a tensioned steel guitar string mounted on an aluminum frame, continuously excited by two modified EBow drivers and monitored by two piezo-based contact microphones. Digitization uses a Focusrite Scarlett 4i4 at $\mathbf{p}=(x,y,z)$6 kHz and $\mathbf{p}=(x,y,z)$7-bit resolution, and inference operates on $\mathbf{p}=(x,y,z)$8 s sliding windows converted either to a $\mathbf{p}=(x,y,z)$9 Mel-spectrogram or a harmonic feature vector. A frozen Audio Spectrogram Transformer pretrained on AudioSet provides a $t_i$0-dimensional embedding, with logistic regression heads for contact occurrence and slip detection and small $t_i$1-layer MLPs for location and force. On held-out tests, contact detection and slip detection reached $t_i$2 accuracy for slip under $t_i$3 m/s, localization MAE was $t_i$4–$t_i$5 mm depending on object rigidity, and force estimation MAE was $t_i$6–$t_i$7 N with end-to-end inference latency of approximately $t_i$8 ms per $t_i$9 s window (Yi et al., 18 Feb 2026).

The same work derives a physics-based simulator that discretizes the string into finite nodes and integrates a damped, driven wave equation with force-dependent tension and damping. After discarding the first approximately $t_j$0 ms of transient, FFT peak trajectories in simulation and real data agreed to within $t_j$1–$t_j$2 across modes $t_j$3 (Yi et al., 18 Feb 2026). This is significant because it links learning-based inference to an interpretable mode-shift mechanism rather than treating the spectrogram solely as a black-box input.

AcousTac targets a different operating regime: electronics-free force-sensitive soft skin. Each taxel is a rigid PLA pipe with a compliant silicone end cap, driven by $t_j$4–$t_j$5 L/min of compressed air and measured with a standard off-board microphone at $t_j$6 kHz. Taxels are tuned by tube length $t_j$7, cap thickness $t_j$8, and either a central hole $t_j$9 or added mass $f(t)$00. For hole-modified caps with $f(t)$01 mm and $f(t)$02 mm, the reported monotonic sensing region is from approximately $f(t)$03 N to approximately $f(t)$04 N with sensitivity approximately $f(t)$05 Hz/N; larger holes with $f(t)$06 mm lower $f(t)$07 to approximately $f(t)$08 N and raise sensitivity to approximately $f(t)$09 Hz/N. Across configurations, the platform is described as tunable across a force range of $f(t)$10–$f(t)$11 N with adjustable sensitivity of approximately $f(t)$12–$f(t)$13 Hz/N, and the update rate is $f(t)$14 Hz with $f(t)$15 ms latency (Li et al., 2023).

The contrast between the two systems is instructive. Sound of Touch is optimized for real-time contact location, normal force, and slip on extended robot surfaces, while AcousTac is optimized for static and quasi-static force sensing on soft robotic surfaces where electronics near the contact are not suitable (Yi et al., 18 Feb 2026, Li et al., 2023). A plausible implication is that active acoustic tactile sensing is not a single architecture but a design space structured by resonator type, excitation mechanism, readout location, and target bandwidth.

4. Learning-based manipulation and multimodal tactile representations

Several systems use the sound of touch as supervision or as a primary sensing stream in robot manipulation.

In “Learning Gentle Grasping Using Vision, Sound, and Touch,” audio is used only at data-collection time to label examples as “gentle” or “non-gentle.” A standard commodity microphone is rigidly mounted on the side of the Allegro Hand and records $f(t)$16 s of mono audio at $f(t)$17 kHz. Gentleness is defined by a hand-set threshold on short-time energy over the full window,

$f(t)$18

with a toy squeak event triggering the non-gentle label. The downstream policy does not consume raw audio; instead, a DenseNet-121-based visuo-tactile model predicts stable and gentle outcomes from RGB, DIGIT images, and candidate actions. On $f(t)$19 samples, the full model achieved $f(t)$20 overall grasp outcome accuracy versus $f(t)$21 for vision-only, and in $f(t)$22 real-world trials it reached approximately $f(t)$23 stable-and-gentle success versus approximately $f(t)$24 for vision-only and $f(t)$25 for random regrasp (Nakahara et al., 11 Mar 2025).

In “Adding internal audio sensing to internal vision enables human-like in-hand fabric recognition with soft robotic fingertips,” the new Minsound fingertip embeds a Knowles SPH0645 MEMS microphone in a soft fingertip shell and captures vibrations from $f(t)$26 Hz to $f(t)$27 kHz at $f(t)$28 kHz. Each camera frame at $f(t)$29 Hz is aligned with the most recent $f(t)$30 audio samples, PSDs are estimated with Welch’s method and truncated to $f(t)$31 bins up to $f(t)$32 kHz, and multimodal features are fused in a transformer with $f(t)$33 layers of multi-head self-attention, $f(t)$34 heads, and $f(t)$35. On a dataset of $f(t)$36 fabrics plus “no fabric,” internal audio only achieved $f(t)$37 fabric classification accuracy, internal plus external audio achieved $f(t)$38, while vision only was $f(t)$39. Under coffee-shop noise at $f(t)$40–$f(t)$41 dB, internal audio alone dropped to $f(t)$42, but adding proprioception raised performance to $f(t)$43 and adding an external microphone reached $f(t)$44 (Andrussow et al., 13 Feb 2026).

UniTouch addresses the sound of touch at the representation-learning level rather than the sensor-design level. Yang et al. align a tactile encoder to the frozen ImageBind image branch, thereby inheriting associations to audio, language, and other modalities already bound to that space. On ObjectFolder 2.0, UniTouch reported zero-shot touch-to-audio retrieval mAP of 37.9, outperforming supervised baselines such as CCA, PLSCA, DSCMR, and DAR in the table reproduced in the paper (Yang et al., 2024). This is notable because no explicit tactile-audio pairs are needed during training.

Taken together, these studies show three distinct roles for touch-generated sound in learning systems: as a self-supervised label for contact quality, as a high-bandwidth sensing modality for material recognition, and as an indirectly aligned embedding target in a unified multimodal space (Nakahara et al., 11 Mar 2025, Andrussow et al., 13 Feb 2026, Yang et al., 2024).

5. Human interaction, affect, and perceptual studies

In affective and social interaction, the sound of touch is used as a cue for gesture type, arousal, valence, and emotional intent. The main reported platforms are Pepper-based studies with a forearm-mounted microphone.

“Sound-Based Recognition of Touch Gestures and Emotions for Enhanced Human-Robot Interaction” uses a single directional microphone near Pepper’s left forearm and a lightweight audio-only MTRCNN. Log-mel spectrograms are computed from $f(t)$45 ms Hamming windows with $f(t)$46 ms hop, $f(t)$47 Mel bands, and per-band mean-variance normalization. The model has three parallel CNN branches with kernel sizes $f(t)$48, $f(t)$49, and $f(t)$50, a fused $f(t)$51-dimensional embedding, $f(t)$52 M parameters, $f(t)$53 MB model size, and $f(t)$54 G FLOPs per $f(t)$55 s input. On the test set with $f(t)$56 s input, mean accuracies were $f(t)$57 for arousal, $f(t)$58 for valence, $f(t)$59 for joint $f(t)$60-way arousal-valence, and $f(t)$61 for $f(t)$62-way gesture recognition. Gesture accuracy peaked at $f(t)$63 with $f(t)$64 s input, while joint emotion peaked at $f(t)$65 with $f(t)$66 s input (Hou et al., 2024).

“Conveying Emotions to Robots through Touch and Sound” records $f(t)$67 s gestures with a Waveshare USB-to-audio microphone mounted adjacent to a $f(t)$68 tactile pad on Pepper’s forearm. Audio features include MFCCs $f(t)$69–$f(t)$70, spectral centroid, spectral bandwidth, zero-crossing rate, and RMSE, reduced to summary statistics and concatenated with tactile features for classification. Among six off-the-shelf classifiers, a linear-kernel SVM achieved the highest accuracy, with $f(t)$71 mean cross-validation accuracy on the training set and $f(t)$72 final test accuracy for $f(t)$73 emotions. “Attention” was the most accurately conveyed emotion, with balanced accuracy of $f(t)$74 (Ren et al., 2024).

“Touch Speaks, Sound Feels” inverts the direction of communication by delivering vibration, sound, or their combination to human participants. Its VibroSleeve uses a $f(t)$75 grid of $f(t)$76 vibration motors on the upper arm synchronized with loudness-normalized $f(t)$77 kHz audio over headphones. In a within-subjects study with $f(t)$78, mean emotion decoding accuracy was $f(t)$79 for touch only, $f(t)$80 for sound only, and $f(t)$81 for the combined condition, with all pairwise differences significant. Gesture decoding was $f(t)$82 for touch only, $f(t)$83 for sound only, and $f(t)$84 for the combined condition (Ren et al., 11 Aug 2025).

Not all perceptual questions yield positive effects for sound. “A Study of Material Sonification in Touchscreen Devices” used granular synthesis of rubbing sounds on a touchscreen tablet and found that the considered audio cues did not significantly contribute to the perception of material qualities, although they increased immersion. Inter-participant agreement was lower in digital conditions than in full-modal physical touch, and dynamic audiovisual interaction increased engagement time relative to visual only and static audiovisual conditions (Martín et al., 2018).

These results delimit the semantics of the sound of touch in HRI and perception. Audio can be highly effective for gesture recognition and can improve affective decoding when paired with haptics, but it does not automatically enrich all perceptual judgments (Hou et al., 2024, Ren et al., 11 Aug 2025, Martín et al., 2018).

6. Acoustic actuation and tactile rendering

The sound of touch also names a reverse mapping: using acoustics to produce tactile sensations or to transmit touch-like structure.

Musinger combines a touch-sensitive recorder with a wearable haptic display. Three force-sensitive resistors under the fingers of one user are sampled at $f(t)$85 Hz and transmitted to a forearm-mounted display of three independent linkages with normal displacement up to $f(t)$86 mm, resolution $f(t)$87 mm, bandwidth from DC to $f(t)$88 Hz, and normal force excursion up to $f(t)$89 N. Four familiar melodies were rendered as tactile patterns, and recognition accuracy reached $f(t)$90 without white noise and $f(t)$91 with white noise (Cabrera et al., 2024).

The programmable acoustic metamaterial uses dual-state linear resonant actuators arranged in a lattice with $f(t)$92 mm and passive-state resonance at $f(t)$93 Hz. By powering some cells as vibration sources and leaving others passive as resonators, it creates self-tuned band gaps and reconfigurable waveguides. For three-bit patterns encoded at cells spaced by $f(t)$94 mm, decoding success exceeded $f(t)$95 for delays of at least $f(t)$96 ms when the carrier was within the band gap near $f(t)$97 Hz, but was approximately $f(t)$98 at $f(t)$99 Hz outside the band gap. In a $i$00-alternative forced-choice study with eight participants, mean correct identification was $i$01 at $i$02 Hz and $i$03 at $i$04 Hz (Daunizeau et al., 2024).

The soft vibrotactile display using sound speakers replaces rigid tactors with standard audio speakers pneumatically coupled through soft silicone tubes to the fingertip. The reported prototype uses Ecoflex 0030, tubing with inner diameter $i$05 mm and length approximately $i$06 mm, and UM1515IA speakers with free-air resonance approximately $i$07 Hz. The authors report empirically verified stimuli up to approximately $i$08 Hz and an $i$09 stimulation array over an $i$10 mm area (Ihara et al., 2024).

In musical haptics, computed fingertip touch and microtouch establish one-to-one mappings between sonic grains and force events. The Cyclotactor tracks fingertip movement at $i$11 Hz over a $i$12 mm range with approximately $i$13 mm resolution, outputs fingerpad-orthogonal force in the range $i$14 N, and supports vibrotactile cues up to $i$15 Hz with $i$16 ms round-trip latency. The KSFT provides fingerpad-parallel friction output in the range $i$17–$i$18 N with approximately $i$19 ms round-trip latency (Jong, 12 Feb 2025). Microtouch pairs each audio grain with either a non-vibratory or vibratory force pulse; in the reported pilot, non-vibratory CT pulses were clearly perceived down to $i$20 ms, vibratory CT pulses required at least $i$21 ms, and intensity ratings increased monotonically with force amplitude and duration (Jong, 2021).

A plausible implication is that acoustic haptics divides into at least three engineering strategies: transmission of symbolic structure through tactile patterns, spatial sculpting of vibration fields through wave control, and direct time-locking of audio synthesis events to haptic transients.

7. Limitations, misconceptions, and research directions

Several limitations recur across the literature. Material non-uniformity and anisotropy complicate calibration in surface-based systems; ALTo explicitly notes wood grain, knots, and hidden metal supports as sources of error, and cross-device synchronization was too noisy for reliable TDOA, motivating intra-device TDOA only (Seshan, 2021). AcousTac has an initial boundary-condition transition below approximately $i$22 N unless mitigated by a hole or added mass, and it requires continuous airflow of $i$23–$i$24 L/min (Li et al., 2023). Sound of Touch with strings identifies tension drift versus sensitivity as a hardware trade-off, and the authors state that two pickups per string are the minimal configuration to decouple $i$25 and $i$26 (Yi et al., 18 Feb 2026). Vibro-Sense shows marked material dependence between impulse localization and friction-based tracking, indicating that a single contact model is unlikely to dominate across tasks (Amri et al., 28 Jan 2026).

A common misconception is that touch-generated sound is always an intended sensing channel. The smartphone acoustic side channel demonstrates the opposite: microphones can recover sensitive user input without any tactile interface being designed for that purpose. Reported mitigations include blocking or muddying audio during secure input, hardware indicators for microphone usage, and application-level jamming (Shumailov et al., 2019).

Another misconception is that adding audio necessarily improves perceptual realism or semantic interpretation. The material-sonification study found no significant improvement in rating material qualities, despite increased immersion (Martín et al., 2018). Likewise, emotion studies report persistent confusions among classes with similar arousal or valence, and some emotions show low intraclass correlation values in how consistently they are conveyed through touch and sound (Ren et al., 2024).

The present trajectory nevertheless points toward broader integration. Vibro-Sense provides open-source datasets, models, and setups (Amri et al., 28 Jan 2026); Sound of Touch provides a physics-grounded simulator for design and interpretation (Yi et al., 18 Feb 2026); UniTouch shows that tactile-audio association can be inherited through a shared embedding space (Yang et al., 2024). This suggests a convergence between physically interpretable acoustic transduction, transformer-based sequence models, and multimodal alignment. If that convergence continues, the sound of touch is likely to remain not a single device class but a unifying research program for contact localization, force sensing, slip detection, material recognition, affective communication, and tactile rendering.