Tactile-Proprioceptive Sensor Fusion for Contact Wrench Estimation in Whole-Body Physical Human-Robot Interaction

Abstract: Direct physical guidance is a natural means of teaching and interacting with robots, and robotic skins make a key contribution by enabling sensitive contact sensing and localization. This paper presents a tactile-proprioceptive sensor fusion framework for natural physical human-robot interaction. Tactile cues from pneumatic skin pads serve as contact indicators that bypass the ambiguity between frictional residues and applied external forces, enabling highly sensitive contact detection without explicit friction identification. We fuse these cues with motor-current-based proprioception to reconstruct multi-axis contact forces on the robot surface. To maintain accuracy during motion, we employ a temporal convolutional network (TCN) to mitigate friction hysteresis during stick-slip transitions, reducing uncertainty at contact onset and yielding smooth, responsive guidance. We validate the approach on a skin-integrated robot arm: (i) multi-axis forces are reconstructed in stationary contacts, and (ii) simultaneous force estimation and kinesthetic teaching are demonstrated. Results indicate improved sensitivity and responsiveness across diverse contact conditions compared with tactile-only and proprioceptive-only baselines, supporting tactile-proprioceptive fusion as a reliable pathway to safe, intuitive physical human-robot interaction.

• The paper combines pneumatic tactile contact detection with proprioceptive torque estimates and a temporal convolutional network (TCN) to compensate friction during whole-body human-robot interaction.
• The framework reduces force-free residual RMSE by 88.6% in static conditions and 54.8% during static-to-kinetic transitions, while cutting the estimation dead band by 53.86%.
• The approach enables responsive admittance-controlled kinesthetic teaching with inexpensive 3D-printed skin and stock servomotors, but remains limited at dynamic-to-static transitions and lacks calibrated torque measurements.

Overview

This paper presents a tactile–proprioceptive sensor fusion framework for estimating contact wrenches during whole-body physical human–robot interaction (pHRI), targeting kinesthetic teaching on a skin-integrated manipulator (2605.28412). The central observation is that motor-current-based external force estimation suffers from large friction-induced residuals—particularly static friction and hysteretic stick–slip transitions—which conventionally force large dead bands that degrade contact responsiveness. The authors exploit pneumatic robot skin as a contact indicator to disambiguate static-friction residuals from true external forces, and feed the resulting friction baseline into a temporal convolutional network (TCN) that compensates transition-phase friction online. The framework is validated on a four-DoF skin-integrated arm, demonstrating multi-axis force reconstruction and simultaneous force estimation during kinesthetic teaching.

Motivation and related approaches

Direct physical guidance requires reliable whole-body contact perception. Existing solutions divide into three camps. Force/torque and joint torque sensors provide accurate measurements but are costly, impact-vulnerable, and reduce structural stiffness. Proprioceptive estimation from motor currents with a dynamic model is durable and inexpensive but incurs significant errors from motor friction and hysteresis, especially in stationary and quasi-static regimes; practitioners typically compensate with large thresholds (a dead band) or Kalman filtering, or more recently with learned friction models. Whole-body robot skins offer high-sensitivity contact detection plus passive mechanical compliance—for example, pneumatic skins whose internal pressure changes under load—but individual pads measure only normal-force magnitude without within-pad localization. Prior tactile–proprioceptive fusion work exists, but the authors argue cost and scalability remain under-addressed; their approach uses only low-cost 3D-printed pneumatic pads and stock servo motors.

System modeling

The platform is an elbow-type serial manipulator modified to four DoFs at the forearm, with a coaxial proximal-roll actuator and no wrist joint, actuated by DYNAMIXEL-P PH54 servos with roughly 500:1 cycloidal reduction communicating over RS-485 at approximately 400 Hz. Skin pads are FDM-printed flexible TPE, chemically smoothed with THF for air-tightness, connected via silicone tubing to Honeywell pressure sensors read by a Teensy 4.0 publishing to ROS 2 via micro-ROS at 1 kHz.

The dynamics follow the standard rigid-body formulation, with measured torque decomposed into inverse-dynamics torque, external torque mapped through the contact-point Jacobian, friction, and residual error. Friction is modeled with a load-dependent Stribeck/Coulomb/viscous model, where Coulomb and viscous coefficients scale with $|G(q)|$ as a load proxy—an approximation the authors justify by noting links dominate loading under gentle pHRI. Crucially, they concede that such models cannot capture stick–slip motion and nonlocal memory hysteresis, and that richer models (LuGre, GMS) demand extensive parameter identification and are environmentally sensitive. They therefore deliberately avoid complex physical friction modeling and delegate hysteresis compensation to learning augmented by tactile cues.

Contact-aware torque estimation

Under static conditions ($\dot{q}=0$), the torque residual equals static friction plus external torque. Tactile signals detect contact onset time $t_{on}$, allowing the pre-contact residual $\tau_{res}(t_{on}^{-})$ to be attributed exclusively to static friction. The external torque is then estimated model-free as the residual increment since onset:

$\hat{\tau}_{ext}(t) = \Delta \tau_{res}(t).$

This is implemented as a three-state finite-state machine: static-without-contact, static-with-contact, and dynamic. By canceling static friction at contact onset, the method bypasses the large dead bands required by purely proprioceptive schemes on high-gear-ratio actuators. The method's validity is confined to the static states: once motion begins, friction transitions from static to kinetic with trajectory-dependent hysteresis, producing a large discontinuity at the state-(ii)-to-(iii) boundary.

TCN-based transition friction compensator

To handle the static-to-kinetic transition, the authors train a TCN whose inputs are the kinematic time series $(q, \dot{q}, \ddot{q})$ together with the FSM-estimated static friction $\hat{\tau}_{fric,s}$ (set to zero when $\dot{q} \neq 0$). Feeding the disambiguated friction estimate rather than the raw residual $\tau_{res}$ is essential: it renders the network input invariant to external forces, preventing the network from "compensating away" the very torque being estimated. Training data are collected contact-free along excitation trajectories, so the measured residual is ground-truth friction.

The architecture follows a multi-head TCN design separating low- and high-frequency residual components through a shared dilated-convolutional backbone and two linear heads, trained with a weighted MSE combination. A kernel size of $k=4$ lengthens the effective time window so the initial friction state persists through the transition phase. The final estimate subtracts both the nominal model and the TCN output from the measured torque. A single dead-band threshold derived from the TCN estimator is used across all states to avoid discontinuities at transitions.

Quantitatively, under force-free conditions (torques reported in mA of motor current because the actuator lacks an official torque constant):

Condition	Model	Avg. RMSE (mA)	Reduction
Static	Nominal	280.17	—
Static	TCN-compensated	31.96	88.6%
Static-to-kinetic	Nominal	270.95	—
Static-to-kinetic	TCN-compensated	124.81	54.8%

Standard deviations closely track RMSEs, indicating low bias. The variance reduction directly enables a smaller estimation dead band—the paper reports a 53.86% dead-band reduction relative to the uncompensated baseline—and hence greater sensitivity to subtle external forces. A known artifact remains immediately after motion stops: because $\dot{q}=0$0 during motion, the TCN's input window initially lacks stationary samples upon returning to rest, causing a short mismatch until the window refills; the authors note this does not impair stability since compliance mode terminates when contact disappears.

Demonstration

Kinesthetic teaching was validated with admittance control using a deliberately small dead band of $\dot{q}=0$1 (~86.6% confidence), where $\dot{q}=0$2 is the transition-region error standard deviation. Such a low threshold would normally be unsafe for proprioception-only systems, but direct tactile contact sensing preserves stability. Three scenarios were shown: (i) repeated weak, slow applied forces revealed that the nominal model produces severe overshoot and undershoot at static-to-kinetic transitions—worst when successive force directions align—while the TCN eliminates undershoot and substantially reduces overshoot; (ii) grasp-guided task compliance, where pneumatic-pad readings alone are unreliable due to contact-area dependence, was handled by estimating contact location from the pads and mapping joint-level torques to task-compliance trajectories; (iii) non-prehensile kinesthetic teaching across three sequential contacts showed the compensator removing static-friction error at contact onset and reducing response overshoot.

Limitations and open questions

The paper is explicit about several constraints. The load-dependent friction model uses $\dot{q}=0$3 as a proxy for transmission load, valid only under gentle interaction forces. The static estimator assumes constant static friction for small loads while at rest. The TCN is unreliable at the dynamic-to-static transition, where the post-motion friction state is unpredictable and the tactile-derived $\dot{q}=0$4 becomes untrustworthy; the authors defer thorough modeling of this regime. Performance gains during sustained motion were observed but left unanalyzed as out of scope. Additionally, all quantitative results are reported in current units rather than physical torque, owing to the absence of a calibrated torque constant, which limits direct comparison with force-calibrated benchmarks.

Conclusion

This work demonstrates that cheap pneumatic tactile cues can serve as contact-onset indicators that decouple static friction from external torque in a model-free manner, and that this decoupled friction estimate is a valuable conditioning input for a learned TCN friction compensator. The combination reduces static-state residual RMSE by 88.6%, transition-region RMSE by 54.8%, and the operational dead band by 53.86%, enabling responsive admittance-controlled kinesthetic teaching at a $\dot{q}=0$5 threshold that would be unstable without tactile contact confirmation. The main open problem is accurate friction characterization at motion termination, where the current pipeline's assumptions break down.