Touch-Based WoZ HRI Study

• The paper introduces a touch-based Wizard-of-Oz framework that utilizes dual-headset VR and precise sensorimotor alignment to simulate advanced HRI.
• A hybrid inverse kinematics system with rigorous calibration protocols ensures retargeted, high-fidelity touch interactions between the operator and the virtual avatar.
• Experimental findings reveal that full facial expressivity and context-specific cues significantly enhance user trust, likability, and perceived safety in HRI studies.

Touch-based Wizard-of-Oz (WoZ) studies in human-robot interaction (HRI) are experimental frameworks in which a human operator (wizard) covertly controls a robot (physical or virtual) to simulate advanced touch-enabled behaviors before autonomous capabilities are available. Such studies are essential for prototyping and evaluating the nuances of touch-rich HRI with precise sensorimotor alignment, embodied social cues, and tightly controlled experimental manipulations. Recent advancements, notably the introduction of dual-headset co-located VR platforms, have enabled researchers to study touch-based interaction with high spatial-temporal fidelity and direct participant feedback, overcoming many of the limitations of purely simulated or physically constrained setups (Wang et al., 18 Jan 2026).

1. Definition and Theoretical Foundation

Touch-based WoZ HRI studies investigate the impact and perception of tactile interactions between humans and robots when the robot’s touch actions are mediated by a human operator. The operator, often hidden from the participant, controls the robot's limb, hand, and facial cues to deliver touch at specific locations and timings. This methodology is used to generate realistic, fine-grained interactions that would be difficult or unsafe to realize with autonomous physical robots, providing controlled variability across experimental conditions. In dual-headset VR WoZ systems, the participant experiences a virtual embodied robot capable of synchronous, physically aligned touch, while the operator, co-located and calibrated in the same spatial frame, delivers the physical touch that is mapped onto the virtual avatar (Wang et al., 18 Jan 2026).

2. System Architecture and Technical Implementation

Modern touch-based WoZ HRI research utilizes sophisticated hardware and software infrastructures to achieve precise alignment between physical touch and virtual representation.

• Dual-Headset Co-Location: Two Meta VR headsets share a single spatial anchor, synchronizing the operator’s physical space with the participant’s virtual environment in a shared world coordinate frame ($W$).
• Tracking and Input Modalities: Comprehensive tracking includes 6-DoF inside-out SLAM for headsets, optical finger/hand skeleton tracking, and blendshape-based face tracking.
• Networked Control Pipeline: A WebSocket server streams and synchronizes operator data (head orientation $R_\text{op}$, palm and fingertip poses $T_\text{palm}$, $T_\text{thumbTip}$, $T_\text{indexTip}$, face blendshapes, gaze angles $\theta_x, \theta_y$) to the participant’s headset, where robot limb and face models are retargeted in real time.
• Motion Retargeting: Hybrid inverse kinematics (IK) solves for arm-to-palm alignment and precise finger articulation.
• Safety and Embodiment: Soft sleeves, virtual safety bubbles, and speed-constrained IK ($\|\mathbf{v}_{\text{end}}\| \leq 0.1$ m/s near participant) enforce physical safety and touch fidelity.

Component	Function	Technical Implementation
Spatial anchor	Co-locates operator and participant	Meta anchor, $\Delta T_\text{handOffset}$ for fine-tuning
Head/hand tracking	Captures wizard's motion	6-DoF SLAM, optical joint tracking
Inverse kinematics	Retargets pose to virtual robot	Levenberg–Marquardt IK, pseudo-inverse Jacobian
Haptic alignment	Ensures congruence of seen/felt touch	3D-printed fingertip covers, palm-level IK

3. Calibration, Alignment, and Retargeting

Precise calibration is critical to ensure that physical touch delivered by the operator is co-registered with the virtual touch perceived by the participant. Key steps include:

• Spatial Anchor Alignment: Both headsets initialize to the same spatial anchor, establishing $T_{\text{anchor}}: H \rightarrow W$.
• Palm Registration: The operator adjusts $\Delta T_{\text{handOffset}} \in SE(3)$ for the virtual robot’s palm to overlay the physical operator’s palm.
• Joint Mapping and IK: The operator’s palm pose in $W$ provides the IK target for the robot arm:

$$\Delta q = J^+(q)\,\text{vee}\left(\log (FK(q)^{-1} \cdot T_\text{target})\right)$$

where $J^+$ is the damped pseudo-inverse of the Jacobian $J$, and $\text{vee}(\cdot)$ computes the twist coordinates.

Facial expressions and gaze are also retargeted using linear mappings from the operator’s blendshape coefficients and gaze angles; equations for vertex and rotation parameters are directly specified in (Wang et al., 18 Jan 2026).

4. Study Design and Experimental Manipulations

WoZ touch-based HRI studies involve carefully scripted protocols to enable experimental control over robot expressivity and behavioral context:

• Manipulated Factors:
  • Expressivity (within-subject):
  • 1. Head only
  • 2. Head+Eyes
  • 3. Head+Eyes+Facial expressions
  • Context (between-subject):
  • A. Functional (“nurse” taking temperature)
  • B. Playful (guessing game)
• Trial Structure: Each participant is assigned to one context and experiences all three expressivity conditions in counterbalanced order. Operator scripts all cues and delivers touch (e.g., drawing a shape on the participant’s palm) in each trial. Speech lines are triggered using a foot pedal for hands-free operation.
• Safety Protocols: Collision detection, physical sleeves, and virtual safety zones around head and torso prevent accidental or unsafe contact.

5. Data Collection and Analytical Methods

Quantitative and qualitative data streams are captured for multimodal analysis:

• Objective Measures:
  • Gaze focus computed by ray-cast intersections at 90 Hz
  • Start/stop of hand collisions
  • Offline-extracted facial action units (Ekman FACS)
• Subjective Metrics:
  • Post-trial Likert questionnaires on likability, perceived competence, comfort, and trust
• Statistical Methods:
  • Mixed ANOVA: expressivity (3) × context (2) for each subjective score
  • Pairwise t-tests (Bonferroni corrected)
  • Correlations between gaze dwell time (e.g., on robot hands) and comfort/trust ratings

6. Observed Results and Empirical Insights

Pilot findings indicate trends with theoretical and applied significance (Wang et al., 18 Jan 2026):

• Full face+eye expressivity conditions elicit higher likability and trust ratings relative to head-only conditions.
• Functional (nurse) scenarios are scored as more competent but less enjoyable than playful contexts.
• Higher expressivity induces longer gaze dwell on robot hands, indicating increased attention to touch behaviors.
• Physical robot-participant contact is perceived as “well aligned” when hybrid IK and haptic alignment techniques are applied.
• No safety-related adverse events were observed.

A plausible implication is that high-fidelity embodied cues and strict spatial alignment are essential for perceived naturalness and comfort in touch-based HRI.

7. Methodological Guidelines and Future Considerations

Synthesized findings provide actionable recommendations for the design and replication of touch-based WoZ HRI studies:

• Co-located spatial anchors ensure drift-free spatial registration, streamlining calibration.
• Hybrid IK approaches (precise palm and two fingertips, direct joint copy for remaining fingers) maximize contact accuracy while reducing instability.
• Operator passthrough MR coupled with live collision-aware IK prevents unsafe contact.
• Scripted operator behaviors and high-frequency event logging enable robust experimental control and post-hoc analysis.
• Registration data, retargeted joint logs, and published face mapping equations enhance study replicability (Wang et al., 18 Jan 2026).

These principles position modern touch-based WoZ HRI as a robust paradigm for evaluating social, functional, and safety aspects of embodied robot touch prior to full autonomy.