ETHOS: A Robotic Encountered-Type Haptic Display for Social Interaction in Virtual Reality (2511.05379v1)

Abstract: We present ETHOS (Encountered-Type Haptics for On-demand Social Interaction), a dynamic encountered-type haptic display (ETHD) that enables natural physical contact in virtual reality (VR) during social interactions such as handovers, fist bumps, and high-fives. The system integrates a torque-controlled robotic manipulator with interchangeable passive props (silicone hand replicas and a baton), marker-based physical-virtual registration via a ChArUco board, and a safety monitor that gates motion based on the user's head and hand pose. We introduce two control strategies: (i) a static mode that presents a stationary prop aligned with its virtual counterpart, consistent with prior ETHD baselines, and (ii) a dynamic mode that continuously updates prop position by exponentially blending an initial mid-point trajectory with real-time hand tracking, generating a unique contact point for each interaction. Bench tests show static colocation accuracy of 5.09 +/- 0.94 mm, while user interactions achieved temporal alignment with an average contact latency of 28.53 +/- 31.21 ms across all interaction and control conditions. These results demonstrate the feasibility of recreating socially meaningful haptics in VR. By incorporating essential safety and control mechanisms, ETHOS establishes a practical foundation for high-fidelity, dynamic interpersonal interactions in virtual environments.

• The paper presents ETHOS, a dynamic encountered-type haptic display for VR that delivers natural physical feedback during social interactions.
• It integrates a KUKA robotic arm with high-fidelity hand tracking and custom passive props to achieve precise spatial registration and adaptive control.
• Quantitative evaluations show sub-centimeter colocation accuracy (~5 mm error) and low interaction latency (<50 ms), supporting realistic gestures.

ETHOS: A Dynamic Robotic Encountered-Type Haptic Display for Social Interaction in Virtual Reality

Motivation and Context

ETHOS (Encountered-Type Haptics for On-demand Social interaction) is presented as a next-generation encountered-type haptic display (ETHD), designed explicitly to facilitate natural, physically meaningful social contact in virtual reality (VR) scenarios. The system targets core limitations in conventional haptic technologies, which predominantly rely on visual and auditory modalities and often lack realistic, precisely timed tactile cues critical for convincing social interactions. ETHOS integrates high-fidelity spatial registration, torque-controlled actuation, safety-aware architecture, and custom passive props to enable realistic interpersonal exchanges such as object handover, fist bump, and high five.

ETHOS System Design

Hardware and Physical Subsystem

ETHOS utilizes a KUKA LBR iiwa 7 R800 robotic manipulator, operated via real-time torque control (FRI, 1000 Hz), supporting both static and dynamic placement of interactable props. The endpoint is equipped with an ATI Mini40 six-axis force–torque sensor and a rapid-exchange mount for passive props crafted for specific interactions:

Figure 1: Fist, open hand, and baton props for rendering fist bump, high five, and handover haptic events, respectively.

The fist and open hand props are silicone castings with wire armature for anatomical compliance, while the baton features 3D-printed structure and electromagnet for controlled release during handovers. This modular approach enables rapid transition between interaction types and supports future extensions to complex social gestures.

Virtual Subsystem and Registration

ETHOS leverages the markerless hand-tracking capabilities of Meta Quest 3 HMD for user localization and interaction detection. Experienced researchers will recognize the significance of robust spatial registration: ETHOS solves this with a ChArUco fiducial system anchored in the robot’s workspace, offering streamlined physical-virtual colocation through a single registration phase. The pipeline utilizes OpenCV and custom averaging logic to achieve reliable transformation matrices for alignment, with pose data exchanged using Google Protocol Buffers over UDP for minimal latency.

Safety Mechanisms

ETHOS implements multi-layer safety, balancing system autonomy with human oversight. The workspace is actively monitored for the user’s head and body position, triggering motion halts or passthrough mode activation as needed. An external operator provides manual override capability, ensuring the safety framework meets requirements for close-proximity operation in immersive VR.

Interaction Sequencing and Control Strategies

ETHOS supports three canonical social interactions: object handover, fist bump, and high five. Interaction flow is structured consistently, beginning with spatial prompts guiding the user to a safe region, followed by avatar engagement, gesture intent cues (arm movement, gaze), haptic prop positioning, and contact detection.

Figure 2: Flowchart illustrating ETHOS's unified sequencing for object handover, fist bump, and high-five interactions.

Static vs. Dynamic Control Modes

ETHOS offers baseline static interaction (prop held at fixed pose) and dynamic tracking mode. In dynamic mode, the robot generates a time-varying trajectory that exponentially blends an initial mid-point between the user’s hand and the prop with live hand tracking, providing a unique contact location for each interaction while ensuring stability and predictivity.

Figure 3: Visualization of the dynamic trajectory control—midpoint initialization, weighted blending toward the user’s hand, and individualized contact point realization.

The control equations are:

$$w(t) = \frac{e^{3\alpha(t)} - 1}{e^3 - 1},\quad \alpha(t) = \min(t_{\text{dynamic}},\, 1)$$

$$x_{\text{target}}(t) = (1 - w(t))\, x_{\text{midTraj}}(t) + w(t)\, x_{\text{hand}}(t)$$

Dynamic control enables adaptation to deviations in user approach trajectories, resulting in more natural social engagement and minimizing misalignment due to tracking uncertainties.

Quantitative Performance Evaluation

Colocation Accuracy

Spatial alignment error was determined through 20 trials using a Vicon motion-capture reference system and fiducial tracking, achieving $5.09 \pm 0.94$ mm error. This places ETHOS well within sub-centimetre colocation standards deemed imperceptible for task context, exceeding the <1 cm distortion threshold for VR haptic interaction realism.

Temporal Alignment

Interaction latency—measured as the difference in time between physical and virtual hand–object contact—was averaged across 600 trials for all interaction types and control conditions. ETHOS attained mean delays below 50 ms in all cases, though the standard deviation occasionally surpassed conservative latency perception thresholds. Specifically:

• Handover: $18.83 \pm 40.65$ ms (static) / $25.72 \pm 35.48$ ms (dynamic)
• Fist bump: $36.01 \pm 35.45$ ms (static) / $24.49 \pm 18.77$ ms (dynamic)
• High five: $23.29 \pm 14.99$ ms (static) / $43.11 \pm 27.02$ ms (dynamic)

These timings are substantially lower than the reported latency perception thresholds in VR for related tasks. Nevertheless, variance in trial outcomes implies that, while average user experience will be unaffected, perceptible discrepancies may arise at the user-level, especially for rapid interactions.

Implications and Future Directions

ETHOS provides a reproducible foundation for dynamic ETHDs in VR, demonstrating that spatial and temporal alignment can be attained through a combination of torque-controlled manipulator, passive props, and simplified fiducial registration. The authors notably address practical limitations encountered in legacy ETHDs—namely, static object rendering, cumbersome hardware, or complex instrumentation—and provide empirical justification for their registration and control strategies.

Subjective evaluation is ongoing, with preliminary qualitative feedback indicating enhanced realism, enjoyment, and avatar connection when physical haptic contact is integrated. However, several limitations remain: prop similarity to biological counterparts was assumed but not quantitatively characterized, interaction force thresholds were heuristically defined, and compliance modelling was not addressed. Future work should include:

• Rigorous subjective studies quantifying perceived presence, naturalness, and trust.
• Objective modelling of prop material properties and ergonomic compliance.
• Extension to multi-user and multi-prop systems for more complex gestures.
• Systematic calibration of force–torque thresholds, dynamic compliance adaptation, and interaction-specific control strategies.
• Broader application in immersive training, telerehabilitation, and VR performance arts.

Conclusion

ETHOS advances the encountered-type haptic display paradigm by enabling dynamic, high-fidelity social contact interactions in VR, supported by robust colocation and sub-perceptual latency metrics. The system architecture is extensible and focused on unencumbered user experience, with modular hardware and safety-aware design. ETHOS lays the groundwork for future research in socially meaningful haptic feedback systems and provides a quantitative and practical blueprint for embedding physicality into virtual social experiences.