Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Effects of Multisensory Feedback on the Perception and Performance of Virtual Reality Hand-Retargeted Interaction (2404.03899v1)

Published 5 Apr 2024 in cs.HC

Abstract: Retargeting methods that modify the visual representation of real movements have been widely used to expand the interaction space and create engaging virtual reality experiences. For optimal user experience and performance, it is essential to specify the perception of retargeting and utilize the appropriate range of modification parameters. However, previous studies mostly concentrated on whether users perceived the target sense or not and rarely examined the perceptual accuracy and sensitivity to retargeting. Moreover, it is unknown how the perception and performance in hand-retargeted interactions are influenced by multisensory feedback. In this study, we used rigorous psychophysical methods to specify users' perceptual accuracy and sensitivity to hand-retargeting and provide acceptable ranges of retargeting parameters. We also presented different multisensory feedback simultaneously with the retargeting to probe its effect on users' perception and task performance. The experimental results showed that providing continuous multisensory feedback, proportionate to the distance between the virtual hand and the targeted destination, heightened the accuracy of users' perception of hand retargeting without altering their perceptual sensitivity. Furthermore, the utilization of multisensory feedback considerably improved the precision of task performance, particularly at lower gain factors. Based on these findings, we propose design guidelines and potential applications of VR hand-retargeted interactions and multisensory feedback for optimal user experience and performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. Grabity: A wearable haptic interface for simulating weight and grasping in virtual reality. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, pages 119–130, 2017.
  2. Aero-plane: A handheld force-feedback device that renders weight motion illusion on a virtual 2d plane. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, pages 763–775, 2019.
  3. Providing haptics to walls & heavy objects in virtual reality by means of electrical muscle stimulation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pages 1471–1482, 2017.
  4. Haptic revolver: Touch, shear, texture, and shape rendering on a reconfigurable virtual reality controller. In Proceedings of the 2018 CHI conference on human factors in computing systems, pages 1–12, 2018.
  5. Mouillé: Exploring wetness illusion on fingertips to enhance immersive experience in vr. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1–10, 2020.
  6. Thor’s hammer: An ungrounded force feedback device utilizing propeller-induced propulsive force. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pages 1–11, 2018.
  7. Torc: A virtual reality controller for in-hand high-dexterity finger interaction. In Proceedings of the 2019 CHI conference on human factors in computing systems, pages 1–13, 2019.
  8. Sensorimotor simulation of redirected reaching using stochastic optimal feedback control. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–17, 2023.
  9. " boundary of illusion": an experiment of sensory integration with a pseudo-haptic system. In Proceedings IEEE Virtual Reality 2001, pages 115–122. IEEE, 2001.
  10. Pseudo-haptic weight: Changing the perceived weight of virtual objects by manipulating control-display ratio. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pages 1–13, 2019.
  11. Anatole Lécuyer. Simulating haptic feedback using vision: A survey of research and applications of pseudo-haptic feedback. Presence: Teleoperators and Virtual Environments, 18(1):39–53, 2009.
  12. The effect of the virtual object size on weight perception augmented with pseudo-haptic feedback. In 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pages 575–576. IEEE, 2021.
  13. Estimating detection thresholds for desktop-scale hand redirection in virtual reality. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pages 47–55. IEEE, 2019.
  14. Sparse haptic proxy: Touch feedback in virtual environments using a general passive prop. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pages 3718–3728, 2017.
  15. Evaluating remapped physical reach for hand interactions with passive haptics in virtual reality. IEEE transactions on visualization and computer graphics, 24(4):1467–1476, 2018.
  16. Haptic retargeting: Dynamic repurposing of passive haptics for enhanced virtual reality experiences. In Proceedings of the 2016 chi conference on human factors in computing systems, pages 1968–1979, 2016.
  17. Hart-the virtual reality hand redirection toolkit. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–7, 2021.
  18. Beyond being real: a sensorimotor control perspective on interactions in virtual reality. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pages 1–17, 2022.
  19. Extending the body for interaction with reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pages 5145–5157, 2017.
  20. The effect of movement direction, hand dominance, and hemispace on reaching movement kinematics in virtual reality. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–18, 2023.
  21. Sensitivity to hand offsets and related behavior in virtual environments over time. ACM Transactions on Applied Perception, 19(4):1–15, 2022.
  22. Joy Paul Guilford. Psychometric methods. 1954.
  23. The measurement and prediction of judgment and choice. Holden-day, ISBN 0816209359, 1968.
  24. George A Gescheider. Psychophysics: the fundamentals. Psychology Press, ISBN 9781138984158, 2013.
  25. What is embodiment? a psychometric approach. Cognition, 107(3):978–998, 2008.
  26. First person experience of body transfer in virtual reality. PloS one, 5(5):e10564, 2010.
  27. Multisensory stimulation can induce an illusion of larger belly size in immersive virtual reality. PloS one, 6(1):e16128, 2011.
  28. Avatar embodiment. towards a standardized questionnaire. Frontiers in Robotics and AI, 5:74, 2018.
  29. The effect of multisensory pseudo-haptic feedback on perception of virtual weight. IEEE Access, 10:5129–5140, 2022.
  30. Elastic-arm: Human-scale passive haptic feedback for augmenting interaction and perception in virtual environments. In 2015 IEEE Virtual Reality (VR), pages 63–68. IEEE, 2015.
  31. Do multisensory stimuli benefit the virtual reality experience? a systematic review. IEEE transactions on visualization and computer graphics, 28(2):1428–1442, 2022.
  32. Gum-gum shooting: Inducing a sense of arm elongation via forearm skin-stretch and the change in the center of gravity. In ACM SIGGRAPH 2018 Emerging Technologies, pages 1–2. 2018.
  33. Seeing and identifying with a virtual body decreases pain perception. European journal of pain, 15(8):874–879, 2011.
  34. The relationship between virtual body ownership and temperature sensitivity. Journal of the Royal Society Interface, 10(85):20130300, 2013.
  35. Need a hand? how appearance affects the virtual hand illusion. In Proceedings of the ACM Symposium on Applied Perception, pages 69–76, 2016.
  36. The body fades away: investigating the effects of transparency of an embodied virtual body on pain threshold and body ownership. Scientific reports, 5(1):1–8, 2015.
  37. Perceptual sensitivity to visual/kinesthetic discrepancy in hand speed, and why we might care. In Proceedings of the ACM symposium on Virtual reality software and technology, pages 3–8, 2006.
  38. Enlarging just noticeable differences of visual-proprioceptive conflict in vr using haptic feedback. In 2015 IEEE World Haptics Conference (WHC), pages 19–24. IEEE, 2015.
  39. Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments. In Proceedings IEEE Virtual Reality (Cat. No. 99CB36316), pages 222–228. IEEE, 1999.
  40. Influence of multisensory feedback on haptic accessibility tasks. Virtual Reality, 10(1):31–40, 2006.
  41. Multimodal virtual environments: response times, attention, and presence. Presence: Teleoperators and virtual environments, 15(5):515–523, 2006.
  42. The visual, the auditory and the haptic–a user study on combining modalities in virtual worlds. In International Conference on Virtual, Augmented and Mixed Reality, pages 159–168. Springer, 2013.
  43. Natural stimuli from three coherent modalities enhance behavioral responses and electrophysiological cortical activity in humans. International Journal of Psychophysiology, 93(1):45–55, 2014.
  44. The effects of substitute multisensory feedback on task performance and the sense of presence in a virtual reality environment. PloS one, 13(2):e0191846, 2018.
  45. The influence of passive haptic feedback and difference interaction metaphors on presence and task performance. Presence, 19(3):197–212, 2010.
  46. Impact of multimodal feedback on simulated ergonomic measurements in a virtual environment: A case study with manufacturing workers. Human Factors and Ergonomics in Manufacturing & Service Industries, 22(2):145–155, 2012.
  47. Design and assessment of a virtual underwater multisensory effects reproducing simulation system. International Journal of Distributed Sensor Networks, 10(7):420428, 2014.
  48. Haptic feedback enhances rhythmic motor control by reducing variability, not improving convergence rate. Journal of Neurophysiology, 111(6):1286–1299, 2014.
  49. The effects of haptic feedback and visual distraction on pointing task performance. International Journal of Human-Computer Interaction, 32(2):89–102, 2016.
  50. Effect of variations in sensory feedback on performance in a virtual reaching task. Presence: Teleoperators & Virtual Environments, 14(4):450–462, 2005.
  51. Relative performance using haptic and/or touch-produced auditory cues in a remote absolute texture identification task. In 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings., pages 151–158. IEEE, 2003.
  52. Multimodal selection techniques for dense and occluded 3d virtual environments. International Journal of Human-Computer Studies, 67(3):237–255, 2009.
  53. An empirical evaluation of visuo-haptic feedback on physical reaching behaviors during 3d interaction in real and immersive virtual environments. ACM Transactions on Applied Perception (TAP), 13(4):1–21, 2016.
  54. The bodily illusion in adverse conditions: Virtual arm ownership during visuomotor mismatch. Perception, 47(5):477–491, 2018.
  55. Multisensory integration in the virtual hand illusion with active movement. BioMed research international, 2016, 2016.
  56. Extending the human body in virtual reality: Effect of sensory feedback on agency and ownership of virtual wings. In Proceedings of the 2016 Virtual Reality International Conference, pages 1–4, 2016.
  57. The importance of sensory feedback to enhance embodiment during virtual training of myoelectric prostheses users. In 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pages 770–771. IEEE, 2021.
  58. Sensation and perception. Cengage Learning, ISBN 130558029X, 2016.
  59. William A Simpson. The method of constant stimuli is efficient. Perception & psychophysics, 44:433–436, 1988.
  60. JASP. JASP (Version 0.16.2)[Computer software], 2022. URL https://jasp-stats.org/.
  61. The influence of different sensory cues as selection feedback and co-location in presence and task performance. Multimedia tools and applications, 68(3):623–639, 2014.
  62. Multisensory roughness perception of virtual surfaces: effects of correlated cues. In 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2004. HAPTICS’04. Proceedings., pages 161–168. IEEE, 2004.
  63. Detecting visuo-haptic mismatches in virtual reality using the prediction error negativity of event-related brain potentials. In Proceedings of the 2019 CHI conference on human factors in computing systems, pages 1–11, 2019.
  64. Neural sources of prediction errors detect unrealistic vr interactions. Journal of Neural Engineering, 19(3):036002, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Hyunyoung Jang (1 paper)
  2. Jinwook Kim (11 papers)
  3. Jeongmi Lee (6 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets