Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Interaction in Metaverse: A Survey (2310.13699v1)

Published 28 Sep 2023 in cs.HC and cs.ET

Abstract: Human-computer interaction (HCI) emerged with the birth of the computer and has been upgraded through decades of development. Metaverse has attracted a lot of interest with its immersive experience, and HCI is the entrance to the Metaverse for people. It is predictable that HCI will determine the immersion of the Metaverse. However, the technologies of HCI in Metaverse are not mature enough. There are many issues that we should address for HCI in the Metaverse. To this end, the purpose of this paper is to provide a systematic literature review on the key technologies and applications of HCI in the Metaverse. This paper is a comprehensive survey of HCI for the Metaverse, focusing on current technology, future directions, and challenges. First, we provide a brief overview of HCI in the Metaverse and their mutually exclusive relationships. Then, we summarize the evolution of HCI and its future characteristics in the Metaverse. Next, we envision and present the key technologies involved in HCI in the Metaverse. We also review recent case studies of HCI in the Metaverse. Finally, we highlight several challenges and future issues in this promising area.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (67)
  1. W. Gan, Z. Ye, S. Wan, and P. S. Yu, “Web 3.0: The future of internet,” in Companion Proceedings of the ACM Web Conference 2023, 2023, pp. 1266–1275.
  2. J. Sun, W. Gan, H.-C. Chao, S. Y. Philip, and W. Ding, “Internet of behaviors: A survey,” IEEE Internet of Things Journal, vol. 10, no. 13, pp. 11 117–11 134, 2023.
  3. J. Sun, W. Gan, Z. Chen, J. Li, and P. S. Yu, “Big data meets metaverse: A survey,” arXiv preprint, arXiv:2210.16282, 2022.
  4. J. Sun, W. Gan, H.-C. Chao, and P. S. Yu, “Metaverse: Survey, applications, security, and opportunities,” arXiv preprint, arXiv:2210.07990, 2022.
  5. D. T. K. Ng, “What is the Metaverse? Definitions, technologies and the community of inquiry,” Australasian Journal of Educational Technology, vol. 38, no. 4, 2022.
  6. S. Mystakidis, “Metaverse,” Encyclopedia, vol. 2, no. 1, pp. 486–497, 2022.
  7. P. Faraboschi, E. Frachtenberg, P. Laplante, D. Milojicic, and R. Saracco, “Virtual worlds (Metaverse): From skepticism, to fear, to immersive opportunities,” Computer, vol. 55, no. 10, pp. 100–106, 2022.
  8. J. Huggett, “Virtually real or really virtual: Towards a heritage metaverse,” Studies in Digital Heritage, vol. 4, no. 1, pp. 1–15, 2020.
  9. Z. Chen, J. Wu, W. Gan, and Z. Qi, “Metaverse security and privacy: An overview,” in IEEE International Conference on Big Data.   IEEE, 2022, pp. 2950–2959.
  10. T.-C. Wu and C.-T. B. Ho, “A scoping review of Metaverse in emergency medicine,” Australasian Emergency Care, 2022.
  11. N. Kovacev, “Metaverse and medicine,” in IEEE Zooming Innovation in Consumer Technologies Conference.   IEEE, 2022, pp. 1–1.
  12. H. Lin, S. Wan, W. Gan, J. Chen, and H.-C. Chao, “Metaverse in education: Opportunities and challenges,” in IEEE International Conference on Big Data.   IEEE, 2022, pp. 2857–2866.
  13. J. Yang, X. Wang, and Y. Zhao, “Parallel manufacturing for industrial Metaverses: A new paradigm in smart manufacturing,” IEEE/CAA Journal of Automatica Sinica, vol. 9, no. 12, pp. 2063–2070, 2022.
  14. H. Ning, H. Wang, Y. Lin, W. Wang, S. Dhelim, F. Farha, J. Ding, and M. Daneshmand, “A survey on the Metaverse: The state-of-the-art, technologies, applications, and challenges,” IEEE Internet of Things Journal, vol. 10, no. 16, pp. 14 671–14 688, 2023.
  15. X. Su, Y. Zhang, Q. Zhao, and L. Gao, “Virtual keyboard: A human-computer interaction device based on laser and image processing,” in IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems.   IEEE, 2015, pp. 321–325.
  16. V. Suma, “Computer vision for human-machine interaction-review,” Journal of Trends in Computer Science and Smart Technology, vol. 1, no. 02, pp. 131–139, 2019.
  17. K. Eason, “Ergonomic perspectives on advances in human-computer interaction,” Ergonomics, vol. 34, no. 6, pp. 721–741, 1991.
  18. L. J. Bannon, “From human factors to human actors: The role of psychology and human-computer interaction studies in system design,” in Readings in Human-computer Interaction.   Elsevier, 1995, pp. 205–214.
  19. J. M. Carroll, “Human–computer interaction: Psychology as a science of design,” International Journal of Human-computer Studies, vol. 46, no. 4, pp. 501–522, 1997.
  20. D. A. Bowman, J. Chen, C. A. Wingrave, J. Lucas, A. Ray, N. F. Polys, Q. Li, Y. Haciahmetoglu, J. S. Kim, S. Kim et al., “New directions in 3d user interfaces,” International Journal of Virtual Reality, vol. 5, no. 2, pp. 3–14, 2006.
  21. C. Falcao, A. C. Lemos, and M. Soares, “Evaluation of natural user interface: a usability study based on the leap motion device,” Procedia Manufacturing, vol. 3, pp. 5490–5495, 2015.
  22. D. Aaron, “Metaverse companies: Who’s involved and who’s investing in 2022,” 2022. [Online]. Available: https://tech.co/news/metaverse-companies-whos-involved-whos-investing
  23. L. Sara, “AR and VR device shipments to skyrocket through 2026,” 2022. [Online]. Available: https://www.insiderintelligence.com/content/ar-vr-device-shipments-skyrocket-through-2026
  24. W. Xu, “User centered design (VI): Human factors approaches for intelligent human-computer interaction,” arXiv preprint arXiv:2111.04880, 2021.
  25. S. Gupta, C. Maple, B. Crispo, K. Raja, A. Yautsiukhin, and F. Martinelli, “A survey of human-computer interaction (HCI) & natural habits-based behavioural biometric modalities for user recognition schemes,” Pattern Recognition, p. 109453, 2023.
  26. A. Z. G. Zahid, I. H. M. S. Al-Kharsan, H. A. Bakarman, M. F. Ghazi, H. A. Salman, and F. N. Hasoon, “Biometric authentication security system using human DNA,” in First International Conference of Intelligent Computing and Engineering.   IEEE, 2019, pp. 1–7.
  27. C. Wan, L. Wang, and V. V. Phoha, “A survey on gait recognition,” ACM Computing Surveys, vol. 51, no. 5, pp. 1–35, 2018.
  28. J. Poddar, V. Parikh, and S. K. Bharti, “Offline signature recognition and forgery detection using deep learning,” Procedia Computer Science, vol. 170, pp. 610–617, 2020.
  29. M. S. Hasan and H. Yu, “Innovative developments in HCI and future trends,” International Journal of Automation and Computing, vol. 14, no. 1, pp. 10–20, 2017.
  30. M. Peissner, V. Doebler, and F. Metze, “Can voice interaction help reducing the level of distraction and prevent accidents,” Meta-Study Driver Distraction Voice Interaction, vol. 24, no. 5, 2011.
  31. A. Poole and L. J. Ball, “Eye tracking in HCI and usability research,” in Encyclopedia of Human Computer Interaction.   IGI global, 2006, pp. 211–219.
  32. M. A. Just and P. A. Carpenter, “Eye fixations and cognitive processes,” Cognitive Psychology, vol. 8, no. 4, pp. 441–480, 1976.
  33. N. Sidorakis, G. A. Koulieris, and K. Mania, “Binocular eye-tracking for the control of a 3D immersive multimedia user interface,” in IEEE 1St Workshop on Everyday Virtual Reality.   IEEE, 2015, pp. 15–18.
  34. O. Palinko, A. L. Kun, A. Shyrokov, and P. Heeman, “Estimating cognitive load using remote eye tracking in a driving simulator,” in Symposium on Eye-tracking Research & Applications, 2010, pp. 141–144.
  35. S. Trösterer, A. Meschtscherjakov, D. Wilfinger, and M. Tscheligi, “Eye tracking in the car: Challenges in a dual-task scenario on a test track,” in The 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 2014, pp. 1–6.
  36. R. J. Adams and B. Hannaford, “Stable haptic interaction with virtual environments,” IEEE Transactions on Robotics and Automation, vol. 15, no. 3, pp. 465–474, 1999.
  37. R. S. Dahiya, G. Metta, M. Valle, and G. Sandini, “Tactile sensing—from humans to humanoids,” IEEE Transactions on Robotics, vol. 26, no. 1, pp. 1–20, 2009.
  38. O. Ozioko and R. Dahiya, “Smart tactile gloves for haptic interaction, communication, and rehabilitation,” Advanced Intelligent Systems, vol. 4, no. 2, p. 2100091, 2022.
  39. D. Camargo-Vargas, M. Callejas-Cuervo, and S. Mazzoleni, “Brain-computer interfaces systems for upper and lower limb rehabilitation: a systematic review,” Sensors, vol. 21, no. 13, p. 4312, 2021.
  40. U. Salahuddin and P.-X. Gao, “Signal generation, acquisition, and processing in brain machine interfaces: A unified review,” Frontiers in Neuroscience, p. 1174, 2021.
  41. T.-K. Huang, C.-H. Yang, Y.-H. Hsieh, J.-C. Wang, and C.-C. Hung, “Augmented reality (AR) and virtual reality (VR) applied in dentistry,” The Kaohsiung Journal of Medical Sciences, vol. 34, no. 4, pp. 243–248, 2018.
  42. S. Mangiante, G. Klas, A. Navon, Z. GuanHua, J. Ran, and M. D. Silva, “VR is on the edge: How to deliver 360 videos in mobile networks,” in the Workshop on Virtual Reality and Augmented Reality Network, 2017, pp. 30–35.
  43. M. J. Park, D. J. Kim, U. Lee, E. J. Na, and H. J. Jeon, “A literature overview of virtual reality (VR) in treatment of psychiatric disorders: recent advances and limitations,” Frontiers in Psychiatry, vol. 10, p. 505, 2019.
  44. N. Elmqaddem, “Augmented reality and virtual reality in education. myth or reality?” International Journal of Emerging Technologies in Learning, vol. 14, no. 3, 2019.
  45. R. T. Azuma, “A survey of augmented reality,” Presence: teleoperators & virtual environments, vol. 6, no. 4, pp. 355–385, 1997.
  46. T. Marescaux, “Holography will bring the Metaverse to life,” 2022. [Online]. Available: https://www.eetimes.eu/holography-will-bring-the-metaverse-to-life/
  47. T. Sluka, “Five ways to address AR’s vergence accommodation conflict,” 2022. [Online]. Available: https://arinsider.co/2022/06/22/5-ways-to-address-ars-vergence-accommodation-conflict/
  48. C. Liangcai, H. Zehao, L. Kexuan, and S. Xiaomeng, “Progress and challenges in dynamic holographic 3D display for the Metaverse,” Infrared and Laser Engineering, vol. 51, no. 1, pp. 20 210 935–1, 2022.
  49. C. Chang, K. Bang, G. Wetzstein, B. Lee, and L. Gao, “Toward the next-generation VR/AR optics: a review of holographic near-eye displays from a human-centric perspective,” Optica, vol. 7, no. 11, pp. 1563–1578, 2020.
  50. L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik, “Towards real-time photorealistic 3D holography with deep neural networks,” Nature, vol. 591, no. 7849, pp. 234–239, 2021.
  51. Y. Zhang, M. Zhang, K. Liu, Z. He, and L. Cao, “Progress of the computer-generated holography based on deep learning,” Applied Sciences, vol. 12, no. 17, p. 8568, 2022.
  52. Q. Jiang, L. Cao, H. Zhang, and G. Jin, “Improve the quality of holographic image with complex-amplitude metasurface,” Optics Express, vol. 27, no. 23, pp. 33 700–33 708, 2019.
  53. J. Li, Q. Smithwick, and D. Chu, “Scalable coarse integral holographic video display with integrated spatial image tiling,” Optics Express, vol. 28, no. 7, pp. 9899–9912, 2020.
  54. J. An, K. Won, Y. Kim, J. Y. Hong, H. Kim, Y. Kim, H. Song, C. Choi, Y. Kim, J. Seo et al., “Slim-panel holographic video display,” Nature communications, vol. 11, no. 1, p. 5568, 2020.
  55. J. D. M.-W. C. Kenton and L. K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in NAACL-HLT, vol. 1, 2019, p. 2.
  56. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in Neural Information Processing Systems, vol. 30, 2017.
  57. A. Radford, K. Narasimhan, T. Salimans, I. Sutskever et al., “Improving language understanding by generative pre-training,” pp. 3–8, 2018.
  58. W. Jiao, W. Wang, J.-t. Huang, X. Wang, and Z. Tu, “Is ChatGPT a good translator? a preliminary study,” arXiv preprint arXiv:2301.08745, 2023.
  59. Ö. Aydın and E. Karaarslan, “OpenAI ChatGPT generated literature review: Digital twin in healthcare,” Available at SSRN 4308687, 2022.
  60. S. Dhelim, T. Kechadi, L. Chen, N. Aung, H. Ning, and L. Atzori, “Edge-enabled metaverse: The convergence of metaverse and mobile edge computing,” arXiv preprint arXiv:2205.02764, 2022.
  61. L. Zhang and J. Chakareski, “Uav-assisted edge computing and streaming for wireless virtual reality: Analysis, algorithm design, and performance guarantees,” IEEE Transactions on Vehicular Technology, vol. 71, no. 3, pp. 3267–3275, 2022.
  62. K. Cao, Y. Liu, G. Meng, and Q. Sun, “An overview on edge computing research,” IEEE Access, vol. 8, pp. 85 714–85 728, 2020.
  63. B. J. King, G. J. Read, and P. M. Salmon, “The risks associated with the use of brain-computer interfaces: a systematic review,” International Journal of Human–Computer Interaction, pp. 1–18, 2022.
  64. S. Zhang, W. Y. B. Lim, W. C. Ng, Z. Xiong, D. Niyato, X. S. Shen, and C. Miao, “Towards green Metaverse networking: Technologies, advancements and future directions,” IEEE Network, pp. 8–9, 2023.
  65. J. Feng and J. Zhao, “Resource allocation for augmented reality empowered vehicular edge Metaverse,” IEEE Transactions on Communications, 2023.
  66. N. H. Chu, D. T. Hoang, D. N. Nguyen, K. T. Phan, E. Dutkiewicz, D. Niyato, and T. Shu, “Metaslicing: A novel resource allocation framework for Metaverse,” IEEE Transactions on Mobile Computing, pp. 3–15, 2023.
  67. H. Du, J. Wang, D. Niyato, J. Kang, Z. Xiong, X. S. Shen, and D. I. Kim, “Exploring attention-aware network resource allocation for customized Metaverse services,” IEEE Network, pp. 6–8, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Hong Lin (14 papers)
  2. Zirun Gan (1 paper)
  3. Wensheng Gan (80 papers)
  4. Zhenlian Qi (10 papers)
  5. Yuehua Wang (2 papers)
  6. Philip S. Yu (592 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now