Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Application of Artificial Intelligence in Hand Gesture Recognition with Virtual Reality: Survey and Analysis of Hand Gesture Hardware Selection (2405.16264v1)

Published 25 May 2024 in cs.HC

Abstract: The ongoing usage of artificial intelligence technologies in virtual reality has led to a large number of researchers exploring immersive virtual reality interaction. Gesture controllers and head-mounted displays are the primary pieces of hardware used in virtual reality applications. This article analyzes the advantages and disadvantages of various hardware tools as well as possible use cases for hand gesture recognition. Several popular data sets used in data pre-processing and hand gesture detection systems are also summarized. This paper's primary objectives are to evaluate the current state of the art in gesture recognition research and to offer potential routes for future study to guide researchers working on immersive virtual reality interaction. Comparing recent study results and taking into account the state of the development of human-computer interaction technology, a number of potential research directions are highlighted, including the following: input data noise should be filtered, a lightweight network should be designed, tactile feedback should be created, and eye and gesture data should be combined.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (91)
  1. A survey of human action analysis in hri applications. IEEE Transactions on Circuits and Systems for Video Technology, 30(7):2114–2128, 2019.
  2. A brief analysis of gesture recognition in vr. In SID Symposium Digest of Technical Papers, volume 51, pages 190–195. Wiley Online Library, 2020.
  3. A system for a hand gesture-manipulated virtual reality environment. In Proceedings of the Annual Conference of the South African Institute of Computer Scientists and Information Technologists, pages 1–10, 2016.
  4. Inam Rehman. Two Hand Gesture Based 3D Navigation in Virtual Environments. International Journal of Interactive Multimedia and Artificial Intelligence, InPress, July 2018.
  5. Jinmo Kim. Vivr: Presence of immersive interaction for visual impairment virtual reality. IEEE Access, 8:196151–196159, 2020.
  6. Object manipulation in immersive virtual environments: Hand Motion tracking technology and snap-to-fit function. Automation in Construction, 124:103594, 2021.
  7. Multimodal learning for human action recognition via bimodal/multimodal hybrid centroid canonical correlation analysis. IEEE Transactions on Multimedia, 21(5):1317–1331, 2018.
  8. Continuous, real-time, tele-monitoring of patients with chronic heart-failure: lessons learned from a pilot study. In Proceedings of the 9th International Conference on Body Area Networks, pages 135–141, 2014.
  9. G3d: A gaming action dataset and real time action recognition evaluation framework. In 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pages 7–12. IEEE, 2012.
  10. Robust part-based hand gesture recognition using kinect sensor. IEEE transactions on multimedia, 15(5):1110–1120, 2013.
  11. Intelligent video surveillance system: 3-tier context-aware surveillance system with metadata. Multimedia Tools and Applications, 57(2):315–334, 2012.
  12. Hand posture and gesture recognition techniques for virtual reality applications: a survey. Virtual Reality, 21(2):91–107, 2017.
  13. 3d panoramic virtual reality video quality assessment based on 3d convolutional neural networks. IEEE Access, 6:38669–38682, 2018.
  14. Skeleton based dynamic hand gesture recognition using lstm and cnn. In 2020 2nd International Conference on Image Processing and Machine Vision, IPMV 2020, page 63–68, New York, NY, USA, 2020. Association for Computing Machinery.
  15. Wearable human computer interface for control within immersive vamr gaming environments using data glove and hand gestures. In 2018 IEEE Games, Entertainment, Media Conference (GEM), pages 1–9. IEEE, 2018.
  16. Haptic illusion glove: A glove for illusionary touch feedback when grasping virtual objects. In Proceedings of Mensch Und Computer 2019, MuC’19, page 565–569, New York, NY, USA, 2019. Association for Computing Machinery.
  17. A Smart Glove of Combining Virtual and Real Environments for Chemical Experiment. In Proceedings of 2020 the 6th International Conference on Computing and Data Engineering, pages 227–231, Sanya China, January 2020. ACM.
  18. A Framework for Hand Gesture Recognition Based on Accelerometer and EMG Sensors. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 41(6):1064–1076, November 2011. Conference Name: IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.
  19. An semg-controlled 3d game for rehabilitation therapies: Real-time time hand gesture recognition using deep learning techniques. Sensors, 20(22):6451, 2020.
  20. A Novel Surface Electromyographic Signal-Based Hand Gesture Prediction Using a Recurrent Neural Network. Sensors, 20(14):3994, January 2020. Number: 14 Publisher: Multidisciplinary Digital Publishing Institute.
  21. Exploring the potential of immersive virtual environments for learning american sign language. In European Conference on Technology Enhanced Learning, pages 459–474. Springer, 2023.
  22. Latern: Dynamic Continuous Hand Gesture Recognition Using FMCW Radar Sensor. IEEE Sensors Journal, 18(8):3278–3289, April 2018. Conference Name: IEEE Sensors Journal.
  23. Improving Classification Accuracy of Hand Gesture Recognition Based on 60 GHz FMCW Radar with Deep Learning Domain Adaptation. Electronics, 9(12):2140, December 2020.
  24. The impact of 2d and 3d gamified vr on learning american sign language. arXiv preprint arXiv:2405.08908, 2024.
  25. Skeleton Based Dynamic Hand Gesture Recognition using LSTM and CNN. In 2020 2nd International Conference on Image Processing and Machine Vision, pages 63–68, Bangkok Thailand, August 2020. ACM.
  26. Lbkt: a lstm bert-based knowledge tracing model for long-sequence data. In 20th International Conference on Intelligent Tutoring Systems: Generative Intelligence and ITS (10/06/24 - 13/06/24), June 2024.
  27. Comparison between full body motion recognition camera interaction and hand controllers interaction used in virtual reality exposure therapy for acrophobia. Sensors, 20(5):1244, 2020.
  28. Integrating lstm and bert for long-sequence data analysis in intelligent tutoring systems. arXiv preprint arXiv:2405.05136, 2024.
  29. Comparative efficacy of 2d and 3d virtual reality games in american sign language learning. In The 31st IEEE Conference on Virtual Reality and 3D User Interfaces. Newcastle University, 2024.
  30. The effects of safety behaviors during exposure therapy for anxiety: Critical analysis from an inhibitory learning perspective. Clinical Psychology Review, 49:1–15, 2016.
  31. Low–cost devices used in virtual reality exposure therapy. Procedia Computer Science, 104:445–451, 2017.
  32. Impact of personalised ai chat assistant on mediated human-human textual conversations: Exploring female-male differences. In Companion Proceedings of the 29th International Conference on Intelligent User Interfaces, pages 78–83, 2024.
  33. Recent trends in hci: A survey on data glove, leap motion and microsoft kinect. In 2018 IEEE International Conference on System, Computation, Automation and Networking (ICSCA), pages 1–5. IEEE, 2018.
  34. Application for hand rehabilitation using leap motion sensor based on a gamification approach. Int. J. Adv. Res. Sci. Eng, 5(2):61–69, 2016.
  35. Towards student behaviour simulation: a decision transformer based approach. In International Conference on Intelligent Tutoring Systems, pages 553–562. Springer, 2023.
  36. User-defined hand gesture interface to improve user experience of learning american sign language. In International Conference on Intelligent Tutoring Systems, pages 479–490. Springer, 2023.
  37. Real-time 3d hand tracking from depth images. In Advanced Materials Research, volume 765, pages 2822–2825. Trans Tech Publ, 2013.
  38. 3d human gesture capturing and recognition by the immu-based data glove. Neurocomputing, 277:198–207, 2018.
  39. Design and motion tracking of a strip glove based on machine vision. Neurocomputing, 288:20–29, 2018.
  40. Developing and evaluating a novel gamified virtual learning environment for asl. In IFIP Conference on Human-Computer Interaction, pages 459–468. Springer, 2023.
  41. User study of vr basic controller and data glove as hand gesture inputs in vr games. In 2017 international symposium on ubiquitous virtual reality (isuvr), pages 1–3. IEEE, 2017.
  42. Improving classification accuracy of hand gesture recognition based on 60 ghz fmcw radar with deep learning domain adaptation. Electronics, 9(12):2140, 2020.
  43. Latern: Dynamic continuous hand gesture recognition using fmcw radar sensor. IEEE Sensors Journal, 18(8):3278–3289, 2018.
  44. Application of fmcw radar for dynamic continuous hand gesture recognition. In 11th EAI International Conference on Mobile Multimedia Communications, page 298. European Alliance for Innovation (EAI), 2018.
  45. Dynamic hand gesture recognition based on micro-doppler radar signatures using hidden gauss–markov models. IEEE Geoscience and Remote Sensing Letters, 18(2):291–295, 2020.
  46. Deephandsvr: Hand interface using deep learning in immersive virtual reality. Electronics, 9(11):1863, 2020.
  47. Data integration by two-sensors in a LEAP-based Virtual Glove for human-system interaction. Multimedia Tools and Applications, 80(12):18263–18277, May 2021.
  48. Toward welding robot with human knowledge: A remotely-controlled approach. IEEE Transactions on Automation Science and Engineering, 12(2):769–774, 2015.
  49. Evaluation of hand-tracking systems in teleoperation and virtual dexterous manipulation. IEEE Sensors Journal, 20(3):1642–1655, 2020.
  50. A three-fingered force feedback glove using fiber-reinforced soft bending actuators. IEEE Transactions on Industrial Electronics, 67(9):7681–7690, 2020.
  51. Enhancing user experience in chinese initial text conversations with personalised ai-powered assistant. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, CHI EA ’24, New York, NY, USA, 2024. Association for Computing Machinery.
  52. Broader and deeper: A multi-features with latent relations bert knowledge tracing model. In European Conference on Technology Enhanced Learning, pages 183–197. Springer, 2023.
  53. Gesture recognition algorithm based on image information fusion in virtual reality. Personal and Ubiquitous Computing, 23(3):487–497, 2019.
  54. Video-based skeletal feature extraction for hand gesture recognition. In Proceedings of the 4th International Conference on Machine Learning and Soft Computing, pages 108–112, 2020.
  55. Comparative study for vision based and data based hand gesture recognition technique. In 2017 International Conference on Intelligent Communication and Computational Techniques (ICCT), pages 26–31, 2017.
  56. Human-machine interaction based on hand gesture recognition using skeleton information of kinect sensor. In Proceedings of the 3rd international conference on applications in information technology, pages 75–79, 2018.
  57. Work-in-progress: A novel data glove for psychomotor-based virtual medical training. In 2021 IEEE Global Engineering Education Conference (EDUCON), pages 1318–1321, 2021.
  58. Smart hand device gesture recognition with dynamic time-warping method. In Proceedings of the International Conference on Big Data and Internet of Thing, pages 216–219, 2017.
  59. Sparse decomposition for data glove gesture recognition. In 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pages 1–5. IEEE, 2017.
  60. Real-time gesture recognition using 3d sensory data and a light convolutional neural network. In Proceedings of the 27th ACM International Conference on Multimedia, pages 401–410, 2019.
  61. Hand gesture recognition using deep feature fusion network based on wearable sensors. IEEE Sensors Journal, 21(1):539–547, 2020.
  62. Comparison of hand gesture inputs of leap motion controller & data glove in to a soft finger. In 2017 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS), pages 62–68. IEEE, 2017.
  63. 3d human gesture capturing and recognition by the immu-based data glove. Neurocomputing, 277:198–207, 2018. Hierarchical Extreme Learning Machines.
  64. A smart glove of combining virtual and real environments for chemical experiment. In Proceedings of 2020 the 6th International Conference on Computing and Data Engineering, pages 227–231, 2020.
  65. Static and dynamic human arm/hand gesture capturing and recognition via multiinformation fusion of flexible strain sensors. IEEE Sensors Journal, 20(12):6450–6459, 2020.
  66. Dynamic hand gesture recognition based on signals from specialized data glove and deep learning algorithms. IEEE Transactions on Instrumentation and Measurement, 70:1–14, 2021.
  67. Sofia Kiriaki Vulgari. Hand Gesture based Telemedicine enabled by Mobile VR. 2019.
  68. A framework for hand gesture recognition based on accelerometer and emg sensors. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 41(6):1064–1076, 2011.
  69. Zhaoxing Li. Deep Reinforcement Learning Approaches for Technology Enhanced Learning. PhD thesis, Durham University, 2023.
  70. Sim-gail: A generative adversarial imitation learning approach of student modelling for intelligent tutoring systems. Neural Computing and Applications, 35(34):24369–24388, 2023.
  71. semg-based continuous hand gesture recognition using gmm-hmm and threshold model. In 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), pages 1509–1514. IEEE, 2017.
  72. Hand gesture recognition using semg signals based on support vector machine. In 2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), pages 230–234. IEEE, 2019.
  73. A survey of collaborative reinforcement learning: interactive methods and design patterns. In Proceedings of the 2021 ACM Designing Interactive Systems Conference, pages 1579–1590, 2021.
  74. A novel surface electromyographic signal-based hand gesture prediction using a recurrent neural network. Sensors, 20(14):3994, 2020.
  75. Hand gesture recognition based on surface electromyography using convolutional neural network with transfer learning method. IEEE Journal of Biomedical and Health Informatics, 25(4):1292–1304, 2020.
  76. A systematic literature review on vision based gesture recognition techniques. Multimedia Tools and Applications, 77(21):28121–28184, November 2018.
  77. Stable hand pose estimation under tremor via graph neural network. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR), pages 226–234. IEEE, 2021.
  78. Hand gesture recognition from depth and infrared kinect data for cave applications interaction. Multimedia Tools and Applications, 76(20):20423–20455, 2017.
  79. Effective inertial hand gesture recognition using particle filtering based trajectory matching. Journal of Electrical and Computer Engineering, 2018, 2018.
  80. Real-time embedded recognition of sign language alphabet fingerspelling in an imu-based glove. In Proceedings of the 4th international Workshop on Sensor-based Activity Recognition and Interaction, pages 1–6, 2017.
  81. Anti-disturbance control based on uncertain data. In Proceedings of the 3rd International Conference on Software Engineering and Information Management, pages 32–35, 2020.
  82. Enhanced appearance-based finger detection and tracking using finite state machine control. Journal of Internet Technology, 21(4):1087–1096, 2020.
  83. Structured pruning of deep convolutional neural networks. ACM Journal on Emerging Technologies in Computing Systems (JETC), 13(3):1–18, 2017.
  84. Sparse convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 806–814, 2015.
  85. A bayesian tensor decomposition approach for spatiotemporal traffic data imputation. Transportation research part C: emerging technologies, 98:73–84, 2019.
  86. Channel pruning for accelerating very deep neural networks. In Proceedings of the IEEE international conference on computer vision, pages 1389–1397, 2017.
  87. Design of digital gloves with feedback for vr. In 2018 IEEE East-West Design & Test Symposium (EWDTS), pages 1–5. IEEE, 2018.
  88. Development of augmented reality serious games with a vibrotactile feedback jacket. Virtual Reality & Intelligent Hardware, 2(5):454–470, 2020.
  89. I see your point: Integrating gaze to enhance pointing gesture accuracy while driving. In Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pages 351–358, 2018.
  90. A user study on mixed reality remote collaboration with eye gaze and hand gesture sharing. In Proceedings of the 2020 CHI conference on human factors in computing systems, pages 1–13, 2020.
  91. Gesture and gaze: Multimodal data in dyadic interactions. In International Handbook of Computer-Supported Collaborative Learning, pages 625–641. Springer, 2021.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets