Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Using Motion Forecasting for Behavior-Based Virtual Reality (VR) Authentication (2401.16649v1)

Published 30 Jan 2024 in cs.LG and cs.CR

Abstract: Task-based behavioral biometric authentication of users interacting in virtual reality (VR) environments enables seamless continuous authentication by using only the motion trajectories of the person's body as a unique signature. Deep learning-based approaches for behavioral biometrics show high accuracy when using complete or near complete portions of the user trajectory, but show lower performance when using smaller segments from the start of the task. Thus, any systems designed with existing techniques are vulnerable while waiting for future segments of motion trajectories to become available. In this work, we present the first approach that predicts future user behavior using Transformer-based forecasting and using the forecasted trajectory to perform user authentication. Our work leverages the notion that given the current trajectory of a user in a task-based environment we can predict the future trajectory of the user as they are unlikely to dramatically shift their behavior since it would preclude the user from successfully completing their task goal. Using the publicly available 41-subject ball throwing dataset of Miller et al. we show improvement in user authentication when using forecasted data. When compared to no forecasting, our approach reduces the authentication equal error rate (EER) by an average of 23.85% and a maximum reduction of 36.14%.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. N. Noah and S. Das, “Exploring evolution of augmented and virtual reality education space in 2020 through systematic literature review,” Computer Animation and Virtual Worlds, vol. 32, no. 3-4, p. e2020, 2021.
  2. F. J. Agbo, I. T. Sanusi, S. S. Oyelere, and J. Suhonen, “Application of virtual reality in computer science education: a systemic review based on bibliometric and content analysis methods,” Education Sciences, vol. 11, no. 3, p. 142, 2021.
  3. D. Hamilton, J. McKechnie, E. Edgerton, and C. Wilson, “Immersive virtual reality as a pedagogical tool in education: a systematic literature review of quantitative learning outcomes and experimental design,” Journal of Computers in Education, vol. 8, no. 1, pp. 1–32, 2021.
  4. S. Shorey and E. D. Ng, “The use of virtual reality simulation among nursing students and registered nurses: A systematic review,” Nurse education today, vol. 98, p. 104662, 2021.
  5. S. Barteit, L. Lanfermann, T. Bärnighausen, F. Neuhann, C. Beiersmann et al., “Augmented, mixed, and virtual reality-based head-mounted devices for medical education: systematic review,” JMIR serious games, vol. 9, no. 3, p. e29080, 2021.
  6. E. Clarke, “Virtual reality simulation—the future of orthopaedic training? a systematic review and narrative analysis,” Advances in Simulation, vol. 6, no. 1, pp. 1–11, 2021.
  7. G. Pizzi, D. Scarpi, M. Pichierri, and V. Vannucci, “Virtual reality, real reactions?: Comparing consumers’ perceptions and shopping orientation across physical and virtual-reality retail stores,” Computers in Human Behavior, vol. 96, no. 0, pp. 1–1, Jul 2019.
  8. L. Xue, C. J. Parker, and H. McCormick, “A virtual reality and retailing literature review: Current focus, underlying themes and future directions,” in Augmented Reality and Virtual Reality.   Berlin, Germany: Springer, 2019, pp. 27–41.
  9. A. G. Campbell, T. Holz, J. Cosgrove, M. Harlick, and T. O’Sullivan, “Uses of virtual reality for communication in financial services: A case study on comparing different telepresence interfaces: Virtual reality compared to video conferencing,” in Future of Information and Communication Conference.   Berlin, Germany: Springer, 2019, pp. 463–481.
  10. S. Weise and A. Mshar, “Virtual reality and the banking experience,” Journal of Digital Banking, vol. 1, no. 2, pp. 146–152, 2016.
  11. J. Muñoz, S. Mehrabi, Y. Li, A. Basharat, L. E. Middleton, S. Cao, M. Barnett-Cowan, J. Boger et al., “Immersive virtual reality exergames for persons living with dementia: User-centered design study as a multistakeholder team during the covid-19 pandemic,” JMIR Serious Games, vol. 10, no. 1, p. e29987, 2022.
  12. S. Mehrabi, J. E. Muñoz, A. Basharat, J. Boger, S. Cao, M. Barnett-Cowan, L. E. Middleton et al., “Immersive virtual reality exergames to promote the well-being of community-dwelling older adults: Protocol for a mixed methods pilot study,” JMIR Research Protocols, vol. 11, no. 6, p. e32955, 2022.
  13. S. Karaosmanoglu, L. Kruse, S. Rings, and F. Steinicke, “Canoe vr: An immersive exergame to support cognitive and physical exercises of older adults,” in CHI Conference on Human Factors in Computing Systems Extended Abstracts.   New York, NY: ACM, 2022, pp. 1–7.
  14. F. A. Alsulaiman and A. El Saddik, “Three-dimensional password for more secure authentication,” IEEE Transactions on Instrumentation and measurement, vol. 57, no. 9, pp. 1929–1938, 2008.
  15. ——, “A novel 3d graphical password schema,” in 2006 IEEE Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems.   Piscataway, NJ: IEEE, 2006, pp. 125–128.
  16. J. Gurary, Y. Zhu, and H. Fu, “Leveraging 3d benefits for authentication,” International Journal of Communications, Network and System Sciences, vol. 10, no. 08, p. 324, 2017.
  17. C. George, D. Buschek, A. Ngao, and M. Khamis, “Gazeroomlock: Using gaze and head-pose to improve the usability and observation resistance of 3d passwords in virtual reality,” in Augmented Reality, Virtual Reality, and Computer Graphics: 7th International Conference, AVR 2020, Lecce, Italy, September 7–10, 2020, Proceedings, Part I 7.   Berlin, Germany: Springer, 2020, pp. 61–81.
  18. Z. Yu, H.-N. Liang, C. Fleming, and K. L. Man, “An exploration of usable authentication mechanisms for virtual reality systems,” in 2016 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS).   Piscataway, NJ: IEEE, 2016, pp. 458–460.
  19. C. George, M. Khamis, E. von Zezschwitz, M. Burger, H. Schmidt, F. Alt, and H. Hussmann, “Seamless and secure vr: Adapting and evaluating established authentication systems for virtual reality,” in NDSS.   San Diego, CA: NDSS, 2017, pp. 1–1.
  20. I. Olade, H.-N. Liang, C. Fleming, and C. Champion, “Exploring the vulnerabilities and advantages of swipe or pattern authentication in virtual reality (vr),” in Proceedings of the 2020 4th International Conference on Virtual and Augmented Reality Simulations.   New York, NY: ACM, 2020, pp. 45–52.
  21. M. Funk, K. Marky, I. Mizutani, M. Kritzler, S. Mayer, and F. Michahelles, “Lookunlock: Using spatial-targets for user-authentication on hmds,” in Ext. Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems.   New York, NY: ACM, 2019, pp. 1–6.
  22. C. George, M. Khamis, D. Buschek, and H. Hussmann, “Investigating the third dimension for authentication in immersive virtual reality and in the real world,” in 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).   Piscataway, NJ: IEEE, 2019, pp. 277–285.
  23. T. Mustafa, R. Matovu, A. Serwadda, and N. Muirhead, “Unsure how to authenticate on your vr headset? come on, use your head!” in Proceedings of the Fourth ACM International Workshop on Security and Privacy Analytics.   New York, NY: ACM, 2018, pp. 23–30.
  24. A. Kupin, B. Moeller, Y. Jiang, N. K. Banerjee, and S. Banerjee, “Task-driven biometric authentication of users in virtual reality (vr) environments,” in MultiMedia Modeling: 25th International Conference, MMM 2019, Thessaloniki, Greece, January 8–11, 2019, Proceedings, Part I 25.   Berlin, Germany: Springer, 2019, pp. 55–67.
  25. K. Pfeuffer, M. J. Geiger, S. Prange, L. Mecke, D. Buschek, and F. Alt, “Behavioural biometrics in vr: Identifying people from body motion and relations in virtual reality,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems.   New York, NY: ACM, 2019, pp. 1–12.
  26. A. Ajit, N. K. Banerjee, and S. Banerjee, “Combining pairwise feature matches from device trajectories for biometric authentication in virtual reality environments,” in 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR).   Piscataway, NJ: IEEE, 2019, pp. 9–97.
  27. R. Miller, A. Ajit, N. K. Banerjee, and S. Banerjee, “Realtime behavior-based continual authentication of users in virtual reality environments,” in 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR).   Piscataway, NJ: IEEE, 2019, pp. 253–2531.
  28. F. Mathis, H. I. Fawaz, and M. Khamis, “Knowledge-driven biometric authentication in virtual reality,” in Ext. Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems.   New York, NY: ACM, 2020, pp. 1–10.
  29. F. Mathis, J. Williamson, K. Vaniea, and M. Khamis, “Rubikauth: Fast and secure authentication in virtual reality,” in Ext. Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems.   New York, NY: ACM, 2020, pp. 1–9.
  30. F. Mathis, J. H. Williamson, K. Vaniea, and M. Khamis, “Fast and secure authentication in virtual reality using coordinated 3d manipulation and pointing,” ACM Transactions on Computer-Human Interaction, vol. 6, no. 1, pp. 1–1, Jan 2021.
  31. R. Miller, N. K. Banerjee, and S. Banerjee, “Within-system and cross-system behavior-based biometric authentication in virtual reality,” in 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW).   Piscataway, NJ: IEEE, 2020, pp. 311–316.
  32. I. Olade, C. Fleming, and H.-N. Liang, “Biomove: Biometric user identification from human kinesiological movements for virtual reality systems,” Sensors, vol. 20, no. 10, p. 2944, 2020.
  33. M. R. Miller, F. Herrera, H. Jun, J. A. Landay, and J. N. Bailenson, “Personal identifiability of user tracking data during observation of 360-degree vr video,” Scientific Reports, vol. 10, no. 1, pp. 1–10, 2020.
  34. R. Miller, N. K. Banerjee, and S. Banerjee, “Using siamese neural networks to perform cross-system behavioral authentication in virtual reality,” in 2021 IEEE Virtual Reality and 3D User Interfaces (VR).   Piscataway, NJ: IEEE, 2021, pp. 140–149.
  35. J. Liebers, M. Abdelaziz, L. Mecke, A. Saad, J. Auda, U. Gruenefeld, F. Alt, and S. Schneegass, “Understanding user identification in virtual reality through behavioral biometrics and the effect of body normalization,” in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems.   New York, NY: ACM, 2021, pp. 1–11.
  36. Z. Zhou, L. Ye, J. Wang, K. Wu, and K. Lu, “Hivt: Hierarchical vector transformer for multi-agent motion prediction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.   Piscataway, NJ: IEEE, 2022, pp. 8823–8833.
  37. Z. Huang, X. Mo, and C. Lv, “Multi-modal motion prediction with transformer-based neural network for autonomous driving,” in 2022 International Conference on Robotics and Automation (ICRA).   Piscataway, NJ: IEEE, 2022, pp. 2605–2611.
  38. Y. Kong and Y. Fu, “Human action recognition and prediction: A survey,” International Journal of Computer Vision, vol. 130, no. 5, pp. 1366–1401, 2022.
  39. Y. Yuan, X. Weng, Y. Ou, and K. M. Kitani, “Agentformer: Agent-aware transformers for socio-temporal multi-agent forecasting,” in Proceedings of the IEEE/CVF International Conference on Computer Vision.   Piscataway, NJ: IEEE, 2021, pp. 9813–9823.
  40. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, pp. 1–1, 2017.
  41. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, vol. 1, pp. 1–1, 2018.
  42. H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond efficient transformer for long sequence time-series forecasting,” in Proceedings of the AAAI conference on artificial intelligence, vol. 35.   Washington, DC: AAAI, 2021, pp. 11 106–11 115.
  43. R. Miller, N. K. Banerjee, and S. Banerjee, “Temporal effects in motion behavior for virtual reality (vr) biometrics,” in 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).   Piscataway, NJ: IEEE, 2022, pp. 563–572.
  44. ——, “Combining real-world constraints on user behavior with deep neural networks for virtual reality (vr) biometrics,” in 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).   Piscataway, NJ: IEEE, 2022, pp. 409–418.
  45. J. M. Jones, R. Duezguen, P. Mayer, M. Volkamer, and S. Das, “A literature review on virtual reality authentication,” in International Symposium on Human Aspects of Information Security and Assurance.   Berlin, Germany: Springer, 2021, pp. 189–198.
  46. A. Giaretta, “Security and privacy in virtual reality–a literature survey,” arXiv preprint arXiv:2205.00208, vol. 1, pp. 1–1, 2022.
  47. S. Stephenson, B. Pal, S. Fan, E. Fernandes, Y. Zhao, and R. Chatterjee, “Sok: Authentication in augmented and virtual reality,” in 2022 IEEE Symposium on Security and Privacy (SP).   Piscataway, NJ: IEEE, 2022, pp. 1552–1552.
  48. F. Alt and S. Schneegass, “Beyond passwords—challenges and opportunities of future authentication,” IEEE Security & Privacy, vol. 20, no. 1, pp. 82–86, 2022.
  49. M. Papadatou-Pastou, E. Ntolka, J. Schmitz, M. Martin, M. R. Munafò, S. Ocklenburg, and S. Paracchini, “Human handedness: A meta-analysis.” Psychological bulletin, vol. 146, no. 6, p. 481, 2020.
  50. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition.   Piscataway, NJ: IEEE, 2016, pp. 770–778.
  51. J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450, vol. 1, pp. 1–1, 2016.
  52. Z. Wang, W. Yan, and T. Oates, “Time series classification from scratch with deep neural networks: A strong baseline,” in 2017 International joint conference on neural networks (IJCNN).   Piscataway, NJ: IEEE, 2017, pp. 1578–1585.
  53. S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in International conference on machine learning.   World: pmlr, 2015, pp. 448–456.
  54. M. Lin, Q. Chen, and S. Yan, “Network in network,” arXiv preprint arXiv:1312.4400, vol. 1, pp. 1–1, 2013.
  55. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, vol. 1, pp. 1–1, 2014.
  56. W. Zeng, M. Liang, R. Liao, and R. Urtasun, “Lanercnn: Distributed representations for graph-centric motion forecasting,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   Piscataway, NJ: IEEE, 2021, pp. 532–539.
  57. Y. Liu, J. Zhang, L. Fang, Q. Jiang, and B. Zhou, “Multimodal motion prediction with stacked transformers,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.   Piscataway, NJ: IEEE, 2021, pp. 7577–7586.
  58. C. Rack, T. Fernando, M. Yalcin, A. Hotho, and M. E. Latoschik, “Who is alyx? a new behavioral biometric dataset for user identification in xr,” arXiv preprint arXiv:2308.03788, 2023.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now