Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

GaitGuard: Towards Private Gait in Mixed Reality (2312.04470v3)

Published 7 Dec 2023 in cs.HC and cs.CR

Abstract: Augmented/Mixed Reality (AR/MR) technologies offers a new era of immersive, collaborative experiences, distinctively setting them apart from conventional mobile systems. However, as we further investigate the privacy and security implications within these environments, the issue of gait privacy emerges as a critical yet underexplored concern. Given its uniqueness as a biometric identifier that can be correlated to several sensitive attributes, the protection of gait information becomes crucial in preventing potential identity tracking and unauthorized profiling within these systems. In this paper, we conduct a user study with 20 participants to assess the risk of individual identification through gait feature analysis extracted from video feeds captured by MR devices. Our results show the capability to uniquely identify individuals with an accuracy of up to 92%, underscoring an urgent need for effective gait privacy protection measures. Through rigorous evaluation, we present a comparative analysis of various mitigation techniques, addressing both aware and unaware adversaries, in terms of their utility and impact on privacy preservation. From these evaluations, we introduce GaitGuard, the first real-time framework designed to protect the privacy of gait features within the camera view of AR/MR devices. Our evaluations of GaitGuard within a MR collaborative scenario demonstrate its effectiveness in implementing mitigation that reduces the risk of identification by up to 68%, while maintaining a minimal latency of merely 118.77 ms, thus marking a critical step forward in safeguarding privacy within AR/MR ecosystems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. 2013. The CPRA. https://thecpra.org/.
  2. 2018. Likert Scale: Definition, Examples & How to Use It. https://www.questionpro.com/blog/what-is-likert-scale/.
  3. 2022. App Quality Criteria - Mixed Reality. https://learn.microsoft.com/en-us/windows/mixed-reality/develop/advanced-concepts/app-quality-criteria-overview.
  4. 2022. Introduction to the Multi-user Capabilities Tutorials - Mixed Reality. https://learn.microsoft.com/en-us/windows/mixed-reality/develop/unity/tutorials/mr-learning-sharing-01.
  5. 2022. Mixed Reality Capture Overview. https://learn.microsoft.com/en-us/windows/mixed-reality/develop/advanced-concepts/mixed-reality-capture-overview.
  6. 2022. Performance - MRTK 2. https://learn.microsoft.com/en-us/windows/mixed-reality/mrtk-unity/mrtk2/performance/perf-getting-started?view=mrtkunity-2022-05.
  7. 2023. Apple Vision Pro. {https://www.apple.com/apple-vision-pro/}.
  8. 2023. Bouvet. https://www.microsoft.com/en-us/p/lens-by-bouvet/9p29ft8z34zs.
  9. 2023. Catapult Mixed Reality Client and Development Application. https://www.makesea.com/download/.
  10. 2023. General Data Protection Regulation (GDPR). https://gdpr-info.eu/.
  11. 2023. HoloLens 2 Hardware. https://learn.microsoft.com/en-us/hololens/hololens2-hardware.
  12. 2023a. HoloLens Microsoft 2. {https://www.microsoft.com/en-us/hololens/}.
  13. 2023b. HoloLens2ForCV. https://github.com/microsoft/HoloLens2ForCV/tree/main/Samples/CameraWithCVAndCalibration.
  14. 2023. Meta Quest 3: Mixed Reality VR Headset. {https://www.meta.com/quest/quest-3/}.
  15. 2023. Microsoft Apps. https://apps.microsoft.com/home?hl=en-US&gl=US.
  16. 2023. OpenCV. {https://docs.opencv.org/3.4/d5/d33/structcv_1_1HOGDescriptor.html}.
  17. 2023. Overview of Dynamics 365 Remote Assist on HoloLens and HoloLens 2. https://learn.microsoft.com/en-us/dynamics365/mixed-reality/remote-assist/overview-hololens.
  18. 2023. Photon Unity Networking for Unity Multiplayer Games PUN2. {https://www.photonengine.com/PUN}.
  19. 2023. Pose Estimation. https://paperswithcode.com/task/pose-estimation.
  20. 2023. Sklearn.Ensemble.GradientBoostingClassifier. https://scikit-learn/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html.
  21. 2023. Sphere XR. https://apps.microsoft.com/detail/9P11TRNWG2HR?hl=en-US.
  22. 2023. What Is Mixed Reality? https://learn.microsoft.com/en-us/windows/mixed-reality/discover/mixed-reality.
  23. Walk to show your identity: gait-based seamless user authentication framework using deep neural network. In The 5th ACM Workshop on Wearable Systems and Applications. 53–58.
  24. Realtime multi-person 2d pose estimation using part affinity fields. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7291–7299.
  25. Aurelio Cappozzo. 1984. Gait analysis methodology. Human movement science 3, 1-2 (1984), 27–50.
  26. Exploring user reactions and mental models towards perceptual manipulation attacks in mixed reality. In USENIX Security, Vol. 18.
  27. Gait-based authentication using a wrist-worn device. In Proceedings of the 13th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services. 208–217.
  28. BystandAR: Protecting Bystander Visual Data in Augmented Reality Systems. In Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services. 370–382.
  29. IMU-based gait recognition using convolutional neural networks and multi-sensor fusion. Sensors 17, 12 (2017), 2735.
  30. GaitPrivacyON: Privacy-preserving mobile gait biometrics using unsupervised learning. Pattern Recognition Letters 161 (2022), 30–37.
  31. Improved cycle detection for accelerometer based gait authentication. In 2010 Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing. IEEE, 312–317.
  32. Juan C Dibene and Enrique Dunn. 2022. HoloLens 2 Sensor Streaming. arXiv preprint arXiv:2211.02648 (2022).
  33. {{\{{LocIn}}\}}: Inferring Semantic Location from Spatial Maps in Mixed Reality. In 32nd USENIX Security Symposium (USENIX Security 23). 877–894.
  34. Full body gait analysis with Kinect. In 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 1964–1967.
  35. Gait authentication and identification using wearable accelerometer sensor. In 2007 IEEE workshop on automatic identification advanced technologies. IEEE, 220–225.
  36. Hidden reality: caution, your hand gesture inputs in the immersive virtual world are visible to all!. In 32nd USENIX Security Symposium (USENIX Security 23). 859–876.
  37. Understanding Person Identification through Gait. arXiv preprint arXiv:2203.04179 (2022).
  38. Sudhakar Kumawat and Hajime Nagahara. 2022. Privacy-Preserving Action Recognition via Motion Difference Quantization. In European Conference on Computer Vision. Springer, 518–534.
  39. Towards security and privacy for multi-user augmented reality: Foundations with end users. In 2018 IEEE Symposium on Security and Privacy (SP). IEEE, 392–408.
  40. Rationalstat LLC. 2023. Mixed Reality Market Is Set to Double by 2030 — Mixed Reality Market Share, Size, Growth, Forecast, 2023-2030 — RationalStat Study. https://www.globenewswire.com/news-release/2023/09/22/2747789/0/en/Mixed-Reality-Market-is-Set-to-Double-by-2030-Mixed-Reality-Market-Share-Size-Growth-Forecast-2023-2030-RationalStat-Study.html.
  41. Video-based pose estimation for gait analysis in stroke survivors during clinical assessments: a proof-of-concept study. Digital Biomarkers 6, 1 (2022), 9–18.
  42. Muhammad Muaaz and Rene Mayrhofer. 2014. Orientation independent cell phone based gait authentication. In Proceedings of the 12th International Conference on Advances in Mobile Computing and Multimedia. 161–164.
  43. Modern methods for affordable clinical gait analysis: theories and applications in healthcare systems. Academic Press.
  44. A reliable gait phase detection system. IEEE Transactions on neural systems and rehabilitation engineering 9, 2 (2001), 113–125.
  45. Comparative abilities of Microsoft Kinect and Vicon 3D motion capture for gait analysis. Journal of medical engineering & technology 38, 5 (2014), 274–280.
  46. Wearable sensor-based real-time gait detection: A systematic review. Sensors 21, 8 (2021), 2727.
  47. SecSpace: prototyping usable privacy and security for mixed reality collaborative environments. In Proceedings of the 2014 ACM SIGCHI symposium on Engineering interactive computing systems. 273–282.
  48. Secure {{\{{Multi-User}}\}} content sharing for augmented reality applications. In 28th USENIX Security Symposium (USENIX Security 19). 141–158.
  49. Warit Sirichotedumrong and Hitoshi Kiya. 2021. A gan-based image transformation scheme for privacy-preserving deep neural networks. In 2020 28th European Signal Processing Conference (EUSIPCO). IEEE, 745–749.
  50. Quantitative gait analysis in Parkinson’s disease: comparison with a healthy control group. Archives of physical medicine and rehabilitation 86, 5 (2005), 1007–1013.
  51. Two-dimensional video-based analysis of human gait using pose estimation. PLoS computational biology 17, 4 (2021), e1008935.
  52. Christopher L Vaughan. 2003. Theories of bipedal walking: an odyssey. Journal of biomechanics 36, 4 (2003), 513–523.
  53. A study on gait-based gender classification. IEEE Transactions on image processing 18, 8 (2009), 1905–1910.
  54. Adversarial framing for image and video classification. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 10077–10078.
  55. Ethnicity classification based on gait using multi-view fusion. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops. IEEE, 108–115.
  56. It’s all in your head (set): Side-channel attacks on ar/vr systems. In USENIX Security.
  57. The detection of age groups by dynamic gait outcomes using machine learning approaches. Scientific reports 10, 1 (2020), 4426.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets