Papers
Topics
Authors
Recent
2000 character limit reached

PrivatEyes: Appearance-based Gaze Estimation Using Federated Secure Multi-Party Computation

Published 29 Feb 2024 in cs.CV and cs.HC | (2402.18970v1)

Abstract: Latest gaze estimation methods require large-scale training data but their collection and exchange pose significant privacy risks. We propose PrivatEyes - the first privacy-enhancing training approach for appearance-based gaze estimation based on federated learning (FL) and secure multi-party computation (MPC). PrivatEyes enables training gaze estimators on multiple local datasets across different users and server-based secure aggregation of the individual estimators' updates. PrivatEyes guarantees that individual gaze data remains private even if a majority of the aggregating servers is malicious. We also introduce a new data leakage attack DualView that shows that PrivatEyes limits the leakage of private training data more effectively than previous approaches. Evaluations on the MPIIGaze, MPIIFaceGaze, GazeCapture, and NVGaze datasets further show that the improved privacy does not lead to a lower gaze estimation accuracy or substantially higher computational costs - both of which are on par with its non-secure counterparts.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (96)
  1. Quantification of Users’ Visual Attention During Everyday Mobile Device Interactions. In Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI). 1–14. https://doi.org/10.1145/3313831.3376449
  2. Shumeet Baluja and Dean Pomerleau. 1993. Non-intrusive gaze tracking using artificial neural networks. Advances in Neural Information Processing Systems 6 (1993).
  3. Assessing differentially private deep learning with membership inference. arXiv preprint arXiv:1912.11328 (2019).
  4. Pradipta Biswas et al. 2021. Appearance-based gaze estimation using attention and difference mechanism. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 3143–3152.
  5. Practical secure aggregation for privacy-preserving machine learning. In proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 1175–1191.
  6. Federated learning and privacy. Commun. ACM 65, 4 (2022), 90–97.
  7. Differential privacy for eye tracking with temporal correlations. Plos one 16, 8 (2021), e0255979.
  8. Eye-tracked Virtual Reality: A Comprehensive Survey on Methods and Privacy Challenges. arXiv preprint arXiv:2305.14080 (2023).
  9. Privacy preserving gaze estimation using synthetic images via a randomized encoding based framework. In ACM symposium on eye tracking research and applications. 1–5.
  10. Andreas Bulling and Hans Gellersen. 2010. Toward mobile eye-based human-computer interaction. IEEE Pervasive Computing 9, 4 (2010), 8–12.
  11. Andreas Bulling and Daniel Roggen. 2011. Recognition of Visual Memory Recall Processes Using Eye Movement Analysis. In Proc. ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp). 455–464. https://doi.org/10.1145/2030112.2030172
  12. EyeContext: Recognition of high-level contextual cues from human visual behaviour. In Proceedings of the sigchi conference on human factors in computing systems. 305–308.
  13. Rofl: Attestable robustness for secure federated learning. arXiv preprint arXiv:2107.03311 (2021).
  14. Quantification of Users’ Visual Attention During Everyday Mobile Device Interactions. In Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI). https://doi.org/10.1145/3313831.3376449
  15. GANT: Gaze analysis technique for human identification. Pattern Recognition 48, 4 (2015), 1027–1038.
  16. Jixu Chen and Qiang Ji. 2008. 3D gaze estimation with a single camera without IR illumination. In 2008 19th International Conference on Pattern Recognition. IEEE, 1–4.
  17. Appearance-based gaze estimation using kinect. In 2013 10th International Conference on Ubiquitous Robots and Ambient Intelligence. IEEE, 260–261.
  18. EIFFeL: Ensuring Integrity for Federated Learning. arXiv preprint arXiv:2112.12727 (2021).
  19. Confidential benchmarking based on multiparty computation. In International Conference on Financial Cryptography and Data Security. Springer, 169–187.
  20. Multiparty computation from somewhat homomorphic encryption. In Annual Cryptology Conference. Springer, 643–662.
  21. FLOD: Oblivious defender for private Byzantine-robust federated learning with dishonest-majority. In European Symposium on Research in Computer Security. Springer, 497–518.
  22. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science 9, 3–4 (2014), 211–407.
  23. Federated Learning for Appearance-based Gaze Estimation in the Wild. In Proc. NeurIPS Workshop on Gaze Meets ML (GMML). 1–11.
  24. An automated behavioral measure of mind wandering during computerized reading. Behavior Research Methods 50, 1 (2018), 134–150.
  25. SAFELearn: Secure aggregation for private federated learning. In 2021 IEEE Security and Privacy Workshops (SPW). IEEE, 56–62.
  26. Model inversion attacks that exploit confidence information and basic countermeasures. In ACM CCS. 1322–1333.
  27. Group privacy for personalized federated learning. https://doi.org/10.48550/ARXIV.2206.03396
  28. Inverting gradients-how easy is it to break privacy in federated learning? Advances in Neural Information Processing Systems 33 (2020), 16937–16947.
  29. Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. In International conference on machine learning. PMLR, 201–210.
  30. Dan Witzner Hansen and Qiang Ji. 2009. In the eye of the beholder: A survey of models for eyes and gaze. IEEE transactions on pattern analysis and machine intelligence 32, 3 (2009), 478–500.
  31. Model inversion attacks against collaborative inference. In Proceedings of the 35th Annual Computer Security Applications Conference. 148–162.
  32. A single camera eye-gaze tracking system with free head motion. In Proceedings of the 2006 symposium on Eye tracking research & applications. 87–94.
  33. Visualizing gaze direction to support video coding of social attention for children with autism spectrum disorder. In 23rd International Conference on Intelligent User Interfaces. 571–582.
  34. Deep models under the GAN: information leakage from collaborative deep learning. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security. 603–618.
  35. Eye-tracking dysfunctions in schizophrenic patients and their relatives. Archives of general psychiatry 31, 2 (1974), 143–151.
  36. Eye movements during everyday behavior predict personality traits. Frontiers in Human Neuroscience (2018), 105.
  37. Building a personalized, auto-calibrating eye tracker from user interactions. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 5169–5179.
  38. Stressclick: Sensing stress from gaze-click patterns. In Proceedings of the 24th ACM international conference on Multimedia. 1395–1404.
  39. Screenglint: Practical, in-situ gaze estimation on smartphones. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 2546–2557.
  40. Eye tracking dysfunction in Alzheimer-type dementia. Neurology 34, 1 (1984), 99–99.
  41. Swati Jindal and Roberto Manduchi. 2022. Contrastive Representation Learning for Gaze Estimation.
  42. Advances and Open Problems in Federated Learning. (2019). https://arxiv.org/abs/1912.04977
  43. Marcel Keller. 2020. MP-SPDZ: A versatile framework for multi-party computation. In Proceedings of the 2020 ACM SIGSAC conference on computer and communications security. 1575–1590.
  44. Overdrive: making SPDZ great again. In Annual International Conference on the Theory and Applications of Cryptographic Techniques. Springer, 158–189.
  45. Daniel Kifer and Ashwin Machanavajjhala. 2011. No free lunch in data privacy. In Proceedings of the 2011 ACM SIGMOD International Conference on Management of data. 193–204.
  46. Nvgaze: An anatomically-informed dataset for low-latency, near-eye gaze estimation. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–12.
  47. Ron Kohavi and George H John. 1995. Automatic parameter selection by minimizing estimated error. In Machine Learning Proceedings 1995. Elsevier, 304–312.
  48. Eye tracking for everyone. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2176–2184.
  49. Appearance-Based Gaze Estimation for ASD Diagnosis. IEEE Transactions on Cybernetics (2022).
  50. {{\{{Kalε𝜀\varepsilonitalic_εido}}\}}:{{\{{Real-Time}}\}} Privacy Control for {{\{{Eye-Tracking}}\}} Systems. In 30th USENIX Security Symposium. 1793–1810.
  51. Membership inference attacks and defenses in supervised learning via generalization gap. ArXiv abs/2002.12062 (2020).
  52. Appearance-based gaze tracking with spectral clustering and semi-supervised gaussian process regression. In Proceedings of the 2013 Conference on Eye Tracking South Africa. 17–23.
  53. Differential privacy for eye-tracking data. In ACM Symposium on Eye Tracking Research & Applications. 1–10.
  54. Oblivious neural network predictions via minionn transformations. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security. 619–631.
  55. Threats, attacks and defenses to federated learning: issues, taxonomy and perspectives. Cybersecurity 5, 1 (2022), 1–19.
  56. {{\{{ML-Doctor}}\}}: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models. In 31st USENIX Security Symposium (USENIX Security 22). 4525–4542.
  57. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics. PMLR, 1273–1282.
  58. Exploiting unintended feature leakage in collaborative learning. In 2019 IEEE symposium on security and privacy (SP). IEEE, 691–706.
  59. Payman Mohassel and Peter Rindal. 2018. ABY3: A mixed protocol framework for machine learning. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security. 35–52.
  60. Payman Mohassel and Yupeng Zhang. 2017. Secureml: A system for scalable privacy-preserving machine learning. In 2017 IEEE symposium on security and privacy (SP). IEEE, 19–38.
  61. Detecting eye position and gaze from a single camera and 2 light sources. In 2002 International Conference on Pattern Recognition, Vol. 4. IEEE, 314–317.
  62. {{\{{FLAME}}\}}: Taming backdoors in federated learning. In 31st USENIX Security Symposium (USENIX Security 22). 1415–1432.
  63. Smooth sensitivity and sampling in private data analysis. In ACM Symposium on Theory of Computing. 75–84.
  64. MIRA–A Gaze-based Serious Game for Continuous Estimation of Alzheimer’s Mental State. In ACM Symposium on Eye Tracking Research and Applications. 1–3.
  65. Domain-Adaptive Full-Face Gaze Estimation via Novel-View-Synthesis and Feature Disentanglement. arXiv:2305.16140 [cs.CV]
  66. A first look into the carbon footprint of federated learning. Journal of Machine Learning Research 24, 129 (2023), 1–23.
  67. Elsa: Secure aggregation for federated learning with malicious actors. In 2023 IEEE Symposium on Security and Privacy (SP). IEEE, 1961–1979.
  68. Adaptive federated optimization. arXiv preprint arXiv:2003.00295 (2020).
  69. Deepsecure: Scalable provably-secure deep learning. In Proceedings of the 55th annual design automation conference. 1–6.
  70. {{\{{Updates-Leak}}\}}: Data set inference and reconstruction attacks in online learning. In 29th USENIX security symposium (USENIX Security 20). 1291–1308.
  71. Gender classification based on eye movements: A processing effect during passive face viewing. Advances in Cognitive Psychology 13, 3 (2017), 232.
  72. Membership inference attacks against machine learning models. In IEEE S&P. 3–18.
  73. Gaze locking: passive eye contact detection for human-object interaction. In Proceedings of the 26th annual ACM symposium on User interface software and technology. 271–280.
  74. Julian Steil and Andreas Bulling. 2015. Discovery of everyday human activities from long-term visual behaviour using topic models. In Proceedings of the 2015 acm international joint conference on pervasive and ubiquitous computing. 75–85.
  75. Privacy-aware eye tracking using differential privacy. In ACM Symposium on Eye Tracking Research & Applications. 1–9.
  76. Learning-by-synthesis for appearance-based 3d gaze estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1821–1828.
  77. Invisibleeye: Mobile eye tracking using multiple low-resolution cameras and learning-based gaze estimation. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 1–21.
  78. Labelled Pupils in the Wild: A Dataset for Studying Pupil Detection in Unconstrained Environments. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications (Charleston, South Carolina) (ETRA ’16). Association for Computing Machinery, New York, NY, USA, 139–142. https://doi.org/10.1145/2857491.2857520
  79. Combining head pose and eye location information for gaze estimation. IEEE Transactions on Image Processing 21, 2 (2011), 802–815.
  80. Roel Vertegaal et al. 2003. Attentive user interfaces. Commun. ACM 46, 3 (2003), 30–33.
  81. Mika Westerlund. 2019. The emergence of deepfake technology: A review. Technology innovation management review 9, 11 (2019).
  82. A 3D Morphable Eye Region Model for Gaze Estimation. In Proc. European Conference on Computer Vision (ECCV). 297–313. https://doi.org/10.1007/978-3-319-46448-0_18
  83. Learning an appearance-based gaze estimator from one million synthesised images. In Proc. ACM International Symposium on Eye Tracking Research and Applications (ETRA). 131–138. https://doi.org/10.1145/2857491.2857492
  84. A methodology for formalizing model-inversion attacks. In 2016 IEEE 29th Computer Security Foundations Symposium (CSF). IEEE, 355–370.
  85. Diversity-sensitive conditional generative adversarial networks. arXiv preprint arXiv:1901.09024 (2019).
  86. User Trust Dynamics: An Investigation Driven by Differences in System Performance. In Proceedings of the 22nd International Conference on Intelligent User Interfaces (Limassol, Cyprus) (IUI ’17). Association for Computing Machinery, New York, NY, USA, 307–317. https://doi.org/10.1145/3025171.3025219
  87. Yu Yu and Jean-Marc Odobez. 2020. Unsupervised representation learning for gaze estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7314–7324.
  88. Training Person-Specific Gaze Estimators from Interactions with Multiple Devices. In Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI). 1–12. https://doi.org/10.1145/3173574.3174198
  89. Eth-xgaze: A large scale dataset for gaze estimation under extreme head pose and gaze variation. In European Conference on Computer Vision. Springer, 365–381.
  90. Everyday eye contact detection using unsupervised gaze target discovery. In Proceedings of the 30th annual ACM symposium on user interface software and technology. 193–203.
  91. Appearance-based Gaze Estimation in the Wild. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 4511–4520. https://doi.org/10.1109/CVPR.2015.7299081
  92. It’s written all over your face: Full-face appearance-based gaze estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 51–60.
  93. MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 41, 1 (2019), 162–175. https://doi.org/10.1109/TPAMI.2017.2778103
  94. The secret revealer: Generative model-inversion attacks against deep neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 253–261.
  95. Age progression/regression by conditional adversarial autoencoder. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5810–5818.
  96. idlg: Improved deep leakage from gradients. arXiv preprint arXiv:2001.02610 (2020).
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now

Whiteboard

Paper to Video (Beta)

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 1 tweet with 1 like about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free