Papers
Topics
Authors
Recent
2000 character limit reached

Do Not DeepFake Me: Privacy-Preserving Neural 3D Head Reconstruction Without Sensitive Images (2312.04106v2)

Published 7 Dec 2023 in cs.CV

Abstract: While 3D head reconstruction is widely used for modeling, existing neural reconstruction approaches rely on high-resolution multi-view images, posing notable privacy issues. Individuals are particularly sensitive to facial features, and facial image leakage can lead to many malicious activities, such as unauthorized tracking and deepfake. In contrast, geometric data is less susceptible to misuse due to its complex processing requirements, and absence of facial texture features. In this paper, we propose a novel two-stage 3D facial reconstruction method aimed at avoiding exposure to sensitive facial information while preserving detailed geometric accuracy. Our approach first uses non-sensitive rear-head images for initial geometry and then refines this geometry using processed privacy-removed gradient images. Extensive experiments show that the resulting geometry is comparable to methods using full images, while the process is resistant to DeepFake applications and facial recognition (FR) systems, thereby proving its effectiveness in privacy protection.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. ICCV, 2021.
  2. A morphable model for the synthesis of 3d faces. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pages 157–164. 2023.
  3. Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16569–16578, 2023.
  4. Lowkey: Leveraging adversarial attacks to protect social media users from facial recognition. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
  5. My face my choice: Privacy enhancing deepfakes for social media anonymization. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1369–1379, 2023.
  6. Spact: Self-supervised privacy preservation for action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20164–20173, 2022.
  7. Privacy-preserving face recognition. In Privacy Enhancing Technologies: 9th International Symposium, PETS 2009, Seattle, WA, USA, August 5-7, 2009. Proceedings 9, pages 235–253. Springer, 2009.
  8. Ted-spad: Temporal distinctiveness for self-supervised privacy-preservation for video anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13598–13609, 2023.
  9. Dynamic neural radiance fields for monocular 4d facial avatar reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8649–8658, 2021.
  10. Instahide: Instance-hiding schemes for private distributed learning. In International conference on machine learning, pages 4507–4518. PMLR, 2020.
  11. Dartblur: Privacy preservation with detection artifact suppression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16479–16488, 2023.
  12. Nersemble: Multi-view radiance field reconstruction of human heads. ACM Trans. Graph., 42(4):161:1–161:14, 2023.
  13. Efficient and privacy-preserving distributed face recognition scheme via facenet. In Proceedings of the ACM Turing Award Celebration Conference-China, pages 110–115, 2021.
  14. Compressing volumetric radiance fields to 1 mb. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4222–4231, 2023.
  15. Privacy-preserving lightweight face recognition. Neurocomputing, 363:212–222, 2019.
  16. Lightweight privacy-preserving ensemble classification for face recognition. IEEE Internet of Things Journal, 6(3):5778–5790, 2019.
  17. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4460–4470, 2019.
  18. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
  19. Gender privacy: An ensemble of semi adversarial networks for confounding arbitrary gender classifiers. In 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), pages 1–10. IEEE, 2018.
  20. Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions on Graphics (ToG), 41(4):1–15, 2022.
  21. Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3504–3515, 2020.
  22. Towards a visual privacy advisor: Understanding and predicting privacy risks in images. In Proceedings of the IEEE international conference on computer vision, pages 3686–3695, 2017.
  23. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 165–174, 2019.
  24. Nerfies: Deformable neural radiance fields. 2021.
  25. H3d-net: Few-shot high-fidelity 3d head reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5620–5629, 2021.
  26. Permutosdf: Fast multi-view reconstruction with implicit surfaces using permutohedral lattices. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8466–8475, 2023.
  27. Efficient privacy-preserving face recognition. In International conference on information security and cryptology, pages 229–244. Springer, 2009.
  28. Does a face mask protect my privacy?: Deep learning to predict protected attributes from masked face images. In Australasian Joint Conference on Artificial Intelligence, pages 91–102. Springer, 2022.
  29. Comprehensive vulnerability evaluation of face recognition systems to template inversion attacks via 3d face reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
  30. Realistic face reconstruction from deep embeddings. In NeurIPS 2021 Workshop Privacy in Machine Learning, 2021.
  31. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2106.10689, 2021a.
  32. Gender obfuscation through face morphing. In 2021 IEEE International Workshop on Biometrics and Forensics (IWBF), pages 1–6. IEEE, 2021b.
  33. Hf-neus: Improved surface reconstruction using high-frequency details. Advances in Neural Information Processing Systems, 35:1966–1978, 2022.
  34. Pet-neus: Positional encoding tri-planes for neural surfaces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12598–12607, 2023.
  35. Privacy-preserving face recognition with outsourced computation. Soft Computing, 20:3735–3744, 2016.
  36. mm3dface: Nonintrusive 3d facial reconstruction leveraging mmwave signals. In Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services, pages 462–474, 2023.
  37. Deformable model-driven neural rendering for high-fidelity 3d reconstruction of human heads under low-view settings. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 17924–17934, 2023.
  38. Facescape: a large-scale high quality 3d face dataset and detailed riggable 3d face prediction. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, pages 601–610, 2020.
  39. Efficient and privacy-preserving online face recognition over encrypted outsourced data. In 2018 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), pages 366–373. IEEE, 2018.
  40. Multiview neural surface reconstruction by disentangling geometry and appearance. Advances in Neural Information Processing Systems, 33:2492–2502, 2020.
  41. Volume rendering of neural implicit surfaces. Advances in Neural Information Processing Systems, 34:4805–4815, 2021.
  42. i3dmm: Deep implicit 3d morphable model of human heads. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12803–12813, 2021.
  43. Nofa: Nerf-based one-shot facial avatar reconstruction. In ACM SIGGRAPH 2023 Conference Proceedings, pages 1–12, 2023.
  44. Adaptive privacy preserving deep learning algorithms for medical data. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1169–1178, 2021.
  45. Neuface: Realistic 3d neural face rendering from multi-view images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
  46. Im avatar: Implicit morphable head avatars from videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13545–13555, 2022.
  47. Personal privacy protection via irrelevant faces tracking and pixelation in video live streaming. IEEE Transactions on Information Forensics and Security, 16:1088–1103, 2020.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up