DuetFace: Collaborative Privacy-Preserving Face Recognition via Channel Splitting in the Frequency Domain (2207.07340v1)

Abstract: With the wide application of face recognition systems, there is rising concern that original face images could be exposed to malicious intents and consequently cause personal privacy breaches. This paper presents DuetFace, a novel privacy-preserving face recognition method that employs collaborative inference in the frequency domain. Starting from a counterintuitive discovery that face recognition can achieve surprisingly good performance with only visually indistinguishable high-frequency channels, this method designs a credible split of frequency channels by their cruciality for visualization and operates the server-side model on non-crucial channels. However, the model degrades in its attention to facial features due to the missing visual information. To compensate, the method introduces a plug-in interactive block to allow attention transfer from the client-side by producing a feature mask. The mask is further refined by deriving and overlaying a facial region of interest (ROI). Extensive experiments on multiple datasets validate the effectiveness of the proposed method in protecting face images from undesired visual inspection, reconstruction, and identification while maintaining high task availability and performance. Results show that the proposed method achieves a comparable recognition accuracy and computation cost to the unprotected ArcFace and outperforms the state-of-the-art privacy-preserving methods. The source code is available at https://github.com/Tencent/TFace/tree/master/recognition/tasks/duetface.

• The paper introduces DuetFace, a method that splits image frequency channels to protect privacy without sacrificing significant recognition accuracy.
• It employs collaborative inference where the client processes high-frequency channels and transfers an attention mask to the server.
• Empirical results show DuetFace closely matches ArcFace's performance on benchmarks while reducing sensitive visual data exposure.

Collaborative Privacy-Preserving Face Recognition through Frequency-Domain Channel Splitting

This paper presents DuetFace, an innovative approach to privacy-preserving face recognition, leveraging collaborative inference in the frequency domain. The authors introduce a method that effectively addresses privacy concerns while maintaining high recognition accuracy through strategic use of high-frequency channel information.

Methodology Overview

The core principle of DuetFace is the collaborative effort between the client and the server, both performing distinct roles in the recognition process. The authors start from a non-trivial observation—face recognition can achieve commendable accuracy using only non-crucial high-frequency channels, allowing for significant privacy protection without substantial loss in recognition performance.

Frequency Domain Channel Splitting

DuetFace begins with a transformation of face images into the frequency domain using block discrete cosine transform (BDCT). This enables the separation of channels based on their energy levels. The server processes the non-crucial channels, which are less visually informative, providing a foundation for privacy as these components reveal minimal sensitive information. This innovative channel splitting addresses privacy concerns by removing visual information from the server-side processing.

Collaborative Inference and Attention Transfer

The method introduces an interactive block designed to alleviate the degradation in attention to facial features that arises from missing visual details. The client-side model, significantly lighter than the server-side, processes crucial channels to derive a feature mask—a refined representation enabling attention transfer. This mask assists the server by realigning its attention to important features, thus compensating for any information loss.

Empirical Results and Performance

DuetFace shows competitive performance, closely matching the recognition rate of unprotected ArcFace across several benchmark datasets. The integration of channel splitting and attention transfer demonstrates efficacy in achieving privacy without compromising accuracy—outperforming existing privacy-preserving approaches in several experiments. The approach offers commendable trade-offs in computational and communication costs.

Implications and Future Directions

The findings have significant theoretical and practical implications, offering a promising path for privacy-preserving deep learning models. By addressing the privacy-accuracy trade-off innovatively, DuetFace inspires further exploration into collaborative models and frequency-domain operations. Future work may explore optimizing channel selection processes and further enhancing attention mechanisms to refine accuracy.

Overall, the DuetFace approach provides a significant contribution to the field of privacy-preserving machine learning by effectively balancing the dual imperatives of accuracy and privacy, setting a strong precedent for future research and application in this domain.