CryptoFace: Encrypted Face Recognition & Privacy

• CryptoFace is a framework that protects facial data by encrypting raw images, features, and templates to ensure privacy in face recognition systems.
• It employs fully homomorphic encryption to enable end-to-end encrypted processing, achieving significant improvements in inference speed and verification accuracy.
• The approach also integrates secure template matching, adversarial obfuscation, and biometric key binding, though high computational cost remains a practical challenge.

Searching arXiv for papers on CryptoFace and related face-privacy/security systems. CryptoFace denotes both a specific fully homomorphic face recognition system and, in current face-security literature, a broader class of mechanisms that protect facial data by encrypting raw images or embeddings, obfuscating shareable face images, or binding cryptographic secrets to facial measurements. In this usage, the central question is not merely recognition accuracy, but which object is protected—pixels, templates, scores, or keys—and under which threat model that protection is achieved (Ao et al., 30 Aug 2025, Serengil et al., 22 Feb 2025, Dong et al., 2023, Dong et al., 2024).

1. Conceptual scope

In the narrow sense, CryptoFace is the system introduced in "CryptoFace: End-to-End Encrypted Face Recognition," described as the first end-to-end encrypted face recognition system with fully homomorphic encryption (FHE) (Ao et al., 30 Aug 2025). In a broader sense suggested by adjacent work, the term can also cover cryptography-based and privacy-preserving face systems that secure different stages of the facial pipeline: feature extraction, template storage, matching, thresholding, public image sharing, or key derivation (Serengil et al., 22 Feb 2025, Bai et al., 2023, Kim et al., 16 Jul 2025).

Paradigm	Representative systems	Protected object
End-to-end encrypted recognition	CryptoFace	Raw images, features, scores, and threshold comparison
Encrypted template matching	CipherFace, CryptoMask, IDFace	Embeddings/templates and similarity computation
Adversarial or obfuscatory face encryption	TIP-IM, FRIA, Forbes	Shared face images or human-recognizable facial appearance
Biometric cryptosystem	WiFaKey	Cryptographic key material bound to facial features

These paradigms are not interchangeable. End-to-end FHE systems keep the entire recognition pipeline in ciphertext. Template-protection systems typically leave feature extraction in plaintext but encrypt storage and matching. Adversarial face-encryption systems retain human-viewable images while attempting to defeat unauthorized face recognition. Biometric cryptosystems convert facial measurements into helper data and recoverable keys rather than performing ordinary identification (Ao et al., 30 Aug 2025, Serengil et al., 22 Feb 2025, Yang et al., 2020, Dong et al., 2024).

2. End-to-end encrypted face recognition

CryptoFace implements the strongest protection model in this group: the client sends only encrypted images, the server performs neural feature extraction, template storage, matching, and threshold comparison directly on ciphertexts, and only the client can decrypt the final result (Ao et al., 30 Aug 2025). The server never sees raw images, feature vectors, similarity scores, or the final decision in plaintext. The underlying cryptographic primitive is CKKS, chosen for approximate arithmetic over real-valued tensors and neural-network inference.

The system architecture is built around a mixture of shallow patch convolutional networks. An image $x$ is split into non-overlapping patches $x_i$, each patch is processed by a dedicated shallow patch CNN $f_{\omega_i}$, local embeddings are concatenated, and a linear fusion layer produces the global embedding

$y = y' A^\top + b.$

This patch-wise decomposition is explicitly motivated by multiplicative-depth reduction and parallel FHE evaluation. The stated consequence is near-resolution-independent latency: scaling from $64\times 64$ to $128\times 128$ increases accuracy while keeping runtime almost constant under sufficient parallelism (Ao et al., 30 Aug 2025).

Matching is based on an encrypted approximation of a cosine-similarity-derived score,

$$\operatorname{Score}(y_1,y_2)=\left\|\frac{y_1}{\|y_1\|}-\frac{y_2}{\|y_2\|}\right\|^2 =2-2\frac{y_1^\top y_2}{\|y_1\|\|y_2\|},$$

with $1/\sqrt{t}$ approximated by a distribution-aware quadratic polynomial fitted on cleartext training data (Ao et al., 30 Aug 2025). The same work reports up to $7\times$ faster inference than state-of-the-art FHE neural networks adapted to face recognition, up to $+8.8\%$ absolute improvement in $x_i$0 verification accuracy, and effective scaling from $x_i$1 to $x_i$2 images with almost constant runtime (Ao et al., 30 Aug 2025).

The principal limitation is cost. Even after architectural depth reduction and a single bootstrapping operation per patch CNN, the reported online verification latency remains on the order of tens of minutes on a large AWS instance, with RAM usage in the hundreds of gigabytes (Ao et al., 30 Aug 2025). This places CryptoFace closer to a feasibility result for maximal confidentiality than to commodity real-time deployment.

3. Encrypted template matching and scalable identification

A second line of work secures facial templates and outsourced matching rather than the full feature extractor. CipherFace encrypts embeddings under FHE, lets the cloud compute encrypted Euclidean or cosine distances, and keeps the secret key on-premises so that only the client decrypts final distances and applies the decision threshold (Serengil et al., 22 Feb 2025). The system uses TenSEAL and CKKS, supports FaceNet128d, FaceNet512d, and VGG-Face embeddings, and reports that in its smaller configuration the homomorphic overhead is on the order of tens of milliseconds per distance computation (Serengil et al., 22 Feb 2025).

CryptoMask combines homomorphic encryption and secure multi-party computation for large-gallery one-to-many search. Its central design choice is a polynomial encoding that places batched inner products at fixed monomial degrees, eliminating expensive homomorphic rotations. It further hides intermediate distance information: the verifier learns only whether there exists a match, whereas existing approaches additionally leak sensitive intermediate distance information (Bai et al., 2023). For a database with 100 million 512-dimensional face vectors, CryptoMask reports approximately $x_i$3 and approximately $x_i$4 speed-ups in terms of computation and communication, respectively (Bai et al., 2023).

IDFace targets the same problem at the level of secure large-scale identification, but introduces two explicit co-design mechanisms for HE efficiency: a template representation transformation and a space-efficient encoding (Kim et al., 16 Jul 2025). The transformation $x_i$5 maps a unit face embedding to a sparse ternary vector in $x_i$6 with exactly $x_i$7 non-zero entries, so that inner products can be reduced to additions and subtractions rather than general multiplications. The encoding then packs many transformed templates into the available HE plaintext space. The reported result is identification from among a database of 1M encrypted templates in 126ms, showing only $x_i$8 overhead compared to identification over plaintexts (Kim et al., 16 Jul 2025).

Taken together, these systems suggest a recurring design principle: efficient encrypted face recognition does not arise from a literal homomorphic implementation of a standard recognizer. It depends on reparameterizing the representation, the distance computation, or both, so that the encrypted computation aligns with the algebraic strengths of BFV, CKKS, or additive HE (Serengil et al., 22 Feb 2025, Bai et al., 2023, Kim et al., 16 Jul 2025).

4. Adversarial and obfuscatory face encryption

A distinct meaning of CryptoFace appears in work on public-image sharing, where “encryption” refers not to ciphertext confidentiality but to corruption of the biometric signal while preserving visual plausibility. "Towards Face Encryption by Generating Adversarial Identity Masks" proposes the targeted identity-protection iterative method (TIP-IM), which overlays an adversarial identity mask $x_i$9 on a face image $f_{\omega_i}$0,

$f_{\omega_i}$1

so that the protected image remains visually near-identical for humans but is shifted toward an authorized or virtual target identity in feature space (Yang et al., 2020). The paper reports $f_{\omega_i}$2 protection success rate against various state-of-the-art face recognition models under practical test scenarios and shows applicability on a commercial API service (Yang et al., 2020).

FRIA, introduced in "Face Encryption via Frequency-Restricted Identity-Agnostic Attacks," addresses two practical weaknesses of earlier adversarial protections: black-box transferability and visual naturalness (Dong et al., 2023). Its identity-agnostic mechanism uses an average feature computed from a crawled dataset as the target, motivated by the observation that representations of the average feature in multiple face recognition models are similar. Its frequency-restricted mechanism uses discrete cosine transform to restrict perturbation in the low-frequency facial regions. The reported outcome is black-box attack success rates of 96% while generating more natural encrypted faces, with additional validation on a real-world black-box commercial API (Dong et al., 2023).

Forbes occupies a related but inverted position. It aims to obfuscate facial appearance recognizable by humans but preserve the identity and attributes decipherable by machines (Kim et al., 2024). The method applies multiple obfuscating transformations, refines their parameters by backpropagation, and optimizes a composite energy including feature-space distance and cosine dissimilarity. It is explicit that Forbes does not use encryption in the cryptographic sense. In evaluation, Forbes-T reports an average correctness rate of 0.2422 in a human study, close to the 0.2 random-choice baseline, while retaining 98.72% face-verification accuracy on LFW and 90.89% on IJB-C (Kim et al., 2024).

These methods share a common operational premise: facial images remain viewable and distributable, but the recognizability of the biometric representation is deliberately distorted. Their security is therefore empirical and model-relative, depending on transferability, perceptual constraints, and the robustness of the target recognizer rather than on semantic security or formal cryptographic hardness (Dong et al., 2023, Yang et al., 2020, Kim et al., 2024).

5. Biometric key generation from face

WiFaKey represents another branch of CryptoFace: face-based biometric cryptosystems that generate or recover cryptographic secrets from noisy facial measurements (Dong et al., 2024). The system is described as a fuzzy commitment–style key binding method specialized for deep face embeddings in unconstrained conditions. A $f_{\omega_i}$3-dimensional embedding $f_{\omega_i}$4 is transformed by AdaMTrans into a binary vector $f_{\omega_i}$5, a random key $f_{\omega_i}$6 is LDPC-encoded to a codeword $f_{\omega_i}$7, and the stored helper data is

$f_{\omega_i}$8

Authentication reconstructs $f_{\omega_i}$9, forms $y = y' A^\top + b.$0, and uses the Neural-MS decoder to recover $y = y' A^\top + b.$1 (Dong et al., 2024).

The two named technical contributions are AdaMTrans and Neural-MS. AdaMTrans combines equal-probability quantization, LSSC binarization, and adaptive random masking to reduce the intra-class bit error rate to the regime that LDPC decoding can correct. Neural-MS is a supervised learning-based neural decoding scheme for 5G NR LDPC codes, positioned between classical min-sum and sum-product decoding (Dong et al., 2024). For $y = y' A^\top + b.$2, the construction yields a 300-bit key.

On the LFW dataset, WiFaKey achieved an average Genuine Match Rate of 85.45% and 85.20% at a 0% False Match Rate for MagFace and AdaFace features, respectively (Dong et al., 2024). The same work reports unlinkability with $y = y' A^\top + b.$3 for all datasets with MagFace and gives conservative system-security estimates in the 80–150-bit range across datasets, including 87 bits for LFW with MagFace and 128 bits for LFW with AdaFace (Dong et al., 2024). In this setting, CryptoFace is not a recognizer operating on encrypted data, but a revocable biometric key infrastructure built on facial embeddings.

6. Security properties, distinctions, and open problems

The literature uses the language of encryption for technically different security objectives. In CryptoFace, CipherFace, CryptoMask, and IDFace, the term refers to actual cryptographic protection: RLWE-based CKKS or BFV, Paillier-style additive HE, encrypted distances, key separation, and sometimes secure comparison protocols (Ao et al., 30 Aug 2025, Bai et al., 2023, Kim et al., 16 Jul 2025). In TIP-IM, FRIA, and Forbes, “face encryption” refers instead to adversarial or perceptual transformation of human-viewable images, typically without a secret key and without formal cryptographic security (Dong et al., 2023, Yang et al., 2020, Kim et al., 2024).

This distinction determines what is protected. End-to-end FHE systems protect raw images, features, scores, and thresholding. Template-protection systems protect embeddings at rest and during matching, but often leave feature extraction outside the encrypted domain. Adversarial systems protect publicly shared images against unauthorized recognizers while preserving social-media usability. Biometric cryptosystems protect helper data and recoverable keys rather than recognition scores (Ao et al., 30 Aug 2025, Serengil et al., 22 Feb 2025, Dong et al., 2024).

Each family also carries characteristic assumptions. CryptoMask assumes honest-but-curious, non-colluding parties and a fully trusted key generator (Bai et al., 2023). IDFace assumes separation between a local server holding the encrypted database and a key server holding the secret key (Kim et al., 16 Jul 2025). CipherFace leaves the representation model in plaintext on-premises and does not address fully encrypted deep learning inference (Serengil et al., 22 Feb 2025). WiFaKey does not provide explicit protection against presentation attacks and does not claim a full composable cryptographic proof (Dong et al., 2024). Adversarial systems face a different limitation: model adaptation, adversarial training, compression, or dedicated detectors can degrade protection, and no formal irreversibility guarantee is implied by high transfer success (Dong et al., 2023, Yang et al., 2020).

A plausible synthesis is that CryptoFace is best understood not as a single algorithmic family but as a layered design space for face security. One layer encrypts computation, another protects templates, another disrupts biometric inference from public imagery, and another binds or derives cryptographic keys from facial measurements. The field’s technical trajectory has therefore moved from secure matching on embeddings, to end-to-end encrypted recognition, to scalable million-template identification, while in parallel developing adversarial and biometric-cryptosystem notions of “face encryption” that address different threat surfaces (Ao et al., 30 Aug 2025, Kim et al., 16 Jul 2025, Dong et al., 2024).