FaceAnonyMixer: Cancelable Face Biometrics

• FaceAnonyMixer is a cancelable face generation framework that uses key-driven latent mixing to anonymize facial identities while preserving discrimination for matching.
• It operates in the latent space of StyleGAN2, replacing identity-specific layers and refining them through per-instance gradient optimization to ensure consistency and irreversibility.
• Empirical evaluation on benchmarks shows enhanced recognition utility, privacy protection, and visual quality compared to existing anonymization methods.

Searching arXiv for the cited papers and closely related work to ground the article. FaceAnonyMixer is a cancelable face generation framework that leverages the latent space of a pre-trained generative model to synthesize privacy-preserving face images from a real face image and a revocable key. Its stated objective is not only face anonymization in the ordinary sense, but compliance with the canonical requirements of cancelable biometrics—revocability, unlinkability, irreversibility, and performance preservation—while remaining directly compatible with existing face recognition systems (Alam et al., 7 Aug 2025). The name has also been used conceptually for the diffusion architecture in “Face Anonymization Made Simple,” where a UNet denoiser and two ReferenceNet branches mix source identity and driving attributes; in the strict sense, however, FaceAnonyMixer denotes the 2025 latent-space cancelable-biometric formulation rather than that earlier diffusion-based anonymizer (Kung et al., 2024).

1. Concept and biometric scope

FaceAnonyMixer is positioned within image-space cancelable face biometrics rather than ordinary de-identification. In this formulation, a protected image $x_p = G(x_r,k)$ must hide the original identity in a released image while preserving sufficient discriminative structure for matching between protected samples of the same person under the same key. This differs from blur, pixelation, and one-way generative anonymization, which may obscure identity but do not inherently satisfy revocability, unlinkability, or irreversibility (Alam et al., 7 Aug 2025).

The framework adopts the ISO/IEC 24745-style operational properties explicitly listed for cancelable biometrics. Let $x_r$ be a real face image, $k$ a revocable key, $x_p = G(x_r,k)$ a protected image, and $F(x_1,x_2)\in\{0,1\}$ a matcher. Then anonymity requires that $P(F(x_r,x_p)=0 \mid x_p = G(x_r,k))$ be high. Identity preservation requires that protected images of the same real identity generated with the same key should match with low false non-match rate, while protected images of different real identities should not match. Unlinkability requires that protected images of the same real face generated under different keys should not match. Irreversibility requires that even if an attacker forms $\hat{x}_r = G^{-1}(G(x_r,k),k)$, the probability that $F(x_r,\hat{x}_r)=0$ remains high (Alam et al., 7 Aug 2025).

This formulation makes FaceAnonyMixer simultaneously an anonymization method and a biometric template protection method. A plausible implication is that its design criteria are stricter than those of systems whose sole objective is low re-identification rate.

2. Latent-space architecture and mixing mechanism

FaceAnonyMixer operates in the $\mathcal{W}^+$ latent space of a pre-trained StyleGAN2 generator. A real image $x_r$ is first inverted by e4e into a latent code $x_r$0. A revocable key $x_r$1 is then used only as a random seed to sample a synthetic latent code $x_r$2 from $x_r$3. The generator $x_r$4 is fixed, as are the auxiliary encoders: ArcFace for identity embeddings and FaRL for identity-agnostic facial attributes (Alam et al., 7 Aug 2025).

The latent is partitioned into semantically differentiated StyleGAN2 layers. Layers $x_r$5 are treated as coarse global structure, layers $x_r$6 as identity-related attributes, and layers $x_r$7 as fine details such as hair, background, and texture. The naïve mixing operator therefore replaces only the identity-related block: $x_r$8 Decoding $x_r$9 yields an initial protected image $k$0 (Alam et al., 7 Aug 2025).

This initial substitution is not regarded as sufficient. The paper states that naïve mixing gives good anonymity but poor identity consistency, because different images of the same person under the same key may not converge to the same pseudo-identity. FaceAnonyMixer therefore refines the mixed latent by per-instance gradient optimization. Augmented views $k$1 of the same subject are inverted, mixed with the same $k$2, and jointly optimized so that the resulting protected images share a consistent protected identity while preserving non-identity attributes (Alam et al., 7 Aug 2025).

Component	Role
StyleGAN2 generator $k$3	Decodes protected latent codes to face images
e4e encoder $k$4	Inverts real images into $k$5
ArcFace encoder $k$6	Supplies identity embeddings for anonymity and consistency losses
FaRL encoder $k$7	Supplies identity-agnostic attribute representation
Key sampler $k$8	Uses the revocable key as RNG seed to sample $k$9

The decisive architectural idea is that revocability is implemented by resampling the synthetic latent block rather than retraining the model. Changing the key changes $x_p = G(x_r,k)$0, which changes the identity-related latent layers, which changes the protected face.

3. Multi-objective latent optimization

After naïve mixing, FaceAnonyMixer optimizes the latent codes with three losses. The anonymity loss explicitly separates the protected identity from the original identity in ArcFace space: $x_p = G(x_r,k)$1 with margin $x_p = G(x_r,k)$2, so the protected and original embeddings are encouraged to have non-positive cosine similarity (Alam et al., 7 Aug 2025).

Identity consistency is enforced across protected augmentations of the same subject under the same key: $x_p = G(x_r,k)$3 This term is central to the framework’s claim of performance preservation. It does not tie a protected face to the real identity; rather, it ties different protected views to one another, thereby inducing a stable pseudo-identity under a fixed key (Alam et al., 7 Aug 2025).

Non-identity facial characteristics are preserved through an attribute loss computed with FaRL: $x_p = G(x_r,k)$4 The total objective is

$x_p = G(x_r,k)$5

with $x_p = G(x_r,k)$6, $x_p = G(x_r,k)$7, and $x_p = G(x_r,k)$8 (Alam et al., 7 Aug 2025).

The loss design is asymmetrical in a deliberate way. Anonymity and identity consistency receive heavy weighting, whereas attribute preservation is weaker. This suggests that the framework treats attribute retention as constrained by, but subordinate to, privacy and protected-domain matchability. The paper’s ablations support this interpretation: without the identity-preserving loss, protected images of the same subject under the same key become inconsistent; without the attribute loss, pose and expression are not preserved; without the anonymity loss, residual identity remains (Alam et al., 7 Aug 2025).

4. Evaluation protocols and empirical performance

FaceAnonyMixer is evaluated on CelebA-HQ and VGGFace2, with additional qualitative results on IJB-C. Privacy is measured primarily by Protection Success Rate (PSR) under multiple face recognition models and false match rate operating points; recognition utility is measured by EER and AUC on matching protected templates; unlinkability is measured both by PSR across keys and with the Gomez-Barrero unlinkability framework; visual quality is measured by FID; commercial compatibility is evaluated with Face++ confidence scores (Alam et al., 7 Aug 2025).

Recognition performance on protected images is reported as stronger than both the original images and CanFG in the cited protocol. On CelebA-HQ, the original images yield EER $x_p = G(x_r,k)$9 and AUC $F(x_1,x_2)\in\{0,1\}$0; CanFG yields EER $F(x_1,x_2)\in\{0,1\}$1 and AUC $F(x_1,x_2)\in\{0,1\}$2; FaceAnonyMixer yields EER $F(x_1,x_2)\in\{0,1\}$3 and AUC $F(x_1,x_2)\in\{0,1\}$4. On VGGFace2, the original images yield EER $F(x_1,x_2)\in\{0,1\}$5 and AUC $F(x_1,x_2)\in\{0,1\}$6; CanFG yields $F(x_1,x_2)\in\{0,1\}$7 and $F(x_1,x_2)\in\{0,1\}$8; FaceAnonyMixer yields $F(x_1,x_2)\in\{0,1\}$9 and $P(F(x_r,x_p)=0 \mid x_p = G(x_r,k))$0 (Alam et al., 7 Aug 2025).

Privacy and unlinkability are evaluated under several false match rate thresholds. For unlinkability, Table 2 reports that at $P(F(x_r,x_p)=0 \mid x_p = G(x_r,k))$1, average PSR across keys is $P(F(x_r,x_p)=0 \mid x_p = G(x_r,k))$2 for FaceAnonyMixer versus $P(F(x_r,x_p)=0 \mid x_p = G(x_r,k))$3 for CanFG; at $P(F(x_r,x_p)=0 \mid x_p = G(x_r,k))$4, the corresponding values are $P(F(x_r,x_p)=0 \mid x_p = G(x_r,k))$5 and $P(F(x_r,x_p)=0 \mid x_p = G(x_r,k))$6. In the Gomez-Barrero framework, the global unlinkability score $P(F(x_r,x_p)=0 \mid x_p = G(x_r,k))$7 on CelebA-HQ drops from $P(F(x_r,x_p)=0 \mid x_p = G(x_r,k))$8 for raw images to $P(F(x_r,x_p)=0 \mid x_p = G(x_r,k))$9 for protected images, with a similar pattern on VGGFace2 (Alam et al., 7 Aug 2025).

Commercial API evaluation is also emphasized. Face++ match confidence for protected versus original images averages $\hat{x}_r = G^{-1}(G(x_r,k),k)$0 for CanFG and $\hat{x}_r = G^{-1}(G(x_r,k),k)$1 for FaceAnonyMixer, with per-dataset values of $\hat{x}_r = G^{-1}(G(x_r,k),k)$2 versus $\hat{x}_r = G^{-1}(G(x_r,k),k)$3 on CelebA-HQ and $\hat{x}_r = G^{-1}(G(x_r,k),k)$4 versus $\hat{x}_r = G^{-1}(G(x_r,k),k)$5 on VGGFace2. The paper summarizes this as “over an 11% gain on commercial API compared to recent cancelable biometric methods” (Alam et al., 7 Aug 2025).

Visual quality is reported through FID. FaceAnonyMixer attains FID $\hat{x}_r = G^{-1}(G(x_r,k),k)$6, whereas CanFG attains $\hat{x}_r = G^{-1}(G(x_r,k),k)$7, which the paper uses to argue that the StyleGAN2 latent-mixing formulation yields sharper and more realistic protected faces (Alam et al., 7 Aug 2025).

Evaluation axis	FaceAnonyMixer result	Comparator noted in paper
CelebA-HQ recognition utility	EER $\hat{x}_r = G^{-1}(G(x_r,k),k)$8, AUC $\hat{x}_r = G^{-1}(G(x_r,k),k)$9	CanFG: $F(x_r,\hat{x}_r)=0$0, $F(x_r,\hat{x}_r)=0$1
VGGFace2 recognition utility	EER $F(x_r,\hat{x}_r)=0$2, AUC $F(x_r,\hat{x}_r)=0$3	CanFG: $F(x_r,\hat{x}_r)=0$4, $F(x_r,\hat{x}_r)=0$5
Unlinkability at $F(x_r,\hat{x}_r)=0$6	PSR $F(x_r,\hat{x}_r)=0$7	CanFG: $F(x_r,\hat{x}_r)=0$8
Visual quality	FID $F(x_r,\hat{x}_r)=0$9	CanFG: $\mathcal{W}^+$0

The t-SNE analyses reported in the paper show protected embeddings clustering tightly by identity while remaining separated across identities. This is consistent with the framework’s objective: not removal of all matchable structure, but conversion of the real identity into a stable, key-dependent protected identity (Alam et al., 7 Aug 2025).

5. Revocation, attack models, and irreversibility

Revocability in FaceAnonyMixer is implemented through the key mechanism rather than through any trainable key-conditioned module. The key $\mathcal{W}^+$1 seeds the random generation of $\mathcal{W}^+$2; a new key produces a new synthetic latent and therefore a new protected identity. Because the model itself does not change, revocation is computationally lightweight: compromised protected templates can be reissued simply by regenerating them under a new key (Alam et al., 7 Aug 2025).

The paper explicitly studies irreversibility under two attack classes. The first is a latent-replacement attack in which an attacker has the anonymized image and the key latent $\mathcal{W}^+$3, inverts the anonymized image back into StyleGAN latent space, replaces the mid-layer block, and decodes a supposed reconstruction. The paper states that these reconstructions do not resemble the original identity. The second, stronger attack trains a Pix2Pix-like mapper from anonymized to original images using many paired examples. Even under this stronger model, Table 4 reports PSR above approximately $\mathcal{W}^+$4 at $\mathcal{W}^+$5 for most models and datasets. On CelebA-HQ, for example, PSR at $\mathcal{W}^+$6 is $\mathcal{W}^+$7 for IRSE50, $\mathcal{W}^+$8 for IR152, $\mathcal{W}^+$9 for FaceNet, and $x_r$0 for MobileFace; corresponding VGGFace2 values remain similarly high (Alam et al., 7 Aug 2025).

Key robustness is also quantified. On IR152 at $x_r$1, five random keys give PSR values $x_r$2, $x_r$3, $x_r$4, $x_r$5, and $x_r$6, with mean $x_r$7 and standard deviation $x_r$8. This indicates that privacy performance does not depend on a narrow subset of favorable keys (Alam et al., 7 Aug 2025).

The framework’s limitations are also explicit. It depends on StyleGAN2 inversion quality, which may degrade under strongly unconstrained inputs. It uses per-image latent optimization of about $x_r$9 Adam steps, so it is not real-time in its reported form. The threat model includes strong reconstruction attacks, but the paper still treats further advanced attacks as an open problem. Future work is stated in terms of computational efficiency and extension to multi-biometric systems (Alam et al., 7 Aug 2025).

6. Relation to adjacent anonymization paradigms

FaceAnonyMixer belongs to a broader shift from direct pixel obfuscation toward structured identity manipulation. Several adjacent paradigms clarify its position.

A conceptually related diffusion-based design appears in “Face Anonymization Made Simple,” where a latent diffusion model based on Stable Diffusion v2.1 uses a UNet denoiser, two ReferenceNet branches, and CLIP image conditioning to mix a “source” identity stream with a “driving” attribute stream. In anonymization mode, the same image is used as both source and driving, and a scalar parameter $x_r$00 attenuates or inverts identity-bearing features while preserving pose, gaze, expression, background, clothing, and accessories. That system is not literally named FaceAnonyMixer, but the paper explicitly notes that, conceptually, it is exactly such a mixer (Kung et al., 2024).

A distinct keyed formulation is KFAAR, “A Key-Driven Framework for Identity-Preserving Face Anonymization,” which combines a head posture-preserving virtual face generation module with a key-controllable virtual face authentication module. KFAAR preserves head posture and facial expression, generates virtual faces using StyleGAN2 and FaceVid2Vid, and allows specialized recognition of the original identity only when the correct user key is provided (Wang et al., 2024). Unlike FaceAnonyMixer, it is not framed as cancelable biometrics in the $x_r$01 mixing sense; its emphasis is the coexistence of anonymity and authorized authentication.

Another keyed and reversible formulation is iFADIT, which uses disentanglement plus a secure flow-based transform to map an identity code into an anonymized version and then invert it with the matching secret. Its central transform is a conditional normalizing flow over identity embeddings rather than direct latent replacement in StyleGAN2 layers (Yuan et al., 8 Jan 2025). Reverse Personalization, by contrast, uses SDXL inversion and negative classifier-free guidance on an identity-conditioning branch to move the denoising trajectory away from the original identity while preserving background and high-level facial attributes (Kung et al., 28 Dec 2025). A further diffusion-based line, “Controllable Localized Face Anonymization Via Diffusion Inpainting,” uses Stable Diffusion inpainting plus an adaptive attribute-guidance module based on FaRL gradients, and supports localized anonymization through editable masks without any additional model training (Salar et al., 18 Sep 2025).

These related systems suggest that contemporary face anonymization research has split into at least four technical families: latent mixing in pre-trained GAN space, keyed virtual-face generation plus specialized authentication, invertible identity transformation, and diffusion-based inversion or inpainting. FaceAnonyMixer is distinctive because it places cancelable biometric requirements at the center of the design and implements revocation through key-derived latent identity replacement rather than through explicit encryption, per-key retraining, or prompt-level guidance (Alam et al., 7 Aug 2025).