VoidFace: Dual Privacy Frameworks for Face Analysis

• VoidFace is a dual-framework that combines a cascading defense against diffusion-based face swapping with a privacy-preserving face recognition system.
• Its diffusion-based defense disrupts identity transfer through targeted perturbations and latent-manifold adversarial optimization with perceptual adaptation.
• The privacy-preserving recognition component employs visual secret sharing, patch-based multi-network training, and cryptographic RTBF protocols to safeguard user data.

VoidFace is a term used for two distinct but influential frameworks in the face analysis domain: (1) a cascading defense against diffusion-based face swapping for privacy protection (Wang et al., 21 Jan 2026), and (2) a privacy-preserving architecture for multi-network face recognition leveraging visual secret sharing and rights management (Muhammed et al., 11 Aug 2025). Both systems address emergent risks in ML-driven face research, and each enforces privacy and data control via mathematically grounded, information-theoretic or adversarial mechanisms.

1. VoidFace for Diffusion-Based Face Swapping Defense

VoidFace (Wang et al., 21 Jan 2026) is a systemic defense that disrupts the identity transfer pathway in state-of-the-art diffusion-based face swapping systems. It addresses the observed structural resilience of face swapping pipelines, which render prior adversarial and image editing-based attack defenses largely ineffective.

1.1 Problem Formulation and Threat Model

Diffusion-based face swapping models exhibit a strict three-stage pipeline:

• Detection (localization): A backbone detector $\Phi$ generates facial bounding boxes using classification $P_{face}\in[0,1]^J$ and regression $\Phi_{reg}(x)\in\mathbb{R}^{J\times4}$ offsets.
• Extraction (semantic encoding): An identity encoder $E$ (e.g., ArcFace) produces a face embedding $\mathcal{C}_{id}=E(x)$ from the aligned crop.
• Generation (conditional diffusion): A U-Net conditional diffusion model denoises the latent $\mathcal{Z}_t$, injected with identity via cross-attention layers:

$$Q^l = \ell_q^l(\mathcal{Z}_t),\quad K^l=\ell_k^l(\mathcal{C}_{id}),\quad V^l=\ell_v^l(\mathcal{C}_{id})$$

Each subsequent stage depends critically on its predecessor, forming a "coupled identity pathway." The defense surface comprises (1) facial bounding-box regression, (2) identity embedding, (3) cross-attention projections, and (4) intermediate generative representations.

1.2 Cascading Pathway Disruption Mechanism

VoidFace injects perturbations at four bottlenecks to induce cascading disruptions:

• Localization Disruption: Masks valid face anchors and manipulates regression outputs. The loss is:

$$\mathcal{L}_{loc} = \exp \left( -\left\| (\Phi_{reg}(x_{adv})-\Phi_{reg}(x_{src})) \odot \mathcal{M}_p \right\|_2 \right)$$

where $\mathcal{M}_p$ restricts the loss to detected faces.

• Identity Erasure: Forces adversarial embeddings toward a null anchor while repelling from the genuine source:

$$\mathcal{L}_{id} = D_{cos}(E(x_{adv}),E(x_{null})) + \max \left(0, m - D_{cos}(E(x_{adv}),E(x_{src})) \right)$$

$P_{face}\in[0,1]^J$0 is cosine distance, and $P_{face}\in[0,1]^J$1 is a margin.

• Attention Decoupling: Maximizes the $P_{face}\in[0,1]^J$2 shift between key/value projections of source and adversarial images in each cross-attention layer:

$P_{face}\in[0,1]^J$3

• Feature Corruption: Adds spatially selective corruption at feature layers $P_{face}\in[0,1]^J$4, focusing on semantically and identity-sensitive regions (from face parsing and Layer-CAM):

$P_{face}\in[0,1]^J$5

Total loss combines the above terms with signed weights: $P_{face}\in[0,1]^J$6

1.3 Latent-Manifold Adversarial Optimization

VoidFace performs adversarial search in the VAE latent $P_{face}\in[0,1]^J$7, rather than pixel space, using Latent-PGD: $P_{face}\in[0,1]^J$8 subject to $P_{face}\in[0,1]^J$9, with $\Phi_{reg}(x)\in\mathbb{R}^{J\times4}$0.

A perceptual adaptive strategy modulates updates via an LPIPS-based mask: spatial masks select less perceptually sensitive regions for perturbation, improving resultant image quality.

1.4 Empirical Evaluation

VoidFace demonstrates strong defense over extensive experiments:

• Victim models: DiffFace, DiffSwap, Face-Adapter, InstantID; transfer to GAN-based SimSwap, InfoSwap.
• Datasets: CelebA-HQ, VGGFace2-HQ.
• Metrics:
  • Attack efficacy: $\Phi_{reg}(x)\in\mathbb{R}^{J\times4}$1 distortion (higher is better), Identity Score Matching (ISM, lower is better), PSNR of swapped outputs.
  • Adversarial image quality: LPIPS, PSNR, FID.

Key performance (DiffFace, CelebA-HQ):

Method	ISM ↓	PSNR (swapped) ↓	LPIPS (adv) ↓	FID ↓
VoidFace	0.3256	27.46 dB	0.1628	32.54
FaceShield	0.3385	~29.1 dB	0.2069	34.55

Swapped outputs from VoidFace-protected faces show severe artifacts or incorrect identities, indicating strong defense. VoidFace retains robustness under JPEG, resizing, and bit-depth reduction and maintains efficacy with GAN-based steganographic swappers.

1.5 Discussion and Limitations

VoidFace uniquely leverages sequential, systemic disruption across the physical, semantic, and generative stages. Its latent-manifold optimization with perceptual adaptation delivers high utility-privacy tradeoff. However, implementation requires white-box access and incurs optimization overhead (~30 PGD steps per image), making extension to black-box or large-scale settings nontrivial. Extreme image transformations, such as heavy occlusion, may bypass perceptual feedback mechanisms (Wang et al., 21 Jan 2026).

2. VOIDFace for Privacy-Preserving Face Recognition

VOIDFace (Muhammed et al., 11 Aug 2025) is a privacy and security-enhanced face recognition training framework. It integrates per-patch visual secret sharing (VSS), distributed storage, and user-controllable rights management for data minimization and strong privacy guarantees.

2.1 Visual Secret Sharing-Based Data Storage

Face images are split by landmark detection into $\Phi_{reg}(x)\in\mathbb{R}^{J\times4}$2 patches (typically left/right eye, left/right eyebrow, nose, mouth), each patch $\Phi_{reg}(x)\in\mathbb{R}^{J\times4}$3. The original image $\Phi_{reg}(x)\in\mathbb{R}^{J\times4}$4 is securely deleted post-extraction.

Each patch is split via $\Phi_{reg}(x)\in\mathbb{R}^{J\times4}$5 minimally refined "perfect" VSS: one randomly generated authentication share ($\Phi_{reg}(x)\in\mathbb{R}^{J\times4}$6) is combined with each patch via XOR to yield a set of private shares ($\Phi_{reg}(x)\in\mathbb{R}^{J\times4}$7):

$\Phi_{reg}(x)\in\mathbb{R}^{J\times4}$8

Each $\Phi_{reg}(x)\in\mathbb{R}^{J\times4}$9 is stored at a separate node, and $E$0 is retained by a trusted third party (TTP). A single share reveals zero information (perfect secrecy), and recovery ($E$1) requires both an authorized node and the TTP.

2.2 Patch-Based Multi-Network Training Architecture

Data is reconstructed in patch form and fed into independent Patch Training Networks (PTN$E$2; MobileNet backbone, 512-d feature). Embeddings are concatenated and aggregated (via a fully connected layer) to a final embedding $E$3. Loss variants include:

• V1: Supervise only on the Aggregator output ($E$4).
• V2: Additional patch-level supervision with cross-entropy loss and optional ArcFace margin per PTN head:

$E$5

Resource-aware federated selection (FedCS, E3CS) chooses non-colluding training participants. Training uses SGD with momentum, cosine annealing, and 20 epochs.

2.3 Right-To-Be-Forgotten (RTBF) Protocol

VoidFace provides user-level, cryptographically enforced RTBF:

1. On registration, TTP stores $E$6 keyed to user.
2. For training, TTP authenticates and releases $E$7 as required.
3. On RTBF invocation, $E$8 is deleted by TTP.
4. Without $E$9, no $\mathcal{C}_{id}=E(x)$0 can be reconstructed for training, ensuring information-theoretic forgetfulness.
5. Orphaned private shares ($\mathcal{C}_{id}=E(x)$1) are garbage collected subsequently.

This protocol is mathematically proven to prevent patch recovery by any coalition lacking $\mathcal{C}_{id}=E(x)$2.

2.4 Security and Privacy Analysis

Security is established for brute-force, statistical, model-inversion (MI), and distributed storage adversaries.

• Brute-force resistance: The probability to guess a full patch by random pixel assignment is negligible:

$\mathcal{C}_{id}=E(x)$3

• Statistical resistance: NPCR (non-overlapping pixel change ratio) over 1,000 encrypted samples remains above 98.5% for all patches. Adjacent-pixel correlation coefficients approach zero.
• Model inversion: With a black-box attack (Nguyen et al.), VoidFace shows 12.1% attack accuracy vs. 82.4% for ArcFace; KNN distance is 2240.30 vs. 1247.28.
• Distributed adversaries: Compromise requires simultaneous access to both $\mathcal{C}_{id}=E(x)$4 and at least one $\mathcal{C}_{id}=E(x)$5 for each patch.

2.5 Empirical Performance and Resource Use

Training and test pipelines employ VGGFace2 (filtered to 1.158M images/8,628 classes). Benchmarks on LFW, CALFW, and AgeDB-30 indicate:

Method	LFW	CALFW	AgeDB-30
Softmax	99.20%	95.30%	94.75%
ArcFace	99.65%	97.10%	96.84%
VOIDFace V1	99.72%	97.45%	97.12%
VOIDFace V2	99.79%	97.92%	97.68%

Storage per share is ≤10 KB (vs 50–200 KB for original images), yielding a ~5× reduction. Training duration increases by ≤10%, attributed to multi-PTN computation, but is parallelizable.

3. Comparative Interpretation and Implications

The two VoidFace systems address distinct classes of privacy threats in face analysis:

• (Wang et al., 21 Jan 2026) targets downstream misuse (face swapping attacks) via proactive, systemic adversarial defense, leveraging the intrinsic stagewise dependence of modern diffusion pipelines.
• (Muhammed et al., 11 Aug 2025) aims at upstream data control during face recognition training, implementing cryptographic secret sharing, distributed processing, and enforceable RTBF.

A plausible implication is that the "VoidFace" paradigm signals a shift toward both data-centric and process-centric defenses for biometric privacy, where adversarial and cryptographic tools are integrated according to the threat surface and operational context.

4. Limitations and Directions for Future Research

(Wang et al., 21 Jan 2026) identifies the need for extending VoidFace to black-box settings, efficient one-shot perturbations for large datasets, and robustness under extreme image modifications. (Muhammed et al., 11 Aug 2025) relies on trusted third party infrastructure and does not explicitly address malicious training nodes or federated learning leakage, suggesting open problems in eliminating central points of failure and further tightening privacy guarantees in collaborative ML settings.

5. Implementation and Reproducibility

Implementation details for both systems are comprehensive and reproducible:

• VoidFace (Face Swapping): Requires white-box access to the target pipeline (detectors, encoders, diffusion U-Net). Losses are injected at the four pathway stages; optimization is in latent space with perceptual modulation.
• VOIDFace (Face Recognition): Uses MobileNet PTNs, DNN aggregator, and VSS-based storage. Full PyTorch code, data splits, and results are available at https://github.com/ajnasmuhammed89/VOIDFace (Muhammed et al., 11 Aug 2025).

6. Bibliographic References and Code Availability

• "Safeguarding Facial Identity against Diffusion-based Face Swapping via Cascading Pathway Disruption" (Wang et al., 21 Jan 2026).
• "VOIDFace: A Privacy-Preserving Multi-Network Face Recognition With Enhanced Security" (Muhammed et al., 11 Aug 2025), including implementation code at https://github.com/ajnasmuhammed89/VOIDFace.