VOIDFace: Privacy-Preserving Face Recognition

• VOIDFace is a face recognition training framework that leverages patch-centric visual secret sharing to provide privacy, enhanced data control, and enforceable RTBF.
• It segments facial images into patches distributed across institutions, ensuring that no single party can reconstruct complete faces without an authentication share.
• Its distributed multi-network architecture fuses localized patch features into a global embedding, maintaining competitive recognition performance and robust security against attacks.

VOIDFace is a privacy-preserving face recognition training framework that replaces replicated full-face datasets with a Visual Secret Sharing (VSS)-based, patch-centric storage and training pipeline. It is designed to eliminate data replication, strengthen data control, enhance security against model inversion, and operationalize the Right-To-Be-Forgotten (RTBF) while maintaining competitive facial recognition performance. The framework combines a multiple-image visual secret sharing formulation tailored to patch-based facial data with a distributed multi-network architecture in which dedicated patch models are fused into a global embedding; experiments are reported on a filtered VGGFace2 setting and evaluated on LFW, CALFW, and AgeDB-30 (Muhammed et al., 11 Aug 2025).

1. Problem setting and design rationale

VOIDFace is motivated by three coupled risks in contemporary face recognition training. First, raw face images are often replicated across multiple institutions or workstations to train separate models, which enlarges the attack surface and complicates database management and oversight. Second, once a subject contributes a face image for dataset preparation, that subject typically loses practical control over how the data is subsequently stored and reused. Third, model inversion attacks can recover training images or closely resembling faces from model outputs, which creates a direct conflict between recognition utility and privacy protection (Muhammed et al., 11 Aug 2025).

The framework addresses these issues by never storing complete face images post-preprocessing. Instead, each front-facing image is segmented into local patches, and each patch is converted into visual shares that are distributed across institutions. Authorized training reconstructs only per-patch data at geographically separated nodes using an authenticated share retained by a trusted third party. When consent is withdrawn, revocation is immediate through deletion of the authentication share, which renders all distributed private shares useless. This suggests that VOIDFace treats revocation as a property of storage design rather than as an after-the-fact dataset maintenance task.

The architectural logic is correspondingly dual. At the storage layer, VSS is used to ensure that neither the trusted third party nor participating institutions alone possess reconstructible facial data. At the model layer, a patch-based multi-network system trains on localized regions rather than on complete faces, and an Aggregator fuses local features into a global representation. The paper presents this combination as a joint response to privacy, security, and data control rather than as a purely cryptographic or purely recognition-oriented modification.

2. Visual secret sharing data model

VOIDFace treats each user’s front-facing image as $N_p$ patch-secrets $P_1, \ldots, P_{N_p}$, with six patches in the reported experiments: both eyebrows, both eyes, nose, and mouth. A trusted third party detects and segments front faces using HOG-based detection with landmarking via dlib, extracts the patches, resizes them to $96 \times 96 \times 3$, and deletes the full face immediately. The full face is therefore never reconstructed or retained after preprocessing (Muhammed et al., 11 Aug 2025).

The sharing rule is a two-share per secret reconstruction scheme. A random authentication share $AS$ of the same size as each patch is generated, and for each patch $i$ the corresponding private share is defined by

$PS_i = P_i \oplus AS.$

Patch reconstruction requires both $AS$ and $PS_i$:

$P_i = AS \oplus PS_i.$

Both shares are random-looking grids, and the paper states that either share alone leaks no information, yielding perfect secrecy akin to one-time pad over images. The minimally refined perfect access structure is $\Gamma_{N_p} = \{(Q^i, F^i)\}_{i=1}^{N_p}$ with minimal qualified set $P_1, \ldots, P_{N_p}$0 and forbidden sets $P_1, \ldots, P_{N_p}$1 (Muhammed et al., 11 Aug 2025).

Scalability is handled by XOR-splitting a private share when the number of storage institutions exceeds the available private shares. If $P_1, \ldots, P_{N_p}$2, then each subshare alone leaks nothing, and reconstruction of the patch requires the XOR of all corresponding subshares together with $P_1, \ldots, P_{N_p}$3. This allows on-demand expansion to additional storage endpoints without changing the basic access rule.

The security analysis is both formal and empirical. With uniformly random $P_1, \ldots, P_{N_p}$4 independent of $P_1, \ldots, P_{N_p}$5, the paper states that $P_1, \ldots, P_{N_p}$6 is uniformly distributed, so $P_1, \ldots, P_{N_p}$7 and $P_1, \ldots, P_{N_p}$8. For brute-force guessing, a single $P_1, \ldots, P_{N_p}$9 share has probability $96 \times 96 \times 3$0. Across 1,000 encryptions per patch, NPCR exceeds 98% for all patches: Left eyebrow 98.87%, Right eyebrow 98.88%, Left eye 98.69%, Right eye 98.75%, Nose 98.54%, Mouth 98.67%. Average adjacent-pixel correlation coefficients in shares are near zero, with the left eyebrow example reported as Horizontal/Vertical/Diagonal $96 \times 96 \times 3$1; analogous near-zero values are reported for other patches (Muhammed et al., 11 Aug 2025).

Storage is also reduced. The authentication share is retained exclusively by the trusted third party, while institutions store only private shares or subshares. Share sizes are reported as $96 \times 96 \times 3$2 kB, in contrast to typical 50–200 kB per compressed face image. The paper characterizes this as a substantial reduction in storage footprint and replication overhead.

3. Patch-based recognition architecture

The recognition model consists of $96 \times 96 \times 3$3 parallel Patch Training Networks (PTNs), one per facial region, and an Aggregator that fuses their outputs into a global representation. Each PTN uses a MobileNet backbone with width multiplier $96 \times 96 \times 3$4 and produces a 512-dimensional embedding. The Aggregator is a fully connected layer that maps the concatenated patch features to a 512-dimensional global embedding (Muhammed et al., 11 Aug 2025).

Two training variants are defined. In V1, learning is single-task: only the Aggregator’s output is supervised for identity classification. In V2, learning is multitask: each PTN has an auxiliary classification head and is supervised with the same identity labels, which the paper describes as strengthening local discriminative learning. The reported results indicate that V2 typically outperforms V1.

The primary training objective is categorical cross-entropy,

$96 \times 96 \times 3$5

where $96 \times 96 \times 3$6 is the one-hot label and $96 \times 96 \times 3$7 is the softmax probability for class $96 \times 96 \times 3$8. Additive angular margin softmax (ArcFace) can also be applied to the classification head to improve inter-class separation, with $96 \times 96 \times 3$9 in the experiments:

$AS$0

The preprocessing and optimization pipeline is explicit. Front-facing images are detected and segmented using HOG-based frontal face detection and landmark extraction; images are normalized by subtracting $AS$1 and scaling by $AS$2. Training uses SGD with momentum $AS$3, batch size $AS$4, cosine annealing from $AS$5 to $AS$6, and 20 epochs. At inference, the six PTN embeddings are concatenated, fused by the Aggregator into a global 512-D embedding, and cosine similarity is used for verification (Muhammed et al., 11 Aug 2025).

The distributed training protocol is inspired by federated learning. Clients are selected resource-aware using FedCS, and volatile or slow clients are excluded via E3CS. After authorization, the trusted third party dispatches the authentication share to $AS$7 independent, non-communicating training nodes $AS$8. Each node reconstructs only its own patch,

$AS$9

or, if subshares are used, $i$0. Each node processes only its reconstructed patch through its PTN and sends features to the Aggregator, or aggregates locally depending on deployment. Computationally, training scales from a single network to $i$1 PTNs plus the Aggregator; inference latency is described as minimal relative to single-network embedding extraction if parallelized.

4. Right-To-Be-Forgotten and data governance

VOIDFace operationalizes RTBF through the authentication share. The $i$2 functions as a revocable capability token for a subject, and the trusted third party validates it before any training event. If a subject withdraws consent, the corresponding $i$3 is deleted immediately. Without $i$4, reconstruction of any patch via $i$5 is impossible, and all private shares become cryptographic noise (Muhammed et al., 11 Aug 2025).

This design changes how erasure propagates. Because VOIDFace never replicates raw images and because future authorized training is impossible without the authentication share, revocation does not require sweeping multiple replicated datasets across institutions. The paper further states that abandoned private shares are periodically identified and purged. It also distinguishes this mechanism from full post-hoc machine unlearning: VOIDFace enables immediate cessation of future contribution without full retraining, while selective forgetting of already learned representations could be layered on top but is not the paper’s focus.

In regulatory terms, the mechanism is framed as operationalizing GDPR Article 17 by making future training technically impossible upon revocation and by centralizing consent in the trusted third party. This suggests a governance model in which data control is encoded directly into the access structure of training data rather than delegated to downstream institutional compliance procedures.

The centrality of the trusted third party is therefore not incidental. It is the sole holder of the authentication share and the enforcement point for revocation. This yields clear control semantics, but it also creates an implementation dependency addressed explicitly in the paper’s limitations.

5. Empirical evaluation and security profile

The experimental evaluation uses VGGFace2 as the base dataset, filtered by a quality-driven frontal-face criterion with $i$6. After filtering, the training set contains 1,158K images across 8,628 identities, with an average of 134 images per identity. Evaluation benchmarks are LFW, CALFW, and AgeDB-30. Baselines are single-network Softmax and ArcFace models with a ResNet50 backbone, trained on the same filtered data for 20 epochs using SGD, a linearly decayed learning rate from 0.01 to $i$7, and ImageNet initialization (Muhammed et al., 11 Aug 2025).

The paper’s summary of recognition performance is qualitative rather than exhaustive in the provided data: VOIDFace is described as effective and robust across benchmarks, with V2 typically outperforming V1, and both outperforming Softmax and ArcFace on many protocols where frontal quality is high and pose is moderate. The result pattern is explicitly tied to the frontal-face assumption.

A more numerically specific comparison is reported for model inversion under a black-box, query-based attack, evaluated using face.evoLve trained on CelebA.

Model	Attack Acc	KNN Dist
ArcFace	82.4%	1247.28
VOIDFace	12.1%	2240.30

These figures are presented as confirmation that recovered samples deviate significantly from private data in VOIDFace (Muhammed et al., 11 Aug 2025).

Encryption robustness is also quantified. Beyond NPCR values above 98% for all patches, entropy is reported as near maximum, approximately 7.98 bits per channel, and adjacent-pixel correlation coefficients are near zero. Together with the formal mutual-information argument and the brute-force probability estimate, these observations support the paper’s claim of information-theoretic security in the one-time pad sense for images, with no key reuse across identities.

Operational overheads are reported in asymptotic and practical terms. Share generation is $i$8 per patch, where $i$9 is the patch size, implemented as XOR against the authentication share. Training cost rises roughly linearly with the number of PTNs and concatenated features. Communication overhead depends on FedCS-based client selection and E3CS-based handling of volatile clients. Inference requires fusion over $PS_i = P_i \oplus AS.$0 patch embeddings but is reported to incur minimal overhead if parallelized.

6. Limitations, adjacent literatures, and terminological scope

The paper identifies several limitations. The authentication share is centralized at a trusted third party, so practical deployments must harden this component, for example with HSMs or secure enclaves. The current system assumes frontal faces; extension to extreme pose and occlusion is identified as future work, potentially through more patches or pose normalization. Scaling to many PTNs and distributed nodes requires orchestration, and the authors note that federated learning, differential privacy, or secure multiparty computation could further strengthen guarantees. Efficient post-hoc machine unlearning for already-trained models is outside the present scope, and share lifecycle management across institutions remains operationally important (Muhammed et al., 11 Aug 2025).

Within adjacent research areas, VOIDFace occupies a specific niche. It is not the same as FaceEraser’s “VOIDFace,” where the term denotes a blank, featureless facial canvas for augmented reality produced by removing eyebrows, eyes, nose, and mouth and replacing them with coherent skin texture (Hua et al., 2021). Nor is it the same as the 2026 “VoidFace” defense against diffusion-based face swapping, which treats swapping as a coupled identity pathway and protects facial identity through cascading pathway disruption across localization, identity embedding, attention injection, and diffusion features (Wang et al., 21 Jan 2026).

There are also neighboring technical intersections without identity of purpose. “In-N-Out” addresses a reconstruction–editability trade-off in 3D-aware GAN inversion by decomposing the scene into in-distribution and out-of-distribution radiance fields; in the supplied synthesis, that decomposition is described as directly aligned with handling occlusions or out-of-distribution facial content in systems like VOIDFace (Xu et al., 2023). FaVoA, by contrast, augments active speaker detection with a face–voice association prior and a gated-bimodal-unit fusion mechanism, using a speech-derived facial representation to rule out non-matching face–voice pairs in ambiguous audiovisual scenes (Carneiro et al., 2021). These connections place VOIDFace among broader efforts to restructure facial processing around privacy, ambiguity, and controllable access, but its distinctive contribution is the integration of VSS-based storage, patch-wise distributed training, and enforceable RTBF in a single face recognition training framework.

Taken together, VOIDFace represents a storage-and-training redesign rather than a minor privacy add-on. Its core claim is that privacy, data control, and recognition utility can be co-engineered by replacing replicated face images with per-patch visual shares, restricting runtime reconstruction to authorized nodes, and tying continued usability of data to the existence of a revocable authentication share.