PerFace Framework: Dual Face Analysis Methods

• PerFace framework is a dual-component approach, combining a perception-aligned face similarity metric for anonymization with parallel MLP architectures for scalable recognition.
• It leverages a triplet loss on human-annotated SimCelebA data to achieve 91.7% triplet-accuracy, improving the balance between anonymity and naturalness.
• Its parallel MLP (OCON) design enables rapid training, fault tolerance, and near-linear scalability in real-time, multiclass face recognition systems.

The PerFace framework encompasses two distinct but influential strands of face analysis research: (1) perceptual face similarity metric learning for face anonymization (Kumagai et al., 24 Sep 2025) and (2) a parallel multilayer perceptron (MLP)–based architecture for scalable, efficient face recognition (Bhowmik et al., 2010). Both are motivated by limitations of conventional, monolithic identity-classification approaches, but they address fundamentally different problems—capturing fine-grained perceptual similarity and achieving scalable, real-time multiclass face classification, respectively—through specialized model design and training protocols.

1. Motivation and Conceptual Foundations

The perceptual-similarity PerFace framework was motivated by the observation that state-of-the-art face-swapping approaches for anonymization are limited by reliance on binary “same/different-identity” recognition models, such as ArcFace, which do not reflect the gradations of similarity perceived by human observers. In anonymization pipelines, it is essential to select swap candidates that optimize the tradeoff between anonymity (visual distinctness) and naturalness (plausibility of facial attributes). PerFace introduces a continuous, perception-aligned face similarity metric to bridge this gap (Kumagai et al., 24 Sep 2025).

The computational PerFace framework addresses slow training convergence of standard MLPs for face recognition as the number of classes or samples increases. Real-time applications such as biometric authentication or surveillance require rapid retraining and operational scalability. The key idea is to decompose the multiclass learning task in a way that allows embarrassingly parallel training jobs, thereby improving throughput and system flexibility (Bhowmik et al., 2010).

2. Dataset Construction and Annotation Protocols

For perceptual similarity learning, PerFace introduces SimCelebA, a dataset of 6,400 human-annotated triplet comparisons. 80 target faces (balanced by gender) were selected from CelebAMask-HQ, from which three face-swapped variants per target were generated using SimSwap, giving 240 triplets per annotator round and totalling 6,400 samples across subsets. Each triplet consists of a reference face (anchor) and two swap images; annotators judged which swap more closely resembles the original target. Rigorous annotation quality controls were implemented: every triplet was assessed by at least three annotators and dummy trials with identical images filtered inattention. Only triplets with full annotator agreement were retained for the high-quality set (designated D2), which was split into training, validation, and test folds (Kumagai et al., 24 Sep 2025).

The computational PerFace framework used standard recognition datasets, e.g., Yale Face Database B, selecting 20 images/class for training and 20 for testing. Images were grayscale preprocessed and subjected to PCA (“eigenface” projection) onto the top 40 eigenvectors, yielding D=40–dimensional features (Bhowmik et al., 2010).

3. Model Architectures and Training Strategies

3.1. Perceptual Similarity Metric Learning

The PerFace model builds on a ResNet50 backbone pretrained with ArcFace on MS1MV3. The ArcFace classification head is removed and a 512-dimensional unit-norm embedding is used:

$x = f(I) \in \mathbb{R}^{512},\quad \|x\|_2 = 1.$

Pairwise similarity is computed using the cosine function:

$\cos(x, y) = x^\top y,$

yielding a value in $[-1, 1]$ that directly reflects perceptual similarity. Model weights are fine-tuned using a relative triplet loss:

$$\mathcal{L} =\sum_{i}\max\left(0,\;[\cos(x_i,x_i^-)-\cos(x_i,x_i^+)] + m\right),$$

where $(x_i, x_i^+, x_i^-)$ are the anchor and its more- and less-similar faces, and $m=0.1$ is a fixed margin. SGD with momentum (0.9), weight decay ($5\times10^{-4}$), and a fixed learning rate of 0.01 are used; batch size is 32, sampled exclusively from D2. Exhaustive triplet enumeration obviates the need for mining. No auxiliary losses are used (Kumagai et al., 24 Sep 2025).

3.2. Parallel MLP-Based Face Classification

The parallel PerFace framework contrasts two multiclass MLP topologies:

• All-Class-in-One-Network (ACON): A single MLP with $C$ outputs (one per class/identity), trained sequentially.
• One-Class-in-One-Network (OCON): $C$ separate, class-specific MLPs, each trained on a binary “class-$i$ vs. rest” task. Each network has a small hidden layer and two outputs.

OCON enables up to $K$ MLPs to be trained in parallel (where $K$ is the number of classes); jobs are distributed to worker nodes on a cluster via scripts or remote shell, without special middleware (Bhowmik et al., 2010).

Both employ standard feed-forward back-propagation. For training example $t$, the error is:

$$E(t) = \frac{1}{2}\sum_{k\in\text{outputs}} (y_k(t) - o_k(t))^2$$

Weights are updated by gradient descent with momentum:

$$\Delta w_{ij}^{(\ell)}(t) = \eta\,\delta_j(t)\,x_i^{(\ell-1)}(t) + \alpha\,\Delta w_{ij}^{(\ell)}(t-1)$$

where $\eta$ is the learning rate, $\alpha$ the momentum term, and $\delta_j(t)$ is the local error gradient (Bhowmik et al., 2010).

4. Experimental Results and Evaluation Metrics

4.1. Perceptual Metric Learning

PerFace achieved 91.7% triplet-accuracy on the SimCelebA test set, surpassing all baseline pretrained models (VGG Face 75.0%, others such as DeepID, FaceNet, ArcFace below 75%). For attribute-based classification on 1,000 unseen queries, PerFace’s embedding distances yielded an AUC of 0.965 for male/female separation (0.933 pre-finetuning) and 0.821 for young/older (0.822 pre-finetuning), indicating retention of salient attribute information in the learned metric. At test time, triplet-accuracy was measured as the fraction of test triplets satisfying $\cos(x_i, x_i^+) > \cos(x_i, x_i^-)$ (Kumagai et al., 24 Sep 2025).

4.2. Parallel Face Recognition

On the Yale B dataset (10 subjects), each OCON subnetwork achieved 100% classification on its test set (10 positives, 10 negatives). The ACON architecture achieved an average of 88% with a range of 70%–100% per class. OCON subnetworks achieved the performance goal ($\epsilon = 10^{-6}$) in $\sim$250,000–300,000 epochs, compared to over 700,000 epochs for ACON, with OCON requiring approximately one-third the wall-clock training time (MATLAB on 4-core Xeon) (Bhowmik et al., 2010).

Architecture	Test Recognition Rate	Training Epochs (to $\epsilon=10^{-6}$)
OCON	100%	250,000–300,000
ACON	88% (range 70-100%)	>700,000 (did not reach $\epsilon$)

5. Application Workflows and Deployment Considerations

5.1. Anonymization via Face Swapping

PerFace enables a two-step anonymization pipeline (Kumagai et al., 24 Sep 2025):

1. Predict the query image’s attributes (gender, age) using an external classifier. Select the attribute group $G^*$ within a fixed candidate pool with minimal maximal distance:

$$D_{I_q,G_i} = \max_{j}\left[1 - \cos\left(f(I_q), f(I_{G_i,j})\right)\right],\quad G^* = \arg\min_{G_i} D_{I_q,G_i}.$$

2. Within $G^*$, sort candidates by similarity in ascending order. Select the (perceptually most dissimilar yet attribute-matched) face for identity replacement.

This ensures the swapped face is both attribute-appropriate and maximally distinct, optimizing the anonymization tradeoff.

5.2. Parallel Operation and Incremental Scalability

In the OCON topology, class-wise decomposition enables:

• Incremental growth: Add new identities by training an additional subnetwork without affecting others.
• Fault tolerance: Failure of a compute node impacts only its assigned class; other classes remain operational.
• Resource-sharing: Idle lab PCs can be leveraged for retraining or background expansion.
• Testing: A probe is evaluated across all $K$ networks in bounded time, as each is compact (Bhowmik et al., 2010).

6. Computational Complexity, Scalability, and Practical Deployment

PerFace’s computational architecture yields significant system-level advantages (Bhowmik et al., 2010):

• Training complexity: In ACON, cost scales monolithically with the number of classes $C$, while in OCON the workload is divided into $K$ independent subproblems. With $K$ processors, wall-time is bounded by the slowest single subjob.
• Scalability: OCON permits near-linear scaling with processor count; adding classes entails only new subnetwork training, not global retraining (as required in ACON).
• Cost-effectiveness: Clusters of commodity workstations are more cost-effective than single high-end servers for large-scale face recognition.

A plausible implication is that the OCON approach remains tractable as the number of classes grows large, provided adequate compute resources.

7. Significance and Context Within the Research Landscape

PerFace’s contribution to perceptual similarity learning underscores the need for metrics that align with human judgments, particularly in privacy-critical applications. Its embedding achieves state-of-the-art triplet-accuracy and improves attribute grouping in controlled anonymization. In parallel, the MLP-based PerFace computational framework demonstrates that class-wise decomposition and parallelization can yield both superior convergence speed and recognition accuracy over conventional monolithic MLPs; this insight generalizes to other high-class-count recognition tasks and motivates further research in distributed neural network training (Kumagai et al., 24 Sep 2025, Bhowmik et al., 2010).

Both approaches represent a decisive shift away from binary identity classification and sequential monolithic training, towards architectures and metrics that accommodate nuanced application requirements—whether optimizing perceptual realism/privacy or achieving scalable, resilient face recognition infrastructure.