MAAD-Face: Scalable Attribute Pipeline

• The pipeline leverages probabilistic label transfer to annotate over 120 million labels on 3.3M face images, surpassing traditional datasets in performance.
• It employs a five-stage architecture with rigorous reliability calibration, thresholding, and cross-source aggregation to ensure high annotation fidelity.
• The approach achieves 91% accuracy, 87% precision, and 94% recall, demonstrating its effectiveness for large-scale, multi-attribute facial analysis.

The MAAD-Face pipeline is an annotation transfer framework for constructing large-scale, high-quality, multi-attribute face datasets by leveraging existing attribute-labeled sources. Designed to scale attribute annotation to millions of face images, MAAD-Face systematically transfers, calibrates, and aggregates binary facial attribute labels from multiple origin datasets to a target corpus—culminating, in its primary instance, in a resource of over 120 million attribute labels spanning 47 distinct categories on 3.3 million VGGFace2 images. The method is distinguished by its probabilistic label transfer protocols, rigorous reliability estimation, and cross-source conflict resolution, resulting in annotations that surpass popular sources in accuracy, precision, and recall (Terhörst et al., 2020).

1. Pipeline Architecture and Stages

MAAD-Face proceeds through five ordered stages to produce high-fidelity attribute annotations:

1. Dataset Preparation and Cleaning:
   • Source datasets (CelebA and LFW) are partitioned subject-disjoint into 80% training and 20% held-out test splits.
   • LFW undergoes an optional cleaning procedure: continuously-scored attribute values near zero are discarded by iteratively selecting threshold intervals until ≥80% of samples nearest the threshold are verified correct by human raters, eliminating approximately 52% of LFW’s annotations.
2. Massive Attribute Classifier (MAC) Training:
   • For each source, a multi-task neural network ("MAC") is trained to predict all binary attributes jointly.
   • The network consists of an input layer (128-d FaceNet embeddings), followed by a shared dense (512-unit) layer (ReLU, BatchNorm, Dropout p=0.5), then one attribute-specific branch per output. Each branch has Dense(512) → ReLU → BatchNorm → Dropout(0.5) → Dense(2) → Softmax.
   • The loss is the sum of cross-entropy penalties for all attributes; training uses Adam (lr=1e-3, linear decay over 200 epochs) with batch sizes of 1024 (CelebA) and 16 (LFW).
3. Reliability Calibration and Threshold Selection:
   • For each attribute, predictions on the held-out source test partition are used to calibrate reliability, which quantifies confidence via stochastic forward passes (Dropout p=0.5, m=100 passes).
   • For each class-probability vector $x = (x_1,...,x_m)$ for attribute $a$:

   $$\mathrm{rel}(x) = \frac{1-\alpha}{m} \sum_{i=1}^{m} x_i - \frac{\alpha}{m^2} \sum_{i=1}^m \sum_{j=1}^m |x_i-x_j|, \quad \alpha=0.5$$

• For each attribute $a$, a reliability threshold $\mathrm{thr}_{\mathrm{source}}^{(a)}$ is chosen as the maximal value such that: the balanced accuracy (on the source test set) for predictions above threshold is at least $\mathrm{acc}_{\min}$, and at least fraction $d_{\min}$ of the target dataset would receive a non-undefined label. In the reference implementation, $\mathrm{acc}_{\min} = 0.90$ and $d_{\min} = 0.50$.

1. Label Transfer to Target:

   • Every VGGFace2 image is processed by each source’s MAC for each attribute $a$, producing a hard label ($+1/-1$) and reliability $r$.
   • If $r < \mathrm{thr}_{\mathrm{source}}^{(a)}$, the label is set to undefined ($0$); otherwise, the predicted sign is kept.
2. Aggregation and Final Cleaning:
   • When an attribute is provided by one source only, that source’s surviving labels become the final annotations.
   • When provided by both sources, the final label is selected from the source whose reliability-to-expected-test-accuracy mapping $\mathrm{acc}_{\mathrm{src}}^{(a)}(r)$ is higher for that particular image.
   • Additional logical rules (e.g., only one out of "young," "middle_aged," "senior" may be true for age-group) are enforced, setting violating labels to undefined.

2. Detailed Algorithmic Procedures

Source Data Processing

• For LFW, original continuous attribute scores are thresholded into ternary labels (+1, -1, 0) with boundary adjustment based on human inspection, to enhance reliability.
• CelebA’s binary annotations are used as-is without further cleaning.

Feature Extraction

• All images are passed through the FaceNet pipeline (face alignment, landmark detection, 160×160 cropping) for consistent, geometry-invariant 128-d embedding extraction.

Training the Multi-Task Classifiers

• MAC architecture: shared base (Dense-512, ReLU, BatchNorm, Dropout), attribute-specific branches. All attributes use softmax outputs for binary classification.
• Cross-entropy loss summed over all active attribute branches.
• Training is implemented using Adam with a linear decay schedule.

Reliability-Driven Label Transfer

• For every target sample and attribute, the MAC outputs m=100 stochastic predictions (using MC Dropout).
• The reliability metric combines mean confidence with dispersion (lower dispersion, higher reliability).
• Only predictions above attribute-specific reliability thresholds are retained.

Cross-Source Aggregation and Plausibility Checks

• If both sources provide a label for attribute $a$ in image $i$, the source with higher expected test-set accuracy for that reliability level is selected.
• Within classes of mutually exclusive attributes (e.g., age categories), any violation (more than one positive) results in all set to undefined.

3. Mathematical Definitions and Thresholding

The critical reliability function—central to all uncertainty handling—is:

$$\mathrm{rel}(x) = \frac{1-\alpha}{m} \sum_{i=1}^m x_i - \frac{\alpha}{m^2}\sum_{i=1}^m\sum_{j=1}^m |x_i-x_j|$$

Label transfer for each source:

$$f(p, r; \mathrm{thr}) = \begin{cases} +1 & \text{if } (r \geq \mathrm{thr}) \wedge (p=+1), \ -1 & \text{if } (r \geq \mathrm{thr}) \wedge (p=-1), \ 0 & \text{if } r < \mathrm{thr}. \end{cases}$$

Aggregation if both sources supply the attribute:

$$l_{\mathrm{target}}^{(a,i)} = \underset{\mathrm{src} \in \{\mathrm{CelebA}, \mathrm{LFW}\}}{\mathrm{arg\,max}}\, \mathrm{acc}_{\mathrm{src}}^{(a)}(r_{\mathrm{src}}^{(a,i)})$$

Threshold selection ensures, for each attribute:

$$\mathrm{BalAcc}_{\mathrm{test}}^{(a)}(r \geq \mathrm{thr}) \geq \mathrm{acc}_{\min} \quad \wedge \quad \frac{\#\{i: r^{(a,i)} \geq \mathrm{thr}\}}{|\mathrm{VGGFace2}|} \geq d_{\min}$$

4. Conflict Handling, Missing Data, and Quality Control

Any attribute value with reliability below threshold is assigned an undefined label. An attribute is excluded from the MAAD-Face release if it is impossible to select a threshold satisfying both the accuracy and coverage requirements. For attributes present in both sources with valid predictions, the label is chosen from the source with the higher expected accuracy for that particular image. The final step involves enforcing hard semantic plausibility (e.g., mutually exclusive age categories), with violations resulting in undefined values for the entire set in question.

Comprehensive human evaluation indicates superior annotation fidelity compared to source datasets: MAAD-Face achieves 91% accuracy, 87% precision, and 94% recall, outperforming both CelebA and LFW (Terhörst et al., 2020).

5. Hyperparameters and Design Choices

• Dropout probability: $p=0.5$ in the MAC (essential for Monte Carlo reliability estimation).
• Stochastic passes for reliability: $m=100$.
• Reliability aggregation weight: $\alpha=0.5$ (equal weight to mean and dispersion).
• Adam optimizer: initial lr $1 \times 10^{-3}$, decaying linearly.
• Training epochs: 200.
• Minimum test accuracy per attribute: $\mathrm{acc}_{\min} = 0.90$.
• Minimum coverage: $d_{\min} = 0.50$.
• Batch size: 1024 (CelebA), 16 (LFW).
• LFW cleaning threshold: manual, tuned such that 9/10 threshold-adjacent samples are annotated correctly by humans.

6. Quantitative Validation and Implementation

On the final held-out test sets, all transferred attributes satisfy the balanced accuracy constraint ($\geq 90\%$). In human validation (over 16,000 judgments), the final annotation set produced by the full pipeline demonstrates per-attribute accuracy, precision, and recall significantly greater than found in the originating label sources.

Aggregate statistics:

• Dataset: VGGFace2 (3.3 million face images).
• Attributes: 47 binary facial attributes.
• Total labels: approximately 123.9 million.
• Mean per-image label count: $37.5 \pm 3.7$ attributes.

Comparison to baselines:

Dataset	Images (M)	Attributes	Total Labels (M)	Per-image (mean ± std)	Human Acc./Prec./Rec.
MAAD-Face	3.3	47	123.9	37.5 ± 3.7	0.91 / 0.87 / 0.94
CelebA	0.2	40	8.2	—	0.85 / 0.83 / 0.89
LFW	0.013	40	0.9	—	0.72 / 0.61 / 0.84

7. Availability, Implementation, and Reproducibility

The MAAD-Face labels are publicly released alongside the VGGFace2 imagery. The pipeline relies on standard frameworks—principal components being the FaceNet embedding extractor and PyTorch for neural network implementation. Pseudocode is provided in the reference, specifying all operations for reliability-driven label transfer, cross-source aggregation, and conflict resolution. These implementation details permit straightforward adaptation and extension to additional datasets or attributes, provided that suitable source annotations and feature extractors are available (Terhörst et al., 2020).