CelebA-Spoof: Large-Scale Face Anti-Spoofing

• CelebA-Spoof is a benchmark dataset that advances face anti-spoofing research with over 625K RGB images and detailed live/spoof and spoof-type annotations.
• Its annotations include 10 spoof-type categories and 40 facial attributes, enabling multi-task learning and nuanced spoof attack analysis.
• The dataset’s subject-disjoint partitions and rigorous evaluation protocols support both intra-dataset and cross-dataset generalization studies.

CelebA-Spoof is a large-scale, richly annotated face anti-spoofing dataset designed to advance research in face presentation attack detection (PAD) and robust model generalization. Developed as an augmentation of the CelebA face dataset, CelebA-Spoof introduces semantically meaningful annotations and unprecedented scale to support the next generation of face anti-spoofing (FAS) research (Zhang et al., 2020).

1. Motivation and Rationale

CelebA-Spoof was created to address four principal limitations of existing FAS benchmarks:

1. Restricted Scale: Previously available FAS datasets were limited in subject count, sample size, and diversity, hampering the training of over-parameterized deep models and enabling overfitting.
2. Limited Diversity: Most benchmarks were acquired in controlled environments with little variation in background, illumination, sensor, or attack presentation, thus failing to capture real-world complexity.
3. Sparse Annotation: Traditional datasets included only binary live/spoof labels, precluding research into spoof-type discrimination, auxiliary-supervised learning, and attribute-conditioned robustness.
4. Generalization Deficit: Methods trained and evaluated on such benchmarks demonstrated significant performance degradation when transferred to new domains or unseen attack styles, exposing a critical gap between lab results and operational deployment.

CelebA-Spoof was proposed to overcome these obstacles through large-scale in-the-wild data collection, attack-type granularity, and rich contextual labeling, ultimately supporting multi-task learning, generalization analysis, and semantic feature disentanglement (Zhang et al., 2020).

2. Dataset Properties and Annotation Structure

2.1 Scale and Diversity

• Subject Count: 10,177 unique identities.
• Total Images: 625,537 RGB-based face images.
• Acquisition Diversity: Data encompasses 8 scenes (2 environment types × 4 illumination conditions), with capture executed using more than 10 distinct sensors.

This scale and acquisition heterogeneity positions CelebA-Spoof as the largest and most diverse publicly available benchmark for FAS at the time of publication.

2.2 Label Schema

Each image is labeled with:

• Binary Live/Spoof Label: The canonical anti-spoofing task target.
• 10 Spoof-Type Labels: Each spoofed image is further annotated according to one of ten attack-type categories, describing aspects such as the physical medium, presentation device, or geometric modification of the spoof carrier (e.g., printed photo, curved display, partial manipulation).
• 40 Facial Attributes: These labels are inherited directly from CelebA and describe canonical semantic properties (e.g., eyeglasses, hair color, facial hair)—enabling auxiliary and bias-aware research.

The annotation design supports multiple research axes: spoof-type-aware learning, semantic subgroup analysis, bias assessment, and multi-task feature learning.

2.3 Partitioning Protocol

• Train/Validation/Test Splits: Subjects are partitioned with no identity overlap between splits, ensuring disjoint subject sets for intra-dataset evaluation (i.e., subject-independent test).
• Identity Disjointness: This partition design is essential for fair benchmarking and robust generalization analysis.

3. Benchmarking Protocol and Metrics

CelebA-Spoof introduces a rigorous evaluation protocol that supports both intra-dataset (subject-disjoint) and cross-dataset generalization studies (Zhang et al., 2020, Zhang et al., 2021).

• Evaluation Tasks:
  • Standard (intra-dataset) evaluation: Training, validation, and testing performed on CelebA-Spoof with disjoint subject splits.
  • Cross-dataset evaluation: Assessing model transfer by training on CelebA-Spoof and evaluating on external datasets, or vice versa.
• Metrics:
  • Accuracy
  • APCER: Attack Presentation Classification Error Rate, i.e., fraction of spoof images misclassified as live.
  • BPCER: Bona Fide Presentation Classification Error Rate, i.e., fraction of live images misclassified as spoof.
  • ACER: Average Classification Error Rate, the mean of APCER and BPCER.

  $$\text{ACER} = \frac{\text{APCER} + \text{BPCER}}{2}$$

Challenge protocols, such as the CelebA-Spoof Challenge 2020, further emphasize TPR at very low FPR (e.g., $\text{TPR}@\text{FPR}=10^{-3}$) to reflect security-sensitive deployments (Zhang et al., 2021).

4. Auxiliary Information Embedding Network (AENet) and Multi-Task Supervision

To leverage the rich annotation schema, the benchmark introduces AENet—a unified multi-task framework designed for FAS (Zhang et al., 2020):

Architecture Overview:

• Shared CNN Backbone: Extracts high-level facial features from cropped and normalized RGB input images.
• Auxiliary Embedding Branches: Embed spoof-type and facial attribute supervisory signals into the feature space.
• Task-Specific Prediction Heads: Independently predict (a) live/spoof label, (b) spoof-type, (c) facial attributes.

Multi-Task Loss:

• Binary anti-spoofing loss: Cross-entropy on the live/spoof label.
• Spoof-type prediction loss: Cross-entropy (one per spoof-type branch).
• Attribute prediction loss: Binary cross-entropy (one per CelebA attribute).
• Total loss:

$$\mathcal{L} = \mathcal{L}_{\text{fas}} + \lambda_1\mathcal{L}_{\text{spoof-type}} + \lambda_2\mathcal{L}_{\text{attr}}$$

Empirical results demonstrate that multi-task auxiliary supervision yields superior anti-spoofing performance, enhances discrimination of hard attacks, and regularizes representation learning to avoid overfitting to dataset idiosyncrasies.

5. Empirical Results and Insights

• Challenge and Difficulty: Even state-of-the-art models fail to saturate CelebA-Spoof, affirming that the benchmark is more challenging than prior datasets. This is evident from intra-dataset testing as well as persistent generalization gaps in cross-dataset scenarios (Zhang et al., 2020, Zhang et al., 2021).
• Auxiliary Supervision Gains: Joint learning with spoof-type and attribute labels clearly improves performance relative to binary-only baselines, particularly on challenging or semantically ambiguous attacks.
• Attack Type Difficulty: Certain categories, such as high-quality replay/screen attacks or attacks with weak post-cropping artifacts, remain more challenging, emphasizing the need for detailed annotation.
• Ensemble Dominance: In the public challenge context, leading solutions consistently employ strong CNN backbones in ensemble configurations, sophisticated face detection/alignment, and heavy data augmentation (Zhang et al., 2021).
• Low-FPR Optimization: Top submissions and protocols focus on maximizing TPR under operationally relevant low-FPR constraints.

6. Limitations and Future Directions

CelebA-Spoof, while a significant advance, presents several residual limitations:

• Generalization Gap: Even at large scale, domain shift and attack diversity remain major challenges; high intra-dataset performance is not predictive of robust real-world deployment.
• RGB-Only Modality: The dataset is limited to RGB images; depth, IR, and temporal signals (e.g., for video-based liveness) are not included, although these are valuable in practical PAD.
• Finite Attack Taxonomy: Only ten spoof types are annotated; ongoing evolution of attack techniques may outpace the dataset categories.
• CelebA-Inherited Bias: As a derivative of CelebA, the dataset could inherit demographic, pose, and style biases, potentially limiting fairness and representational completeness.
• Necessity for Richer Domain Variation: Additional improvements are possible through increased sensor/environment heterogeneity, further attack realism, and subgroup fairness analysis.

Planned future research directions include collection of broader and more diverse PAD benchmarks, deployment of multimodal and temporal PAD approaches, and protocols emphasizing hidden test sets and cross-domain evaluation (Zhang et al., 2021).

7. Position Relative to the State of the Art

CelebA-Spoof occupies a central role as the first large-scale, richly annotated, and publicly available image-based FAS dataset with attack granularity and subject diversity sufficient for benchmarking modern deep architectures and auxiliary-supervised learning pipelines. Its challenge protocols and hidden-test evaluation have shifted the field toward greater empirical rigor and comparability (Zhang et al., 2021). While true deployment robustness still requires improvements in attack coverage, sensor diversity, and cross-domain generalization, CelebA-Spoof remains the reference benchmark for attribute-aware, large-scale, image-based anti-spoofing research (Zhang et al., 2020Zhang et al., 2021).