---
title: 'Celeb-DF++: Robust DeepFake Video Benchmark'
url: https://www.emergentmind.com/topics/celeb-df-e6fe6c9c-4c26-4dde-8faa-23c3d524fe95
type: topic
---

# Celeb-DF++: Robust DeepFake Video Benchmark

Celeb-DF++ is a large-scale, challenging video DeepFake benchmark specifically constructed to facilitate research in generalizable DeepFake forensics. Unlike earlier datasets, Celeb-DF++ emphasizes broad forensic coverage by encompassing a wide variety of forgery types—including face-swap, face-reenactment, and talking-face forgeries—created by diverse state-of-the-art generation techniques. Its rigorous protocols and granular evaluation strategies are designed to address the persistent gaps in cross-method, cross-quality, and cross-dataset generalization of DeepFake detectors, thereby aligning the benchmark with real-world detection needs [2507.18015].

## 1. Motivation and Scope

The primary motivation for Celeb-DF++ is to address the shortcomings of existing DeepFake video datasets, which generally comprise a limited set of forgery types—most typically early face-swap methods—thus impeding the development of detectors robust to the ever-evolving landscape of generative attacks. The dataset is constructed to serve three key objectives:

- Comprehensive scenario coverage: Including the three most common forgery types—face-swap (FS), face-reenactment (FR), and talking-face (TF)—each prevalent in real-world media manipulation incidents.
- Diversity of methods: Incorporating 22 recent DeepFake generation methods differing in architecture (autoencoders, GANs, diffusion models, neural rendering), target facial regions (whole face, organs, 3D shape, landmarks, audio‐driven motion), and pipeline design.
- Generalizability assessment: Establishing evaluation protocols that stress not only within-method accuracy but also cross-forgery, cross-quality (compression-robustness), and cross-dataset generalization, reflecting real deployment conditions [2507.18015].

## 2. Dataset Construction and Statistics

Celeb-DF++ expands significantly on previous benchmarks, both in real and forged content volume and in technical heterogeneity.

### 2.1 Real Video Collection

- 590 celebrity interview videos sourced from YouTube.
- 59 distinct identities, balanced in gender, age, and ethnicity.
- Each video averages approximately 13 s at 30 fps, yielding a total of about 15 million frames.

### 2.2 Forged Video Generation

Celeb-DF++ includes 53,196 DeepFake videos, spanning three principal manipulation scenarios and 22 DeepFake methods. Each method contributes roughly 1,900–3,000 forged videos.

| Scenario     | Number of Methods | Approximate Videos | Methods                                                                  |
|--------------|-------------------|-------------------|--------------------------------------------------------------------------|
| Face-swap (FS)        | 8       | ~16,000           | Celeb-DF original, SimSwap, InSwapper, HifiFace, GHOST, UniFace, MobileFaceSwap, BlendFace   |
| Face-reenactment (FR) | 7       | ~14,000           | DaGAN, TPSMM, MCNET, HyperReenact, LIA, FSRT, LivePortrait                |
| Talking-face (TF)     | 7       | ~20,000           | SadTalker, IP-LAP, AniTalker, EDTalk, Real3D-Portrait, EchoMimic, FLOAT    |

### 2.3 Preprocessing and Quality Control

- Pipeline: Face detection → cropping and alignment → resizing to 256×256.
- Color-transfer post-processing minimizes donor–target mismatches.
- Detail-preserving masks (eyebrow/lip-driven convex hulls with smooth blending) suppress boundary artifacts.
- Temporal smoothing of facial landmarks via Kalman filtering reduces flickering.

These steps are designed to maximize the realism and technical comparability of forgeries across diverse generation methods [2507.18015].

## 3. Evaluation Protocols and Metrics

Celeb-DF++ introduces three principal evaluation protocols and rigorously defined metrics to measure detector performance under conditions that stress generalizability.

### 3.1 Metrics

- **Frame-level ROC AUC**: Calculated across all sampled frames ($N=32$ per video), using per-frame “fake” probabilities $s_i$.
- **Video-level ROC AUC**: The average $\bar s = (1/N)\sum_i s_i$ is used per video; AUC is computed over these averages.
- **Area Under the Curve (AUC)**:
  $$
  \mathrm{AUC} = \int_{0}^{1} \mathrm{TPR}(f)\,df
  $$
  where $f$ is the false-positive rate.
- **Equal Error Rate (EER)**:
  $$
  \mathrm{EER} = \{\epsilon \mid \mathrm{FAR}(\epsilon) = \mathrm{FRR}(\epsilon)\}
  $$
  with $\mathrm{FAR}$, $\mathrm{FRR}$ the false‐accept and false‐reject rates.

### 3.2 Evaluation Protocols

- **GF-eval (Generalized-Forgery):** Train on original Celeb-DF (single FS method); test on all 21 other methods (FS, FR, TF).
- **GFQ-eval (Generalized-Forgery across Quality):** As GF-eval, but with H.264 compression (c35, c45) applied to test videos.
- **GFD-eval (Generalized-Forgery across Datasets):** Train on FaceForensics++(HQ); test on all 22 methods of Celeb-DF++.

Each protocol isolates distinct aspects of generalization: cross-method, cross-quality, and cross-dataset respectively [2507.18015].

## 4. Benchmark Results and Analysis

Celeb-DF++ exposes fundamental weaknesses in current DeepFake detectors.

### 4.1 Zero-shot Difficulty

When detectors are trained on FaceForensics++(HQ) and tested directly (zero-shot) on Celeb-DF++:

- Frame-level AUC drops from 76.5% (Celeb-DF) to 69.6% (Celeb-DF++).
- Video-level AUC drops from 81.6% to 73.8%.

### 4.2 GF-eval (Protocol 1)

Across 8 representative detectors:

- Frame-level AUC: 71.7%
- Video-level AUC: 72.1%
- Best model (Effort): ≈83% (frame), 84.4% (video), but with sharp drop on FR/TF scenarios.

### 4.3 GFQ-eval (Protocol 2)

- H.264 compression (c35):
  - Frame-level AUC ≈68.2% (−3.5% drop)
  - Video-level AUC drops by 2.2%
- Compression (c45):
  - Frame-level AUC ≈63.8%
  - Video-level AUC ≈69.9%

### 4.4 GFD-eval (Protocol 3)

- Frame-level AUC: 69.4%
- Video-level AUC: 73.7%
- Performance highest for FS (similar to FF++), weaker for FR and TF.

### 4.5 Scenario-specific Observations

- FS-trained detectors generalize poorly to FR and TF.
- FR⇄TF generalization is unexpectedly strong; models trained on one often surpass 95% AUC on the other, indicative of shared motion cues.
- FS is the “easiest” scenario; audio-driven TF (e.g., EchoMimic, SadTalker) presents the greatest challenge, with some AUC values <60%.

## 5. Analysis of Detector Failings

Even top-performing detectors (ForAda, Effort, ProDet, CFM) exhibit major degradations when confronting methods, compressions, or domains unseen during training.

- High-frequency artifacts and blending-boundary cues, effective for classical FS detection, become unreliable for modern FR/TF forgeries or after compression.
- Realistic TF generation, which involves audio→landmarks→rendering, suppresses many spatial/temporal artifacts and requires new forensic features (e.g., multi-scale motion cues, audio-visual consistency, self-supervised anomaly learning).
- The observed gaps underscore the challenge of forging detectors that are robust to both method diversity and post-processing typical in the wild [2507.18015].

## 6. Forward Directions and Open Research Problems

Celeb-DF++ illuminates several key trajectories for advancing generalizable DeepFake forensics:

- Joint training on FS, FR, and TF with balanced sampling to avoid overfitting to a single scenario.
- Development of

Source: https://www.emergentmind.com/topics/celeb-df-e6fe6c9c-4c26-4dde-8faa-23c3d524fe95