MesoInception-4: Compact CNN for Deepfake Detection

• MesoInception-4 is a compact convolutional neural network designed to detect facial video forgeries by focusing on mid-level mesoscopic texture patterns.
• It employs lightweight, dilated inception modules to capture multi-scale features, enhancing robustness against video compression and cosmetic perturbations.
• The architecture achieves high frame-level and video-level accuracies while being evaluated for fairness and robustness using metamorphic testing methods.

MesoInception-4 is a highly compact convolutional neural network (CNN) designed specifically for the detection of facial video forgeries such as Deepfake and Face2Face manipulations. It targets the mesoscopic scale of image analysis, focusing on mid-level texture patterns that are less vulnerable to video compression artifacts than fine-grained details, yet more discriminative than high-level semantics such as identity. Combining inception-style multi-branch modules with mesoscopic design, MesoInception-4 achieves high accuracy in both frame-level and video-level forgery detection, while remaining efficient enough for real-time deployment on commodity hardware (Afchar et al., 2018). Its performance and behavior under fairness-sensitive perturbations, such as makeup application, have recently become a focus in the evaluation of deepfake detectors (Pu et al., 2022).

1. Architectural Overview

MesoInception-4 extends the baseline Meso-4 network by integrating two lightweight inception modules at its front-end. The architecture is structured as follows (Afchar et al., 2018, Pu et al., 2022):

• Input: 256×256 RGB face crop.
• Stage 1 (Inception-1):
  • Four parallel branches:
  • 1×1 convolution ($a_1=1$)
  • 1×1 convolution → 3×3 dilated convolution (dilation 4, $b_1=4$)
  • 1×1 convolution → 3×3 dilated convolution (dilation 4, $c_1=4$)
  • 1×1 convolution ($d_1=1$)
  • Concatenation along channel axis yields 10 feature maps.
  • BatchNorm, ReLU, 2×2 MaxPool.
• Stage 2 (Inception-2):
  • Identical structure to Stage 1 with $a_2=1, b_2=4, c_2=4, d_2=2$ (total 11 output channels).
  • BatchNorm, ReLU, 2×2 MaxPool.
• Stage 3: Conv2D (5×5, 16 filters) → BatchNorm → ReLU → MaxPool2D (2×2).
• Stage 4: Conv2D (5×5, 16 filters) → BatchNorm → ReLU → MaxPool2D (2×2).
• Dense Head: Flatten (4096 units) → Dense(4) + ReLU + Dropout(0.5) → Dense(1) + Sigmoid.

The total parameter count is approximately 28,615, making MesoInception-4 several orders of magnitude smaller than standard deep networks for image classification.

Architectural Rationale

• Mesoscopic Design: By restricting the depth and focusing on 4 principal feature-extraction layers, the architecture accentuates mesoscopic visual cues—blurriness, medium-scale texture inconsistencies—associated with forgeries.
• Dilated Inception Modules: Parallel branches with dilated convolutions sample texture at multiple receptive field scales, increasing robustness against aggressive video compression.
• Compact Head: A minimal dense head with early flattening prevents overfitting and enables real-time inference.

2. Mathematical Formalism

MesoInception-4 adopts standard convolutional building blocks with a focus on low parameter count and multi-scale representation:

• 2D Convolution:

$$(X * W)\,(i,j) = \sum_{m=0}^{K-1}\sum_{n=0}^{K-1} X(i+m,\,j+n)\,W(m,n)$$

• Dilated Convolution:

$$(f^{(d)} * x)[i,j] = \sum_{m=-1}^{1}\sum_{n=-1}^{1} f[m,n]\,x[i - d\,m,\; j - d\,n]$$

• Concatenation:

For feature maps $F^{(1)},...,F^{(4)}$ from inception branches, output is

$$F[i,j,:] = [ F^{(1)}[i,j,:],\ F^{(2)}[i,j,:],\ F^{(3)}[i,j,:],\ F^{(4)}[i,j,:] ]$$

• Batch Normalization: For feature-map $k$,

$$\hat{x}_{i,j,k} = \frac{x_{i,j,k}-\mu_k}{\sqrt{\sigma_k^2+\epsilon}};\quad y_{i,j,k} = \gamma_k \hat{x}_{i,j,k} + \beta_k$$

• Activation:

$\mathrm{ReLU}(z) = \max(0, z)$

• Loss:
  • Frame-level training: Mean squared error $l(a,y) = 0.5(a-y)^2$ (Afchar et al., 2018)
  • Binary cross-entropy with $L_2$ regularization:

  $$\mathcal{L}(\theta) = -\frac{1}{N}\sum_{i=1}^N\left[y_i\ln p_\theta(y_i|x_i)+(1-y_i)\ln(1-p_\theta(y_i|x_i))\right] + \lambda\|\theta\|_2^2$$

  (Pu et al., 2022)

3. Training Methodology

Dataset Construction

• FaceForensics (Face2Face, compression rate 23):

  • ~5,000 face images, balanced by gender.
  • Splits: 3,168 training (1,584 each male/female), 352 validation, 1,930 testing.
• Preprocessing:
  • Face detection (Viola-Jones), alignment (dlib 68 landmarks), crop to 256×256, pixel scaling.
  • On-the-fly augmentation: random zoom (±10%), rotation (±10°), horizontal flip, brightness/hue jitter.

Optimization Strategy

• Optimizer: Adam with β₁=0.9, β₂=0.999.
• Batch size: 75.
• Learning rate schedule: Initial $10^{-3}$, reduced by 10× every 1,000 iterations (lower bound $10^{-6}$).
• Epochs: 30–50 (early stopping on validation loss flattening).

4. Detection Performance

Core Metrics

• Frame-level Detection:
  • Deepfake, test set: Recall (forged) 93.4%, Recall (real) 90.0%, Overall accuracy 91.7%.
  • Face2Face: Compression 0 (lossless) 96.8%, Compression 23 (light) 93.4%, Compression 40 (strong) 81.3%.
  • Area under ROC: ≈0.98 (Deepfake), ≈0.96 (Face2Face, light compression).
• Video-level Aggregation:
  • Deepfake: 98.4% (MesoInception-4).
  • Face2Face (compression 23): 95.3%.
• Ablation against Meso-4:

MesoInception-4 yields higher recall for forged samples (+5 pp) and overall accuracy (+2.6 pp) on Deepfake. Under challenging compression, dilated filters may excessively smooth or overshoot fine details.

Summary Table: Frame-Level Performance (Afchar et al., 2018)

Task / Metric	Meso-4 (%)	MesoInception-4 (%)
Deepfake Recall (F)	88.2	93.4
Deepfake Recall (R)	90.1	90.0
Deepfake Accuracy	89.1	91.7
Face2Face Lvl 23 Acc.	92.4	93.4

Robustness to Perturbation and Fairness Assessment

Metamorphic testing using realistic makeup perturbations reveals:

• Gender Bias: Even with balanced data, a ≈4% bias (higher accuracy on female faces) persists on an unperturbed test set (Pu et al., 2022).
• Makeup Sensitivity: Application of makeup (eyeliner, eyeshadow, blush, lipstick), synthesized via 68-landmark mapping, degrades detection accuracy by up to 30%. Lipstick perturbation yields maximal disruption, indicating a model weakness for “salient region” artifacts around the mouth.
• Bias Factor: Perturbation increases the absolute difference $$|\text{Accuracy}_\text{male} - \text{Accuracy}_\text{female}|$$, indicating widened gender gap in adversarial conditions.

5. Metamorphic Testing for Fairness and Robustness

A case study (Pu et al., 2022) probed MesoInception-4 for robustness and group fairness under identity-preserving input transformations:

• Metamorphic Relations (MRs):
  • MR₁: Full-face, adaptive makeup should not shift detection prediction > ε.
  • MR₂: Varying global intensity (light/medium/heavy) should not over-influence output.
  • MR₃: Single-region makeup (eyes/cheeks/lips) should not cause significant response change absent forgery.
• Evaluation:
  • Classic metrics (Accuracy, Precision, Recall, F1-score).
  • Bias factor per gender as fairness indicator.

Findings show that beyond absolute accuracy, the network’s susceptibility to innocuous, population-sensitive perturbations exposes fairness and reliability limitations.

6. Significance, Constraints, and Research Directions

MesoInception-4 demonstrates that compact, mesoscopic CNNs with multi-scale inception modules can achieve high accuracy and efficiency for facial forgery detection under real-world conditions (Afchar et al., 2018). However, model robustness and demographic fairness are not assured by architecture alone. The network’s vulnerability to cosmetic perturbations and its measurable gender bias—even when explicitly balancing the training data—highlight open challenges:

• Achieving invariant detection performance under semantically neutral but realistic appearance changes.
• Quantifying and mitigating demographic error gaps under distributional shifts.
• The broader implication is that forensic models must be stress-tested using metamorphic testing paradigms, as done in (Pu et al., 2022), to surface hidden vulnerabilities and inform quality assurance in deployments.

7. Related Architectures and Comparative Context

MesoInception-4’s design philosophy departs from deep, high-capacity CNNs—such as those based on ResNet or VGG—for reasons of computational and forensic robustness (Afchar et al., 2018). It deliberately sidesteps deep semantic abstraction in favor of mid-level cues, which are preserved under common video degradations. Ablation studies confirm the advantage of inception-based features at mesoscopic scales, particularly in the context of Deepfake detection, and highlight the delicate trade-off between sensitivity to forgeries and resilience to benign perturbations.