Capsule-Forensics: Digital Forgery Detection

• Capsule-Forensics is a media forensics technique that uses hierarchical capsule networks and dynamic routing to detect subtle digital forgeries.
• It processes facial images with convolutional layers and statistical pooling to extract pose vectors for reliable classification of real versus fake content.
• The method employs optimized training, data augmentation, and multimodal capsule stacks to enhance the detection of replay, deepfake, and CGI forgeries.

Capsule-Forensics denotes a family of media forensics techniques that leverage capsule networks—notably the dynamic routing-by-agreement paradigm—to robustly detect digital forgeries in images and videos. These methods extend capsule networks’ original intent beyond inverse graphics, applying the architecture’s hierarchical feature aggregation to extract the subtle, distributed artifacts characteristic of spoofed media, including replay, print, CGI, deepfake, and instruction-guided edits (Nguyen et al., 2018, Nguyen et al., 2019, Nguyen et al., 27 Apr 2025).

1. Architectural Foundations and Network Design

In its prototypical implementation, Capsule-Forensics processes a pre-cropped facial RGB image (e.g., $128\times128$) with the first three convolutional blocks of VGG-19, producing a $16\times16\times256$ feature tensor (Nguyen et al., 2018). This latent feature map is partitioned across multiple (often three) primary capsule branches, each realized as a lightweight CNN with statistical pooling (mean and variance per channel) to form a pose vector $u_i$ for capsule $i$. These primary capsules are then dynamically routed to output capsules corresponding to semantic classes, such as “real” and “fake.”

The core component, the dynamic routing algorithm, transfers pose information $u_i$ from each primary capsule to high-level capsules $v_j$ through trainable transformation matrices $W_{ij}$. Routing proceeds over $r=2$ iterations as follows:

• Transformations with added Gaussian noise $\mathcal{N}(0,0.01)$ during training augment the weights.
• Coupling coefficients $c_{ij}$ are computed via softmax over output capsules, normalized for each primary capsule.
• Output capsules aggregate votes as $16\times16\times256$0, squashed to normalize length.
• Agreement scores update logits $16\times16\times256$1 as $16\times16\times256$2.

Prediction is extracted by dimension-wise softmax across output capsule vectors, followed by averaging:

$16\times16\times256$3

where $16\times16\times256$4 is the capsule dimension, and $16\times16\times256$5 denotes the $16\times16\times256$6th component pair of the “real” and “fake” capsules. Binary cross-entropy loss outperforms the margin loss in this application (Nguyen et al., 2018, Nguyen et al., 2019).

Extensions like CapsFake (Nguyen et al., 27 Apr 2025) increase representational power via multimodal capsule stacks: visual (image-based), textual (caption-based), and frequency (DCT-domain) features, each projected into shared low-dimensional capsule space and routed competitively to class capsules.

2. Training Procedures and Datasets

Standard Capsule-Forensics pipelines utilize Adam or AdamW optimizers, typical learning rates ($16\times16\times256$7 or $16\times16\times256$8), batch sizes of 32–64, and moderate epoch counts ($16\times16\times256$950). Data augmentation includes random flips and photometric transforms to increase data variety (Nguyen et al., 2018).

Dataset coverage is broad:

• REPLAY-ATTACK: printed and video replays for anti-spoofing.
• FaceForensics, FaceForensics++: facial reenactment (Face2Face), deepfakes, face swapping, under variable compression