Forgery Mixup in Deepfake Detection

• The paper introduces Fo-Mixup as a frequency-domain augmentation method that diversifies training forgeries by mixing amplitude spectra to counteract spectral bias.
• It employs a learned binary frequency mask to selectively blend spectral regions, paired with dual consistency regularization using CAMs and vMF hyperspherical embeddings.
• Experimental results show that Fo-Mixup improves cross-domain detection performance notably, with AUC gains from 69.2% to 83.6% when integrated with full consistency regularization.

Forgery Mixup (Fo-Mixup) is a frequency-domain data augmentation procedure introduced as part of FreqDebias for deepfake detection. It is designed to mitigate a model bias termed spectral bias, in which detectors overly rely on specific frequency bands and therefore generalize poorly to unseen forgery types. In its concrete formulation, Fo-Mixup dynamically diversifies the frequency characteristics of training forgeries by mixing amplitude spectra under a learned binary frequency mask while preserving the phase of one source image; within FreqDebias, it is paired with dual consistency regularization based on CAMs and vMF hyperspherical embeddings (Kashiani et al., 26 Sep 2025).

1. Spectral bias and the rationale for Fo-Mixup

The stated motivation for Fo-Mixup is that standard CNN-based deepfake detectors often “over-rely” on a narrow set of dominant frequency bands when distinguishing real from fake. The dominant bands are not constant across forgery methods: DeepFake artifacts may concentrate in very low frequencies, Face2Face in mid frequencies, and NeuralTextures also in higher bands. A detector tuned to one such range therefore generalizes poorly to new forgeries (Kashiani et al., 26 Sep 2025).

Fo-Mixup is presented as a response to the need for frequency-domain augmentation. By dynamically altering the amplitude spectrum of training forgeries, it forces the model to see a broader distribution of spectral cues. The method is explicitly described as breaking spectral shortcuts and encouraging learning of more robust, semantically meaningful features that span multiple frequency bands. Within the larger FreqDebias framework, this augmentation is one of two complementary strategies, the other being dual consistency regularization.

A common misunderstanding would be to treat Fo-Mixup as ordinary sample interpolation. In the deepfake-detection setting, the operation is not defined as convex mixing of raw image tensors or of labels. Instead, it operates in Fourier space, selectively preserving some frequency regions from one forgery and mixing other regions between two forgeries. This distinction is central to its role as a debiasing mechanism rather than a generic vicinal-risk augmentation (Kashiani et al., 26 Sep 2025).

2. Mathematical definition in the frequency domain

Let $x_i$ and $x_j$ be two input forgery images with identical spatial dimensions. The 2D discrete Fourier transform and its inverse are defined as

$F(x) = A(x,u,v)\cdot e^{\,i\cdot P(x,u,v)},$

where $A(x)\in\mathbb{R}^{H\times W}$ is the amplitude spectrum and $P(x)\in\mathbb{R}^{H\times W}$ is the phase spectrum.

Fo-Mixup picks a binary frequency mask $B\in\{0,1\}^{H\times W}$ and a mixing coefficient $\xi\sim \mathrm{Uniform}[0,1]$. The mixed amplitude is

$$\hat A(x_{ij}) = A(x_i)\odot B + \big[(1-\xi)A(x_i) + \xi A(x_j)\big]\odot (1-B),$$

where $\odot$ denotes element-wise multiplication.

An optional perturbation is then introduced in amplitude space:

$$\tilde A(x_{ij}) = N(1,\sigma^2)\odot \hat A(x_{ij}).$$

Finally, the augmented image is reconstructed by keeping the original phase of $x_j$0:

$x_j$1

The formal structure therefore separates amplitude manipulation from phase preservation. This suggests that the augmentation is intended to diversify spectral statistics while anchoring reconstruction to the phase content of $x_j$2. The paper states this in operational terms by defining the final image through the inverse transform of the modified amplitude and the original phase (Kashiani et al., 26 Sep 2025).

3. Mask construction and synthesis procedure

The algorithmic realization of Fo-Mixup is more specific than simple random masking. The pseudocode begins by computing the FFT of each channel of $x_j$3 and $x_j$4, obtaining $x_j$5 and $x_j$6. The amplitude spectrum $x_j$7 is then partitioned into $x_j$8 angular segments by discretizing

$x_j$9

For each segment $F(x) = A(x,u,v)\cdot e^{\,i\cdot P(x,u,v)},$0, the mean log spectrum is computed as

$F(x) = A(x,u,v)\cdot e^{\,i\cdot P(x,u,v)},$1

The set $F(x) = A(x,u,v)\cdot e^{\,i\cdot P(x,u,v)},$2 is clustered by k-means into $F(x) = A(x,u,v)\cdot e^{\,i\cdot P(x,u,v)},$3 clusters. This produces $F(x) = A(x,u,v)\cdot e^{\,i\cdot P(x,u,v)},$4 binary masks $F(x) = A(x,u,v)\cdot e^{\,i\cdot P(x,u,v)},$5, each selecting one cluster’s segments. For each mask $F(x) = A(x,u,v)\cdot e^{\,i\cdot P(x,u,v)},$6, a “filtered image” is produced by inverse FFT:

$F(x) = A(x,u,v)\cdot e^{\,i\cdot P(x,u,v)},$7

These filtered images are scored with a pre-trained detector and ranked by CE loss using OHEM. The algorithm keeps the top-$F(x) = A(x,u,v)\cdot e^{\,i\cdot P(x,u,v)},$8 clusters and randomly chooses one mask $F(x) = A(x,u,v)\cdot e^{\,i\cdot P(x,u,v)},$9 from these, after which the amplitude mixing, optional perturbation, and inverse reconstruction are performed.

Several implementation details are specified. Clustering uses $A(x)\in\mathbb{R}^{H\times W}$0 clusters, and mask selection keeps top $A(x)\in\mathbb{R}^{H\times W}$1 by OHEM. The mixing coefficient $A(x)\in\mathbb{R}^{H\times W}$2 is sampled in $A(x)\in\mathbb{R}^{H\times W}$3 per pair. The amplitude noise $A(x)\in\mathbb{R}^{H\times W}$4 is Gaussian $A(x)\in\mathbb{R}^{H\times W}$5, described as multiplicative noise with mean $A(x)\in\mathbb{R}^{H\times W}$6 and standard deviation approximately $A(x)\in\mathbb{R}^{H\times W}$7. Segmentation uses $A(x)\in\mathbb{R}^{H\times W}$8 angular segments with $A(x)\in\mathbb{R}^{H\times W}$9 and $P(x)\in\mathbb{R}^{H\times W}$0 chosen to uniformly tile the spectrum. The augmentation is applied to every forgery image during synthesis, while real images use standard augmentations only (Kashiani et al., 26 Sep 2025).

4. Role inside FreqDebias and consistency-driven training

Fo-Mixup is not used in isolation in the full framework. Starting from a training set $P(x)\in\mathbb{R}^{H\times W}$1, synthetic forgery samples are generated by

$P(x)\in\mathbb{R}^{H\times W}$2

with labels inherited as $P(x)\in\mathbb{R}^{H\times W}$3. Low-confidence synthesized samples are then filtered via Shannon entropy, keeping the top $P(x)\in\mathbb{R}^{H\times W}$4.

The resulting training procedure applies dual consistency regularization on pairs $P(x)\in\mathbb{R}^{H\times W}$5. The classification term is

$P(x)\in\mathbb{R}^{H\times W}$6

Local consistency matches CAMs from $P(x)\in\mathbb{R}^{H\times W}$7 and $P(x)\in\mathbb{R}^{H\times W}$8 via JS divergence. Global consistency matches vMF hyperspherical embeddings from $P(x)\in\mathbb{R}^{H\times W}$9 and $B\in\{0,1\}^{H\times W}$0. Network updates minimize

$B\in\{0,1\}^{H\times W}$1

In this design, Fo-Mixup supplies spectrally diversified forgery samples, while the dual consistency losses constrain the representations learned from original and synthesized inputs. The paper explicitly states that local supervision is provided by CAMs and global supervision by a vMF distribution on a hyperspherical embedding space, and that this dual CR mitigates over-reliance on certain frequency components by promoting consistent representation learning under both local and global supervision (Kashiani et al., 26 Sep 2025).

5. Experimental behavior and practical operating points

The reported ablations isolate both the contribution of Fo-Mixup alone and its comparison with other Fourier-space augmentations. Standalone Fo-Mixup, without CR losses, improves cross-domain performance relative to a baseline using standard augmentation only. Under the full FreqDebias setting, Fo-Mixup also outperforms the listed alternatives Amplitude-Mix and Amplitude-Swap.

Setting	Method	Result
CDFv2, standalone augmentation	Baseline (ResNet-34, standard aug only)	AUC = 69.2%
CDFv2, standalone augmentation	+ Fo-Mixup & CE alone	AUC = 74.8%; EER from 36.0% to 31.3%
Full CR	Amplitude-Mix (AM)	AUC = 76.0%
Full CR	Amplitude-Swap (AS)	AUC = 78.5%
Full CR	Fo-Mixup	AUC = 83.6%

Confidence sampling is also quantitatively characterized. Keeping the top $B\in\{0,1\}^{H\times W}$2 of synthesized samples by prediction entropy adds approximately $B\in\{0,1\}^{H\times W}$3 AUC gain over using all synthesized samples. Tuning $B\in\{0,1\}^{H\times W}$4, the fraction of kept samples, is reported to be best at $B\in\{0,1\}^{H\times W}$5; extremes such as $B\in\{0,1\}^{H\times W}$6 or $B\in\{0,1\}^{H\times W}$7 degrade performance.

The practical recommendations follow directly from these ablations. They specify using $B\in\{0,1\}^{H\times W}$8 clusters and $B\in\{0,1\}^{H\times W}$9 top hard clusters to focus mask selection on the most informative spectral regions, sampling the mixing ratio $\xi\sim \mathrm{Uniform}[0,1]$0 uniformly in $\xi\sim \mathrm{Uniform}[0,1]$1 for maximum diversity, applying light amplitude noise with mean $\xi\sim \mathrm{Uniform}[0,1]$2 and low variance, and filtering synthesized samples by entropy while keeping roughly the top $\xi\sim \mathrm{Uniform}[0,1]$3 most confident. The same source also states that Fo-Mixup should be combined with consistency regularization—local CAM and global vMF—for best cross-domain generalization, and that there is no modification to inference cost because all Fo-Mixup and auxiliary heads are removed at test time (Kashiani et al., 26 Sep 2025).

6. Terminological scope and relation to earlier mixup literature

The name “Fo-Mixup” also appears in a separate line of discussion on deep-learning-based side-channel attacks, but there it has a different status. The underlying baseline is standard mixup, defined for traces $\xi\sim \mathrm{Uniform}[0,1]$4 and one-hot labels $\xi\sim \mathrm{Uniform}[0,1]$5 by sampling

$\xi\sim \mathrm{Uniform}[0,1]$6

This is framed under the Vicinal Risk Minimization principle, and the paper reports that mixup can enhance attack performance, especially for insufficient profiling traces (Luo et al., 2021).

Within that discussion, “Towards a Specialized Forgery Mixup (Fo-Mixup)” introduces possible Fo-Mixup ideas rather than a fixed, experimentally validated algorithm under that name. The listed ideas include targeted pairing by key-hypothesis, adaptive mixing coefficient, region-aware mixup, label smoothing with domain prior, and forgery by synthetic noise injection. The same source further states that a Fo-Mixup implementation might proceed by identifying predicted leakage points, building pairs only under constraints such as equal Hamming weight or equal least-significant bit, sampling $\xi\sim \mathrm{Uniform}[0,1]$7 from pair-specific Beta distributions, mixing only in a local window, and creating softened labels with a small neural-network mapping.

This establishes an important terminological distinction. In deepfake detection, Fo-Mixup is a concrete Fourier-domain augmentation with explicit formulas, pseudocode, and ablation results. In the side-channel context, the same label is used for a proposed specialization of generic mixup, framed through “possible Fo-Mixup ideas” and “might proceed” language rather than a finalized procedure. A plausible implication is that the term has acquired a method-specific meaning in the FreqDebias literature while remaining a broader design motif in adjacent augmentation discussions (Luo et al., 2021).