Unveiling Deepfakes: A Frequency-Aware Triple Branch Network for Deepfake Detection

Abstract: Advanced deepfake technologies are blurring the lines between real and fake, presenting both revolutionary opportunities and alarming threats. While it unlocks novel applications in fields like entertainment and education, its malicious use has sparked urgent ethical and societal concerns ranging from identity theft to the dissemination of misinformation. To tackle these challenges, feature analysis using frequency features has emergedas a promising direction for deepfake detection. However, oneaspect that has been overlooked so far is that existing methodstend to concentrate on one or a few specific frequency domains,which risks overfitting to particular artifacts and significantlyundermines their robustness when facing diverse forgery patterns. Another underexplored aspect we observe is that different features often attend to the same forged region, resulting in redundant feature representations and limiting the diversity of the extracted clues. This may undermine the ability of a model to capture complementary information across different facets, thereby compromising its generalization capability to diverse manipulations. In this paper, we seek to tackle these challenges from two aspects: (1) we propose a triple-branch network that jointly captures spatial and frequency features by learning from both original image and image reconstructed by different frequency channels, and (2) we mathematically derive feature decoupling and fusion losses grounded in the mutual information theory, which enhances the model to focus on task-relevant features across the original image and the image reconstructed by different frequency channels. Extensive experiments on six large-scale benchmark datasets demonstrate that our method consistently achieves state-of-the-art performance. Our code is released at https://github.com/injooker/Unveiling Deepfake.

• The paper introduces a triple-branch architecture that leverages adaptive selection of spatial and frequency cues using mutual information constraints.
• It employs dynamic frequency channel selection and cross-channel enhancement to fuse top-K and next-K DCT features for optimal detection.
• Empirical results demonstrate superior AUC performance and cross-dataset generalization compared to state-of-the-art deepfake detectors.

Frequency-Aware Triple Branch Networks for Robust Deepfake Detection

Introduction

The proliferation of deepfake technologies has escalated the challenge of distinguishing between authentic and manipulated visual content. While prior deepfake detection paradigms have largely focused on spatial feature inconsistencies or the exploitation of hand-crafted frequency bands, their effectiveness is increasingly undermined by advances in generative models and by their intrinsic lack of generalization across diverse manipulation types. The paper "Unveiling Deepfakes: A Frequency-Aware Triple Branch Network for Deepfake Detection" (2604.17477) introduces a principled architecture and learning framework that addresses these challenges via a triple-branch network capturing both spatial and multi-level frequency cues, constrained using mutual information-driven objectives for feature diversity and discrimination.

Frequency Domain Motivation

Deepfake manipulations often leave subtle artifacts not easily captured in the pixel domain but manifesting in characteristic frequency bands when images are transformed via Discrete Cosine Transform (DCT). The authors empirically observe that distinct and complementary forgery cues are revealed across separate frequency channels, advocating for the explicit and adaptive modeling of multiple (dynamically selected) frequency bands, rather than restricting to pre-defined frequency ranges. This insight leads to the central architectural design principle: exploiting the spatial–frequency complementarity for comprehensive artifact exploitation.

Figure 1: The use of the discrete cosine transform (DCT) for frequency decomposition reveals distinct and complementary forgery cues across channels, justifying adaptive frequency selection and cross-channel enhancement.

Network Architecture

The proposed framework is a structured triple-branch network comprising:

• RGB Branch: Extracts features from the original image, targeting spatial and texture information.
• Primary Frequency Branch: Focuses on features reconstructed from the top-$K$ DCT frequency channels, capturing highly discriminative spectral cues.
• Secondary Frequency Branch: Processes reconstructions from the next-$K$ highest frequency channels (i.e., channels $K\!+\!1$ to $2K$), providing complementary but moderately informative spectral evidence.

A core module, Dynamic Frequency Channel Selection (DFCS), is designed using nested attention to adaptively select, for each input, the most informative frequency subsets. The output from both frequency branches is then fused by a Cross Frequency Channel Enhancement (CFCE) module, which exploits cosine-similarity-based channel reweighting to maximize complementary information and minimize redundant feature encoding.

To ensure that spatial and frequency features contribute non-overlapping forensic evidence, a Feature Decoupling Module (FDM) is introduced. This module uses mutual information minimization losses to guide the network towards extracting distinct, task-relevant representations from the RGB and fused frequency features. The final stage, the Global Fusion Module (GFM), applies multi-scale feature fusion and global information alignment, suppressing label-irrelevant or redundant activations for a compact and discriminative representation.

Figure 2: End-to-end framework schematic, illustrating DFCS, CFCE, FDM, and GFM integration for spatial and multi-level frequency feature utilization.

Figure 3: The GFM employs multi-scale feature fusion and alignment to enhance discriminativity and suppress redundancy in the joint representation.

Theoretical Foundation: Mutual Information-Driven Losses

The architecture is explicitly regularized using objectives derived from mutual information theory. The FDM minimizes mutual information between features extracted by the RGB and merged frequency branches, subject to preserving label-relevant content. Practically, the loss is implemented by maximizing a lower bound related to the conditional Kullback–Leibler divergence of predicted class distributions, evaluated on ablated versions of the feature vector (with branch components masked). The GFM further introduces a global information alignment loss, minimizing the difference in mutual information between pre- and post-compression fused features with respect to the label.

Empirical Evaluation

Benchmark Protocols

The framework is evaluated on six large-scale deepfake datasets (e.g., FF++, DF40, CDF2, DFDC) under both in-dataset and cross-dataset protocols. Performance is primarily reported using AUC, with supplementary reporting of ACC and F1 scores.

Numerical Results

In-dataset: The method achieves superior AUCs across standard benchmarks: 0.990 (FF++/C23) and 0.999 (CDF2), exceeding state-of-the-art baselines such as "Exposing the Deception" and "FreqBlender" by consistent margins.

Cross-dataset: The network displays pronounced generalization: 0.796 (CDF1), 0.872 (CDF2), 0.777 (DFDC-P), 0.735 (DFDC), outperforming strong baselines that frequently exhibit AUC drops $>0.05$ when transferred to unseen distributions.

Modularity and Ablation

Ablations reveal that the primary performance gain is contributed by the DFCS module and the multimodal triple-branch fusion, with CFCE and mutual information-driven FDM/GFM providing further additive improvements in both robustness and generalization. Notably, naive RGB or frequency-only branches are substantially outperformed by the full architecture, indicating the efficacy of the compositional design.

Figure 4: In-dataset performance metrics (AUC/ACC/F1) as a function of selected frequency channel count; saturation at $K\approx8$ demonstrates optimal artifact coverage.

Figure 5: Cross-dataset sensitivity of performance metrics to channel number, establishing an optimal window ($K=6$–$8$) for balancing discriminativity and efficiency.

Interpretability

Grad-CAM visualizations demonstrate the orthogonality and specialization of attention maps across branches: the RGB branch typically emphasizes texture and boundary inconsistencies, while frequency branches focus on complementary spectral regions, with minimal spatial overlap. This specialized branching explains improved generalization and robustness, especially on manipulations with varied or diffuse artifacts.

Figure 6: Visualization of attention maps from different branches across multiple deepfake datasets, exhibiting distinct and non-overlapping focus on forgery-related regions.

Robustness and Limitations

The model is empirically robust to practical degradations such as lossy compression and noise corruptions, with modest metric drops under augmentation. However, domain shifts inducing large frequency distribution changes—such as aggressive downscaling or re-encoding—still pose challenges for frequency-aware approaches. The architecture incurs significantly higher computation cost (parameters/FLOPs) due to the triple-branch structure, motivating future efficiency optimizations.

Implications and Future Directions

The triple-branch frequency-aware paradigm establishes that adaptive, dynamically-selected spectral feature modeling, coupled with mutual information constraints, is critical for detecting increasingly sophisticated and diverse forgery artifacts. Practically, the design achieves strong generalization and interpretability, making it viable for real-world deployment where manipulation characteristics and input quality vary widely.

Future research could focus on:

• Designing lightweight variants of the frequency branches for deployment in resource-constrained environments.
• Exploring advanced domain adaptation schemes or frequency-invariant representations to further enhance robustness against severe distribution shifts.
• Extending the mutual information-based decoupling paradigm to text, audio, or multimodal deepfake detection where the complementarity of spatial-frequency (or analogous) features remains under-exploited.

Conclusion

A frequency-aware triple branch deepfake detector with mutual information-driven objectives outperforms state-of-the-art alternatives in both in-domain and cross-domain deepfake detection benchmarks. The system robustly exploits the complementarity of spatial and multi-level frequency features, raising the standard for generalization, interpretability, and empirical reliability in the detection of sophisticated visual forgeries (2604.17477).