Spatial-Frequency Fusion Network (SFMFNet)

• The paper presents SFMFNet, a lightweight deepfake detection model that jointly models spatial textures and frequency artifacts for robust real-time performance.
• It leverages multi-scale feature fusion with residual-enhanced blur pooling and token-selective cross attention to capture manipulation cues while reducing computational cost.
• Extensive evaluations show SFMFNet achieves an in-domain AUC of 0.9867 on FF++ and an average cross-domain AUC of 0.8682 with 6.64M parameters and 1.27 GFLOPs.

Searching arXiv for SFMFNet and closely related spatial-frequency fusion papers to ground the article with current citations. Spatial-Frequency Aware Multi-Scale Fusion Network (SFMFNet) is a lightweight deepfake detection architecture designed for real-time deployment while maintaining strong accuracy and cross-domain robustness. It targets the practical gap between detectors that achieve high accuracy on standard benchmarks and the computational constraints of live streaming, video conferencing, and online moderation. The network is organized around three ideas: joint modeling of spatial textures and frequency artifacts, multi-scale feature fusion across low-, mid-, and high-level representations, and efficient downsampling and cross-level interaction. In the reported evaluation, SFMFNet achieves 1.27 GFLOPs with 6.64M parameters, an in-domain AUC of 0.9867 on FF++, and a cross-domain average AUC of 0.8682 across DFD, CDF2, DFDCP, DFDC, and UADFV (Lv et al., 28 Aug 2025).

1. Problem setting and design objectives

SFMFNet is motivated by the observation that many forgeries introduce both subtle spatial anomalies and frequency-domain artifacts. The model therefore does not treat deepfake detection as a purely spatial recognition problem. Instead, it is constructed to jointly leverage spatial textures and frequency artifacts, with the explicit goal of enhancing sensitivity to subtle manipulations while keeping the network sufficiently compact for real-time use (Lv et al., 28 Aug 2025).

The design rationale is multi-part. First, multi-scale feature fusion is used because forged cues may appear at different semantic levels: low-level textures, mid-level local inconsistencies, and higher-level semantic irregularities. Second, downsampling is treated as a potentially destructive operation; the model therefore introduces a residual-enhanced blur pooling structure to retain key semantic cues during feature reduction. Third, feature interaction across levels is handled by token-selective cross attention rather than dense attention, which reduces cost while preserving cross-level alignment. A plausible implication is that SFMFNet treats efficiency not as a post hoc compression step, but as a first-class architectural constraint.

A common oversimplification is to describe SFMFNet as merely a lightweight CNN augmented by a frequency module. The reported architecture is broader: spatial-frequency hybrid awareness, residual-preserving downsampling, and token-selective multi-level interaction are all integral to its behavior, rather than optional add-ons (Lv et al., 28 Aug 2025).

2. Network organization and feature hierarchy

The overall architecture uses a shallow CNN backbone to extract hierarchical features at low, mid, and high levels. Each scale is enhanced by a Spatial-Frequency Hybrid Aware (SFHA) module. Enhanced low-level features are then processed by a Downsample with BlurPool (DBP) module, low- and mid-level features are fused, and the resulting representation is combined with compressed high-level features through a Token-Selective Cross Attention (TSCA) module. The network ends with global pooling and an MLP head for binary classification (Lv et al., 28 Aug 2025).

Component	Function	Technical role
SFHA	Scale-wise enhancement	Fuses spatial and frequency attentions through a gated mechanism
DBP	Downsampling	Preserves semantics and edges during feature reduction
TSCA	Multi-level interaction	Performs efficient token-level cross attention between fused and high-level features

This arrangement gives SFMFNet a distinctly hierarchical fusion pattern. Low- and mid-level features contribute local texture and manipulation traces, while high-level features contribute more compressed semantic information. The fusion is therefore not a single concatenation step; it is distributed across the network, with each stage serving a different representational purpose.

This multi-level treatment places SFMFNet within a broader class of fusion architectures that avoid one-shot aggregation. However, in SFMFNet the immediate target is not image reconstruction or segmentation, but discriminative detection under strict latency and model-size budgets. That emphasis on efficient discriminative fusion is central to the network’s identity (Lv et al., 28 Aug 2025).

3. Spatial-frequency hybrid aware module

The SFHA module is the core mechanism for joint spatial-frequency modeling. Its purpose is to amplify regions likely to contain forgeries by fusing frequency-domain and spatial-domain attentions using a learnable gate. The frequency branch applies channel-wise Haar wavelet transform to the input feature map and then uses 3D and 2D convolutions to produce a frequency attention map:

$$\mathrm{M}_{\mathrm{freq}} = \mathcal{K}_f\left(\mathcal{U}\left(\mathrm{ReLU}\left(\tilde{\mathcal{K}}_f(\mathrm{Haar}(\mathrm{X}_{\mathrm{low}}))\right)\right)\right)$$

The spatial branch aggregates channel-wise global-average and max-pooled responses, concatenates normalized coordinate maps, and applies a 2D convolution:

$$\mathrm{M}_{\mathrm{spa}} = \mathcal{K}_s \cdot \mathrm{CoordAdd}\left(\left[\frac{1}{C}\sum_{c=1}^{C}\mathrm{X}_{\mathrm{low}},\ \max_{c=1}^{C}\mathrm{X}_{\mathrm{low}}\right]\right)$$

These two attentions are linearly weighted by learned coefficients $\alpha$ and $\beta$, concatenated, and passed through a gating convolution and sigmoid:

$$\mathrm{G}_{\mathrm{low}} = \sigma\left(\mathcal{K}_g \cdot [\alpha \cdot \mathrm{M}_{\mathrm{freq}},\ \beta \cdot \mathrm{M}_{\mathrm{spa}}]\right)$$

The resulting gating map modulates the input features with a residual connection:

$$\mathrm{Y}_{\mathrm{low}} = \mathrm{X}_{\mathrm{low}} + \mathrm{G}_{\mathrm{low}} \odot \mathrm{X}_{\mathrm{low}}$$

The module is applied at different scales, optionally with varied convolution kernel sizes for scale-specific priors (Lv et al., 28 Aug 2025).

The reported interpretation of this design is that frequency information captures artifacts that are less obvious in raw spatial features alone, while spatial attention preserves localization and coordinate-sensitive structure. This suggests that SFMFNet operationalizes “frequency awareness” primarily as feature modulation rather than as a standalone frequency classifier. The use of Haar wavelets is also notable: the model extracts sub-band information but immediately reinserts it into a gated discriminative pathway, rather than reconstructing a frequency-domain image.

4. Downsampling and token-selective cross-scale fusion

A second defining component is the DBP module. Conventional max pooling and average pooling are reported to harm edges and fine structures, which are important for manipulation detection. DBP therefore uses a blur-pooled main path and a residual blur-pooled projection path:

$$\mathrm{D}_{\mathrm{main}} = \mathrm{ReLU}(\mathrm{BN}(\mathcal{K}_m(\mathcal{B}(\mathrm{Y}_{\mathrm{low}}))))$$

$$\mathrm{D}_{\mathrm{res}} = \mathrm{BN}(\mathcal{K}_r(\mathcal{B}(\mathrm{Y}_{\mathrm{low}})))$$

$$\mathrm{D}_{\mathrm{low}} = \mathrm{ReLU}(\mathrm{D}_{\mathrm{main}} + \mathrm{D}_{\mathrm{res}})$$

This structure is intended to preserve low-level cues and sharp boundaries during spatial reduction. The visual analysis reported for DBP states that it preserves sharper contours in feature maps than average pooling (Lv et al., 28 Aug 2025).

Cross-level interaction is then handled by TSCA. Fused low/mid features and projected high-level features are separately mapped into queries, keys, and values:

$$\begin{aligned} \mathrm{Q}_1 &= \mathcal{K}_q(\mathrm{F}_{\mathrm{fuse}}) \ \mathrm{K}_2 &= \mathcal{K}_k(\mathrm{F}_{\mathrm{proj}}) \ \mathrm{V}_2 &= \mathcal{K}_v(\mathrm{F}_{\mathrm{proj}}) \end{aligned}$$

Instead of dense attention over all positions, TSCA performs spatial average pooling to select representative tokens:

$$\mathrm{M}_{\mathrm{spa}} = \mathcal{K}_s \cdot \mathrm{CoordAdd}\left(\left[\frac{1}{C}\sum_{c=1}^{C}\mathrm{X}_{\mathrm{low}},\ \max_{c=1}^{C}\mathrm{X}_{\mathrm{low}}\right]\right)$$0

Cross attention is then computed on the reduced token set:

$$\mathrm{M}_{\mathrm{spa}} = \mathcal{K}_s \cdot \mathrm{CoordAdd}\left(\left[\frac{1}{C}\sum_{c=1}^{C}\mathrm{X}_{\mathrm{low}},\ \max_{c=1}^{C}\mathrm{X}_{\mathrm{low}}\right]\right)$$1

The attended result is reprojected and fused with the original features by a weighted residual form:

$$\mathrm{M}_{\mathrm{spa}} = \mathcal{K}_s \cdot \mathrm{CoordAdd}\left(\left[\frac{1}{C}\sum_{c=1}^{C}\mathrm{X}_{\mathrm{low}},\ \max_{c=1}^{C}\mathrm{X}_{\mathrm{low}}\right]\right)$$2

The benefit claimed for TSCA is a favorable computation–accuracy tradeoff. In ablation, replacing TSCA with channel concatenation or standard CCA lowers accuracy, whereas TSCA yields a reported gain of about 3% AUC at less than 1% FLOPs increase (Lv et al., 28 Aug 2025).

5. Optimization objectives and gating regularization

SFMFNet is trained with a composite objective that combines classification, gate supervision, and cross-scale regularization. The classification term is binary cross-entropy, denoted $$\mathrm{M}_{\mathrm{spa}} = \mathcal{K}_s \cdot \mathrm{CoordAdd}\left(\left[\frac{1}{C}\sum_{c=1}^{C}\mathrm{X}_{\mathrm{low}},\ \max_{c=1}^{C}\mathrm{X}_{\mathrm{low}}\right]\right)$$3. The gate regularization term is defined as mean squared error between pooled gating responses and the ground-truth label for each scale:

$$\mathrm{M}_{\mathrm{spa}} = \mathcal{K}_s \cdot \mathrm{CoordAdd}\left(\left[\frac{1}{C}\sum_{c=1}^{C}\mathrm{X}_{\mathrm{low}},\ \max_{c=1}^{C}\mathrm{X}_{\mathrm{low}}\right]\right)$$4

A consistency loss encourages stable gating responses across scales:

$$\mathrm{M}_{\mathrm{spa}} = \mathcal{K}_s \cdot \mathrm{CoordAdd}\left(\left[\frac{1}{C}\sum_{c=1}^{C}\mathrm{X}_{\mathrm{low}},\ \max_{c=1}^{C}\mathrm{X}_{\mathrm{low}}\right]\right)$$5

The total objective is the weighted combination

$$\mathrm{M}_{\mathrm{spa}} = \mathcal{K}_s \cdot \mathrm{CoordAdd}\left(\left[\frac{1}{C}\sum_{c=1}^{C}\mathrm{X}_{\mathrm{low}},\ \max_{c=1}^{C}\mathrm{X}_{\mathrm{low}}\right]\right)$$6

The paper attributes an interpretability function to these auxiliary terms: the gating/map regularization encourages attention maps to consistently highlight forged regions (Lv et al., 28 Aug 2025). This is relevant because frequency-aware detection models can otherwise remain difficult to interpret, especially when their decisions depend on subtle artifact distributions rather than obvious visual inconsistencies.

6. Reported performance, ablations, and interpretability

On FF++, SFMFNet reports an AUC of 0.9867, described as the best among the compared methods. In cross-domain evaluation on DFD, CDF2, DFDCP, DFDC, and UADFV, it reports an average AUC of 0.8682, which is also stated to be the highest among lightweight and many larger models. The model’s computational footprint is 1.27 GFLOPs with 6.64M parameters, and it is reported to outperform UCF, Xception, F3Net, and related baselines that use substantially more computation and parameters (Lv et al., 28 Aug 2025).

The ablation findings are structurally informative. Removing SFHA causes the largest performance drop, indicating that joint spatial-frequency attention is the most consequential module. Removing DBP reduces accuracy by losing edge and detail information during downsampling. Replacing TSCA with simpler fusion strategies lowers performance, reinforcing the importance of efficient multi-level token interaction. The cross-domain evaluation reports a pronounced gain on DFDCP, with about a 10% improvement over UCF (Lv et al., 28 Aug 2025).

The paper also reports t-SNE evidence that SFMFNet produces better-separated clusters for real and fake samples than F3Net, Xception, and AlexNet. This supports the claim that the fused representation is not merely compact, but discriminatively organized. A plausible implication is that the combination of frequency-aware gating and cross-scale interaction improves both feature saliency and domain transferability, rather than only improving benchmark accuracy.

7. Position within spatial-frequency fusion research

SFMFNet belongs to a broader research trajectory that couples spatial and frequency representations instead of relying exclusively on one domain. In multi-exposure image fusion, MEF-SFI introduced a deep Fourier-based framework with a spatial path and a frequency path, arguing that amplitude captures illumination information while phase represents structural content (Yang et al., 2023). In remote sensing segmentation, SFFNet used wavelet decomposition and a Multiscale Dual-Representation Alignment Filter to bridge the semantic gap between spatial and frequency features (Yang et al., 2024). In image deblurring, SFAFNet proposed adaptive spatial-frequency fusion through a gated block with learnable low-pass filtering and cross-attention (Gao et al., 20 Feb 2025). In multi-modal image fusion, ISFM emphasized that simple serial or parallel spatial-frequency fusion is insufficient without interaction, and proposed interactive guidance between frequency and spatial features (Zhu et al., 4 Feb 2026).

Against this backdrop, SFMFNet is specialized for real-time deepfake detection rather than restoration, segmentation, or multimodal fusion. Its frequency awareness is wavelet-based and attention-driven; its multi-scale fusion is mediated by token-selective cross attention; and its efficiency strategy includes residual-enhanced blur pooling. These choices distinguish it from heavier reconstruction-oriented spatial-frequency networks. A plausible implication is that SFMFNet demonstrates how spatial-frequency fusion can be reformulated as an efficient discriminative architecture, not only as a generative or dense-prediction framework.

Another important distinction is task pressure. Many spatial-frequency models prioritize reconstruction fidelity, segmentation IoU, or fusion quality metrics. SFMFNet instead operates under a real-time detector constraint, where feature selectivity, downsampling behavior, and cross-domain generalization are central. In that sense, it extends the spatial-frequency fusion paradigm into practical detection settings where latency and parameter count are part of the scientific problem, not merely engineering afterthoughts (Lv et al., 28 Aug 2025).