Source-Target Frequency Fusion Module

• Source-Target Frequency Fusion Module is a neural network substructure that integrates amplitude and phase information from distinct modalities via frequency-domain techniques.
• It employs FFT, wavelet transforms, and component-wise attention to blend and reconstruct cross-modal features with enhanced structural fidelity and noise suppression.
• Empirical results demonstrate its improved performance in multi-modal image fusion, remote sensing, and medical imaging through efficient global context modeling.

A Source-Target Frequency Fusion Module is a class of neural network substructure designed to integrate complementary information from two data streams—typically distinct sensors, modalities, or domains—by operating in the frequency domain. Such modules exploit the explicit decomposition of signals (images, features, or k-space measurements) into frequency components, typically amplitude and phase, to modulate, blend, and reconstruct cross-modal content with controlled structural fidelity, adaptability to noise, and computational efficiency. This architectural theme appears under various names: Frequency-Domain Fusion Module (FDFM), Cross-Modal Frequency Fusion, Selective Frequency Fusion, Dynamic Filter Fusion, Frequency-Decoupled Fusion, Wavelet-aware Frequency Enhancement, and Source-Target Frequency Fusion. Implementations blend discrete Fourier or wavelet transforms, small convolutional branches, component-wise attention, and residual connections. These modules play pivotal roles in state-of-the-art systems for image fusion, multi-source data classification, MRI and remote sensing reconstruction, and cross-domain adaptation, consistently increasing accuracy, robustness, and scalability relative to purely spatial or vanilla self-attention methods (Zheng et al., 9 Jul 2025, Hu et al., 30 Oct 2024, Li et al., 5 Dec 2025, Zhao et al., 6 Jul 2025, Zou et al., 27 Jun 2024, Li et al., 18 Dec 2025, Zhang et al., 4 Jun 2025, Zhang et al., 12 Jun 2025, Sun et al., 25 Mar 2025).

1. Fundamental Principles and Motivations

The design of Source-Target Frequency Fusion Modules is driven by several failures of conventional spatial-domain fusion. Long-range spatial dependencies require costly operations—quadratic-cost self-attention or very large convolutional kernels—that only approximate true global context, while crucial modality-specific cues (texture, edges, repetitive patterns) reside in the frequency spectrum and are either diluted or missed entirely. Frequency-domain fusion enables:

• Efficient global modeling: FFTs and frequency-domain convolutions capture $H\times W$ context with $\mathcal{O}(HWC^2)$ cost by pointwise multiplication of frequency bins instead of $\mathcal{O}(H^2W^2C^2)$ for full spatial convolutions (Zheng et al., 9 Jul 2025).
• Disentanglement of complementary features: Amplitude encodes texture, style, and global statistics; phase preserves structural information. Component-wise fusion rules adaptively blend these according to saliency, degradation, or modality alignment, directly addressing domain gaps and boosting generalizability (Li et al., 5 Dec 2025, Li et al., 18 Dec 2025).
• Structural fidelity and degradation suppression: By guiding fusion with semantic or modality-specific prompts and learning sub-band adaptive filters (e.g., wavelet, DCT, or dynamic frequency kernels), modules can suppress cross-source degradations and align details inaccessible in the spatial domain (Zhang et al., 4 Jun 2025, Zhang et al., 5 Sep 2025).

2. Mathematical Formulations and Algorithmic Structures

The canonical Source-Target Frequency Fusion Module instantiates several precise algorithmic motifs:

Fourier-based Fusion: Given feature maps $$F_{\text{src}}, F_{\text{tgt}}\in\mathbb{R}^{C\times H\times W}$$,

1. FFT: $F^{\text{freq}}_* = \mathcal{F}(F_*)$.
2. Amplitude/Phase Split: $A_* = |F^{\text{freq}}_*|$, $P_* = \angle F^{\text{freq}}_*$.
3. Amplitude Fusion: $$A_{\text{fuse}} = O([A_{\text{src}}, A_{\text{tgt}}, \text{Prompt}_A])$$, via small CNNs or attention blocks (Li et al., 5 Dec 2025).
4. Phase Fusion: $$P_{\text{fuse}} = O([P_{\text{src}}, P_{\text{tgt}}, \text{Prompt}_P])$$.
5. Reconstruction: $$F_{\text{fused}} = \mathcal{F}^{-1}[A_{\text{fuse}}\cdot e^{i P_{\text{fuse}}}]$$.

Wavelet-based Fusion: Feature maps are decomposed by DWT into $(LL, LH, HL, HH)$. Modules apply cross-attention between sub-bands, guided correction of prompt-based degradations, and then inverse transform and residual skip (Zhang et al., 4 Jun 2025, Zhang et al., 5 Sep 2025).

Dynamic Filter Construction: Soft MLP-generated weights over a bank of frequency kernels blend spectral context adaptively per instance (Zhao et al., 6 Jul 2025).

Component-Exchange via Kolmogorov-Arnold Networks (KAN): Sine and cosine terms across modalities are swapped to maximize cross-modal complementarity (Zuo et al., 9 Nov 2025).

Attention and Nonlinearities: Frequency–domain fusion is frequently interleaved with spatial attention, channel shuffling, and MHSA at sub-band or channel group level. Learnable prompts (generated from domain masks or vision-language guides) modulate fusion weights and are often projected with small CNNs or MLPs (Li et al., 18 Dec 2025, Zhang et al., 5 Sep 2025).

3. Architectural Integration and Module Variants

The integration of Source-Target Frequency Fusion Modules varies by application:

• Iterative dual-branch fusion (RPFNet): Frequency Domain Fusion Modules (FDFM) operate alongside Residual Prior Modules and Cross Promotion Modules in a loop, ensuring local texture preservation and global context with mixed spatial-frequency modeling (Zheng et al., 9 Jul 2025).
• Parallel spatial-frequency design (SFDFusion): Frequency and spatial features are merged via concatenation and decoded jointly, allowing losses to flow across both domains simultaneously and preventing any branch from dominating (Hu et al., 30 Oct 2024).
• Hierarchical filter fusion (HAPNet/DFFNet): Frequency filter banks are dynamically generated and fused at each feature hierarchy, with channel/spatial attention blocks and inter-domain channel shuffling (Luo et al., 22 Aug 2024, Zhao et al., 6 Jul 2025).
• Prompt-guided cross-modal fusion (UniFS): Adaptive mask-based learning generates feature-domain prompts for amplitude/phase fusion that robustly generalize across k-space sampling patterns (Li et al., 5 Dec 2025).
• Wavelet-based intra/inter-frequency enhancement (WIFE-Fusion): Self-attention and combinatory interactions mine same-frequency and heterogeneous subband cues across modalities for precise aggregation (Zhang et al., 4 Jun 2025).

4. Empirical Results and Comparative Impact

Source-Target Frequency Fusion Modules yield quantifiable gains in qualitative sharpness, mutual/visual information, structural similarity, and downstream detection or segmentation metrics. For example:

• SFDFusion's FDFM improves Mutual Information (MI) from 3.006 to 3.914 and Visual Information Fidelity (VIF) from 0.753 to 1.011 on MSRS, outperforming seven previous methods (Hu et al., 30 Oct 2024).
• UniFS achieves PSNR of 34.93 dB on cross-acceleration factor extrapolation vs. 32.84 dB for FSMNet, demonstrating robustness to unseen k-space patterns (Li et al., 5 Dec 2025).
• DFFNet's combination of Dynamic Filter Block and Spectral-Spatial Adaptive Fusion yields up to 4.8% boost in Overall Accuracy on Houston2013 compared to either component alone (Zhao et al., 6 Jul 2025).
• FCEKAN module in SFFR provides a 2.4% mAP50 gain on multispectral UAV detection by selective frequency component exchange (Zuo et al., 9 Nov 2025).
• Frequency-decoupled fusion in FUSE reduces Abs.Rel error by 14–25% and RMSELog by 10–33% on depth estimation tasks, facilitating zero-shot adaptability to real-world perception challenges (Sun et al., 25 Mar 2025).

5. Implementation Constraints, Computational Efficiency, and Practical Design

Modules leverage $\mathcal{O}(N\log N)$ FFTs, complex-valued convolutions (often implemented via real channel stacking), depthwise separable networks, and parameter-efficient small FC/conv blocks.

• FFT/IFFT operations are implemented via PyTorch's torch.fft or similar frameworks; empirical runtime is negligible, supporting real-time deployment (Hu et al., 30 Oct 2024, Zheng et al., 9 Jul 2025).
• Fusion parameters (kernel weights, prompts, attention heads) are densely learnable but kept small to limit computational overhead. Modules such as those in SFDFusion and DFFNet use under 0.14M parameters and <45 GFLOPs (Hu et al., 30 Oct 2024, Zhao et al., 6 Jul 2025).
• Learnable mask-based prompt layers adapt to varying domain gaps; dynamic kernel generation allows per-instance adaptation, supporting generalization to out-of-distribution scenarios (Li et al., 5 Dec 2025, Li et al., 18 Dec 2025).
• Channel shuffling and multi-scale branch aggregation are implemented by standard reshape-transpose or group convolutions (Zhao et al., 6 Jul 2025, Zhang et al., 4 Jun 2025).
• Hyperparameter choices—wavelet scales, filter bank size, prompt embedding dimension—directly affect the balance between structural fidelity and detail aggregation (Zhang et al., 4 Jun 2025, Zhang et al., 5 Sep 2025).

6. Application Domains and Generalization Properties

Source-Target Frequency Fusion Modules are central to systems for:

• Multimodal image fusion (infrared+visible): FDFM, FSAT, GFMSE, and wavelet-aware fusion modules sharply enhance targets, edges, and texture for challenging fused scenes (Zheng et al., 9 Jul 2025, Hu et al., 30 Oct 2024, Zhang et al., 12 Jun 2025, Zhang et al., 4 Jun 2025, Zhang et al., 5 Sep 2025).
• Multi-source remote sensing and classification: Frequency-aware fusion modules increase data classification accuracy for HSI/SAR/LiDAR inputs across land cover types (Luo et al., 22 Aug 2024, Zhao et al., 6 Jul 2025).
• Medical imaging (multi-contrast MRI): Frequency fusion with prompt-guided amplitude/phase blending generalizes across undersampling patterns, boosting PSNR/SSIM and reconstruction robustness (Li et al., 5 Dec 2025, Zou et al., 27 Jun 2024).
• Domain adaptation and segmentation: Source-Target Frequency Fusion in AFDAN accelerates transfer by synthesizing intermediate domain-blended frequency representations, increasing IoU by 2–4% (Li et al., 18 Dec 2025).
• Event-based and dynamic vision: Frequency-decoupled fusion strategies for joint image-event depth estimation resolve spatiotemporal frequency mismatches, yielding strong zero-shot and adverse condition robustness (Sun et al., 25 Mar 2025).

7. Limitations, Open Problems, and Future Directions

Current modules are typically limited by:

• The simplicity of analytic fusion rules (e.g., direct amplitude addition, basic dynamic filters) versus potentially richer learned non-linear or adaptive operators.
• The absence of explicit frequency-domain normalization or activation protocols, risking domain drift or instability in particular regimes (Hu et al., 30 Oct 2024).
• Insufficient quantification of frequency loss landscapes; FFT-guided or wavelet-guided supervision is often implicit and under-explored.
• Rigorous comparative studies of frequency-domain modules against advanced spatial transformers, especially in multi-sensor, non-image applications.

Active research directions include leveraging large-scale masking and prompt-guided fusion for domain adaptation (Li et al., 5 Dec 2025, Li et al., 18 Dec 2025), architectural exploration of multi-resolution fusion (wavelets vs. DCT vs. FFT), and systematic integration into 3D, video, or non-vision domains.

---

The Source-Target Frequency Fusion Module is an integral structure for cross-modal data integration, grounded in precise spectral manipulation, learnable component-wise fusion, and efficient global context modeling, with demonstrated empirical gains in accuracy, robustness, and efficiency across multiple scientific domains (Zheng et al., 9 Jul 2025, Hu et al., 30 Oct 2024, Li et al., 5 Dec 2025, Zhao et al., 6 Jul 2025, Zou et al., 27 Jun 2024, Li et al., 18 Dec 2025, Zhang et al., 4 Jun 2025, Zhang et al., 12 Jun 2025, Sun et al., 25 Mar 2025).