Dual-frequency Fusion in Multimodal Research

• Dual-frequency fusion is a method that separates low-frequency global structures from high-frequency details to fuse complementary modalities effectively.
• It employs techniques such as FFT/IFFT, wavelet transforms, and Gaussian filtering to preserve texture, edges, and modality-specific cues.
• The approach uses cross-modal gating, state-space models, and discrepancy maps to prevent any single modality from dominating the fused output.

Multimodal dual-frequency fusion denotes a class of fusion strategies that does not treat heterogeneous modalities as a single undifferentiated feature source, but instead explicitly separates, approximates, or emphasizes complementary frequency components—typically low-frequency structure and high-frequency detail—and then recombines them through cross-modal interaction. In the recent literature, this formulation appears in infrared-visible image fusion, visible-infrared object detection, RGB-thermal semantic segmentation, medical image fusion, and even text-image rumor detection, with implementations based on FFT/IFFT, discrete wavelet transforms, Gaussian filtering, bilinear decomposition, and discrepancy-guided state-space exchange (Sun et al., 9 Jan 2026, Wu et al., 13 Nov 2025, Zhang et al., 4 Jun 2025, Zhu et al., 4 Feb 2026, Lao et al., 2023). Across these settings, the central objective is stable: preserve global consistency and modality-salient structure while avoiding the loss of fine textures, edges, boundaries, or thermal targets that occurs when one modality dominates the other.

1. Core formulation and problem setting

Multimodal fusion is motivated by the fact that different sensors capture complementary yet domain-specific cues. Infrared-visible fusion seeks images with prominent targets and rich texture details; RGB-thermal segmentation addresses adverse lighting conditions where RGB alone is insufficient; medical fusion combines intensity and anatomical detail; and multimodal rumor detection treats text and image as heterogeneous carriers of semantically correlated evidence (Sun et al., 9 Jan 2026, Hu et al., 2024, Canıtez et al., 25 May 2026, Lao et al., 2023). A recurring difficulty is information imbalance: one modality can overpower the other, producing fused outputs that either over-prioritize infrared intensity at the cost of visible detail or preserve visible structure while diminishing thermal salience (Sun et al., 9 Jan 2026, Wu et al., 13 Nov 2025).

Within this setting, “dual-frequency” usually refers to an explicit or implicit partition between low-frequency and high-frequency information. Low-frequency components are used for global brightness, smooth structure, silhouettes, or thermal outlines, while high-frequency components are used for edges, textures, boundaries, and fine-grained detail (Wu et al., 13 Nov 2025, Zhang et al., 4 Jun 2025, Canıtez et al., 25 May 2026). Some methods realize this literally through sub-band decomposition. Others use a proxy. DIFF-MF, for example, constructs a feature discrepancy map

$$Diff = \tanh \left|F_{vi}^{share} - F_{ir}^{share}\right|$$

and uses it to recalibrate modality-specific features, describing this discrepancy-driven signal as the basis for balancing thermal targets and visible textures (Sun et al., 9 Jan 2026). This suggests that dual-frequency fusion is not restricted to classical spectral transforms; it can also be instantiated as modality difference modeling when the difference map is used to guide complementary feature extraction.

A second defining property is that dual-frequency fusion is rarely only a decomposition step. The decomposition is typically followed by cross-modal exchange, adaptive gating, or reconstruction constraints designed to prevent simple averaging of heterogeneous cues. This is explicit in works that criticize purely spatial-domain fusion, purely local convolutions, or simple serial/parallel spatial-frequency combination without interaction (Hu et al., 2024, Zhu et al., 4 Feb 2026).

2. Frequency representations and decomposition mechanisms

The literature uses several distinct mechanisms to expose low- and high-frequency structure.

Method	Frequency representation	Characteristic operation
DIFF-MF (Sun et al., 9 Jan 2026)	Discrepancy-guided “dual-frequency” signal	$\tanh |F_{vi}^{share}-F_{ir}^{share}|$ gates private branches
FreDFT (Wu et al., 13 Nov 2025)	Fourier-domain attention	$\mathcal{F}(Q)\odot\mathcal{F}(K)$ with IFFT reconstruction
FSRU (Lao et al., 2023)	1D DFT spectrum for text and image	spectrum compression and cross-modal spectrum co-selection
SFDFusion (Hu et al., 2024)	FFT amplitude and phase	separate fusion of amplitude and phase, then IFFT
WIFE-Fusion (Zhang et al., 4 Jun 2025)	DWT sub-bands	$LL$, $LH$, $HL$, $HH$ with IFSA and IFI
AdaSFFuse (Wang et al., 21 Aug 2025)	Adaptive Approximate Wavelet Transform	learnable $K_{LL},K_{LH},K_{HL},K_{HH}$ grouped convolutions
ISFM (Zhu et al., 4 Feb 2026)	DWT low/high multi-scale fusion	LFFB and HFFB over wavelet bands
FMRFusion (Zhoua et al., 6 Jun 2026)	Bilinear frequency decomposition	Transformer-style low-frequency and MS-SPM high-frequency branches
RGB-T segmentation (Canıtez et al., 25 May 2026)	Gaussian low/high split	$T_{low}=G_\sigma\circledast T_i'$, $T_{high}=T_i'-T_{low}$

FFT-based systems formulate fusion directly in the Fourier domain. In FreDFT, queries and keys are transformed by 2D FFT, correlated by element-wise multiplication, and returned to the spatial domain by inverse FFT:

$\tanh |F_{vi}^{share}-F_{ir}^{share}|$0

after which the resulting weights modulate the other modality’s values (Wu et al., 13 Nov 2025). SFDFusion instead separates amplitude and phase,

$\tanh |F_{vi}^{share}-F_{ir}^{share}|$1

fuses them independently, and reconstructs a spatial feature by IFFT (Hu et al., 2024). FSRU applies a one-dimensional DFT along the sequence or patch dimension for text and image, compresses the power spectrum, and performs cross-modal spectrum co-selection before inverse transformation (Lao et al., 2023).

Wavelet-based systems operate on explicitly localized frequency bands. WIFE-Fusion decomposes each modality into one low-frequency subband and three high-frequency subbands via DWT, treating structure and detail separately (Zhang et al., 4 Jun 2025). ISFM also uses DWT and then applies distinct low-frequency and high-frequency fusion blocks (Zhu et al., 4 Feb 2026). AdaSFFuse replaces fixed wavelet filters with learnable analysis filters and 2D kernels

$\tanh |F_{vi}^{share}-F_{ir}^{share}|$2

followed by sub-band-specific dilated convolutions (Wang et al., 21 Aug 2025).

Other formulations are domain-specific but frequency-aware in the same sense. RGB-thermal segmentation uses a fixed Gaussian filter with $\tanh |F_{vi}^{share}-F_{ir}^{share}|$3 and $\tanh |F_{vi}^{share}-F_{ir}^{share}|$4 to obtain low- and high-frequency thermal features (Canıtez et al., 25 May 2026). FMRFusion uses a bilinear decomposition in which shallow features are split into low-frequency base features through a Transformer-style block and high-frequency detail features through a Multi-Scale Structural Perception Module (Zhoua et al., 6 Jun 2026). These designs broaden the concept of dual-frequency fusion beyond classical Fourier analysis.

3. Cross-modal interaction operators

Once frequency structure has been exposed, fusion quality depends on how the modalities interact. The recent literature proposes several families of interaction operators.

A first family uses cross-modal spectral weighting. FSRU summarizes each modality’s compressed spectrum by

$\tanh |F_{vi}^{share}-F_{ir}^{share}|$5

then broadcasts these summary vectors to re-weight the other modality:

$\tanh |F_{vi}^{share}-F_{ir}^{share}|$6

The paper explicitly frames this as dual-frequency fusion because each modality’s spectral bands are modulated by the other modality’s global spectral priorities (Lao et al., 2023). FreDFT adopts a related but detection-oriented formulation: MFDA computes frequency-domain affinities, while FDFFL reassembles mixed-scale low-, mid-, and high-band information from depth-wise convolutions with kernel sizes $\tanh |F_{vi}^{share}-F_{ir}^{share}|$7 (Wu et al., 13 Nov 2025).

A second family uses cross-modal interaction within and across sub-bands. WIFE-Fusion’s Intra-Frequency Self-Attention performs cross-modal attention inside the same band, for example

$\tanh |F_{vi}^{share}-F_{ir}^{share}|$8

while Inter-Frequency Interaction concatenates channel-attended low-frequency features from one modality with spatial-attended high-frequency features from the other (Zhang et al., 4 Jun 2025). ISFM similarly separates low-frequency fusion from high-frequency fusion, but then goes further by using the resulting frequency-complementary feature to guide spatial fusion through a Frequency-Guided Gate (Zhu et al., 4 Feb 2026).

A third family uses discrepancy-driven or exchange-based interaction. DIFF-MF first recalibrates modality-private features using the discrepancy map:

$\tanh |F_{vi}^{share}-F_{ir}^{share}|$9

$\mathcal{F}(Q)\odot\mathcal{F}(K)$0

It then applies a Channel-Exchange Module with cross-modal state-space exchange and an adaptive gate, followed by a Spatial-Exchange Module that interleaves infrared and visible features along rows, columns, or concatenation paths (Sun et al., 9 Jan 2026). This design is explicitly intended to prevent either modality from “drowning out” the other.

These mechanisms collectively show that dual-frequency fusion is not merely a decomposition problem. It is a routing and reweighting problem in which low-frequency and high-frequency components are selectively exchanged, aligned, and recombined. This is also why several papers argue against simple serial or parallel spatial-frequency pipelines without interaction (Zhu et al., 4 Feb 2026).

4. State-space and Mamba-based formulations

A major current direction is the replacement of quadratic attention with state-space models, especially Mamba or VSS-based components, while retaining explicit spatial-frequency interaction.

DIFF-MF is a representative state-space design. Its Channel-Exchange Module flattens modality features into token sequences, splits projected tokens into SSM analogues of $\mathcal{F}(Q)\odot\mathcal{F}(K)$1, $\mathcal{F}(Q)\odot\mathcal{F}(K)$2, and $\mathcal{F}(Q)\odot\mathcal{F}(K)$3, and exchanges channels in both directions before adaptive channel re-weighting. Its Spatial-Exchange Module realigns features through row-wise, column-wise, and concatenation-based patterns, then processes them with VSS blocks whose scans are linear recurrences. The paper states that the overall spatial fusion is $\mathcal{F}(Q)\odot\mathcal{F}(K)$4 rather than $\mathcal{F}(Q)\odot\mathcal{F}(K)$5, and reports total FLOPs of approximately $\mathcal{F}(Q)\odot\mathcal{F}(K)$6 versus approximately $\mathcal{F}(Q)\odot\mathcal{F}(K)$7 for a Restormer-style transformer (Sun et al., 9 Jan 2026).

SFMFusion also integrates Mamba with explicit spatial and frequency enhancement. Its three-branch architecture couples an infrared-reconstruction branch, a visible-reconstruction branch, and a multi-modal fusion branch. The core Spatial-Frequency Enhanced Mamba Block is

$\mathcal{F}(Q)\odot\mathcal{F}(K)$8

where MMB introduces mixed-scale spatial context into 2D Selective-Scan Mamba, CEB models cross-channel dependencies with dual-pool gating, and FEB operates in the Fourier domain on amplitude and phase (Sun et al., 10 Nov 2025). The Dynamic Fusion Mamba Block then uses a learned spatial weight map

$\mathcal{F}(Q)\odot\mathcal{F}(K)$9

to shift emphasis between infrared and visible reconstruction features (Sun et al., 10 Nov 2025).

AdaSFFuse embeds spatial-frequency reasoning directly inside a Mamba2-style SSD module. Its Spatial-Frequency Mamba block contains a spatial branch $LL$0 based on $LL$1 convolution and a frequency branch $LL$2 based on FFT, learnable thresholding, and IFFT:

$LL$3

The concatenated spatial and frequency outputs are split into $LL$4 and processed by a 2D State-Space Duality update (Wang et al., 21 Aug 2025).

ISFM places a Modality-Specific Extractor built from Vision State-Space Modules ahead of Multi-scale Frequency Fusion and Interactive Spatial-Frequency Fusion. Its Frequency-Guided Mamba computes modality-specific spatial features through 2D-SSM, then modulates them with frequency-derived gates before residual fusion (Zhu et al., 4 Feb 2026). Across these designs, state-space modeling is used to preserve global dependency modeling at linear complexity, while the frequency pathway corrects Mamba’s lack of full spatial and frequency perceptions, a limitation explicitly identified in SFMFusion (Sun et al., 10 Nov 2025).

5. Optimization objectives and empirical behavior

Dual-frequency fusion models are typically trained with composite objectives that couple structural similarity, intensity preservation, edge or gradient fidelity, and task-specific auxiliary terms. DIFF-MF uses

$LL$5

where $LL$6 matches fused-image gradients to the maximum source-image gradients and $LL$7 matches intensity to a modality combination $LL$8 (Sun et al., 9 Jan 2026). WIFE-Fusion uses an analogous weighted sum of intensity, texture, and SSIM losses with $LL$9, $LH$0, and $LH$1 (Zhang et al., 4 Jun 2025). SFMFusion adds two reconstruction losses to the fusion loss,

$LH$2

so that image reconstruction acts as an auxiliary task for multimodal fusion (Sun et al., 10 Nov 2025).

Some methods introduce explicitly frequency-aware supervision. SFDFusion adds a frequency-domain fusion loss

$LH$3

where $LH$4 is a pre-computed infrared saliency mask and $LH$5 is the IFFT reconstruction from the fused spectrum (Hu et al., 2024). FSRU is distinctive in combining classification cross-entropy with intra-modal and inter-modal contrastive objectives, thereby enforcing both unimodal spectral discriminability and cross-modal alignment (Lao et al., 2023).

Empirically, reported improvements are measured not only by generic fusion scores such as Entropy (EN), Standard Deviation (SD), Spatial Frequency (SF), Average Gradient (AG), Mutual Information (MI), Visual Information Fidelity (VIF), and $LH$6, but also by downstream detection or segmentation performance. DIFF-MF reports, for example, on M³FD: EN $LH$7, SD $LH$8, SF $LH$9, MI $HL$0, VIF $HL$1, and AG $HL$2; on TNO: EN $HL$3, SD $HL$4, SF $HL$5, MI $HL$6, VIF $HL$7, and AG $HL$8; and on DroneVehicle: EN $HL$9, SD $HH$0, SF $HH$1, MI $HH$2, VIF $HH$3, and AG $HH$4 (Sun et al., 9 Jan 2026). FreDFT reaches mAP50 $HH$5 on FLIR, $HH$6 on LLVIP, and $HH$7 on M³FD, with ablations reporting that MFDA outperforms spatial cross-attention by approximately $HH$8 mAP and FDFFL beats a standard MLP by $HH$9 mAP (Wu et al., 13 Nov 2025). SFMFusion attains the best average rank on six benchmarks and reports, on MSRS, EN $K_{LL},K_{LH},K_{HL},K_{HH}$0, SD $K_{LL},K_{LH},K_{HL},K_{HH}$1, SF $K_{LL},K_{LH},K_{HL},K_{HH}$2, AG $K_{LL},K_{LH},K_{HL},K_{HH}$3, MI $K_{LL},K_{LH},K_{HL},K_{HH}$4, VIF $K_{LL},K_{LH},K_{HL},K_{HH}$5, $K_{LL},K_{LH},K_{HL},K_{HH}$6 $K_{LL},K_{LH},K_{HL},K_{HH}$7, and Avg.Rank $K_{LL},K_{LH},K_{HL},K_{HH}$8 (Sun et al., 10 Nov 2025). WIFE-Fusion reports downstream object detection improvement on MSRS to mAP@[.5:.95] $K_{LL},K_{LH},K_{HL},K_{HH}$9, compared with $T_{low}=G_\sigma\circledast T_i'$0 for IR alone and $T_{low}=G_\sigma\circledast T_i'$1 for VI alone (Zhang et al., 4 Jun 2025). For RGB-thermal semantic segmentation, the lightest variant of the frequency-guided fusion architecture reports $T_{low}=G_\sigma\circledast T_i'$2 mIoU on MFNet and $T_{low}=G_\sigma\circledast T_i'$3 mIoU on PST900 with $T_{low}=G_\sigma\circledast T_i'$4 parameters (Canıtez et al., 25 May 2026).

Taken together, these results indicate a consistent empirical pattern: dual-frequency fusion tends to improve both perceptual fusion metrics and downstream task metrics when the model explicitly preserves the trade-off between low-frequency structure and high-frequency detail.

6. Scope, misconceptions, limitations, and extensions

The scope of multimodal dual-frequency fusion is broader than infrared-visible image fusion alone. The surveyed systems cover infrared-visible fusion, RGB-NIR fusion, CT-MRI and PET-MRI medical fusion, multi-exposure fusion, multi-focus fusion, RGB-thermal semantic segmentation, visible-infrared object detection, and text-image rumor detection (Zhang et al., 4 Jun 2025, Wang et al., 21 Aug 2025, Canıtez et al., 25 May 2026, Wu et al., 13 Nov 2025, Lao et al., 2023). Some authors also state that the underlying mechanism can be adapted to audio-visual speech separation, LiDAR-camera fusion, and medical MRI-CT alignment when complementary bandwidths exist (Wu et al., 13 Nov 2025).

A common misconception is that dual-frequency fusion is synonymous with wavelet decomposition. The recent literature contradicts this directly: methods use FFT amplitude-phase fusion, Gaussian low/high separation, adaptive learnable wavelets, bilinear base-detail decomposition, and discrepancy maps between modalities (Hu et al., 2024, Canıtez et al., 25 May 2026, Wang et al., 21 Aug 2025, Zhoua et al., 6 Jun 2026, Sun et al., 9 Jan 2026). A second misconception is that adding a frequency branch is sufficient. ISFM explicitly argues that many methods rely on simple serial or parallel spatial-frequency fusion without interaction, and proposes interactive guidance instead (Zhu et al., 4 Feb 2026). WIFE-Fusion and FreDFT similarly make interaction—not decomposition alone—the core mechanism (Zhang et al., 4 Jun 2025, Wu et al., 13 Nov 2025).

The principal limitations are also recurrent. AdaSFFuse states that it requires roughly pixel-aligned inputs and that large geometric misalignments still challenge the sub-band summation; it also identifies single-scale wavelet levels as a constraint (Wang et al., 21 Aug 2025). WIFE-Fusion notes that DWT uses a fixed filter bank and that the number of WIFE blocks introduces compute and memory overhead (Zhang et al., 4 Jun 2025). ISFM reports moderate GFLOPs of $T_{low}=G_\sigma\circledast T_i'$5 and $T_{low}=G_\sigma\circledast T_i'$6 parameters, and notes that DWT/IDWT adds implementation complexity and restricts the architecture to power-of-2 spatial dimensions (Zhu et al., 4 Feb 2026). These are not incidental implementation details; they define the current trade space between interpretability, efficiency, alignment robustness, and reconstruction fidelity.

The proposed extensions follow directly from these limitations. WIFE-Fusion suggests learnable or multi-level wavelet transforms, extending IFSA/IFI to more than two modalities, and incorporating frequency-aware perceptual losses or adversarial training (Zhang et al., 4 Jun 2025). AdaSFFuse proposes multi-level AdaWAT, end-to-end task-aware fusion, upstream geometric alignment modules, and non-linear wavelet bases via small MLPs (Wang et al., 21 Aug 2025). FreDFT suggests replacing conventional cross-attention with Dual-Frequency Fusion Transformer layers in broader multimodal settings (Wu et al., 13 Nov 2025). A plausible implication is that future work will continue to move away from fixed, hand-crafted frequency partitioning toward adaptive decomposition, interactive cross-domain gating, and tighter coupling with downstream objectives.

In contemporary usage, then, multimodal dual-frequency fusion is best understood not as a single architecture, but as a design principle: isolate or approximate structurally distinct frequency content, let modalities exchange complementary evidence under learned gates or state-space dynamics, and reconstruct a fused representation that preserves both global coherence and modality-specific high-frequency detail.