---
title: 'GD²Fusion: Guided Dual-Domain Image Fusion'
url: https://www.emergentmind.com/topics/guided-dual-domain-fusion-gd-2fusion
type: topic
---

# GD²Fusion: Guided Dual-Domain Image Fusion

Guided Dual-Domain Fusion (GD\(^2\)Fusion) is a degradation-aware infrared-visible image fusion framework that synergistically integrates vision-language models (VLMs) for degradation perception with dual-domain (frequency/spatial) joint optimization. It is formulated for dual-source degraded scenarios in which visible images may suffer from low-light or overexposure and infrared images may suffer from low contrast or noise, and it is explicitly proposed as an end-to-end alternative to decoupled pre-enhancement-to-fusion pipelines that lead to cross-stage representation mismatch, residual artifact propagation, and error accumulation [2509.05000].

## 1. Problem formulation and scope

The framework addresses infrared-visible image fusion (IVIF) with two registered inputs, an infrared image and a visible-light image, with the goal of producing a fused image that preserves salient infrared targets, visible details, and overall structural consistency. In the paper’s notation, the image inputs are
\[
I_{ir}, I_{vi} \in \mathbb{R}^{B \times 3 \times H \times W},
\]
and the prompt inputs are
\[
P_{ir}, P_{vi} \in \mathbb{R}^{B \times w}.
\]
The task departs from the standard high-quality-input assumption by explicitly modeling compound degradation across both modalities rather than a single degraded source [2509.05000].

The motivating critique is directed at pipelines that first pre-enhance each degraded source image with dedicated models and then feed the enhanced outputs into a fusion network. In the described setting, such cascades require manual selection of enhancement models, degradation-specific tuning, and repeated execution for multiple degradation types. The framework instead treats degradation perception, degradation suppression, and cross-modal fusion as a single optimization problem. This suggests a shift from restoration-then-fusion modularity to degradation-aware fusion-native design, but the paper grounds that shift in the concrete claim that the decoupled pipeline causes error accumulation rather than in a separate theoretical formalization [2509.05000].

The degradation taxonomy stated for the main benchmark consists of four representative combinations: visible low-light plus infrared low-contrast, visible low-light plus infrared noise, visible overexposure plus infrared low-contrast, and visible overexposure plus infrared noise. The method is therefore organized around heterogeneous degradations across modalities rather than a shared corruption model. That distinction is central to the meaning of “guided” in GD\(^2\)Fusion: the system is intended to distinguish fusion-relevant content from modality-specific degradation patterns before and during fusion, not merely afterward [2509.05000].

## 2. Network structure and dual-domain decomposition

The architecture comprises a shallow feature extraction stage, two frequency-domain branches for infrared and visible modalities, one spatial-domain fusion branch, and a final image reconstruction head. Shallow features are produced as
\[
F_{ir} = SFE(I_{ir}), \quad F_{vi} = SFE(I_{vi}),
\]
with
\[
F_{ir}, F_{vi} \in \mathbb{R}^{B \times C \times H \times W}.
\]
Prompt embeddings are extracted with frozen CLIP:
\[
F^{p}_{ir}, F^{p}_{vi} = CLIP(P_{ir},P_{vi}),
\]
where
\[
F^{p}_{ir}, F^{p}_{vi} \in \mathbb{R}^{B \times d}.
\]
These embeddings guide both major branches of the model [2509.05000].

The frequency-domain path is implemented by the Guided Frequency Modality-Specific Extraction (GFMSE) module. For each modality, shallow features are transformed by discrete wavelet transform (DWT) into one low-frequency sub-band and three high-frequency sub-bands:
\[
LL \in \mathbb{R}^{B \times C \times \frac{H}{2} \times \frac{W}{2}},
\qquad
LH, HL, HH \in \mathbb{R}^{B \times C \times \frac{H}{2} \times \frac{W}{2}}.
\]
For infrared, the initialization is
\[
L^0_{ir}=LL_{ir}, \quad H^0_{ir}=Concat(LH_{ir},HL_{ir},HH_{ir}),
\]
followed by stacked GFMSE layers,
\[
L^{l}_{ir}, H^{l}_{ir} = GFMSE(L^{l-1}_{ir},H^{l-1}_{ir},F^p_{ir}),
\qquad
F^{l}_{ir}=IWT(L^{l}_{ir},H^{l}_{ir}),
\]
with an analogous path for the visible branch. The low-frequency band is described as carrying luminance distribution and structural contours, whereas \(LH/HL/HH\) carry texture and edge information [2509.05000].

The spatial-domain path is implemented by the Guided Spatial Modality-Aggregated Fusion (GSMAF) module. Its initial fused feature is
\[
F^0_{fu}=Cat(F_{ir},F_{vi}),
\]
and the first stage computes
\[
F^1_{local},F^1_{global}=GSMAF(F^0_{fu},F^p_{ir},F^p_{vi}).
\]
At deeper stages,
\[
F' = Cat(F^{l-1}_{local},F^{l-1}_{global},F^{l-1}_{ir},F^{l-1}_{vi}),
\]
\[
F^l_{local},F^l_{global}=GSMAF(F',F^p_{ir},F^p_{vi}), \quad l\in[2,L].
\]
Final reconstruction is
\[
I_{fu}=Re(Cat(F^L_{local},F^L_{global},F^L_{ir},F^L_{vi})),
\]
where \(Re(\cdot)\) is a three-layer \(3\times 3\) convolutional reconstruction head with ReLU [2509.05000].

| Component | Domain | Stated role |
|---|---|---|
| GFMSE | Frequency | Frequency-domain degradation perception and suppression; discriminative extraction of fusion-relevant sub-band features |
| GSMAF | Spatial | Cross-modal degradation filtering and adaptive multi-source feature aggregation |
| Re | Reconstruction | Three-layer \(3\times3\) convolutional head with ReLU |

The text also notes a typographical inconsistency in the GSMAF equations: the local branch is defined by parallel \(3\times3\), \(5\times5\), and \(7\times7\) convolutions, while the Transformer path corresponds to the global branch. This suggests that the intended decomposition is local multi-scale convolution plus global Transformer aggregation, even though one printed equation repeats \(F_{local}\) where \(F_{global}\) is contextually indicated [2509.05000].

## 3. Guidance mechanisms and degradation-aware conditioning

The guidance signal is provided by CLIP-derived prompt embeddings, which are used as channel-wise affine conditioning vectors rather than as labels or dense maps. In GFMSE, a modality-specific prompt embedding \(F^{p}_{in}\) is converted into low-frequency and high-frequency affine parameters:
\[
\alpha^{low}, \beta^{low} = MLP(F^{p}_{in}),
\]
\[
\alpha^{high}, \beta^{high} = MLP(Pad(F^{p}_{in})).
\]
These condition the low- and high-frequency features through residual affine modulation:
\[
F^{low}_{guided} = \alpha^{low} \times F^{low}_{in} + \beta^{low} + F^{low}_{in},
\]
\[
F^{high}_{guided} = \alpha^{high} \times F^{high}_{in} + \beta^{high} + F^{high}_{in}.
\]
The parameter shapes are
\[
\alpha^{low}, \beta^{low} \in \mathbb{R}^{B \times C \times 1 \times 1},
\qquad
\alpha^{high}, \beta^{high} \in \mathbb{R}^{3B \times C \times 1 \times 1}.
\]
The module is intended to support infrared contrast correction, visible luminance compensation, structural feature enhancement, suppression of high-frequency degradations such as infrared noise, and preservation of fine detail [2509.05000].

After guidance, both low- and high-frequency streams are processed with
\[
F^{low}_{out} = TrmBlock^{M}(ConvBlcok^{N}_{3}(F^{low}_{guided})),
\]
\[
F^{high}_{out} = TrmBlock^{M}(ConvBlcok^{N}_{3}(F^{high}_{guided})),
\]
where the implementation uses \(N=3\) stacked \(3\times3\) convolution plus LeakyReLU blocks and \(M=2\) Transformer layers. The paper’s dual-domain rationale is explicit: low-frequency sub-bands are suitable for luminance, contrast, and coarse structure, while high-frequency sub-bands are suitable for edges, fine textures, and degradations such as noise [2509.05000].

In GSMAF, degradation-aware conditioning is aggregated across modalities. The prompt embeddings are first fused:
\[
F^{p}_{fu} = Proj(Cat(F^{p}_{ir},F^{p}_{vi})),
\]
then mapped to affine parameters
\[
\alpha_{fu}, \beta_{fu} = MLP(F^{p}_{fu}),
\qquad
\alpha_{fu}, \beta_{fu} \in \mathbb{R}^{B \times C \times 1 \times 1}.
\]
The spatial input
\[
F^{in}_{fu} \in \mathbb{R}^{B \times 4C \times H \times W}
\]
is first passed through channel modulation,
\[
F^{cm}_{fu} = CM(F^{in}_{fu}),
\]
with \(CM(\cdot)\) described as channel attention plus \(1\times1\) convolution, and then conditioned as
\[
F^{guided}_{fu} = \alpha_{fu} \times F^{cm}_{fu} + \beta_{fu} + F^{cm}_{fu}.
\]
Local aggregation uses multi-scale convolutions with
\[
k=\{3,5,7\},
\]
followed by
\[
F'' = Cat(ConvBlock^N_{k}(F^{guided}_{fu})),
\]
\[
F_{local} = GN\text{-}LReLU(Conv_1(F'') + F^{guided}_{fu}),
\]
while the global branch is associated in the text and figure with
\[
F_{global} = TrmBlock^M(F^{guided}_{fu}).
\]
Prompt aggregation therefore serves as cross-modal degradation filtering in the spatial branch, whereas prompt-conditioned affine modulation in GFMSE serves as modality-specific degradation perception and suppression in the frequency branch [2509.05000].

The prompt construction strategy itself is under-described. The framework specifies textual guidance inputs \(P_{ir}\) and \(P_{vi}\), and the figures reportedly display the infrared and visible textual prompts, but the method section does not reproduce the exact prompt strings, does not state whether prompt selection is automatic, and does not specify the CLIP variant. This suggests that the guidance mechanism is architecturally central but textually underspecified at the level of prompt engineering [2509.05000].

## 4. Objective function, supervision, and implementation profile

Training combines three losses: intensity, texture, and color. The intensity loss is
\[
L_{int} = \frac{1}{HW} \left\| I_{fu}-\max(I^{ref}_{ir},I^{ref}_{vi}) \right\|_{1},
\]
where the fused image is matched to the per-pixel maximum of the degradation-free infrared and visible references. The texture loss is
\[
L_{text} = \frac{1}{HW}\left\| \nabla I_{fu}-\max(\nabla I^{ref}_{ir},\nabla I^{ref}_{vi}) \right\|_{1},
\]
with \(\nabla\) the Sobel gradient operator. The color loss constrains chrominance in YCbCr space:
\[
L_{color}=\frac{1}{HW}\left\|(I_{fu})_{CbCr}-(I^{ref}_{vi})_{CbCr}\right\|_{1}.
\]
The total objective is
\[
L = \gamma L_{int} + \lambda L_{text} + \theta L_{color},
\]
with
\[
\gamma=5,\quad \lambda=5,\quad \theta=6.
\]
The losses therefore prioritize salient intensity inheritance, texture preservation, and visible-compatible color fidelity [2509.05000].

The use of \(I^{ref}_{ir}\) and \(I^{ref}_{vi}\) implies supervised training with degradation-free source references for the degradation-aware setting. The paper does not mention SSIM loss, perceptual loss, adversarial loss, frequency-domain reconstruction loss, or explicit degradation regularization. The optimizer is not explicitly stated in the provided text, which is a nontrivial omission because the framework otherwise reports a detailed training schedule [2509.05000].

Implementation settings are specified as follows. The degradation-aware model is trained for 200 epochs, and the high-quality-input-assumption model for 100 epochs, using random \(96\times96\) patches, batch size 16, and learning rate \(2.5\times10^{-4}\). Architectural hyperparameters are \(L=3\) stacked layers, \(N=3\) convolution blocks, \(M=2\) Transformer blocks, and GSMAF local kernels
\[
K=\{3,5,7\}.
\]
Training is performed on a single NVIDIA GeForce RTX 4090D GPU, and CLIP remains frozen during training and inference [2509.05000].

## 5. Evaluation, ablations, and downstream effects

The degradation-aware setting is evaluated on the EMS dataset, which provides 2,278 registered source image pairs for training, with four evaluation subsets: 69 low-light plus low-contrast pairs, 69 low-light plus noise pairs, 35 overexposure plus low-contrast pairs, and 35 overexposure plus noise pairs. High-quality-input evaluation uses LLVIP, with 11,025 randomly sampled training images and 50 testing images, and MSRS, with 45 images for fusion evaluation and 80 annotated image pairs for object detection. For degraded comparison, methods that assume clean inputs are preceded by pre-enhancement, including IAT for visible brightness adjustment, MIRNet for infrared contrast enhancement, and SwinIR for infrared denoising [2509.05000].

On high-quality-input evaluation, the framework remains strong despite its degradation-oriented formulation. On MSRS it achieves
\[
FMI = 0.9530,\quad MI = 2.2290,\quad VIF = 0.9927,\quad Q_{abf} = 0.6383,\quad Q_w = 0.9167,\quad Q_y = 0.8543,
\]
with the best scores on \(FMI\), \(MI\), \(Q_{abf}\), \(Q_w\), and \(Q_y\), and second on \(VIF\). On LLVIP it is best on all six reported metrics:
\[
FMI = 0.9273,\quad MI = 2.5573,\quad VIF = 0.9881,\quad Q_{abf} = 0.7351,\quad Q_w = 0.9206,\quad Q_y = 0.7761.
\]
The qualitative summary attributes these results to better preservation of salient infrared targets, infrared structural contours when visible structure is weak, visible details and edges, balanced modality integration, and color fidelity [2509.05000].

The main target regime is degraded fusion. On low-light plus low-contrast, the framework reports
\[
AG = 4.5904,\quad EI = 47.6333,\quad SD = 43.1937,\quad SF = 14.0132,
\]
with best values on \(AG\), \(EI\), and \(SF\), and second on \(SD\). On overexposure plus low-contrast it reports
\[
AG = 6.9128,\quad EI = 71.8015,\quad SD = 52.3444,\quad SF = 19.6723,
\]
which are best across all four metrics. On low-light plus noise the reported values are
\[
AG = 4.5412,\quad EI = 47.3899,\quad SD = 43.3764,\quad SF = 13.9295,
\]
again best on \(AG\), \(EI\), and \(SF\), and second on \(SD\). On overexposure plus noise it achieves
\[
AG = 6.6836,\quad EI = 69.5317,\quad SD = 52.9414,\quad SF = 19.0150,
\]
best on all four reported metrics. The paper characterizes these outcomes as superior fusion performance compared with existing algorithms and strategies in dual-source degraded scenarios [2509.05000].

Ablation results support the necessity of both domains. Removing GFMSE yields
\[
AG: 4.4151,\quad EI: 45.8626,\quad SD: 40.7929,\quad SF: 13.1751,
\]
while removing GSMAF yields
\[
AG: 4.0314,\quad EI: 42.0138,\quad SD: 38.3505,\quad SF: 12.4980.
\]
The spatial-domain drop is larger, but both modules are required for the reported optimum. The paper also states that without texture loss, sharpness and detail drop significantly, and without color loss, color fidelity worsens and metrics decrease. The selected hyperparameters \(L=3\), \(N=3\), \(M=2\), and \(K=\{3,5,7\}\) are presented as a practical balance between performance and complexity [2509.05000].

The fused images also improve downstream detection on MSRS with YOLOv5, producing
\[
\text{mAP@0.85} = 0.588,\quad \text{mAP@0.90} = 0.336,\quad \text{mAP@[0.5,0.95]} = 0.659.
\]
These values exceed those obtained from raw infrared and visible images and from all compared fusion methods. This suggests that GD\(^2\)Fusion is not only a perceptual fusion model but also a representation-improving front end for later semantic tasks, although the paper does not provide runtime, FLOPs, or parameter-count comparisons in the provided text [2509.05000].

## 6. Position within the broader dual-domain fusion literature

GD\(^2\)Fusion belongs to a broader methodological lineage in which fusion is improved by explicitly separating representational domains and then coupling them with guidance. A closely related formulation is SFIGF, “Simultaneous Feature and Image Guided Fusion,” which performs guided fusion simultaneously in the feature domain and the image domain for guided image restoration. SFIGF uses a GF-inspired cross-attention module for feature-level fusion and a GF-like image-domain branch of the form
\[
Q_{Im} = A_{Im} \circ I + B_{Im},
\]
thereby pairing contextual recovery with direct detail-preserving image-level guidance [2312.08853].

A second cluster uses spatial-frequency decomposition. MEF-SFI treats multi-exposure image fusion as a spatial-frequency integration problem, arguing that local spatial operators recover textures and edges but struggle with image-wide illumination consistency, while the frequency domain offers a natural global view for balancing exposure and illumination distributions. The network therefore couples a spatial path and a frequency path through repeated Spatial-Frequency Fusion Modules, with a dual-domain loss to preserve complementary information in both domains [2312.10604]. FUSION in underwater image enhancement makes an analogous claim, describing a dual-domain framework in which the spatial branch restores local textures and the FFT-based frequency branch captures global structural information, with a Frequency Guided Fusion module injecting frequency-derived features into spatial reconstruction [2504.01243].

The same design principle extends beyond image fusion. DUN-DD for portable MRI reconstruction uses parallel Fourier-domain and image-domain branches inside a five-stage physics-guided unrolled network, then combines them through an attention-based residual U-Net. There the dual domains are k-space and image space, and the “guidance” comes jointly from the MRI forward model and learned attention-based fusion [2602.19829]. In multimodal autonomous-driving perception, DDHFusion combines voxel and BEV domains, with sparse image-voxel construction, cross-modal Mamba alignment in both domains, voxel-to-BEV compensation, and BEV-guided voxel decoding; this suggests that dual-domain fusion can also be organized around geometric representations rather than transform domains [2503.08992].

These comparisons indicate that GD\(^2\)Fusion is specific in name and task—degradation-aware infrared-visible image fusion with CLIP-conditioned frequency/spatial processing—but general in methodological orientation. A plausible implication is that the term now denotes not only the particular IVIF framework of [2509.05000] but also a recognizable design pattern: one domain is used to model information that is globally structured, physically grounded, or degradation-selective, while a second domain captures local detail, structural aggregation, or task-level consistency. In that broader sense, GD\(^2\)Fusion is part of a wider shift from single-space fusion toward explicitly guided cross-domain optimization.

Source: https://www.emergentmind.com/topics/guided-dual-domain-fusion-gd-2fusion