Dual Interactive Fusion Module (DIFM)

• DIFM is a dual-stream fusion mechanism that interactively exchanges and refines features using bidirectional cross-attention and global spatial attention.
• It processes original and enhanced image streams separately to retain critical diagnostic cues while minimizing artifacts from enhancement.
• Empirical results show that integrating DIFM boosts segmentation performance, achieving Dice scores up to 93.25% on benchmark datasets.

Dual Interactive Fusion Module (DIFM) denotes a class of dual-stream fusion mechanisms in which complementary feature sources are exchanged, reweighted, and recombined through explicitly interactive operators rather than by naïve concatenation or addition. In the literature summarized here, the explicit name is introduced in the Dual Interaction Network (DIN) for medical image segmentation, where DIFM fuses original and enhanced image features through bidirectional cross-image attention and global spatial attention at four encoder scales (Noh et al., 7 Sep 2025). Closely related mechanisms appear under other names in automatic speech recognition, multimodal medical imaging, and retinal vessel segmentation, where the same general objective is to preserve distinct feature distributions while learning selective cross-stream interactions instead of rigid serial composition or undifferentiated fusion (Lai et al., 2023).

1. Problem setting and design objective

The canonical DIFM formulation is motivated by a specific failure mode in medical image segmentation: original images preserve diagnostically important cues, whereas enhanced images can emphasize boundaries and subtle structures but may also distort or suppress useful information. Conventional fusion strategies, such as feature concatenation, can therefore fail to exploit complementarity while suppressing enhancement-induced artifacts. DIFM is introduced to address this by processing original and enhanced images in separate encoders, fusing low- and high-level features per scale, performing bidirectional cross-image attention, and then refining the exchanged information with global spatial attention (Noh et al., 7 Sep 2025).

In DIN, the enhanced image is produced by fuzzy image enhancement, and the architecture uses two image-specific encoders with no weight sharing, both based on ConvNeXt-base. Features are extracted at four hierarchical scales, and a DIFM is inserted at each scale in the skip-connection path that feeds a SegFormer-style MLP decoder. This arrangement is intended to preserve the distinct feature statistics of the original and enhanced views while progressively combining shallow edge and texture cues with deeper shape and semantic information (Noh et al., 7 Sep 2025).

A broader architectural rationale also appears in related work. InterFormer in ASR replaces rigid serial local-global composition with a parallel dual-branch design and introduces explicit bidirectional interactions and selective fusion; Swin-Res-Net fuses Swin Transformer and Res2Net streams level by level to couple global context and multi-scale local detail; DRIFA-Net uses dual attention paths for within-modality and cross-modality fusion. These systems do not all use the term DIFM, but they share the same core premise: complementary representations should interact before final fusion, and the interaction should be selective rather than uniform (Lai et al., 2023).

2. Canonical formulation in the Dual Interaction Network

At each encoder stage, DIFM first merges a low-level feature and a high-level feature within each branch. For branch $b \in \{o,e\}$, with original branch $o$ and enhanced branch $e$, the merged feature is

$$\tilde{f}^{(b)} = \mathrm{conv}\big(\mathrm{concat}(\hat{f}^{(b)}_{\text{low}}, \hat{f}^{(b)}_{\text{high}})\big).$$

This intra-branch merging makes boundary detail and semantics jointly available before cross-image interaction. DIFM then computes bidirectional cross-attention in local windows. With projected tensors

$$Q_b = \phi_Q(\tilde{f}^{(b)}), \qquad K_b = \phi_K(\tilde{f}^{(b)}), \qquad V_b = \phi_V(\tilde{f}^{(b)}),$$

and scaled dot-product attention

$$\operatorname{Att}(Q,K,V) = \mathrm{softmax}\left(\frac{QK^\top}{\sqrt{d}}\right)V,$$

the two directional exchanges are defined as

$$F^{o\rightarrow e} = \operatorname{Att}(Q_o, K_e, V_e) + Q_o, \qquad F^{e\rightarrow o} = \operatorname{Att}(Q_e, K_o, V_o) + Q_e.$$

The residual additions preserve branch-specific information while allowing each stream to attend to corresponding spatial information in the other stream. The paper describes this as exploiting low- to high-level features implicitly associated with edges, blobs, and object shapes (Noh et al., 7 Sep 2025).

Each directional feature then passes through a gated feed-forward refinement. For a generic feature $F$,

$$\tilde{f}_1,\tilde{f}_2 = \mathrm{chunk}\!\left(\mathrm{conv}\!\left(\mathrm{LN}(F)\right)\right),$$

$$\hat{F} = \mathrm{conv}\!\Big(\mathrm{GELU}(\tilde{f}_1)\otimes\tilde{f}_2\Big) + F.$$

This FFN uses layer normalization, channel expansion, channel splitting, multiplicative gating, and a residual connection. The result is then refined by global spatial attention to compensate for continuity loss introduced by window partitioning:

$o$0

The final DIFM output at a given scale is the sum of the two refined directional features,

$o$1

Within this formulation, the defining characteristics of DIFM are the symmetry of the two attention directions, the use of local-window cross-attention for efficiency, and the post-attention global refinement that restores cross-window spatial coherence (Noh et al., 7 Sep 2025).

3. Multi-scale placement, loss design, and optimization

DIN instantiates four DIFMs, one per encoder scale, and the fused outputs are delivered to the decoder as skip features. The paper characterizes the shallow scales as capturing edges, textures, and fine boundaries, the mid scales as capturing blobs and organ parts, and the deep scales as capturing global shapes and semantics. An ablation on the number of DIFMs shows a monotonic improvement, peaking with four modules (Noh et al., 7 Sep 2025).

The training objective combines region-based and boundary-aware supervision. The segmentation terms are cross-entropy and Dice loss:

$o$2

$o$3

To improve contour fidelity, DIN adds a multi-scale boundary loss based on Sobel-gradient extraction. With average pooling at scales $o$4 and a Sobel-based operator $o$5, the boundary term is

$o$6

The total loss is

$o$7

The reported implementation uses AdamW, 300 epochs, learning rate $o$8, batch size $o$9, and $e$0 resizing with random rotation and horizontal and vertical flips. Evaluation on ACDC and Synapse uses mean Dice, and Synapse additionally uses HD95. The attention head count, window size, and exact projection dimensions inside DIFM are not specified in the paper (Noh et al., 7 Sep 2025).

4. Empirical behavior and ablation evidence

DIN reports quantitative gains on both ACDC and Synapse. On ACDC, which contains 100 patients with a 70/10/20 train/validation/test split and classes RV, Myo, and LV, DIN with DIFM and boundary loss achieves Dice 93.25% overall, with RV 92.16, Myo 91.08, and LV 96.50. On Synapse, which contains 30 volumes with 18 train and 12 test cases over 8 organs, DIN achieves Dice 85.49% and HD95 10.74; the paper states that this is second-best Dice behind 3D nnFormer at 86.57% Dice and 10.63 HD95 (Noh et al., 7 Sep 2025).

Dataset	Reported result	Context
ACDC	Dice 93.25%	RV 92.16, Myo 91.08, LV 96.50
Synapse	Dice 85.49%, HD95 10.74	8 organs; second-best Dice behind 3D nnFormer

The ablation studies isolate the contribution of each DIFM component. On ACDC Dice, the concatenation baseline without DIFM is 92.50. Adding dual cross-attention yields 92.55, adding feature merging yields 92.81, adding global spatial attention yields 92.75, and combining all three yields 92.91. Adding the multi-scale boundary loss further raises performance to 93.25. Directional ablations show 93.14 for enhanced$e$1original only, 93.10 for original$e$2enhanced only, and 93.25 for dual interaction, indicating that the symmetric design contributes more than either one-way exchange alone (Noh et al., 7 Sep 2025).

Input-configuration and fusion-strategy comparisons further constrain the interpretation of DIFM’s benefit. On ACDC Dice, only original images yield 92.88, only enhanced images yield 92.66, original+original through DIFM yields 92.92, original+HE yields 92.97, original+CLAHE yields 93.02, original+Zero-DCE yields 93.05, and original+fuzzy yields 93.25. The paper also states that input fusion degrades relative to non-fusion, concatenation layer fusion reaches 92.90, and DIFM reaches 93.25. When plugged into SwinUNet and FCT, DIFM improves them by +0.61 and +0.47, respectively (Noh et al., 7 Sep 2025).

5. Related mechanisms under different names

The explicit term DIFM is not universal. In InterFormer for ASR, the corresponding mechanism is split into a Bidirectional Feature Interaction Module (BFIM) and a Selective Fusion Module (SFM). BFIM performs local-to-global and global-to-local modulation through sigmoid gating and pointwise convolution,

$e$3

while the convolution branch also uses Dynamic ReLU with parameters conditioned on the global feature,

$e$4

SFM then concatenates local and global features, reduces channel dimension, expands to branch-specific gates, applies softmax competition, and optionally applies Squeeze-and-Excitation. The paper states directly that InterFormer does not use the name “Dual Interactive Fusion Module (DIFM),” but that BFIM + SFM are the direct equivalents of a DIFM-like design (Lai et al., 2023).

In DRIFA-Net, the term DIFM is likewise absent, but the paper’s dual robust information fusion attention can be mapped to the concept. Multi-branch Fusion Attention (MFA) enhances each modality by combining Hierarchical Information Fusion Attention and Channel-wise Local Information Attention, whereas Multimodal Information Fusion Attention (MIFA) learns shared cross-modality gating through multimodal global and local pooling paths. The resulting branch-level and multimodal attention maps are formed through weighted combinations and sigmoid gating rather than transformer-style dot-product attention. The paper explicitly notes that it does not use the term DIFM, but that MFA and MIFA map naturally to interactive within-modality fusion and interactive cross-modality fusion, respectively (Dhar et al., 2024).

In Swin-Res-Net for retinal vessel segmentation, the analogous construct is called the “Two-Path Interactive Fusion Module” and is implemented as Fu-Blocks 1–4. At each encoder level, outputs from a Swin Transformer path and a Res2Net path are passed through CBR blocks, concatenated, and processed by learned projection stacks; deeper Fu-Blocks additionally ingest the previous fusion result, and Fu-Block 4 introduces a Horblock based on recursive gated convolution. The paper maps these Fu-Blocks to a DIFM interpretation because they repeatedly fuse global-context and multi-scale local-detail streams level by level (Yang et al., 2024).

Taken together, these papers show that the name DIFM is narrower than the underlying design pattern. The surveyed mechanisms differ in operator choice—cross-attention in DIN, multiplicative gating in InterFormer, pooled dual attention in DRIFA-Net, and concatenation/projection plus Horblock in Swin-Res-Net—but they all implement dual-path interactive fusion rather than simple feature accumulation (Noh et al., 7 Sep 2025).

6. Interpretation, misconceptions, and limitations

A recurrent misconception is to treat DIFM as a single standardized block. The literature summarized here does not support that reading. The explicit DIFM of DIN is a bidirectional cross-image attention module with global spatial refinement, but closely related modules in other domains use different interaction operators and different fusion topologies. This suggests that “DIFM” functions more reliably as a family resemblance term than as a unique architectural primitive (Noh et al., 7 Sep 2025).

Another misconception is that stronger interaction necessarily requires full global attention. InterFormer reaches its interactive fusion through lightweight pointwise convolution, sigmoid gating, and Dynamic ReLU rather than cross-attention, and the paper states that BFIM overhead is minor relative to MHSA. On Aishell-1, InterFormer improves Conformer from 4.6/5.1 to 4.4/4.9 CER with 46.8M parameters versus 46.2M, an increase of about 0.6M parameters. By contrast, DIN targets accuracy over efficiency: it reports 323.31M parameters and 1860.09 GFLOPs for two ConvNeXt-base encoders, four DIFMs, and an MLP decoder, and the authors note future work on lightweight variants (Lai et al., 2023).

The reported limitations are correspondingly domain-specific. In DIN, the attention head count, window size, and exact projection dimensions are not specified, and the model depends on separate original and enhanced streams. In InterFormer, the interactions are described as simple rather than full cross-attention, and the reported gains are modest but consistent. In DRIFA-Net, parameter counts, FLOPs, and latency are not reported. In Swin-Res-Net, window size, exact stage channels, batch size, and ablations of the fusion module are not provided (Dhar et al., 2024).

The most stable conclusion across these works is methodological rather than nominal: dual interactive fusion is used when the input streams are complementary but distributionally distinct, and when naïve fusion is likely to discard, blur, or over-transfer critical information. Where this principle is instantiated through symmetry, selective weighting, residual exchange, or multi-scale placement, the observed effect is improved utilization of complementary signals rather than the mere enlargement of feature dimensionality (Yang et al., 2024).