Adaptive Dual-stream Fusion Module

• ADFM is a neural module that adaptively fuses two distinct feature streams using data-dependent gating mechanisms.
• It employs fusion operations like summation, concatenation, or convolutional mixing to optimize robustness against noise and modality-specific challenges.
• ADFM architectures are applied across domains such as vision, signal processing, EEG decoding, and medical imaging, resulting in significant performance gains.

An Adaptive Dual-stream Fusion Module (ADFM) refers to a class of neural modules that adaptively combine two distinct streams of information—usually representing heterogeneous feature spaces, modalities, or temporal/spatial cues—using data-dependent, learnable weighting mechanisms. Across vision, signals, and medical AI, ADFM designs serve as the critical nexus at which parallel pathways are merged, with their fusion weights and gating functions optimized to emphasize the most informative or robust features in a context-dependent manner.

1. Core Architectural Principles

At their foundation, all ADFM designs instantiate three essential elements: (i) two parallel pathways producing complementary representations; (ii) a gating or weighting system—learned, adaptive, attention-based, or spatially varying—that mediates how much each stream contributes to the fused output; and (iii) a fusion (aggregation) operation, performed either via summation, concatenation, or learned convolutional mixing.

A representative form is realized in SKANet, which integrates Time-Frequency Image (TFI) and Power Spectral Density (PSD) streams. The TFI and PSD flows are separately processed by stacks of Selective Kernel (SK) blocks (themselves containing Asymmetric Convolution Blocks, ACBs), prior to a Squeeze-and-Excitation (SE) fusion head that applies learnable channelwise weights over the concatenated feature vectors. The TFI path captures transient spectral features, while the PSD path models stationary spectral content; the SE block computes per-channel soft attention, enabling dynamic recalibration at the fusion stage (Zeng et al., 19 Jan 2026).

Similarly, in video and EEG, DualEngage and LI-DSN use transformer or backbone networks to extract person-level and scene-level, or temporal and spatial, embeddings, which are then combined by adaptive gating networks—either by softmax-weighted sums or channelwise attention pooling (Chowdhury et al., 11 Apr 2026, Yue et al., 2 Apr 2026).

The fusion rule differs across domains: in image synthesis, fusion may occur spatially via per-pixel softmax gating, as in DLSF’s Adaptive Global Fusion (AGF) (Chen et al., 16 Jul 2025), or by channelwise or vector concatenation followed by a gating MLP as in engagement recognition (Chowdhury et al., 11 Apr 2026).

2. Mathematical Formulations and Gating Mechanisms

A ubiquitous feature of ADFM is the use of learned attention or gating weights. Formally, given two feature vectors or tensors $F_1, F_2$ (with possible spatial, temporal, or channel dimensions), the fusion operation takes the form: $$F_\text{fused} = \alpha \cdot F_1 + \beta \cdot F_2,\quad [\alpha, \beta] = \mathrm{softmax}(f_\text{gate}(F_1, F_2))$$ where $f_\text{gate}$ is a parameterized function (often a lightweight MLP) that computes the input-dependent logits for the softmax. This weighted sum generalizes to spatially varying weights (e.g., in image fusion or COD) and can be extended to more than two streams: $$F_\text{fused} = \sum_{i=1}^{K} \alpha_i \cdot F_i,\;\; \sum_i \alpha_i = 1$$ as seen in outcome prediction for precision medicine (Chen et al., 30 Oct 2025).

Advanced attention modules incorporate multi-head mechanisms (as in cross-attention for boundary-texture fusion (Mbonu et al., 19 May 2026)), spatial softmax (MHENet (Niu et al., 3 Apr 2026)), or channel-/patch-wise pooling (LI-DSN (Yue et al., 2 Apr 2026), IAIFNet (Yang et al., 2023)). In U-Nets for image synthesis, AGF and DSF implement 2D softmax-and-sigmoid gating across spatial maps (Chen et al., 16 Jul 2025).

For cases requiring spatially-varying mixing, e.g., RGB-D fusion in camouflage detection, $$F^m_i = W^r_i \otimes \hat{R}_i + W^d_i \otimes \hat{D}_i$$ where $W^r_i, W^d_i \in \mathbb{R}^{1 \times H\times W}$ are per-location softmax weights (Niu et al., 3 Apr 2026).

3. Domain-Specific Instantiations and Implementation

ADFM variants are widely deployed across distinct modalities:

• Signal & RF: SKANet’s ADFM fuses TFI and PSD features for compound GNSS jamming classification. Its dual-stream feature extractors employ dynamic receptive fields (SK-ACB blocks), with fusion via a channelwise SE block. This enables robust discrimination even at low JNR (–20 dB), demonstrating 96.99% OA (Zeng et al., 19 Jan 2026).
• EEG decoding: LI-DSN’s ADFM aggregates spatially (per-channel static weights) and temporally (softmax temporal attention) across tokenized feature matrices, concatenating the results before final task-specific classification (Yue et al., 2 Apr 2026).
• Medical imaging: HADS-Net applies multi-head cross-attention to gate boundary (edge) features into global texture embeddings, followed by residual plus layer norm, before classification (Mbonu et al., 19 May 2026).
• RGB-D & Multimodal Vision: MHENet’s ADFM at each scale computes cross-modal excitation, spatial weight generation (spatial softmax over modalities), then gated fusion, followed by context refinement and multi-scale aggregation (Niu et al., 3 Apr 2026). IAIFNet’s ADFM leverages differential features and channelwise gating for fusing visible and infrared feature streams under challenging illumination (Yang et al., 2023). In DS_FusionNet for plant disease recognition, adaptive deformable convolutions enable fine spatial alignment and fusion between CNN backbones (Song et al., 29 Apr 2025).
• Synthetic tasks: AGF/DSF in DLSF spatially or globally gate between coarse latent and fine-detail latent, optimizing for global semantic alignment and local texture fidelity (Chen et al., 16 Jul 2025).
• Outcome prediction: For clinical prediction from scarce data, ADFM concatenates softmax-gated projections of high-fidelity, low-fidelity, and deep feature-derived representations, allowing for interpretable fusion reflecting data confidence (Chen et al., 30 Oct 2025).

4. Empirical Impact and Ablative Analysis

A consistent outcome across ADFM employing works is quantifiable, often substantial, performance gain over both single-stream and naïve (e.g., concatenation) fusion approaches.

• SKANet: Eliminating the SE fusion head or one stream degrades OA by 0.8–3.5% (to as low as 93.45%) (Zeng et al., 19 Jan 2026).
• DualEngage: Gated fusion yields ~8% macro-F1 improvement over simple concatenation; single-stream models underperform gated dual-stream by 31–42 absolute F1 points (Chowdhury et al., 11 Apr 2026).
• MHENet: Removing ADFM drops structural similarity and boundary accuracy; spatially adaptive fusion improves S, E, F, and MAE metrics relative to direct concatenation (Niu et al., 3 Apr 2026).
• Precision Medicine: Removal of any ADFM stream (low-fidelity prediction, low-fidelity features, or high-fidelity inputs) increases MSE by 10–22% (Chen et al., 30 Oct 2025).
• Depth Estimation: Elementwise attention-based ADFM reduces KITTI AbsRel to 0.042, a 33% reduction over naïve concat (Meng et al., 2024).

Ablations, across modalities, reveal that learned adaptive gating outperforms fixed-weight, convex, or static-layer fusion.

5. Design Patterns, Theoretical Advantages, and Limitations

Notable properties of ADFM include:

• Data-dependent gating: Weights are dynamically modulated by the joint or individual content of each stream, either globally, channelwise, or pixelwise. This allows the model to prioritize, for instance, depth over RGB for foreground boundaries in camouflaged scenes, or boundary over texture for subtle ultrasound findings.
• Interpretable weighting: In some predictors, the softmax attention coefficients themselves can be interpreted as model trust in various information sources (e.g., clinical features vs. simulator outputs in treatment prediction).
• Suppression of spurious or noisy features: Adaptive fusion can downregulate noisy channels, uninformative patches, or unreliable modalities (as with cross-attention in IAIFNet or channel attention in MHENet).
• Optimization under challenging regimes: Applications in low SNR (SKANet), scarce data (precision medicine (Chen et al., 30 Oct 2025)), or multimodal uncertainty (RGB-D, IR-VIS) benefit from adaptive weighting, as non-stationary local conditions are reflected in the fusion map.

A plausible implication is that where data modalities have complementary but nonuniformly reliable information content, adaptive dual-stream fusion modules offer a quantifiable gain in robustness and generalization. They may, however, increase parameter count and require additional care in regularizing or initializing attention/gating functions.

6. Comparative Summary and Tabular Overview

Domain	Streams/Modalities	Gating/Fusion Mechanism	Key Metric/Impact	arXiv ID
GNSS Interference	TFI + PSD	Channelwise SE after SK-ACB	OA 96.99% (–3.5% without gating)	(Zeng et al., 19 Jan 2026)
Engagement Recognition	Motion + Scene R3D	2-way softmax MLP, weighted sum	↑8% F1 over concat; motion/scene dynamic	(Chowdhury et al., 11 Apr 2026)
EEG Decoding	Temporal + Spatial	Softmax-pool over time, channelwise	–4% acc. w/o gating; best in ablation	(Yue et al., 2 Apr 2026)
RGB-D Camouflage	Texture + Geometry	Spatial softmax per pixel	+1–3% in S/E/F; spatially adaptive	(Niu et al., 3 Apr 2026)
IR-VIS Fusion	Infrared + Visible	Channelwise gating + cross-attn	↑Q_abf/SF/SSIM; better low-light mAP	(Yang et al., 2023)
Image Synthesis	Coarse + Detail Latent	Per-pixel softmax (AGF/DSF)	↓FID ~1.4; ↑Inception Score/recall	(Chen et al., 16 Jul 2025)
Depth Estimation	Single + Multi-View	Per-voxel attention-gated product	↓AbsRel 33% vs. concat	(Meng et al., 2024)
Precision Med. Prediction	High-fid + dual low-fid	3-way softmax, concatenated fuse	10-22% ↑ MSE if any path dropped	(Chen et al., 30 Oct 2025)
Plant Disease Recognition	Two CNN backbones	Deformable conv + channel pooling	+0.43% PlantDisease; ↑ small-sample acc	(Song et al., 29 Apr 2025)
Few-shot Detection	Conv + FC (for query/support)	Task-specific scalar gates	+4.5 AP₅₀ vs. single-branch	(Liu et al., 2020)

7. Future Directions and Open Challenges

Current ADFM research explores increasingly domain-specific gating policies (e.g., multi-head, hierarchical fusion per feature scale; multi-way fusion beyond two streams), as well as integration with transformer architectures and non-convolutional backbones. Open questions remain about optimal placement of fusion (early-, middle-, or late-fusion), scalability to higher modality counts, and interpretability in clinical and scientific applications.

Additionally, theoretical work on learnability, expressivity, and regularization of adaptive gating parameters—as well as methods for quantifying and visualizing the learned attention weights in practical deployments—remains an active area for further study.