Gated Adaptive Fusion Module (GAFM)

• GAFM is a family of learned fusion mechanisms that use input-dependent gating to replace fixed combination rules.
• It encompasses various approaches—from sigmoid-based interpolation to softmax-normalized expert weighting—to selectively integrate multimodal features.
• GAFM enhances robustness and performance in tasks like sentiment analysis, action recognition, tracking, and remote sensing by regulating information flow.

A Gated Adaptive Fusion Module (GAFM) denotes a family of learned fusion mechanisms that replace fixed combination rules—such as simple concatenation, averaging, or static weighting—with input-dependent gating over multiple feature streams. In the literature, the term is not standardized: some papers explicitly define a GAFM, while others describe closely related units under names such as Gated Multimodal Unit (GMU), Gated Fusion Unit (GFU), Gated Projection Fusion (GPF), Gated Feature Fusion Module (GFFM), or Cross-Attention Gated (CAG) module. Across these variants, the common function is to regulate how much information from one source should be preserved, suppressed, or injected into another, often to improve robustness under noise, missing modalities, misalignment, or modality-specific degradation (Arevalo et al., 2017, Liu et al., 2021, Song et al., 2024, Wen et al., 20 Aug 2025, You et al., 15 Apr 2026).

1. Terminology, lineage, and scope

One early and influential formulation is the GMU, introduced as an internal unit for multimodal learning that “learns to decide how modalities influence the activation of the unit using multiplicative gates” (Arevalo et al., 2017). In that model, the gate is computed from the concatenated modalities and interpolates between modality-specific hidden states, establishing the basic pattern later reused across many GAFM-like designs. In parallel, two-stream action-recognition work framed gated fusion explicitly through mixture-of-experts language: the spatial and temporal streams act as experts, and a gating network produces sample-specific fusion weights rather than relying on a fixed ratio chosen on a validation set (Zhu et al., 2017).

Subsequent work expanded this principle across domains. In deformable object tracking, gating was used to arbitrate between original convolutional features and deformable features, because deformable convolution was helpful under pose change and nonrigid deformation but not uniformly reliable under occlusion or illumination change (Liu et al., 2018). In dynamic saliency prediction, gated fusion replaced static spatial-temporal fusion with a content-dependent, spatially varying rule over RGB and optical flow streams (Kocak et al., 2021). In remote sensing, the GFU in MultiModNet addressed unequal, redundant, noisy, missing, or contradictory modalities by letting the primary modality guide the secondary modality during early merging (Liu et al., 2021).

More papers apply the same core idea to middle-level LiDAR-radar fusion on BEV feature maps, adaptive fusion in multimodal sentiment analysis, RGB-event tracking, cross-domain audio representation, and CSI prediction (Song et al., 2024, Wen et al., 20 Aug 2025, Wu et al., 2 Oct 2025, You et al., 15 Apr 2026, Lin et al., 25 Jun 2026). This suggests that GAFM is best understood not as a single canonical layer, but as a recurrent design principle for context-sensitive arbitration among heterogeneous representations.

2. Core computational forms

A common GAFM form is sigmoid-gated interpolation between two latent representations. The GMU defines modality-specific hidden states

$$h_v = \tanh\left(W_v\cdot x_v\right), \qquad h_t = \tanh\left(W_t\cdot x_t\right),$$

computes

$$z = \sigma\left(W_z \cdot \left[x_v, x_t\right] \right),$$

and fuses them as

$h = z * h_v + \left(1 - z \right) * h_t.$

Here the gate decides, per example, whether the visual or textual transformed state should dominate the fused representation (Arevalo et al., 2017).

PGF-Net applies the same logic after cross-modal extraction rather than on raw modalities. Its adaptive gate is computed from the concatenation of the self-attended text state and the cross-attended multimodal state,

$$g = \sigma\left(W_g [H_{\text{text}}; H_{\text{cross}}] + b_g\right),$$

with final fusion

$$H_{\text{fused}} = g \odot H_{\text{text}} + (1-g)\odot H_{\text{cross}}.$$

The paper emphasizes that the gate is computed per hidden dimension, so values near 1 preserve the original linguistic feature and values near 0 favor multimodal enrichment (Wen et al., 20 Aug 2025). A closely related formulation appears in the CSI predictor, where each GRU hidden state is fused with a global Luong attention context through

$$g_\tau = \sigma\!\left(W_2 \operatorname{ReLU}(W_1 u_\tau)\right), \qquad h'_\tau = g_\tau \odot h_\tau + (1-g_\tau)\odot c,$$

with $u_\tau=[h_\tau;c]$ and a bottleneck reduction ratio $r=4$ (Hussain et al., 7 May 2026).

A second family uses softmax-normalized expert weighting. In adaptive fusion for human action recognition, the gating network produces raw scores $w_i$, normalizes them as

$$z_i = \frac{e^{w_i - \max(\mathbf{w})}}{\sum_{j=1}^{n} e^{w_j - \max(\mathbf{w})}}, \qquad \sum_{j=1}^{n}z_j = 1,$$

and computes

$$z = \sigma\left(W_z \cdot \left[x_v, x_t\right] \right),$$0

The same broad mixture-of-experts structure is used in AECF, where a simplex-weight vector $$z = \sigma\left(W_z \cdot \left[x_v, x_t\right] \right),$$1 weights modality-specific projections, but the entropy regularization coefficient $$z = \sigma\left(W_z \cdot \left[x_v, x_t\right] \right),$$2 is instance-dependent and derived from predictive variance (Yudistira, 4 Dec 2025, Chlon et al., 21 May 2025).

A third family applies gates at spatial or channel granularity. LiRaFusion concatenates LiDAR and radar BEV feature maps,

$$z = \sigma\left(W_z \cdot \left[x_v, x_t\right] \right),$$3

predicts channel-specific weight maps

$$z = \sigma\left(W_z \cdot \left[x_v, x_t\right] \right),$$4

modulates each branch by element-wise multiplication, and concatenates the gated outputs into the fused feature map (Song et al., 2024). GFSalNet generates a probability map $$z = \sigma\left(W_z \cdot \left[x_v, x_t\right] \right),$$5, defines complementary gates $$z = \sigma\left(W_z \cdot \left[x_v, x_t\right] \right),$$6 and $$z = \sigma\left(W_z \cdot \left[x_v, x_t\right] \right),$$7, and fuses spatial and temporal saliency representations by weighted summation (Kocak et al., 2021). GDT predicts a spatial gate $$z = \sigma\left(W_z \cdot \left[x_v, x_t\right] \right),$$8 and forms

$$z = \sigma\left(W_z \cdot \left[x_v, x_t\right] \right),$$9

thereby blending deformable and original convolutional features online (Liu et al., 2018).

Cross-attention-gated forms generalize these ideas further. AG-Fusion computes bidirectional cross-attention between camera and LiDAR BEV windows, generates a pixel-wise gate

$h = z * h_v + \left(1 - z \right) * h_t.$0

and fuses the directional representations as

$h = z * h_v + \left(1 - z \right) * h_t.$1

GAFusion similarly generates a BEV fusion weight map $h = z * h_v + \left(1 - z \right) * h_t.$2 from projected LiDAR and camera features and uses it inside a LiDAR-guided adaptive fusion transformer (Liu et al., 27 Oct 2025, Li et al., 2024).

3. Placement within network architectures

GAFM-like modules are usually inserted after modality-specific encoding and before task heads, but their exact placement varies substantially. In PGF-Net, the gate appears inside each Cross-Attention Gated Fusion Layer, after text self-attention and cross-attention and before the post-fusion adapter. The paper describes a hierarchical encoder in which early layers are plain text-only Transformer layers and fusion begins from a configurable depth $h = z * h_v + \left(1 - z \right) * h_t.$3; in the reported setup, fusion starts at the first BERT layer with $h = z * h_v + \left(1 - z \right) * h_t.$4, the backbone is bert-base-uncased, LoRA rank is 32, and the post-fusion adapter bottleneck dimension is 64 (Wen et al., 20 Aug 2025).

In MultiModNet, the GFU sits between the modality-specific encoders and the later fusion/decoder pathway. The PAF module first extracts fused contextual features $h = z * h_v + \left(1 - z \right) * h_t.$5 from the primary modality, and the GFU then uses $h = z * h_v + \left(1 - z \right) * h_t.$6 to generate a gate for the secondary modality’s low-level features $h = z * h_v + \left(1 - z \right) * h_t.$7. The updated $h = z * h_v + \left(1 - z \right) * h_t.$8 feeds the remaining encoder layers and also serves as one of the inputs to PAF, so the gate acts as an early merging or feature-bridging block rather than the final fusion stage (Liu et al., 2021).

In LiRaFusion, GAFM is explicitly a middle-fusion module. An early-fusion branch stacks LiDAR and radar at the input level and processes them through a joint voxel feature encoder and sparse convolution backbone, while a parallel PointPillars-based radar backbone produces a radar feature map. GAFM then fuses the two BEV feature maps before the FPN neck and detection head (Song et al., 2024). GAFusion organizes the pipeline even more deeply: LiDAR guidance first improves camera 3D features through sparse depth guidance and LiDAR occupancy guidance, MSDPT enlarges receptive fields, and only then does the LiDAR-guided adaptive fusion transformer fuse camera and LiDAR BEV features (Li et al., 2024).

Other placements are task-specific. GFSalNet uses gated fusion as the final integration mechanism after its spatial and temporal streams, multi-level information block, spatial attention, and channel-wise attention (Kocak et al., 2021). MambaTrack performs modality-specific feature extraction with a Vision Mamba/static SSM branch for RGB and a DSSM branch for events, then applies GPF before the tracking head (You et al., 15 Apr 2026). The CSI predictor applies gated fusion after a GRU encoder and Luong attention but before the Dimension-wise Separable Linear Head (Hussain et al., 7 May 2026). This distribution of placements suggests that the gate is usually introduced where the architecture first exposes a semantically meaningful tension between complementary but unequal feature sources.

4. Functional role: robustness, selectivity, and information-flow regulation

Across domains, the stated motivation for GAFM is rarely mere feature aggregation; it is selective control under unequal modality quality. PGF-Net explicitly argues that naive fusion can let noisy non-linguistic signals overwhelm text or dilute useful multimodal cues by treating all sources equally. Its gate is therefore framed as a dynamic controller that arbitrates between original linguistic semantics and cross-attended multimodal enrichment, especially because audio and visual streams can be incomplete, misaligned, or noisy while text is often the most stable carrier of semantic content (Wen et al., 20 Aug 2025).

The same logic appears in action recognition and remote sensing, but with different failure modes. Adaptive fusion for human action recognition argues that “simple averaging” or fixed weights are suboptimal because the best modality contribution changes from sample to sample, and because visual, motion, audio, depth, skeleton, or sensor cues do not remain equally reliable across contexts (Yudistira, 4 Dec 2025). MultiModNet states that unequal, redundant, noisy, missing, or contradictory modalities make simple summation or concatenation too crude; its GFU allows the primary modality to guide the secondary modality so that supplementary information is retained without amplifying noise (Liu et al., 2021).

In tracking and event-based fusion, the gate often manages reliability shifts over time. GDT notes that deformable convolution helps under pose change, rotation, and nonrigid deformation, but can degrade scale estimation or localization under severe occlusion or illumination change; the gate decides when deformable features should influence the original appearance representation (Liu et al., 2018). MambaTrack generates its gate from event density and RGB confidence, making fusion simultaneously density-aware and confidence-aware. The same event density signal $h = z * h_v + \left(1 - z \right) * h_t.$9 is also used to modulate the dynamic state transition in the event branch, coupling temporal modeling and fusion intensity (You et al., 15 Apr 2026).

Some recent work extends the role of gating from robustness to calibration. AECF places an entropy-gated fusion layer on top of frozen encoders, adapts the entropy coefficient per instance, enforces monotone calibration across all modality subsets through Contrastive Expert Calibration, and drives an Adaptive Curriculum Masking scheme directly from training-time gate entropy (Chlon et al., 21 May 2025). This suggests a broader interpretation of GAFM: beyond feature selection, gating can be used to shape confidence behavior under missing-input conditions.

5. Empirical behavior across application domains

In multimodal sentiment analysis, the contribution of gating is directly quantified. On MOSI, full PGF-Net reports MAE 0.691, Corr 0.809, Acc-7 49.4, Acc-2 86.8, and F1 86.9, whereas removing the gate degrades performance to MAE 0.710, Corr 0.796, Acc-7 47.5, Acc-2 85.7, and F1 85.8; the full model does so with only 3.09M trainable parameters (Wen et al., 20 Aug 2025). AGFN reports, on CMU-MOSI, Acc-2 82.75, F1 82.68, Acc-7 48.69, and MAE 71.02; removing the Information Entropy Gate lowers Acc-2 to 81.95, Acc-7 to 45.36, and worsens MAE to 73.13, while removing the Modality Importance Gate gives Acc-2 82.56, Acc-7 46.35, and MAE 72.53 (Wu et al., 2 Oct 2025).

In video understanding, saliency, and action recognition, adaptive gating consistently outperforms static fusion. GFSalNet reports that replacing gated fusion with a standard $$g = \sigma\left(W_g [H_{\text{text}}; H_{\text{cross}}] + b_g\right),$$0 convolution lowers performance on UCF-Sports from AUC-J 0.914, CC 0.526, NSS 3.333, SIM 0.382, and KLDiv 1.516 to AUC-J 0.900, CC 0.480, NSS 2.913, SIM 0.353, and KLDiv 1.676 (Kocak et al., 2021). In human action recognition with RGB and optical flow, the best fixed weighting shown is 90.02% at RGB/Flow $$g = \sigma\left(W_g [H_{\text{text}}; H_{\text{cross}}] + b_g\right),$$1, whereas the gated version achieves 91% (Yudistira, 4 Dec 2025). In two-stream ConvNets on UCF101, Gated TSN reaches 94.5% mean accuracy, compared with 94.0% for original TSN with 2 modalities and 94.2% for TSN with 3 modalities; on split 1, gated fusion gives 94.11 versus 93.81 for weighted averaging and 93.96 for SCI (Zhu et al., 2017).

In 3D detection and tracking, the same pattern appears under spatially varying modality reliability. LiRaFusion reports NDS/mAP of 66.59/59.45 for middle fusion alone and 66.69/60.11 for the full model, compared with 65.74/59.39 for FUTR3D-LO; channel-specific gating also improves over channel-constant gating, 66.59/59.45 versus 66.01/58.84 (Song et al., 2024). GAFusion compares fusion strategies on nuScenes val and reports 70.85/72.82 for ADD, 70.92/72.88 for CAT, 71.63/73.32 for LGFT, and 71.79/73.43 for LGAFT, with the full model reaching 73.6% mAP and 74.9% NDS on the nuScenes test set (Li et al., 2024). AG-Fusion reports, for the Bucket category on E3D, 52.62 AP$$g = \sigma\left(W_g [H_{\text{text}}; H_{\text{cross}}] + b_g\right),$$2 with ConvFuser, 67.54 with fixed $$g = \sigma\left(W_g [H_{\text{text}}; H_{\text{cross}}] + b_g\right),$$3, 61.09 with fixed $$g = \sigma\left(W_g [H_{\text{text}}; H_{\text{cross}}] + b_g\right),$$4, and 77.50 with AdaptiveGate; the paper also reports 93.92% accuracy on KITTI and a 24.88% improvement over the baseline on E3D (Liu et al., 27 Oct 2025). In tracking, GDT improves OTB-2013 success from 0.701 for the baseline to 0.711 when the gate is added on top of deformable convolution (Liu et al., 2018), while MambaTrack drops from SR 42.5% and PR 54.0% to SR 41.9% and PR 53.4% when GPF is removed (You et al., 15 Apr 2026).

Remote sensing, audio representation, CSI prediction, and socioeconomic inference show similarly task-specific gains. In MultiModNet, GFU improves mF1 by about +1.1 to +1.4% over point-wise summation and concatenation, gives about +2.2 to +3.3% on small objects, and speeds training convergence by about 2× (Liu et al., 2021). WQ-Fusion reports an overall score of 0.836, compared with 0.820 for Concat., 0.829 for Adapt. and Trans., 0.832 for Gated Trans., 0.796 for Qwen2-Audio-7B, and 0.754 for Whisper-large (Lin et al., 25 Jun 2026). The CSI predictor reaches an average NMSE of $$g = \sigma\left(W_g [H_{\text{text}}; H_{\text{cross}}] + b_g\right),$$5 dB with 26% fewer parameters and approximately 2.3x higher inference throughput than a dimension-matched LinFormer baseline; its gated fusion ablation improves over fixed linear fusion from $$g = \sigma\left(W_g [H_{\text{text}}; H_{\text{cross}}] + b_g\right),$$6 dB with 1.32M parameters to $$g = \sigma\left(W_g [H_{\text{text}}; H_{\text{cross}}] + b_g\right),$$7 dB with 1.24M parameters (Hussain et al., 7 May 2026). In poverty prediction from satellite imagery, the Gated-Attention Feature-Fusion based framework reports a 75% $$g = \sigma\left(W_g [H_{\text{text}}; H_{\text{cross}}] + b_g\right),$$8 score and states improvements of 74% over Jean et al., 55% over Li et al., and 60% over Rekha et al. (Ramzan et al., 2024). AECF, finally, reports that on AV-MNIST and MS-COCO it improves masked-input mAP by +18 pp at a 50% drop rate while reducing ECE by up to 200%, with 1% run-time overhead and frozen backbones (Chlon et al., 21 May 2025).

6. Conceptual distinctions and recurring misconceptions

A recurring source of confusion is terminological. MultiModNet explicitly names its block GFU rather than GAFM, MambaTrack uses GPF, SFAFNet uses GFM inside GSFFBlock, PGF-Net describes an Adaptive Gated Arbitration mechanism, and AG-Fusion distinguishes between the framework name AG-Fusion and the internal CAG module (Liu et al., 2021, You et al., 15 Apr 2026, Gao et al., 20 Feb 2025, Wen et al., 20 Aug 2025, Liu et al., 27 Oct 2025). Treating these modules as identical would be inaccurate; treating them as unrelated would also miss their shared gating logic.

Another misconception is that GAFM always fuses raw modalities directly. In PGF-Net, the gate operates on the self-attended text state and the cross-attended multimodal state, not on raw audio and video (Wen et al., 20 Aug 2025). In MultiModNet, the gate updates the secondary modality’s low-level feature map using fused contextual features from the primary modality (Liu et al., 2021). In LiRaFusion and GAFusion, gating occurs on BEV feature maps after substantial modality-specific processing and, in GAFusion, after explicit geometry guidance (Song et al., 2024, Li et al., 2024). In CSI prediction, the gate blends local recurrent features with a global attention context rather than multiple external sensors (Hussain et al., 7 May 2026).

It is also incorrect to assume that all gates are normalized convex weights. Some formulations use softmax so that weights sum to 1, as in mixture-of-experts fusion for action recognition and AECF (Yudistira, 4 Dec 2025, Chlon et al., 21 May 2025). Many others use independent sigmoid gates, complementary gates $$g = \sigma\left(W_g [H_{\text{text}}; H_{\text{cross}}] + b_g\right),$$9 and $$H_{\text{fused}} = g \odot H_{\text{text}} + (1-g)\odot H_{\text{cross}}.$$0, or even ReLU-based nonnegative weights that do not need to sum to 1, as in the gating ConvNet for two-stream action recognition (Kocak et al., 2021, Zhu et al., 2017). The granularity likewise varies: scalar, vector, per hidden dimension, per channel, per spatial location, or per pixel.

The broader literature therefore supports a precise but non-monolithic definition. A GAFM is not a single equation, a single placement, or a single modality recipe. It is a design pattern in which fusion is made conditional on learned estimates of contextual relevance, reliability, uncertainty, or complementarity. The most stable common property is that the module regulates information flow rather than merely stacking features, and the strongest empirical evidence appears when modality quality varies across samples, locations, time steps, or environmental conditions (Arevalo et al., 2017, Song et al., 2024, Wen et al., 20 Aug 2025, Wu et al., 2 Oct 2025).