Attention Mechanism-based Fusion Algorithm (AMFA)

• AMFA is a family of fusion methods that replaces static addition with adaptive, attention-based weighting of heterogeneous representations.
• It utilizes techniques like weighted complementary mixing and cross-attention to dynamically integrate features in domains such as infrared–visible imaging and multimodal recognition.
• Empirical results show improved performance over fixed-fusion baselines, though challenges remain in generalization and optimal multi-source integration.

Attention Mechanism-based Fusion Algorithm (“AMFA”; Editor’s term) denotes a class of fusion methods in which attention, gating, or content-aware routing is used to combine heterogeneous representations instead of relying on fixed addition or concatenation. In the supplied literature, the term is most directly associated with infrared–visible image fusion through AMFusionNet, which is described as using multiple convolutional kernels, parallel attention mechanisms, and the multi-scale structural similarity (MS-SSIM) loss for IVIF (Xu, 2023). More broadly, closely related designs appear in feature fusion, multimodal emotion recognition, explanation-map generation, tracking, robotics, and multi-sensor ego-motion estimation, which suggests that AMFA is best understood as a design family rather than a single canonical architecture (Dai et al., 2020, Ntrougkas et al., 2023, Moughnieh et al., 23 Jun 2026).

1. Conceptual scope

A central premise of AMFA-style methods is that fusion is not a neutral algebraic step. In “Attentional Feature Fusion,” feature fusion is described as “an omnipresent part of modern network architectures,” yet it is often implemented by “simple operations, such as summation or concatenation,” which “might not be the best choice” (Dai et al., 2020). The AMFA viewpoint replaces such fixed aggregation with adaptive, data-dependent weighting.

This design appears at several granularities. Some systems fuse branches or layers inside a CNN, as in AFF and TAME (Dai et al., 2020, Ntrougkas et al., 2023). Some fuse modalities, such as image and text for outfit recommendation, audio and video for emotion recognition, RGB and depth for robot manipulation, or infrared and visible imagery for image fusion (Laenen et al., 2019, Zhou et al., 2021, Shen et al., 2024, Li et al., 2024). Others fuse entire sensors, as in ADM-Fusion, which dynamically weights RGB, LiDAR, IMU, and radar for ego-motion estimation (Moughnieh et al., 23 Jun 2026). Inference: the unifying feature is not the application domain but the substitution of static fusion rules by attention-mediated selection.

A common misconception is that AMFA is synonymous with transformer cross-attention. The surveyed work is broader. It includes channel attention, spatial attention, operation-wise weighting, hierarchical branch fusion, factorized bilinear pooling, and content-aware routing (Qin et al., 27 Apr 2025, Charitidis et al., 2021, Zhou et al., 2021, Lu et al., 2024). Inference: “attention mechanism-based fusion” names a functional role—deciding what to fuse and how much—rather than a single operator.

2. Architectural primitives and fusion operators

One canonical AMFA form is weighted complementary mixing. AFF defines fusion for two aligned feature maps $$\mathbf{X}, \mathbf{Y} \in \mathbb{R}^{C \times H \times W}$$ as

$$\mathbf{Z} = \mathbf{M}(\mathbf{X} \uplus \mathbf{Y}) \otimes \mathbf{X} + \left(1 - \mathbf{M}(\mathbf{X} \uplus \mathbf{Y})\right) \otimes \mathbf{Y},$$

where $\mathbf{M}(\cdot)$ is an MS-CAM-based attention module and $\uplus$ denotes initial integration, instantiated in the simplest case as $\mathbf{X} + \mathbf{Y}$ (Dai et al., 2020). This formulation makes fusion a soft selection problem rather than a fixed sum.

A second form is joint spatial–channel recalibration. MIA-Mind takes an intermediate tensor $\mathbf{X} \in \mathbb{R}^{C \times H \times W}$, computes a channel descriptor

$$\mathbf{z}_c = \frac{1}{H \cdot W}\sum_{i=1}^{H}\sum_{j=1}^{W}\mathbf{X}_{c,i,j},$$

and a spatial descriptor

$$\mathbf{M}_{i,j} = \frac{1}{C}\sum_{c=1}^{C}\mathbf{X}_{c,i,j}.$$

The channel branch uses a two-layer bottleneck MLP, the spatial branch uses a $7\times7$ convolution, and the joint attention is formed multiplicatively:

$$\mathcal{A}_{c,i,j} = \mathbf{w}_c[c]\cdot \mathbf{w}_s[i,j].$$

The output is then

$$\mathbf{Z} = \mathbf{M}(\mathbf{X} \uplus \mathbf{Y}) \otimes \mathbf{X} + \left(1 - \mathbf{M}(\mathbf{X} \uplus \mathbf{Y})\right) \otimes \mathbf{Y},$$0

This explicitly couples “what” is important with “where” it is important (Qin et al., 27 Apr 2025).

A third form is complementarity-oriented cross-attention. CrossFuse argues that ordinary cross-attention emphasizes correlation, whereas infrared–visible fusion requires emphasis on complementarity or uncorrelation. Its defining modification is

$$\mathbf{Z} = \mathbf{M}(\mathbf{X} \uplus \mathbf{Y}) \otimes \mathbf{X} + \left(1 - \mathbf{M}(\mathbf{X} \uplus \mathbf{Y})\right) \otimes \mathbf{Y},$$1

so that cross-attention attends to less-correlated features rather than the most similar ones (Li et al., 2024). This is paired with modality-specific self-attention, a shift operation between self-attention blocks, and a decoder that injects intensity-aware shallow and deep skip features.

A fourth form is multiplicative multimodal interaction. In AM-FBP, global audio and video emotion vectors are first produced by attention, then fused with factorized bilinear pooling:

$$\mathbf{Z} = \mathbf{M}(\mathbf{X} \uplus \mathbf{Y}) \otimes \mathbf{X} + \left(1 - \mathbf{M}(\mathbf{X} \uplus \mathbf{Y})\right) \otimes \mathbf{Y},$$2

with adaptive weights

$$\mathbf{Z} = \mathbf{M}(\mathbf{X} \uplus \mathbf{Y}) \otimes \mathbf{X} + \left(1 - \mathbf{M}(\mathbf{X} \uplus \mathbf{Y})\right) \otimes \mathbf{Y},$$3

This makes modality weighting sample-dependent without adding new trainable parameters in AG-FBP (Zhou et al., 2021).

A fifth form is operation-wise or structure-wise routing. OwAF uses parallel operations—$$\mathbf{Z} = \mathbf{M}(\mathbf{X} \uplus \mathbf{Y}) \otimes \mathbf{X} + \left(1 - \mathbf{M}(\mathbf{X} \uplus \mathbf{Y})\right) \otimes \mathbf{Y},$$4, $$\mathbf{Z} = \mathbf{M}(\mathbf{X} \uplus \mathbf{Y}) \otimes \mathbf{X} + \left(1 - \mathbf{M}(\mathbf{X} \uplus \mathbf{Y})\right) \otimes \mathbf{Y},$$5, $$\mathbf{Z} = \mathbf{M}(\mathbf{X} \uplus \mathbf{Y}) \otimes \mathbf{X} + \left(1 - \mathbf{M}(\mathbf{X} \uplus \mathbf{Y})\right) \otimes \mathbf{Y},$$6, and $$\mathbf{Z} = \mathbf{M}(\mathbf{X} \uplus \mathbf{Y}) \otimes \mathbf{X} + \left(1 - \mathbf{M}(\mathbf{X} \uplus \mathbf{Y})\right) \otimes \mathbf{Y},$$7 convolutions, average pooling, and max pooling—and weights them by attention before concatenation and a $$\mathbf{Z} = \mathbf{M}(\mathbf{X} \uplus \mathbf{Y}) \otimes \mathbf{X} + \left(1 - \mathbf{M}(\mathbf{X} \uplus \mathbf{Y})\right) \otimes \mathbf{Y},$$8 convolution (Charitidis et al., 2021). AFter extends the idea to dynamic fusion-structure selection through SEU, CEU, CMEU$$\mathbf{Z} = \mathbf{M}(\mathbf{X} \uplus \mathbf{Y}) \otimes \mathbf{X} + \left(1 - \mathbf{M}(\mathbf{X} \uplus \mathbf{Y})\right) \otimes \mathbf{Y},$$9, and CMEU$\mathbf{M}(\cdot)$0, with a router

$\mathbf{M}(\cdot)$1

that predicts combination weights for fusion units (Lu et al., 2024). ADM-Fusion applies the same principle at sensor level by combining per-sensor experts with a content-aware router and a mixture

$\mathbf{M}(\cdot)$2

inside separate translation and rotation branches (Moughnieh et al., 23 Jun 2026).

3. Optimization regimes and objective design

AMFA-style systems typically couple the fusion operator to a task-specific objective rather than training attention in isolation. In CrossFuse, training is explicitly two-stage. Stage 1 trains two auto-encoders, one for infrared and one for visible, with

$\mathbf{M}(\cdot)$3

where $\mathbf{M}(\cdot)$4. Stage 2 freezes the encoders and trains CAM plus the decoder with

$\mathbf{M}(\cdot)$5

where $\mathbf{M}(\cdot)$6; the intensity term preserves locally dominant modality content and the gradient term preserves stronger edge/detail responses (Li et al., 2024).

TAME uses a different regime because the target CNN is frozen and only the attention module is optimized. Its training loss is

$\mathbf{M}(\cdot)$7

with $\mathbf{M}(\cdot)$8, $\mathbf{M}(\cdot)$9, $\uplus$0, and $\uplus$1 for the area term exponent (Ntrougkas et al., 2023). Here attention is supervised to produce compact, smooth, class-discriminative explanation maps rather than generic saliency.

Temporal AMFA designs embed fusion into sequence modeling. The Differential Attention Fusion Model divides the input into forward, center, and backward parts, computes difference features $\uplus$2 and $\uplus$3, applies neighbor attention, then a sliding fusion mechanism and a residual layer with a 1D convolution and stacked LSTMs (Li et al., 2022). ADM-Fusion trains adaptive routing with frame-to-frame and frame-to-global pose losses, together with a balance regularizer

$\uplus$4

to prevent router collapse (Moughnieh et al., 23 Jun 2026).

The IVIF paper on AMFusionNet states, at abstract level, that the integration of the MS-SSIM loss “refines network training” and “optimizes the model for IVIF task,” but the supplied material does not include the underlying equations, architecture description, or numerical benchmarks beyond that statement (Xu, 2023). This absence is bibliographically relevant because it limits article-level reconstruction of that specific model.

4. Representative application domains

The literature shows that AMFA-style fusion is not domain-specific. It recurs wherever heterogeneous signals, scales, or layers must be combined under variable relevance.

Domain	Representative system	Key fusion target
Infrared–visible image fusion	AMFusionNet; CrossFuse	thermal saliency and visible texture
General CNN feature fusion	AFF; iAFF	same-layer, short-skip, long-skip features
Audio-visual emotion recognition	AM-FBP	global and intra-trunk emotion cues
RGBT tracking	AFter	dynamic RGB–thermal fusion structure
Multi-sensor ego-motion	ADM-Fusion	RGB, LiDAR, IMU, radar streams
Explainability	TAME	multi-layer CNN feature maps
Tampering localization	OwAF	five forensic localization maps

Beyond these examples, attention-based fusion is used for outfit recommendation, where image regions and description-level or word-level features are fused into an item representation for compatibility prediction (Laenen et al., 2019). It is also used in super-resolution, where non-local information and second-order features are fused with multi-scale $\uplus$5, $\uplus$6, and $\uplus$7 convolutions (Lyn et al., 2020). In robotics, Focal-CVAE fuses RGB and depth through a mixed focal attention module and reduces sequence-processing cost with saliency attention (Shen et al., 2024). In GPR defect detection, MCGA-Net combines MCFF in the neck with GAM in the backbone to improve multi-scale and global-context modeling (Lv et al., 25 Dec 2025).

This distribution across domains suggests a stable taxonomy. Some AMFA systems fuse modalities; some fuse layers; some fuse operations; some fuse sensors; and some fuse tasks, as in ADM-Fusion’s coupled translation and rotation branches (Moughnieh et al., 23 Jun 2026). The common engineering problem is inconsistent reliability across sources.

5. Empirical behavior and recurring performance patterns

Quantitatively, AMFA-style designs often outperform fixed-fusion baselines, but the character of the gain depends on the task. In infrared–visible image fusion, CrossFuse reports on TNO: EN = 6.8389, SD = 73.4712, MI = 13.6779, FMI$\uplus$8 = 0.3866, FMI$\uplus$9 = 0.9044, and SCD = 1.7659, with best EN, best SD, best MI, and best FMI$\mathbf{X} + \mathbf{Y}$0. On VOT-RGBT it reports EN = 6.8908, SD = 77.1780, MI = 13.7816, FMI$\mathbf{X} + \mathbf{Y}$1 = 0.3827, FMI$\mathbf{X} + \mathbf{Y}$2 = 0.9061, and SCD = 1.6635, with best SD and best FMI$\mathbf{X} + \mathbf{Y}$3 (Li et al., 2024).

In feature recalibration, MIA-Mind reports 82.9% accuracy on CIFAR-10, 78.7% accuracy and Dice coefficient 0.876 on ISBI2012, and 91.9% accuracy on CIC-IDS2017, together with 0.831 precision, 0.829 recall, and 0.828 F1 for CIFAR-10, and 0.989 precision, 0.745 recall, and 0.849 F1 for CIC-IDS2017 (Qin et al., 27 Apr 2025). In general CNN feature fusion, AFF and especially iAFF improve over addition and concatenation with modest overhead; the appendix reports ResNet-50 at 4.1 GFlops and AFF-ResNet-50 at 4.3 GFlops, a 4.88% FLOP increase, while ImageNet top-1 error for iAFF-ResNet-50 is 20.4 with 35.1M parameters (Dai et al., 2020).

In multimodal recognition, AM-FBP achieves the best reported test accuracy of 63.09% on AFEW and 75.49% on IEMOCAP (Zhou et al., 2021). In tracking, AFter reports 90.1 PR / 66.7 SR on RGBT234 and 84.9 PR / 72.5 SR on VTUAV, with ablations showing that removing routers substantially degrades performance (Lu et al., 2024). In ego-motion estimation, adding radar in ADM-Fusion’s four-sensor configuration reduces average translational RPE from 26.93 mm to 9.57 mm, about a 64% reduction, while only slightly increasing rotational RPE (Moughnieh et al., 23 Jun 2026).

The empirical record is not uniformly positive. OwAF is strongest on DEFACTO, with Macro-F1 = 0.912, F1 (tampered) = 0.829, and IoU = 0.707, and it also exceeds the knowledge-based fusion baseline on CASIA v2.0. However, on the First IFS-TC dataset, Splicebuster remains best and the knowledge-based fusion baseline exceeds OwAF (Charitidis et al., 2021). This indicates that attention-based fusion can substantially improve in-distribution performance while still exposing generalization gaps.

Inference: recurring gains appear when attention addresses one of three bottlenecks—scale mismatch, modality imbalance, or unreliable fixed fusion structure. The strongest results typically arise when the fusion policy itself becomes adaptive, as in iAFF, AFter, and ADM-Fusion (Dai et al., 2020, Lu et al., 2024, Moughnieh et al., 23 Jun 2026).

6. Limitations, misconceptions, and open directions

One limitation is terminological. AMFA is not a standardized architecture name in the surveyed corpus. It is closest, in nomenclature, to AMFusionNet in IVIF (Xu, 2023), but the broader literature spans attention masks, split attention, bilinear pooling, routing, and cross-task coupling (Qin et al., 27 Apr 2025, Zhou et al., 2021, Moughnieh et al., 23 Jun 2026). Inference: the term is useful as a taxonomic umbrella but should not be mistaken for a single reproducible reference implementation.

A second limitation is generalization under distribution shift. OwAF performs strongly on DEFACTO and CASIA v2.0 but not on First IFS-TC, and the paper explicitly notes overfitting to the manipulations seen in DEFACTO (Charitidis et al., 2021). ADM-Fusion likewise notes that adaptive routing offers less benefit when sensor quality is uniformly high and similar across modalities, especially in clean simulation (Moughnieh et al., 23 Jun 2026). CrossFuse states that its CAM is “simple and efficient,” but “does not dramatically transform performance in the transformer framework,” leaving room for sparse representation and metric learning (Li et al., 2024).

A third limitation is that adding more branches or more layers does not automatically improve fusion. TAME reports that adding a second feature-map set improves results over a single set, but adding a third set is not always monotonic because earlier layers may inject extra noise (Ntrougkas et al., 2023). AFF observes that the initial integration step can itself become a bottleneck, motivating iterative attentional feature fusion rather than a single attention stage (Dai et al., 2020).

The IVIF case that names AMFusionNet also illustrates a bibliographic limitation. The supplied material contains the abstract-level claim that the method uses multiple convolutional kernels, parallel attention mechanisms, and MS-SSIM loss, and that experiments show improvement on publicly available datasets, but it does not contain the actual method description, equations, training setup, experiments, or ablation results needed for a full technical reconstruction (Xu, 2023). For encyclopedia purposes, this means the AMFA concept is more fully documented by the surrounding attention-fusion literature than by that single paper alone.

Open directions are stated explicitly in several works. MIA-Mind proposes future extension to large-scale datasets, adaptive attention fusion strategies, and distributed deployment (Qin et al., 27 Apr 2025). CrossFuse proposes sparse representation and metric learning (Li et al., 2024). ADM-Fusion motivates further robustness under changing environmental conditions and sensor degradation (Moughnieh et al., 23 Jun 2026). A plausible implication is that the next phase of AMFA research will be less about introducing yet another attention block and more about calibrating when, where, and at what granularity fusion should occur.