---
title: Mutual-Guided Fusion Module (MGFM)
url: https://www.emergentmind.com/topics/mutual-guided-fusion-module-mgfm
type: topic
---

# Mutual-Guided Fusion Module (MGFM)

Searching arXiv for the cited MGFM-related papers to ground the article in published sources.
I’m checking the available arXiv records for the MGFM variants mentioned in the source material.
Mutual-Guided Fusion Module (MGFM) denotes a class of learned fusion blocks used to combine information from multiple feature streams, but the acronym is not tied to a single standardized architecture in the arXiv literature. In the cited works, MGFM refers to at least three distinct modules: a **feature mutual mapping fusion module** for unsupervised image fusion, a **Multi-view Global-based Fusion Module** for multi-view echocardiogram video segmentation, and a **Multi-Head Attention Gated Fusion Module** for event-image semantic segmentation; a closely related mutual-learning fusion block also appears in gait recognition [2201.10152], [2309.11144], [2507.21971], [2407.14812]. Across these settings, the common objective is to replace fixed add-or-concat rules with input-dependent interactions that exchange complementary information across modalities, views, or subnetworks.

## 1. Terminological scope

In the cited literature, MGFM is a paper-specific acronym rather than a canonical module family with a single agreed implementation. The following usages appear explicitly in the provided sources.

| Paper | Expansion | Core mechanism |
|---|---|---|
| [2201.10152] | Feature Mutual Mapping Fusion Module | Global map from pairwise dot-product similarity and Softmax |
| [2309.11144] | Multi-view Global-based Fusion Module | Two-stage non-local fusion across views and across time |
| [2507.21971] | Multi-Head Attention Gated Fusion Module | Bidirectional multi-head attention plus spatial-channel gating |

A related but differently named formulation appears in GaitMA, where a **Mutual Learning Module** performs feature fusion through cross-attention and is coupled with a Wasserstein loss [2407.14812]. In Feature Fusion Learning (FFL), the detailed source material also describes a fusion block as MGFM, implemented by depthwise convolution, pointwise convolution, BatchNorm, and ReLU, and embedded in an online mutual knowledge-distillation framework [1904.09058].

This distribution of usages suggests that “MGFM” functions primarily as a compact local name for a fusion module whose defining property is **mutual**, **cross-branch**, or **globally informed** feature interaction, rather than as a single architecture.

## 2. Recurrent design principles

A recurring motivation is the inadequacy of manually designed fusion rules. In unsupervised image fusion, the stated limitations are “manually designed fusion function” and “input-independent network learning”; fixed rules such as pixel-wise addition, concatenation followed by a small network, or hand-tuned weighting cannot adapt on the fly to local differences in saliency, noise levels, or semantic content [2201.10152]. In multi-view echocardiogram analysis, “unsophisticated fusion strategies can even lower performance,” and in event-image segmentation the task is complicated by sparse and noisy event streams and by the need to fuse them with dense image features that differ in structure and representation [2309.11144], [2507.21971].

The principal architectural response is **input-conditioned fusion**. In the image-fusion MGFM, a global map measures the connections of pixels between the input source images; in GL-Fusion, scaled-dot-product attention is applied across views and then across time; in EIFNet, each modality attends to the other and is then combined by a learnable gating map; in GaitMA, one modality is treated as queries and the other as keys and values, with a symmetric reverse pass [2201.10152], [2309.11144], [2507.21971], [2407.14812]. By contrast, FFL uses a convolutional fusion block rather than explicit attention, but it is still learned jointly with the subnetworks and coupled to bidirectional distillation losses that make the fused branch and the sub-network branches teach one another [1904.09058].

A second recurrent principle is **bidirectionality**. The image-fusion MGFM computes two maps, \(Map^A\) and \(Map^B\); EIFNet models event-to-image and image-to-event interactions; GaitMA applies cross-attention in both directions; FFL includes ensemble-to-fusion distillation and fusion-to-sub-networks distillation [2201.10152], [2507.21971], [2407.14812], [1904.09058]. A plausible implication is that the “mutual” aspect of MGFM is often realized either as symmetric attention or as reciprocal supervision.

A third principle is **integration with a larger task-specific scaffold** rather than isolated use. The image-fusion module sits inside a dual-branch multi-scale autoencoder with feature aggregation; GL-Fusion passes MGFM outputs to dense cycle loss and to a local fusion module; EIFNet places MGFM after modality-adaptive recalibration in each encoder stage; FFL couples its fusion block to classifier heads and online knowledge distillation [2201.10152], [2309.11144], [2507.21971], [1904.09058].

## 3. Feature mutual mapping in unsupervised image fusion

In “Unsupervised Image Fusion Method based on Feature Mutual Mapping” [2201.10152], MGFM is the central fusion block of an unsupervised adaptive image fusion method. The stated goal is to replace hand-crafted or fixed fusion rules with a data-driven and spatially adaptive mechanism. Two separate encoders in a “dual-branch multi-scale autoencoder” extract multi-scale features \(f^A_l\) and \(f^B_l\), and at each level \(l\) the MGFM computes attention-style maps \(M^A_l,M^B_l\in\mathbb{R}^{N\times N}\), where \(N=H_l\cdot W_l\). The fused feature at level \(l\) is then
\[
G_l = M^A_l\cdot F^A_l + M^B_l\cdot F^B_l .
\]

At a given scale, the module takes source-image features \(f^A,f^B\in\mathbb{R}^{C\times H\times W}\), flattens them spatially to \(F^A,F^B\in\mathbb{R}^{C\times N}\), and computes raw affinity
\[
M = (F^A)^T\cdot F^B \in\mathbb{R}^{N\times N},
\]
so that \(M_{ij}=\langle F^A_{:,i},F^B_{:,j}\rangle\) measures dot-product similarity between position \(i\) in \(A\) and position \(j\) in \(B\). Each row is normalized by Softmax,
\[
\hat M_{ij}
= \frac{\exp\bigl(M_{ij}\bigr)}{\sum_{k=1}^{N}\exp\bigl(M_{ik}\bigr)},
\]
and the paper sets
\[
Map^A=\hat M,\quad Map^B=(\hat M)^T .
\]
The fused feature is written as
\[
G = Map^A\,F^A + Map^B\,F^B,
\]
or in spatial notation,
\[
G(p)=\sum_q Map^A_{p,q}\,f^A(q)+\sum_q Map^B_{p,q}\,f^B(q).
\]

The surrounding architecture uses independent, non-weight-sharing branches. Each branch contains a \(3\times3\) convolution, two residual blocks, and down-sampling by stride-2 convolution after each block, producing three scales: \((H,W)\), \((H/2,W/2)\), and \((H/4,W/4)\). The deepest feature maps enter the MGFM block first; the resulting fused feature \(G_3\) is up-sampled to the next scale and densely concatenated with encoder features at \(H/2\times W/2\), and a second MGFM can then run at that scale. “Feature aggregation” denotes the up-sampling of all deeper fused maps to the current scale and their concatenation with current encoder features so that multi-scale cues are preserved.

Training uses a single **variance-guided SSIM** loss. For each sliding window \(W\), with source patches \(I_X,I_Y\) and fused patch \(I_F\),
\[
Var\_SSIM(I_X,I_Y,I_F\mid W)=
\begin{cases}
SSIM(I_X,I_F)&\text{if }\sigma^2(I_X)>\sigma^2(I_Y),\\
SSIM(I_Y,I_F)&\text{otherwise.}
\end{cases}
\]
The final loss is
\[
L_{var\_SSIM}=1-\tfrac1N\sum_W Var\_SSIM(I_X,I_Y,I_F\mid W).
\]
The reported training setup is: **40 000 aligned infrared/visible pairs from KAIST**, patch size **\(256\times256\)**, **Adam**, learning rate **\(1e{-}4\)**, and **4 epochs**. During fusion of new tasks such as multi-focus and medical image fusion, **no re-training is done**.

The ablations isolate the contribution of the module and its context. Replacing the 3-scale encoder/aggregator with VIF-Net’s single-scale encoder causes **EI, SF, and VIF** to drop by **5–15%**. Using “mean-guided SSIM” yields **CE=1.39, VIF=0.65**, while “variance-guided SSIM” yields **CE=1.20, VIF=0.76**. For the fusion rule itself, the paper reports **Addition: \(Q^{ab/f}=0.454\), \(SD=91.3\)**; **Concat: \(Q^{ab/f}=0.544\), \(SD=89.9\)**; **MGFM: \(Q^{ab/f}=0.558\), \(SD=93.1\)**. A 3-layer encoder performs best, while deeper 4-layer and 5-layer variants bring no gain or slight degradation. Within this formulation, MGFM is therefore a self-attention-style, globally coupled fusion rule embedded in a multi-scale autoencoder [2201.10152].

## 4. Fusion learning and bidirectional distillation

In “Feature Fusion for Online Mutual Knowledge Distillation” [1904.09058], the source material describes an MGFM inside the Feature Fusion Learning (FFL) framework. Here the module does not use explicit pairwise attention; instead, it combines feature maps from \(N\) parallel sub-networks through a lightweight learned operator. If \(h_i\in\mathbb{R}^{C_i\times H\times W}\) is the last-layer feature map of the \(i\)-th sub-network, the inputs are first brought to a common spatial resolution and concatenated,
\[
H=\mathrm{concat}(h_1,h_2,\ldots,h_N)\in\mathbb{R}^{M\times H\times W},
\quad M=\sum_i C_i.
\]
The concatenated tensor passes through two stages:
\[
Z = DWConv_{3\times3}(H),\quad \hat Z = ReLU(BN(Z)),
\]
\[
F = PWConv_{1\times1}(\hat Z)\in\mathbb{R}^{N_f\times H\times W},\quad \hat F = ReLU(BN(F)).
\]
The pointwise channel count \(N_f\) is chosen either to match the smaller of \(\{C_i\}\) or \(C_1+C_2\), depending on the case.

The module is trained jointly with classifier heads under mutual knowledge distillation. Each sub-network has cross-entropy loss
\[
L_{cls}^{(k)}=-\sum_{i=1}^{m} y_i\log[\sigma(z_k;T=1)_i],
\]
and the fused classifier has
\[
L_{cls}^{fuse}=-\sum_{i=1}^{m} y_i\log[\sigma(z_f;T=1)_i].
\]
The ensemble logit is
\[
z_e=(1/N)\sum_{k=1}^N z_k .
\]
The source material defines **ensemble-to-fusion distillation**
\[
L_{KD}^{e\rightarrow fuse}=KL(\sigma(z_e;T)\,\|\,\sigma(z_f;T)),
\]
and **fusion-to-sub-networks distillation**
\[
L_{KD}^{fuse\rightarrow k}=KL(\sigma(z_f;T)\,\|\,\sigma(z_k;T)).
\]
A total objective is written as
\[
L_{total}
= \sum_{k=1}^N [L_{cls}^{(k)}+\lambda_1 L_{KD}^{fuse\rightarrow k}]
+L_{cls}^{fuse}
+\lambda_2\sum_{k=1}^N L_{KD}^{k\rightarrow fuse}.
\]
The original FFL paper multiplies both distillation terms by \(T^2\), giving effectively \(\lambda_1=\lambda_2=T^2\); the typical choice is **\(T=3\)** and thus **\(\lambda_1=\lambda_2=9\)**.

Optimization is end-to-end. On CIFAR-10/100 the reported settings are **Nesterov-SGD**, momentum **0.9**, weight decay **\(1\times10^{-4}\)** for sub-networks and **\(1\times10^{-5}\)** for MGFM, initial learning rate **0.1**, drops to **0.01** at epoch **150** and **0.001** at epoch **225**, total epochs **300**, batch size **128**, and linear ramp-up of the distillation weight in the first **80 epochs**. On ImageNet, the learning rate decays by a factor of **0.1 every 30 epochs** for **90 epochs**, batch size **256**, with ramp-up over the first **20 epochs**.

The ablation results indicate that both the fusion module and reciprocal distillation matter. On CIFAR-100 with two ResNet-32 sub-networks, **DualNet fused error: 27.49% → FFL fused: 25.56%**, and **DualNet sub-nets average: 34.91% → FFL sub-nets: 27.06%**. Removing the fusion module and replacing it by average gives **fused 26.10%, sub 27.46%**; removing EKD yields **fused 27.03%, sub 28.36%**; removing FKD yields **fused 27.29%, sub 31.04%**. On ImageNet with ResNet-34, the reported **Top-1/Top-5** errors are **vanilla 26.69/8.58**, **ONE-E 24.48/7.31**, and **FFL 23.91/7.17**. In this setting, MGFM is therefore best understood as a convolutional fusion operator whose “mutual” character derives from the surrounding online knowledge-distillation loop rather than from explicit attention [1904.09058].

## 5. Global, cross-modal, and gated MGFM variants

In GL-Fusion, the **Multi-view Global-based Fusion Module** is the first of two fusion stages and is designed “to ‘broadcast’ global context from every other view” back into each view while also exploring “the cyclic relationship of different heartbeat cycles” [2309.11144]. For \(V\) views, each encoder produces
\[
F^{(i)}\in\mathbb{R}^{D\times h\times w\times T}.
\]
The reported setting uses **\(V=3\)**, **\(D=256\)**, **\(h=w=14\)**, **\(T=40\)**, and attention dimensions **\(d=64\)** and **\(d_t=64\)**. The first stage applies cross-view attention at each fixed spatio-temporal slot \((n,t)\) with learned projections \(W^Q,W^K,W^V\), computes
\[
A^{view}_{i\rightarrow j}(n,t)=softmax_j\Bigl[\frac{Q_i(n,t)\cdot K_j(n,t)}{\sqrt d}\Bigr],
\]
and updates each view by
\[
G^{view}_i(n,t)=F^{(i)}[:,(n,t)] + \sum_{j=1}^V A^{view}_{i\rightarrow j}(n,t)V_j(n,t).
\]
The second stage applies temporal attention within each view,
\[
A^{time}_{i,t\rightarrow s}(n)=softmax_s\Bigl[\frac{Q^t_i(n,t)\cdot K^t_i(n,s)}{\sqrt{d_t}}\Bigr],
\]
followed by
\[
G^{cycle}_i(n,t)=G^{view}_i(n,t)+\sum_{s=1}^T A^{time}_{i,t\rightarrow s}(n)V^t_i(n,s).
\]
The output \(\overline{F}_{global}^{(i)}\) has the same shape as the input feature volume. The paper reports **Baseline Avg Dice = 74.46%**, **+ MGFM only = 80.20%**, **+ MLFM only = 78.41%**, and **MGFM + MLFM = 82.29%**.

In EIFNet, the **Multi-Head Attention Gated Fusion Module** appears after the Modality-Adaptive Recalibration Module in each encoder stage and takes recalibrated event and image features
\[
E_{rec}, I_{rec}\in\mathbb{R}^{B\times C\times H\times W}.
\]
Its first stage performs bidirectional standard transformer multi-head attention:
\[
E' = E_{rec} + MultiHead(Q=E_{rec\_flat},K=I_{rec\_flat},V=E_{rec\_flat}),
\]
\[
I' = I_{rec} + MultiHead(Q=I_{rec\_flat},K=E_{rec\_flat},V=I_{rec\_flat}),
\]
where flattening reshapes \((B,C,H,W)\) to \((B,N,C)\) with \(N=H\cdot W\). The second stage constructs a joint tensor
\[
F_{fused}=Concat[E',I']\in\mathbb{R}^{B\times 2C\times H\times W},
\]
computes a channel-domain gating signal
\[
A_c = ReLU(BN(Conv_{1\times1}(GAP(F_{fused})))),
\]
and a spatial-domain gating signal
\[
A_s = ReLU(BN(Conv_{7\times7}(F_{fused}))).
\]
These are summed, passed through another \(1\times1\) convolution and Softmax across the two modality channels to form
\[
G\in\mathbb{R}^{B\times2\times H\times W},
\]
which is split into \(G^e\) and \(G^i\) and used for fusion:
\[
F_{out}=E'\odot G^e + I'\odot G^i.
\]
A LayerNorm and FFN with residual then produce the final \(\hat F\). The source material states **number of heads \(h\): e.g. 8**, **per-head dimension \(d_k=C/h\)**, **Conv\(_{1\times1}(2C\rightarrow2)\)** and **Conv\(_{7\times7}(2C\rightarrow2)\)** in the gating branch, and an FFN with hidden size **\(4C\)**. On DDD17, **adding MGFM to the baseline yields an mIoU jump from \(\approx73.4\%\rightarrow74.56\%\)**; **Baseline + MARM + MGFM gives 76.36% mIoU**; and the full EIFNet reaches **76.55% mIoU and 96.19% pixel accuracy** [2507.21971].

A closely related, though differently named, formulation appears in GaitMA [2407.14812]. After the co-attention alignment module, the **Mutual Learning Module** takes aligned silhouette and skeleton features \(Y_1,Y_2\in\mathbb{R}^{B\times N\times d}\), applies single-head cross-attention in both directions,
\[
A_{12}=Softmax(S_{12}),\quad
Y'_1=LayerNorm(A_{12}Y_2 + Y_1),
\]
\[
A_{21}=Softmax(S_{21}),\quad
Y'_2=LayerNorm(A_{21}Y_1 + Y_2),
\]
and couples the fusion with a 2-Wasserstein loss
\[
\mathcal{L}_w
=\|\mu-\mu^*\|_2^2 + \|\Sigma^{1/2}-{\Sigma^*}^{1/2}\|_F^2.
\]
The reported ablation on Gait3D shows **+ skeleton heatmaps → +3.8% Rank-1**, **+ CAM → +0.4%**, **+ MLM → +0.8%**, and **+ Wasserstein loss → +0.8%**, with **mAP from 48.9% to 55.4%** and **Rank-1 from 59.9% to 66.1%**.

## 6. Ablation evidence, misconceptions, and significance

The empirical record in the cited sources is consistent on one point: simple fusion is weaker than learned, input-dependent fusion. In unsupervised image fusion, MGFM outperforms simple addition and concatenation-plus-convolution on \(Q^{ab/f}\) and \(SD\) [2201.10152]. In FFL, replacing the fusion module by average degrades both fused-classifier and sub-network performance [1904.09058]. In GL-Fusion, MGFM alone adds **5.74 percentage points** of Avg Dice over the baseline; in EIFNet, MGFM improves mIoU over the baseline and contributes to the full model’s reported state-of-the-art DDD17 result [2309.11144], [2507.21971].

A common misconception is that MGFM denotes one fixed architectural block. The cited literature shows otherwise. One MGFM computes a global \(N\times N\) map from dot-product similarities between two source images; another is a depthwise-pointwise convolutional fusion operator inside mutual distillation; another is a two-stage non-local module across views and time; another is a multi-head attention block followed by spatial-channel gating [2201.10152], [1904.09058], [2309.11144], [2507.21971]. This suggests that the acronym is best interpreted contextually.

Another misconception is that “global” or “mutual” fusion necessarily removes task-specific structure. The cited implementations are explicitly designed to avoid that outcome. GL-Fusion states that MGFM broadcasts global context “without ever convolving away fine spatial detail,” and the image-fusion network preserves multi-scale cues through feature aggregation in the decoder [2309.11144], [2201.10152]. EIFNet likewise delays fusion until after modality-adaptive recalibration and then uses a gating map to decide, at each pixel, whether to rely more on event or image features [2507.21971].

Taken together, these modules indicate a broader design pattern rather than a single recipe. MGFM-type blocks couple **cross-branch interaction**, **adaptive weighting**, and **task-coupled supervision**; what changes across papers is the concrete mechanism—global mutual mapping, non-local view/time attention, multi-head attention with gating, or convolutional fusion with reciprocal distillation. A plausible implication is that the enduring contribution of MGFM lies less in a fixed implementation than in the systematic replacement of static fusion heuristics by learned, mutually informed feature integration.

Source: https://www.emergentmind.com/topics/mutual-guided-fusion-module-mgfm