---
title: 'MM-FusionNet: Multi-Modal Fusion Paradigm'
url: https://www.emergentmind.com/topics/mm-fusionnet
type: topic
---

# MM-FusionNet: Multi-Modal Fusion Paradigm

Searching arXiv for "MM-FusionNet" and closely related naming variants to ground the article in current records.
to=arxiv_search.search  大发快三和值ી  仲博json
{"query":"MM-FusionNet OR MMFNet OR \"KAN-Mamba FusionNet\" OR \"FusionNet\"","max_results":10,"sort_by":"relevance"}{"query":"MM-FusionNet OR MMFNet OR \"KAN-Mamba FusionNet\" OR \"FusionNet\"","max_results":10,"sort_by":"relevance"}
Looking up exact arXiv entries for the principal "MM-FusionNet" variants and the naming collisions around "FusionNet".
to=arxiv_search.search  玩北京赛车  ирызjson
{"query":"\"MM-FusionNet\"","max_results":20,"sort_by":"submittedDate"}{"query":"\"MM-FusionNet\"","max_results":20,"sort_by":"submittedDate"}
MM-FusionNet is not a single canonical architecture in the arXiv record. The title appears explicitly in a 2025 framework for multi-modal fake news detection with large vision-language models [2508.05684], while closely related or informally equivalent usages include MMF-Net for multi-focus image fusion [1910.13136], MMFNet for multi-modality MRI segmentation of nasopharyngeal carcinoma [1812.10033], and KAN-Mamba FusionNet for medical image segmentation, for which an equivalence note states that “MM-FusionNet is equivalent to the proposed KAN-Mamba FusionNet” [2411.11926]. This suggests that the term functions less as a unique model identifier than as a recurrent label for architectures that fuse heterogeneous signals, branches, or learned representations.

## 1. Terminological scope and naming

The naming space around MM-FusionNet is unusually heterogeneous. In one case, “MM-FusionNet” is the official paper title for a Large Vision-Language Model-based detector of multi-modal fake news whose central component is the Context-Aware Dynamic Fusion Module (CADFM) [2508.05684]. In another, the official name is “MMF-Net,” expanded once as “Matte Model Fusion Net,” and the text explicitly states that it is not called “MM-FusionNet” in the paper itself [1910.13136]. For nasopharyngeal carcinoma segmentation, the official title is “MMFNet: A Multi-modality MRI Fusion Network for Segmentation of Nasopharyngeal Carcinoma,” and the accompanying summary notes that the same architecture has been referred to as “MM-FusionNet” in other venues [1812.10033]. The “KAN-Mamba FusionNet” paper states that the document does not define a separate MM-FusionNet and that the term is being used as an equivalence label for the proposed architecture [2411.11926].

A broader historical backdrop is provided by earlier “FusionNet” usage in connectomics, where the name denoted a deep, fully residual encoder-decoder CNN with summation-based skip connections for electron microscopy segmentation rather than a multimodal model [1612.05360]. Additional papers reuse the “FusionNet” label for 3D abdominal CT abnormality detection, enhanced beam prediction, low-light enhancement, physics-aware multispectral learning, and passive DOA sensing, but these are distinct task-specific systems rather than variants of a single standardized blueprint [1908.07654].

A practical consequence is that “MM-FusionNet” should be interpreted with strong task context. In some papers it denotes multi-modal sensor fusion, in others multi-branch representation fusion, and in still others multi-model output fusion.

## 2. LVLM-based MM-FusionNet for fake news detection

The most explicit use of the exact title is “MM-FusionNet: Context-Aware Dynamic Fusion for Multi-modal Fake News Detection with Large Vision-Language Models” [2508.05684]. Its task is binary classification of real versus fake news from paired text and image inputs, motivated by two failure modes: imbalance, in which one modality is much more informative than the other, and contradiction, in which text and image conflict. The system uses a fine-tuned LLM such as Vicuna-7B or Mistral-7B for text, a CLIP-based ViT-B/16 for images, shared linear projections into a common semantic space, and CADFM for fusion.

CADFM performs bi-directional cross-modal attention in both directions and then computes dynamic modality weights with a learnable gating network. In the notation given in the paper, projected text and image features are
$$
H_t = T P_t,\qquad H_v = V P_v,
$$
followed by text-to-image and image-to-text attention, residual integration,
$$
H_t' = H_t + Attn_{t\leftarrow v},\qquad H_v' = H_v + Attn_{v\leftarrow t},
$$
and pooled summaries $p_t$, $p_v$, $c_{t\leftarrow v}$, and $c_{v\leftarrow t}$. A cosine consistency score
$$
s = \frac{\langle p_t, p_v\rangle}{\|p_t\|\|p_v\|}
$$
may be included in the gating input. The gating network then produces modality weights $\alpha_t$ and $\alpha_v$, yielding a fused representation such as
$$
f = \alpha_t p_t + \alpha_v p_v
$$
or a residual-augmented variant using the cross-attended context vectors.

The dataset is the Large-scale Multi-modal Fake News Dataset (LMFND), described as containing approximately 80,000 samples and balanced at approximately 1:1 fake versus real. Images are resized to $224\times224$ and normalized; both encoders are fine-tuned with AdamW, learning rate $1\times10^{-5}$, batch size $32$, and $10$ epochs with early stopping on validation F1.

The reported result is state-of-the-art performance with F1 $= 0.938$. The paper reports text-only F1 $= 0.917$, image-only F1 $= 0.885$, simple concatenation F1 $= 0.925$, and fixed-weight bi-directional co-attention without gating F1 $= 0.933$. Robustness experiments show F1 $= 0.887$ with text missing, $0.919$ with image missing, $0.927$ under text noise, and $0.933$ under image noise. In a 500-sample subset, human annotators achieved F1 $= 0.945$ versus the model’s F1 $= 0.938$. Within this lineage, MM-FusionNet is defined by sample-specific, context-conditioned modality prioritization rather than fixed fusion weights.

## 3. Medical-imaging variants

In medical imaging, the MM-FusionNet label covers several substantially different fusion strategies. One variant is the 3D abdominal CT system “FusionNet: Incorporating Shape and Texture for Abnormality Detection in 3D Abdominal CT Scans,” which detects pancreatic ductal adenocarcinoma by combining a binary pancreas mask with the CT volume [1908.07654]. The pipeline is two-stage: pancreas segmentation produces a binary mask, and a dual-stream 3D CNN then fuses shape and texture information. The architecture search spans fusion depth $\alpha \in \{1,\ldots,6\}$ and operator $\beta \in \{+, *, \oplus\}$ over $18$ candidate architectures. The selected model is FusionNet3*, defined by fusion at the third convolutional layer using element-wise multiplication. On $200$ normal scans and $136$ PDAC scans under 4-fold cross-validation, FusionNet3* achieved sensitivity $92.65\%$, specificity $97.00\%$, AUC $97.72\%$, and F1 $94.03\%$, improving more than $5\%$ specificity and more than $10\%$ sensitivity over single-branch models. The fusion mechanism is explicitly motivated as complementarity between organ morphology and CT texture.

A second medical variant is MMFNet for nasopharyngeal carcinoma segmentation from T1, T2, and CET1 MRI [1812.10033]. Its backbone is a 3D multi-encoder, single-decoder network in which each modality has its own encoder and the decoder receives fused skip features. The fusion block consists of 3D-CBAM followed by a Residual Fusion Block. Channel attention uses average, max, and standard-deviation pooling with separate MLPs, while spatial attention uses average, max, and standard-deviation pooling along the channel axis. The self-transfer training strategy first pretrains three single-modality models, initializes the modality-specific encoders from them, freezes encoders for five epochs, and then fine-tunes the entire network. On 149 NPC patients, the full model with self-transfer achieved mean DSC $72.38\%\pm10.99$, ASD $2.07$ mm $\pm2.32$, and HD $18.31$ mm $\pm16.73$, outperforming 3D U-Net and input-level fusion baselines. Inference is reported as approximately $9$ seconds per patient.

A third medical variant is KAN-Mamba FusionNet, which combines Kolmogorov-Arnold Networks with Mamba state-space modeling inside a U-shaped segmentation architecture [2411.11926]. Its characteristic component is the KAMBA block:
$$
PatchEmbed \rightarrow KANB \rightarrow \Psi(\cdot) \rightarrow SSM \rightarrow CBAM \rightarrow \text{residual fusion},
$$
where $\Psi$ is a Bag-of-Activations layer over ReLU, Tanh, Softplus, GELU, and SiLU. The model inserts the modified Mamba-KAN block after the first convolutional stage, places additional KAN blocks in the bottleneck, and uses a BCE-plus-Dice loss. Reported complexity is $6.37$M parameters and $13.96$ GFLOPs. Mean results over three runs are IoU/F1 $62.41/76.41$ on BUSI, $76.95/86.66$ on Kvasir-Seg, and $84.92/91.82$ on GlaS. The BUSI ablation rises from Mamba+MLP at $58.81/73.27$ to the full KAN-Mamba FusionNet at $62.41/76.41$.

These three systems share the use of distinct feature pathways and nontrivial fusion operators, but they solve different problems: binary diagnosis, 3D tumor segmentation, and generic medical segmentation.

## 4. Image-fusion lineages and related usage

The official MMF-Net for multi-focus image fusion is a cascaded boundary-aware network built around an $\alpha$-matte boundary defocus model [1910.13136]. The problem is all-in-focus image synthesis from source images focused at different depths, with particular emphasis on the focused/defocused boundary. The model has two sub-networks: an Initial Fusion sub-net that predicts a guidance map $Gmap$, and a Boundary Fusion sub-net that refines only the boundary region. The arithmetic is explicit:
$$
Fusion_{Ini} = Gmap \times I_1 + (1-Gmap)\times I_2,
$$
$$
Bmap = 1 - |2\times Gmap - 1|,
$$
$$
Fusion_{Fin} = Fusion_{Ini} + Bmap \times (\text{BoundarySubNetOutput}).
$$
Training uses 4,000 synthetic image pairs generated from 200 foreground objects and 1,200 COCO backgrounds. On the Lytro dataset of 20 pairs, MMF-Net achieves the best average AG, LIF, MSD, and GLD among the compared methods, with AG $2.9189$, LIF $0.4071$, MSD $0.1114$, and GLD $14.4388$. Runtime is approximately $0.27$ seconds per $520\times520$ pair on a single GTX 1080 Ti GPU.

A distinct but conceptually related line is FNet with IFNet, an interpretable multi-modal image fusion framework based on $\ell_0$-regularized multi-modal convolutional sparse coding [2411.04519]. FNet separates modality-unique features $u_1$ and $u_2$ and common features $c$, then reconstructs the fused image as
$$
I_f = G_c(c) + G_{u_1}(u_1) + G_{u_2}(u_2).
$$
IFNet inverts this process during training to reconstruct the source images from the fused image. The architecture uses algorithm unrolling via the LZSC block, $9\times9$ kernels, $64$ filters, and $N=4$ iteration modules. Training is two-stage on 1,444 MSRS visible-infrared pairs with Adam, learning rate $10^{-4}$, 20,000 iterations per stage, and batch size $16$. The paper reports leading performance on MI, VIF, Qabf, and SSIM across five multi-modal fusion tasks, along with the best downstream YOLOv5 mAP@[0.5:0.95] on M3FD among the compared fusion methods.

Other “FusionNet” papers extend the nomenclature into adjacent image-restoration and representation-learning settings. For low-light image enhancement, “FusionNet: Multi-model Linear Fusion Framework for Low-light Image Enhancement” linearly combines the outputs of Retinexformer, CIDNet, and ESDNet with weights constrained to sum to $1$, reporting first place in the CVPR 2025 NTIRE Low Light Enhancement Challenge and aggregate complexity of $13.54$M parameters and $50.29$ GFLOPs [2504.19295]. For multi-spectral and thermal Earth observation, “FusionNet: Physics-Aware Representation Learning for Multi-Spectral and Thermal Data via Trainable Signal-Processing Priors” uses five unimodal DGCNN branches, intermediate concatenation, and channel attention; DGCNN reaches $88.7\%$ accuracy on the SWIR ratio and FusionNet reaches $90.6\%$, outperforming the listed baselines across five spectral configurations [2512.19504]. These systems are not official MM-FusionNet models, but they show how the “FusionNet” label has become attached to a wide range of fusion mechanisms.

## 5. Sensing and communications variants

A formally titled sensing variant appears in passive direction-of-arrival estimation for a green heterogeneous hybrid analog-digital massive MIMO receiver [2411.06927]. The pipeline has three stages: coarse DOA estimation from a fully digital subarray, candidate angle generation from H$^2$AD groups with GMaxCS or GMinD clustering, and final fusion via either Iteration Weighted Fusion or MM-FusionNet. In the MM-FusionNet variants, the input is a low-dimensional vector
$$
x = concat([\hat\theta_F],[\hat\theta_{c,1},\ldots,\hat\theta_{c,P}]),
$$
and a lightweight fully connected network outputs the final DOA estimate, trained with MSE:
$$
L_{MSE} = \frac{1}{Q}\sum_{q=1}^{Q}(\tilde\theta_q-\theta_q)^2.
$$
The reported simulation setup uses $P=3$ groups, $(M_1,M_2,M_3)=(7,11,13)$, $K_1=K_2=K_3=16$, an FD array of $128$ antennas, $H=100$ snapshots, true DOA $\theta_0=41^\circ$, and $T=3000$ Monte Carlo trials. All four proposed methods reach the CRLB at SNR $\ge 0$ dB, while MM-fusionNet-GMaxCS and MM-fusionNet-GMinD approach the CRLB already at approximately $-5$ dB. At $-20$ dB, MM-FusionNet accuracy is reported as approximately $8\times$ better than MM-IWF.

A related communications use of the base name appears in mmWave beam prediction from sub-6GHz CSI and a few mmWave pilots [2009.02655]. Here FusionNet is a dual-input neural network with separate fully connected branches for sub-6GHz and mmWave inputs, followed by a classifier network. The mmWave input uses only $\tilde N_m \ll N_m$ active antennas and can exploit delay-domain sparsity through a 1-D DFT along the frequency axis. With typical simulated layer sizes of $2048$ neurons per feature-extractor layer, $L_m=4$, $L_s=6$, and $L_c=3$, the example configuration has approximately $51$M parameters. The paper reports that Top-3 accuracy is approximately $100\%$ even at sub-6 SNR $=0$ dB, that even $\tilde N_m=2$ yields approximately a $5$ dB sub-6 SNR gain in Top-1 accuracy relative to the sub-6-only baseline, and that for sub-6 SNR approximately $5$ dB with $\tilde N_m=4$, the achievable rate shows almost no loss compared to ideal beam selection.

These sensing and communications papers broaden the operational meaning of “fusion.” The modalities are not necessarily images or sensor channels; they may be coarse estimates, clustered candidates, partial channel measurements, or cross-band CSI.

## 6. Recurrent design patterns, misconceptions, and significance

A common misconception is that MM-FusionNet denotes a single, standard architecture. The record summarized here does not support that reading. The exact name is attached to a fake-news detector [2508.05684], an equivalent label for a KAN-Mamba segmentation model [2411.11926], an informal alias for MMFNet in MRI segmentation [1812.10033], and a designated fusion stage in passive DOA sensing [2411.06927]. Related work uses MMF-Net, FusionNet, or FNet instead.

Another misconception is that fusion in these systems is uniformly realized by concatenation. The operators differ substantially. The PDAC detector searches over addition, multiplication, and concatenation, with multiplicative mid-level fusion performing best [1908.07654]. The fake-news model uses bi-directional cross-modal attention followed by dynamic gating [2508.05684]. MMFNet for NPC segmentation uses 3D attention with a residual fusion block [1812.10033]. Multi-focus MMF-Net uses arithmetic initial fusion plus boundary-gated residual correction [1910.13136]. The low-light enhancement framework uses a weighted linear sum of model outputs constrained by $\sum_i k_i = 1$ [2504.19295]. The DOA version uses a small FCNN over angle features rather than pixel or token fusion [2411.06927].

A further recurring pattern is that “modality” is defined operationally, not ontologically. In the PDAC system, the two modalities are a binary pancreas mask and a CT volume rather than two sensor types [1908.07654]. In the fake-news model they are text and image [2508.05684]. In NPC segmentation they are T1, T2, and CET1 MRI [1812.10033]. In passive DOA sensing they are FD coarse angles and inferred H$^2$AD true angles [2411.06927]. In the low-light framework they are the outputs of three independently trained enhancement models operating in different color spaces [2504.19295].

Across these works, a consistent technical theme is the preservation of modality-specific processing before fusion. Dual encoders, multi-encoder backbones, unimodal branches, or separate FC subnetworks are used to avoid premature collapse of heterogeneous signals. This suggests that “MM-FusionNet” is best understood as a design philosophy centered on controlled interaction between complementary representations, with the fusion locus—early, mid-level, late, or output-level—chosen according to the structure of the task and the reliability of the constituent modalities.

Source: https://www.emergentmind.com/topics/mm-fusionnet