MAFR: Multi-Modal Attention-Driven Fusion Restoration

• The paper demonstrates that leveraging cross-modal attention enables adaptive, restoration-aware fusion of RGB images and point clouds for effective industrial anomaly detection.
• MAFR architectures integrate modality-specific encoding, a shared latent fusion space, and attention-guided decoders to suppress degradation and recover reliable features.
• Empirical results across HDR, medical, and infrared-visible domains reveal significant improvements in quantitative metrics and real-world performance due to attention-driven restoration.

Multi-Modal Attention-Driven Fusion Restoration (MAFR) denotes a family of models in which heterogeneous inputs are fused under explicit, learned control over cross-modal contribution, while restoration, reconstruction, or anomaly-sensitive recovery is performed jointly rather than as an isolated preprocessing step. In the narrowest sense, the term names the unsupervised industrial anomaly detection framework that synthesizes a shared latent space from RGB images and point clouds and restores modality-specific features with attention-guided decoders (Ali et al., 20 Oct 2025). In a broader technical usage established by adjacent work, MAFR describes a recurring design pattern: modality-specific encoding, a shared or progressively blended fusion space, attention or attention-like weighting over correspondence and contribution, and a restoration objective that suppresses degradation, missingness, saturation, noise, or misalignment while preserving complementary structure (Chen et al., 2021, Li et al., 25 Jun 2026, Su et al., 28 Jun 2025).

1. Emergence and conceptual scope

The modern MAFR pattern emerged from several converging lines of work. Multi-exposure HDR restoration introduced attention-guided fusion under saturation and motion, exemplified by APNT-Fusion’s two-stream design and its motion, saturation, and scale attention modules (Chen et al., 2021). Cross-attention-guided dense fusion generalized attention-based correspondence learning across infrared-visible, multi-exposure, multi-focus, and MRI-PET settings (Shen et al., 2021). Spatial-frequential cross-attention then extended the idea to frequency-aware medical fusion in AdaFuse (Gu et al., 2023). Later models broadened the formulation to missing modalities, prompt-conditioned restoration, adverse-weather fusion, and large-scale multi-task remote sensing, while the explicit acronym MAFR was formalized for 2D-3D industrial anomaly detection (Fan et al., 2024, Ma et al., 10 Mar 2025, Ali et al., 20 Oct 2025, Cui et al., 7 Apr 2026).

This development suggests that MAFR is better understood as an architectural paradigm than as a single canonical network. Some instances are fully unsupervised and anomaly-oriented, some are supervised fusion-restoration systems, and some are presented as conceptual extensions of multi-modal fusion mechanisms from classification to restoration. The unifying property is not a fixed backbone, but the coupling of cross-modal interaction with restoration-aware reconstruction.

Work	Modalities / task	Defining mechanism
APNT-Fusion (Chen et al., 2021)	Short/medium/long exposure HDR restoration	Two-stream MEF+NFT, MS-HDR matching, motion/saturation/scale attention
CADNIF (Shen et al., 2021)	IR-VIS, MRI-PET, multi-exposure, multi-focus	Cross-attention-guided dense fusion with auxiliary long-range branch
AdaFuse (Gu et al., 2023)	Multi-modal medical image fusion	Spatial-frequential cross-attention with Fourier-guided fusion
LURE (Ma et al., 10 Mar 2025)	Degradation-aware IR-VIS fusion	Unified latent feature space with text-guided attention
UniFuse (Su et al., 28 Jun 2025)	Medical fusion under degradation and misalignment	Prompt-guided alignment and restoration-fusion in one stage
Explicit MAFR (Ali et al., 20 Oct 2025)	RGB + point cloud anomaly detection	Shared fusion encoder with attention-guided modality decoders
Adverse-weather MAFR (Li et al., 25 Jun 2026)	Infrared-visible fusion under weather degradation	Mask-guided feature restoration and cross-modal interaction

2. Canonical architectural formulation

A recurrent MAFR architecture begins with modality-specific or modality-aware feature extraction, followed by a shared fusion stage, and ends with restoration-oriented decoding. In the explicit RGB-3D formulation, dense 2D and projected 3D features are fused by a shared encoder,

$$E_{\text{fus}} = \phi_{\text{enc}}(E_{\text{2D}}, E_{\text{3D}}),$$

and reconstructed by modality-specific decoders,

$$\hat{E}_{\text{2D}} = \psi_{\text{2D}}(E_{\text{fus}}), \qquad \hat{E}_{\text{3D}} = \psi_{\text{3D}}(E_{\text{fus}}),$$

with anomaly evidence derived from the reconstruction discrepancy between original and restored features (Ali et al., 20 Oct 2025). This encoder-fusion-decoder pattern recurs, with different parameterizations, in FusionNet’s dual encoders and alpha-blending head, in APNT-Fusion’s MEF and NFT streams, and in UniFuse’s DAPL-OUFR-FA-UFRF stack (Sun et al., 14 Sep 2025, Chen et al., 2021, Su et al., 28 Jun 2025).

A second common element is an explicit model of contribution. In adverse-weather infrared-visible fusion, the fused output is treated as

$$\mathrm{Fuse} = M \odot \mathrm{VI} + (1-M) \odot \mathrm{IR} + \epsilon,$$

with a stabilized mask inversion

$$M(x) = \Pi_{[0,1]}\left(\frac{\mathrm{Fuse}(x)-\mathrm{IR}(x)}{\mathrm{VI}(x)-\mathrm{IR}(x)+\mathrm{Fuse}(x)+\delta}\right),$$

so that the mask becomes both a training signal and an interaction controller (Li et al., 25 Jun 2026). FusionNet uses the same principle in image space with a learned pixel-wise alpha map,

$$I_{\text{fused}}(x) = \alpha(x)\, I_{\text{ir}}(x) + (1-\alpha(x))\, I_{\text{visY}}(x),$$

where $\alpha$ is predicted from attended features rather than solved analytically (Sun et al., 14 Sep 2025). These formulations make the fusion rule spatially varying and directly interpretable as modality responsibility.

A third architectural axis is latent-space unification. LURE defines a Unified Latent Feature Space $\mathcal{Z}$ with the invariance condition $$\forall z \in \mathcal{Z}, \forall c \in \mathcal{C}, P(c \mid z) = P(c)$$, so degradation identity becomes statistically indistinguishable in latent space (Ma et al., 10 Mar 2025). LLaRS performs a different form of unification by aligning heterogeneous channels to learnable slots using Sinkhorn-Knopp optimal transport before expert routing and attention fusion (Cui et al., 7 Apr 2026). UniFuse addresses the same problem through prompt-conditioned feature unification and contrastive cross-modal alignment rather than explicit attention (Su et al., 28 Jun 2025). The shared implication is that MAFR systems often require a normalization layer between raw modality space and the fusion operator, whether that normalization is geometric, photometric, or semantic.

3. Attention, gating, and correspondence mechanisms

The defining technical signature of MAFR is not simply “using attention,” but using attention to resolve correspondence, conflict, and trust. APNT-Fusion separates these roles into a triad. Motion attention suppresses misaligned content relative to the medium exposure, saturation attention distinguishes missing content caused by saturation from discrepancies caused by motion, and scale attention enforces blending consistency across decoder levels (Chen et al., 2021). CADNIF’s cross-attention module serves a related purpose in a simpler unsupervised setting:

$$A_i = \mathrm{Attent}(I_i, \mathrm{Concat}(I_i, I_j)), \qquad A_j = \mathrm{Attent}(I_j, \mathrm{Concat}(I_i, I_j)),$$

so each source is gated using information from the paired source rather than self-attention alone (Shen et al., 2021). AdaFuse extends cross-attention into the Fourier domain and then re-fuses spatial and frequency branches through its Cross-Attention Fusion block, thereby making high-frequency transfer an explicit fusion target rather than a by-product (Gu et al., 2023).

Other models specialize attention to modality reliability. MDA constructs “continuous attention” across EUS, WLE, and report modalities, with nested scaled dot-product operations that dynamically reduce weight on low-correlation, missing, or intrinsically noisy modalities (Fan et al., 2024). FusionNet learns a modality-aware attention mask $A \in [0,1]^{C \times H \times W}$ and a separate alpha map, so feature-level competition and image-level blending are disentangled (Sun et al., 14 Sep 2025). Adverse-weather MAFR uses mask-guided cross-modal cross-attention in which the mask modulates queries rather than logits: masked visible and infrared features generate queries, while keys and values come from fused features. This construction ties modality interaction to the current estimate of per-pixel contribution, rather than to a fixed attention prior (Li et al., 25 Jun 2026).

A common misconception is that MAFR requires transformer-style self- or cross-attention everywhere. The published literature does not support that restriction. UniFuse explicitly states that it does not employ self-/cross-attention; instead, Spatial Mamba provides global aggregation and ALSN provides prompt-conditioned gating that is described as akin to attention experts (Su et al., 28 Jun 2025). M2Restore similarly combines sparse MoE routing with edge-aware Dynamic Gated Feature Fusion, where

$$\hat{E}_{\text{2D}} = \psi_{\text{2D}}(E_{\text{fus}}), \qquad \hat{E}_{\text{3D}} = \psi_{\text{3D}}(E_{\text{fus}}),$$0

and treats both expert routing and branch fusion as attention-like control mechanisms (Wang et al., 9 Jun 2025). LLaRS generalizes this further by routing slot-aligned features through convolutional experts, channel-mixing experts, and LoRA-adapted attention experts, with routing weights driven jointly by image content and language prompts (Cui et al., 7 Apr 2026). This suggests that, in practice, MAFR is defined by adaptive contribution control, not by a single attention primitive.

4. Restoration objectives and optimization regimes

MAFR training objectives vary with task, but they consistently optimize restoration and fusion together. APNT-Fusion trains in the tone-mapped HDR domain with

$$\hat{E}_{\text{2D}} = \psi_{\text{2D}}(E_{\text{fus}}), \qquad \hat{E}_{\text{3D}} = \psi_{\text{3D}}(E_{\text{fus}}),$$1

thereby optimizing perceptual HDR quality while inferring a linear HDR output (Chen et al., 2021). The explicit RGB-3D MAFR model instead uses a composite unsupervised loss:

$$\hat{E}_{\text{2D}} = \psi_{\text{2D}}(E_{\text{fus}}), \qquad \hat{E}_{\text{3D}} = \psi_{\text{3D}}(E_{\text{fus}}),$$2

where $$\hat{E}_{\text{2D}} = \psi_{\text{2D}}(E_{\text{fus}}), \qquad \hat{E}_{\text{3D}} = \psi_{\text{3D}}(E_{\text{fus}}),$$3 is built from ZNSSD, $$\hat{E}_{\text{2D}} = \psi_{\text{2D}}(E_{\text{fus}}), \qquad \hat{E}_{\text{3D}} = \psi_{\text{3D}}(E_{\text{fus}}),$$4 is edge-aware, and $$\hat{E}_{\text{2D}} = \psi_{\text{2D}}(E_{\text{fus}}), \qquad \hat{E}_{\text{3D}} = \psi_{\text{3D}}(E_{\text{fus}}),$$5 compares average-pooled local structure (Ali et al., 20 Oct 2025). In its ablation, $$\hat{E}_{\text{2D}} = \psi_{\text{2D}}(E_{\text{fus}}), \qquad \hat{E}_{\text{3D}} = \psi_{\text{3D}}(E_{\text{fus}}),$$6 is the strongest individual term, while the full combination yields the highest I-AUROC and AUPRO@1%.

Other systems add task-aware priors directly to the objective. FusionNet combines global MSE, gradient preservation, entropy encouragement, and target-aware ROI fidelity:

$$\hat{E}_{\text{2D}} = \psi_{\text{2D}}(E_{\text{fus}}), \qquad \hat{E}_{\text{3D}} = \psi_{\text{3D}}(E_{\text{fus}}),$$7

so weak bounding-box supervision biases fusion toward semantically important objects such as pedestrians and vehicles (Sun et al., 14 Sep 2025). Adverse-weather MAFR uses a Mask-Guided Learning Strategy and a Task-coupled Degradation-Aware Strategy:

$$\hat{E}_{\text{2D}} = \psi_{\text{2D}}(E_{\text{fus}}), \qquad \hat{E}_{\text{3D}} = \psi_{\text{3D}}(E_{\text{fus}}),$$8

with $$\hat{E}_{\text{2D}} = \psi_{\text{2D}}(E_{\text{fus}}), \qquad \hat{E}_{\text{3D}} = \psi_{\text{3D}}(E_{\text{fus}}),$$9 decaying during training so Pseudo Ground Truth stabilizes early learning without fixing the final fused image to a pseudo target (Li et al., 25 Jun 2026). AdaFuse couples content loss to a structure loss composed of logarithmic structural tensor discrepancy and SSIM, while CADNIF uses unsupervised pixel and gradient consistency to each source image (Gu et al., 2023, Shen et al., 2021).

A separate line of work optimizes MAFR under prompt, multi-task, or missing-modality regimes. LURE uses a two-stage curriculum: Stage 1 learns a degradation-invariant unified latent space with reconstruction, task, unified feature alignment, and text-classification losses, and Stage 2 freezes the encoders and learns fusion in that space (Ma et al., 10 Mar 2025). UniFuse optimizes prompt classification, modality unification, registration regularization, and restoration-fusion losses jointly in a single-stage pipeline (Su et al., 28 Jun 2025). LLaRS performs step-level Dynamic Weight Adjustment across eleven tasks and augments reconstruction with routing classification and MoE load-balancing losses, thereby stabilizing a single foundation model across restoration and fusion settings (Cui et al., 7 Apr 2026). By contrast, MDA itself remains a classification model; its extension to restoration is stated as a plausible architectural adaptation, not as a published restoration experiment (Fan et al., 2024).

5. Application domains and empirical profile

The empirical record shows that MAFR-like designs are not confined to one domain. In HDR restoration, APNT-Fusion achieves the best DeepHDR test results among the listed methods with PSNR-$$\mathrm{Fuse} = M \odot \mathrm{VI} + (1-M) \odot \mathrm{IR} + \epsilon,$$0 43.96, PSNR-L 41.69, SSIM-$$\mathrm{Fuse} = M \odot \mathrm{VI} + (1-M) \odot \mathrm{IR} + \epsilon,$$1 0.9957, and SSIM-L 0.9914, while ablations show substantial degradation when MS-HDR, NFT, motion attention, or scale attention are removed (Chen et al., 2021). In explicit unsupervised MAFR for industrial anomaly detection, the mean image-level AUROC reaches 0.972 on MVTec 3D-AD and 0.901 on Eyecandies; element-wise multiplication of 2D and 3D anomaly maps outperforms addition, max, or single-modality maps (Ali et al., 20 Oct 2025).

Clinical and medical settings reveal a different strength profile. MDA reaches 98.9% on the gastrointestinal multi-center dataset with full modalities, 95.4% when one modality is randomly missing, and 97.9% under intrinsic report noise, indicating that continuous cross-modal attention can reallocate trust under missingness and contradiction (Fan et al., 2024). UniFuse reports 395 GFLOPs per case, yet improves substantially over staged baselines across BraTS2020, SynthRAD2023, and FDG-PET/CT, while prompt-guided alignment yields better Q_ssim than CorrMLP on BraTS2020 (Su et al., 28 Jun 2025). AdaFuse achieves the highest values across all five reported metrics on CT-MRI, leads EN and CC on PET-MRI, and attains the best EN and FMI on SPECT-MRI, supporting the claim that spatial-frequential cross-attention is particularly effective for preserving high-frequency medical detail (Gu et al., 2023).

Infrared-visible and adverse-weather fusion further broaden the picture. FusionNet reports SSIM 0.87, MSE 0.012, Entropy 7.42, and ROI-SSIM 0.84 on M3FD, with alpha maps serving as direct visualizations of where infrared dominates over visible luminance (Sun et al., 14 Sep 2025). The adverse-weather mask-guided model consistently ranks top-2 across snow, rain, and haze metrics and shows the largest average ablation drop, about 6.9%, when mask-guided cross-modal cross-attention is removed; it also delivers the best YOLOv7 detection performance on M3FD (Li et al., 25 Jun 2026). LURE reaches mIOU 81.22 for downstream semantic segmentation on MSRS, and its degradation-aware formulation remains competitive across low-light, haze, overexposure, low-contrast, and super-resolution settings, including combined degradations such as OE+LC and LL+LC (Ma et al., 10 Mar 2025).

At larger scale, M2Restore obtains the best average PSNR/SSIM, 31.65/0.936, across Outdoor-Rain, Snow100K-L, and Raindrop, while retaining a 0.17 s inference time on 224×224 inputs (Wang et al., 9 Jun 2025). LLaRS extends the MAFR pattern into a foundation-model regime, reporting an average PSNR of 37.60, average SSIM of 0.9172, average SAM of 0.0644, and average ERGAS of 4.4069 across eleven remote-sensing restoration and fusion tasks (Cui et al., 7 Apr 2026). These results suggest that the combination of alignment, adaptive contribution control, and restoration-aware decoding scales from pairwise fusion to prompt-conditioned, multi-task settings.

6. Limitations, misconceptions, and open directions

Several limitations recur across the literature. Some MAFR systems assume registration or near-registration. FusionNet relies on geometrically aligned M3FD pairs and explicitly identifies misalignment as a failure mode (Sun et al., 14 Sep 2025). APNT-Fusion improves robustness to misalignment but notes that its MS-HDR assumption can fail when a saturated background is occluded (Chen et al., 2021). UniFuse addresses deformable misalignment explicitly, yet still notes sensitivity to extreme misalignments and severe corruption (Su et al., 28 Jun 2025). In the RGB-3D anomaly setting, the method depends on accurate projection of point features into the image plane and therefore masks invalid 3D regions during anomaly fusion (Ali et al., 20 Oct 2025).

Another misconception is that MAFR necessarily “recovers” absent information by generating a missing modality. MDA does not perform imputation or generative synthesis; its robustness to missing modalities arises purely from attention-based reweighting (Fan et al., 2024). The same caution applies to other adaptive gating systems: attention can suppress unreliable inputs, but it does not by itself invent physically absent evidence. A related interpretability issue is that attention weights and alpha masks are informative but incomplete. FusionNet’s interpretability evidence is qualitative, and MDA notes that attention explanations can be sensitive to architectural choices (Sun et al., 14 Sep 2025, Fan et al., 2024). Thus, the interpretability claims in MAFR are strongest when they are tied to explicit contribution maps, disease-specific modality weights, or ablation-backed task behavior.

Computational cost remains a practical constraint. APNT-Fusion reports no runtime and acknowledges the overhead of multi-scale patch matching (Chen et al., 2021). The explicit RGB-3D MAFR notes increased training time due to CBAM in the decoders (Ali et al., 20 Oct 2025). The adverse-weather mask-guided model reports about 59.7M parameters and about 242 GFLOPs, with HTB contributing notably to the cost (Li et al., 25 Jun 2026). UniFuse reports about 395 GFLOPs per case, while LLaRS increases parameter count from 11.85M for a plain U-Net baseline to 83.64M for the full expert model (Su et al., 28 Jun 2025, Cui et al., 7 Apr 2026).

The open research directions identified in the papers are largely convergent. Proposed next steps include hybrid Mamba-attention blocks, uncertainty-aware and deformable alignment, lighter attention or refinement modules, extension beyond two modalities, and better prompt or mask generation mechanisms (Su et al., 28 Jun 2025, Ali et al., 20 Oct 2025, Li et al., 25 Jun 2026). A plausible implication is that future MAFR systems will become less tied to a single modality pair and more centered on transferable latent harmonization, expert routing, and explicit reliability estimation. The literature already points toward that trajectory through LLaRS’s language-conditioned multi-task formulation, LURE’s degradation-invariant latent space, and the explicit MAFR framework’s extension path to additional modalities such as thermal or hyperspectral sensing (Cui et al., 7 Apr 2026, Ma et al., 10 Mar 2025, Ali et al., 20 Oct 2025).