Intensity-Guided Fusion Mechanisms

• Intensity-guided fusion mechanisms are algorithms that leverage local intensity statistics to adaptively blend information from multiple modalities.
• They employ techniques such as weighted least-squares, guided filtering, and nonlinear gating to enhance details in tasks like hyperspectral pansharpening and multi-exposure fusion.
• The approach integrates mathematical models and attention-based modulation to improve robustness and performance in dynamic scene denoising and multi-modal image restoration.

Intensity-guided fusion mechanisms constitute a class of algorithms and architectures where pixel/voxel, local patch, or spatial–spectral “intensity” statistics are leveraged to guide information transfer, alignment, or blending between multiple modalities or exposures. These mechanisms play a foundational role in hyperspectral pansharpening, multi-exposure fusion, multi-modal 3D–4D data denoising, radar–camera perception, multi-source image restoration, and multi-illumination infrared–visible fusion. The central technical innovation in intensity-guided fusion is the explicit use of intensity-derived weights, gates, or attention scores to modulate the fusion process, thereby increasing adaptivity and reducing modality bias.

1. Mathematical Foundations and Canonical Formulations

The intensity-guided fusion paradigm encompasses frameworks based on weighted least-squares, guided filtering, nonlinear gating, attention modulation, and hybrid mixtures-of-experts. Mathematically, typical forms involve weighted convex combinations, local linear models, or intensity-gated attention.

Weighted Least-Squares Image Fusion: For multi-exposure fusion, the variational energy is

$$E[I] = \sum_k \int_\Omega W_k(x)\, [I(x)-I_k(x)]^2 \, dx,$$

subject to the constraint $\sum_k W_k(x)=1$, yielding pixelwise fusion $I_f(x)=\sum_k W_k(x) I_k(x)$ where $W_k(x)$ reflects local information content such as entropy (Singh et al., 2022).

Guided Filtering Formulation: In guided image restoration, guided filtering assumes a local linear relation in a window $\omega_k$,

$Q_j = a_k I_j + b_k, \quad j \in \omega_k,$

with $(a_k, b_k)$ chosen by minimizing reconstruction error with a regularization on $a_k$ (Liu et al., 2023).

Nonlinear Gating and Attention: In hyperspectral pansharpening, nonlinear “fish-distribution” gates $q(x,y)$ based on the PAN intensity and the standard deviation of abundance encode adaptive injection strengths: $$q = \sigma(\text{ReLU}(y - (k_1 x + b_1))) + \sigma(\text{ReLU}((k_2 x+b_2)-y))$$ guiding both linear and nonlinear detail transfer (Li et al., 2022).

Intensity-Gated Attention: In cross-modal camera–radar fusion, deformable attention module sampling offsets and attention logits are modulated by continuous “intensity” maps $$\mathcal{I}^{\mathrm{Cam}}, \mathcal{I}^{\mathrm{Radar}}$$ for spatial alignment and fusion (Mishra et al., 17 Dec 2025).

2. Key Application Domains

Intensity-guided fusion mechanisms are broadly employed across the following domains:

• Hyperspectral Pansharpening: Injecting high spatial detail from a panchromatic sensor into a low-resolution HSI using a PAN Detail Inject Network (PDIN) that exploits intensity–abundance statistical relations and pixelwise nonlinear weighting (Li et al., 2022).
• Multi-Exposure Image Fusion: Combining LDR images into an enhanced dynamic range output by fusing local well-exposed regions using entropy-derived intensity weights, with or without prior CLAHE preprocessing (Singh et al., 2022).
• Guided Image Restoration: Simultaneous feature and image (intensity) guided fusion through deep networks inspired by guided filter and cross-attention (Liu et al., 2023).
• 3D/4D Dynamic Scene Denoising: Intensity-guided spatiotemporal fusion using intensity similarities to weight spatial and temporal smoothing for dynamic point cloud sequences (Zhang et al., 2017).
• Radar–Camera Fusion for 3D Perception: Intensity-aware cross-attention for camera and radar BEV features, using confidence or RCS-derived intensity as guidance for both deformable offsets and attention weight scaling (Mishra et al., 17 Dec 2025).
• Illumination-Dependent Multi-Modality Fusion: Gated mixture-of-experts approach for visible–infrared fusion where illumination intensity probability guides expert weighting and asymmetric cross-attention fuses at multiple depths (Jinfu et al., 27 Jul 2025).

3. Representative Architectures and Algorithmic Patterns

Several architectures exemplify the intensity-guided fusion principle:

Panchromatic Detail Injection (PDIN) in Pgnet: Combines a nonlinear STD-adaptive weight $q$ (from abundance–intensity relationship) with a linear PAN-guided mapping for abundance correction and spatial detail injection, applied recursively during upsampling and in deep pixelwise attention (Li et al., 2022).

Entropy-Weighted Multi-Exposure Pyramid Fusion: Employs local entropy normalization to derive fusion weights, applies CLAHE–Rayleigh histogram equalization to enforce local intensity balance, and aggregates exposures via weighted Laplacian pyramid summation for spatial regularity (Singh et al., 2022).

Simultaneous Feature and Image Guided Fusion (SFIGF): Integrates image-domain guided filter-style pixel fusion with feature-domain guided cross-attention, both informed by guidance intensity and covariance statistics within feature and spatial domains (Liu et al., 2023).

Intensity-Guided Deformable Cross-Attention (IMKD): Radar intensity maps (aggregated RCS, Doppler) and camera confidence maps (learned 1×1 conv-sigmoid) modulate cross-attention offsets and scaling. Cross-attention is performed with radar features as queries and camera features as key/value, gated by learned intensity functions (Mishra et al., 17 Dec 2025).

Illumination Gates for Modality Routing: Classifies input illumination into high/low using a CNN, then blends the outputs of two chiral transformer expert stacks (with opposite cross-attention directionality) according to illumination probabilities (Jinfu et al., 27 Jul 2025).

Architecture	Guidance Intensity Source	Key Fusion Mechanism
Pgnet/PDIN (Li et al., 2022)	PAN intensity, abundance STD	Nonlinear gating + linear
Multi-Exposure (Singh et al., 2022)	Patch entropy (CLAHE preprocessing)	Entropy-weighted sum
SFIGF (Liu et al., 2023)	Guidance image intensity, features	GF-inspired CA and ImGF
3D/4D Fusion (Zhang et al., 2017)	Image intensity, spatial similarity	Bilateral, temporal avg
IMKD (Mishra et al., 17 Dec 2025)	Radar RCS/Doppler, camera BEV scores	Intensity-gated attention
MoCTEFuse (Jinfu et al., 27 Jul 2025)	ResNet-illum. classifier (visible)	Expert gating, ACA

4. Performance, Robustness, and Ablation Results

Empirical studies consistently report performance improvements for intensity-guided fusion mechanisms compared to non-guided or naive alternatives, in both quantitative metrics and qualitative assessments.

• Pgnet/PDIN: Achieves PSNR = 36.27dB versus 32.38dB with no PDIN, and significantly lower spectral artifacts on Chikusei $r=16$ dataset. Nonlinear gating alone gives 35.74dB; adding linear PAN weight yields +0.53dB (Li et al., 2022).
• Multi-Exposure Fusion: Visual evidence demonstrates superior shadow and highlight preservation, reduced haloing, and seamless region selection by local entropy weights. No direct PSNR/SSIM reported but qualitatively matches or exceeds methods such as Mertens et al. and Goshtasby (Singh et al., 2022).
• Intensity-Guided 4D Fusion: Reduces mean surface roughness by 40–60% compared to seven baselines under both spatial and intensity noise up to 10% variance. Maintains robustness to motion and noise (Zhang et al., 2017).
• IMKD Intensity-Guided Fusion: Stage-3 intensity-aware fusion improves mAP by +3.1% (from 43.4% to 46.5%) and NDS by +1.8%, with full pipeline giving 61.0% mAP and 67.0% NDS on nuScenes (Mishra et al., 17 Dec 2025).
• MoCTEFuse: Full illumination-gated mixture achieves EN = 6.73, SD = 43.16, MI = 3.63, VIF = 1.04 (MSRS, DroneVehicle), and mAP = 70.93% (MFNet), outperforming variants without HI/LI experts or the competitive loss (Jinfu et al., 27 Jul 2025). Object detection mAP improvements are substantial.

5. Architectural Trade-offs and Design Parameters

Intensity-guided fusion introduces design choices affecting expressivity, cost, and robustness:

• Linear vs Nonlinear Guidance: Strictly linear injection is simpler but omits enrichment captured by nonlinear statistical gating, resulting in ~0.5dB PSNR drop (Pgnet Table VII) (Li et al., 2022).
• Local vs Global Measurement: Patchwise entropy or local statistics provide adaptivity but can introduce artifacts under severe noise; global measures offer more regularity but less selectivity (Singh et al., 2022, Liu et al., 2023).
• Attention Modulation Granularity: Deformable attention with intensity gating allows fine spatial adaptivity; naive fusion can cause blending artifacts and lose edge sharpness (Mishra et al., 17 Dec 2025).
• Expert Gating Specificity: Illumination classification at inference enables dynamic adaptation but requires careful gating model training and reliable scene statistics (Jinfu et al., 27 Jul 2025).
• Complexity Considerations: Computational cost scales with number of attention windows, pyramid levels, or deformable sampling locations. Neighborhood attention reduces $\mathcal{O}(H^2W^2)$ cost to $\mathcal{O}(HWk^2)$ in SFIGF (Liu et al., 2023). PDIN is lightweight ($<0.1$s/patch on V100, $0.05$M params) (Li et al., 2022).

6. Theoretical and Empirical Impact

Intensity-guided fusion incorporates domain priors—through statistical measures, filtering, or confidence maps—into deep and classical pipelines, promoting context-sensitive and artifact-resilient information integration. Its effects include:

• Enhanced detail preservation, especially at intensity transitions, edges, and modality boundaries.
• Modality complementarity retention, avoiding information collapse or suppression prevalent in direct concatenation/matching regimes (Mishra et al., 17 Dec 2025).
• Dynamic selectivity to context conditions such as scene illumination (MoCTEFuse (Jinfu et al., 27 Jul 2025)) or local exposure (entropy-based fusion (Singh et al., 2022)).
• Improved downstream performance for detection and recognition: e.g., MoCTEFuse gives AP$_{0.5}$ = 0.9280 on MFNet vs. 0.8051 (IR alone).

A plausible implication is that intensity-guided fusion, by acting as a statistical and confidence-based “routing” mechanism, provides a generalizable strategy for future multimodal architectures, particularly as sensor and illumination diversities increase.

7. Implementation Guidelines and Typical Hyperparameters

Practical deployment of intensity-guided fusion mechanisms requires careful tuning of:

• Local window/patch size: 3×3 for entropy (image fusion), 5–11 pixel radius for 4D fusion spatial neighborhoods (Singh et al., 2022, Zhang et al., 2017).
• Regularization and loss weights: $\epsilon$ in guided filtering, $\alpha, \beta, \gamma$ for loss terms (e.g., intensity/gradient/SSIM in MoCTEFuse) (Jinfu et al., 27 Jul 2025).
• Number of experts and attention heads: Two in MoCTEFuse, four self-attention/aggregation blocks in SFIGF (Jinfu et al., 27 Jul 2025, Liu et al., 2023).
• Scale of Gaussian kernels: $\delta_d$, $\delta_g$, $\delta_f$ for spatiotemporal weights in 4D fusion (typical values 1–3 mm spatial, 0.05–0.15 intensity) (Zhang et al., 2017).
• Learning rates and schedules: Adam optimizer with decayed learning rate or cosine annealing, batch sizes as reported in original papers (Li et al., 2022, Mishra et al., 17 Dec 2025, Jinfu et al., 27 Jul 2025).

Extensive ablation across these settings is required to balance detail retention, robustness, and computational efficiency for targeted applications.

---

Principal references: (Li et al., 2022, Singh et al., 2022, Liu et al., 2023, Zhang et al., 2017, Mishra et al., 17 Dec 2025, Jinfu et al., 27 Jul 2025).