Uncertainty-Guided Multi-branch Fusion

• UMF is a design family where multiple computational branches are fused based on uncertainty signals, enabling adaptive feature rectification and decision-making.
• It employs various uncertainty measures—evidential, aleatoric, and entropic—to modulate branch contributions, ensuring reliable fusion across scales, modalities, and views.
• UMF techniques enhance robustness and performance in applications such as deraining, adaptive classification, multi-modal detection, and 3D panoptic segmentation by addressing branch-quality mismatches.

to=functions.arxiv_search 天天彩票与你同行 json {"query":"\"uncertainty-guided\" multi-branch fusion uncertainty-aware fusion multi-branch", "max_results": 10} to=functions.arxiv_search .scalablytyped json {"query":"(Tong et al., 2023) OR (Zhang et al., 2024) OR (Zhang et al., 10 Apr 2026) OR (Wang et al., 16 Mar 2026) OR (Zhang et al., 2023) OR (Lou et al., 2023) OR (Dai et al., 10 May 2026) OR (Mohan et al., 22 Feb 2026)", "max_results": 10} Uncertainty-guided Multi-branch Fusion (UMF) denotes a class of architectures in which multiple computational branches—such as multi-scale pathways, early-exit classifiers, modality-specific detectors, view-specific experts, or dual-domain encoders—are not fused uniformly, but are fused according to branch- or pixel-level uncertainty signals. In the works surveyed here, uncertainty is used as an active control variable for feature rectification, routing, weighting, arbitration, or suppression, rather than as a purely diagnostic confidence score. A 2023 deraining formulation, UMFFNet, characterizes this direction by learning a probability mapping distribution between paired images to estimate uncertainty, introducing an uncertainty feature fusion block (UFFB), and combining multi-scale features for collaborative rain removal (Tong et al., 2023). Subsequent formulations extend the same logic to adaptive image classification, multi-modal and multi-view perception, challenging-tailed sequence learning, medical segmentation, and 3D panoptic segmentation (Zhang et al., 2024, Zhang et al., 2023, Lou et al., 2023, Zhang et al., 10 Apr 2026, Wang et al., 16 Mar 2026, Dai et al., 10 May 2026, Mohan et al., 22 Feb 2026).

1. Conceptual scope and motivating rationale

UMF emerges from a shared diagnosis: deterministic fusion is brittle when branch quality is heterogeneous. In deraining, this brittleness is framed as reconstruction error caused by deterministic outputs that ignore epistemic uncertainty; well-calibrated uncertainty is proposed as a mechanism for alleviating prediction errors and assisting measurement devices in mitigating risks and improving usability (Tong et al., 2023). In autonomous-driving detection, the same issue appears as branch-quality mismatch under adverse conditions: existing fusion schemes may ignore the quality of each fusion input and therefore degrade when one or more sensors are corrupted (Lou et al., 2023). In dual-domain segmentation, deterministic cross-domain fusion is described as “blind cross-domain fusion” that can overfit to label noise (Dai et al., 10 May 2026).

The common architectural move is to retain branch diversity while making inter-branch interaction conditional on reliability. Depending on the domain, the branches may represent scales, exits, modalities, views, or experts. The uncertainty signal may be epistemic, aleatoric, evidential, entropic, or derived from feature instability, but its operational role is consistent: it modulates contribution to a fused representation or decision.

This suggests that UMF is best understood not as a single canonical algorithm, but as a design family whose unifying principle is reliability-conditioned fusion. The most important variation across implementations lies in how uncertainty is represented and whether fusion authority is assigned locally by each branch or globally by a dedicated arbitration mechanism.

2. Representative realizations across domains

The literature represented here spans several non-equivalent but structurally related UMF instantiations.

System	Branches	Fusion mechanism
UMFFNet	Multi-scale deraining features	UFFB dynamically enhances acquired features using uncertainty and fuses feature information from multiple scales (Tong et al., 2023)
CDM / GCDM	Multiple classifier heads in adaptive networks	Evidential uncertainty from earlier heads improves the current head through uncertainty-aware decision fusion (Zhang et al., 2024)
UMF for multi-modal object detection	RGB branch and Depth branch	Uncertainty-aware multi-source NMS with variance-weighted box fusion (Zhang et al., 2023)
UMoE	LiDAR and camera proposal experts	Expert towers plus gating network produce uncertainty-weighted proposal confidences (Lou et al., 2023)
TMUR	View-private experts plus one collaborative expert	Unified router observes global multi-view context and generates sample-level expert weights (Zhang et al., 10 Apr 2026)
UME	Multiple LoRA experts	DST-guided dynamic weights fuse expert evidences and route hard samples to later experts (Wang et al., 16 Mar 2026)
UGDD-Net	Spatial and frequency branches	Uncertain target features are rectified by reliable source features via bi-directional deformable cross-attention (Dai et al., 10 May 2026)
UP-Fuse	LiDAR and camera range-view features	Camera features are uncertainty-modulated before deformable attention into LiDAR queries (Mohan et al., 22 Feb 2026)

Across these systems, branch structure serves different purposes. In adaptive classification, branches correspond to exits at different depths and therefore to variable compute budgets. In multi-view classification and multi-expert sequence learning, branches embody conditional specialization. In multi-modal detection and panoptic perception, branches reflect sensor complementarity. In medical segmentation and deraining, branches capture complementary representations of the same scene or image.

A notable consequence is that “fusion” itself is heterogeneous. Some systems fuse predictions or evidences, some fuse detection proposals, and others fuse intermediate features. The uncertainty-guided principle is therefore orthogonal to the level of representation at which fusion occurs.

3. Uncertainty representations and fusion operators

A major line of UMF uses evidential deep learning or Dempster–Shafer Theory (DST). In CDM/GCDM, each classifier head produces logits $p^{(c)}$, evidence is obtained by

$$e_k^{(c)}=\mathrm{SoftPlus}\!\bigl(p_k^{(c)}\bigr), \qquad \alpha_k^{(c)}=e_k^{(c)}+1,$$

and uncertainty is defined through the Dirichlet strength $S^{(c)}=\sum_k \alpha_k^{(c)}$ as

$u^{(c)}=\frac{K}{S^{(c)}}.$

Belief masses $b_k^{(c)}=(\alpha_k^{(c)}-1)/S^{(c)}$ are then fused sequentially, with additional balance terms introduced to reduce “fusion saturation” and “fusion unfairness” (Zhang et al., 2024). TMUR adopts the same evidential form for multi-view experts, but contests the assumption that independently produced evidences are numerically comparable across views (Zhang et al., 10 Apr 2026). UME similarly maps expert logits to evidence, defines belief masses and total uncertainty by $u^m=K/S^m$, and then uses DST-inspired conflict and uncertainty propagation to derive dynamic expert weights before computing fused evidence $\hat e=\sum_m s_m e^m$ (Wang et al., 16 Mar 2026).

A second line models aleatoric uncertainty directly in regression outputs. Zhang et al. formulate each box coordinate as Gaussian, for example

$b_x \sim \mathcal N(\mu_x,\sigma_x^2),$

and train by negative log-likelihood with loss attenuation. At fusion time, box coordinates are updated by variance-weighted averaging, e.g.

$$x_{\rm fused}=\frac{\sum_j x_j/\sigma_{x,j}^2}{\sum_j 1/\sigma_{x,j}^2},$$

inside an uncertainty-aware multi-source NMS procedure (Zhang et al., 2023). Here the uncertainty signal is not an auxiliary branch statistic; it is embedded directly into the detection head and propagated into geometric fusion.

A third line uses uncertainty to modulate feature interaction. In UGDD-Net, entropy-based uncertainty maps are computed from test-time augmentation ensembles, normalized to $[0,1]$, and used to separate an “uncertain target” from a “reliable source”:

$$e_k^{(c)}=\mathrm{SoftPlus}\!\bigl(p_k^{(c)}\bigr), \qquad \alpha_k^{(c)}=e_k^{(c)}+1,$$0

After deformable alignment, guided cross-attention injects corrections from the reliable branch into the uncertain branch (Dai et al., 10 May 2026). UP-Fuse takes a related but asymmetric route: camera-feature instability under corruption is predicted, converted to

$$e_k^{(c)}=\mathrm{SoftPlus}\!\bigl(p_k^{(c)}\bigr), \qquad \alpha_k^{(c)}=e_k^{(c)}+1,$$1

and then translated into a camera fusion weight $$e_k^{(c)}=\mathrm{SoftPlus}\!\bigl(p_k^{(c)}\bigr), \qquad \alpha_k^{(c)}=e_k^{(c)}+1,$$2 before deformable attention from LiDAR queries into uncertainty-modulated camera features (Mohan et al., 22 Feb 2026).

UMoE occupies an intermediate position between decision-level and proposal-level fusion. It estimates classification uncertainty via Shannon entropy of averaged softmax outputs under MC-Dropout, estimates regression uncertainty via total variance plus learned data uncertainty, encodes these raw uncertainties into comparable scalar cues, and feeds them with proposal scores into modality-specific expert towers and a gating network that outputs refined proposal confidences (Lou et al., 2023). This architecture explicitly acknowledges that raw uncertainty values are often not directly usable across modalities without additional encoding.

4. Optimization, regularization, and calibration strategies

UMF methods generally do not rely on a single loss term. Their optimization schemes combine task supervision with uncertainty calibration, branch guidance, or routing regularization.

In CDM/GCDM, evidential outputs are trained with an EDL regression loss, and GCDM adds a Jensen–Shannon guidance term in which the last classifier guides earlier classifiers through softened outputs at two temperatures. The full objective is

$$e_k^{(c)}=\mathrm{SoftPlus}\!\bigl(p_k^{(c)}\bigr), \qquad \alpha_k^{(c)}=e_k^{(c)}+1,$$3

with fusion applied only at test time rather than during training (Zhang et al., 2024). This training design aims to improve early-exit quality without removing inter-head diversity.

TMUR optimizes fused and view-wise evidential objectives jointly:

$$e_k^{(c)}=\mathrm{SoftPlus}\!\bigl(p_k^{(c)}\bigr), \qquad \alpha_k^{(c)}=e_k^{(c)}+1,$$4

The load-balancing term discourages router collapse to a small subset of experts, while the diversity term decorrelates private experts. The key architectural claim is that fusion arbitration should be learned from the global multi-view context, not delegated to view-local uncertainty alone (Zhang et al., 10 Apr 2026).

UME introduces a more selective training policy. It combines negative marginal likelihood, evidence KL regularization, hierarchical classification loss, and contrastive loss into a per-expert objective, then sums these only over experts satisfying the threshold condition $$e_k^{(c)}=\mathrm{SoftPlus}\!\bigl(p_k^{(c)}\bigr), \qquad \alpha_k^{(c)}=e_k^{(c)}+1,$$5:

$$e_k^{(c)}=\mathrm{SoftPlus}\!\bigl(p_k^{(c)}\bigr), \qquad \alpha_k^{(c)}=e_k^{(c)}+1,$$6

This gating rule is used to encourage later experts to specialize on hard samples (Wang et al., 16 Mar 2026).

In UMoE, training is explicitly two-stage. Sensor-specific detectors are first trained as probabilistic detectors with heteroscedastic direct modeling and MC-Dropout. The detectors are then fixed, and UMoE plus the downstream proposal-fusion network are trained end-to-end using the standard 3D detection objective of the host fusion pipeline (Lou et al., 2023). In UP-Fuse, uncertainty estimation is supervised directly by a Huber regression term on feature instability and optimized jointly with a panoptic set-based loss:

$$e_k^{(c)}=\mathrm{SoftPlus}\!\bigl(p_k^{(c)}\bigr), \qquad \alpha_k^{(c)}=e_k^{(c)}+1,$$7

This couples reliability prediction to downstream segmentation quality (Mohan et al., 22 Feb 2026).

UGDD-Net uses a curriculum. For the first 20 epochs, only base segmentation losses on the “glance” outputs are applied; afterward, the full uncertainty-guided machinery, including UGBFF and UGGR, is enabled (Dai et al., 10 May 2026). This staging indicates that uncertainty-guided fusion can destabilize early optimization if introduced before branch representations become reasonably structured.

5. Empirical behavior across tasks

The empirical record is consistently organized around two themes: nominal performance and robustness under branch corruption, imbalance, or budget variation.

In adaptive classification, CDM and GCDM report $$e_k^{(c)}=\mathrm{SoftPlus}\!\bigl(p_k^{(c)}\bigr), \qquad \alpha_k^{(c)}=e_k^{(c)}+1,$$8 to $$e_k^{(c)}=\mathrm{SoftPlus}\!\bigl(p_k^{(c)}\bigr), \qquad \alpha_k^{(c)}=e_k^{(c)}+1,$$9 accuracy improvement on ImageNet datasets under varying computing resources, and the ablations attribute part of this gain to the balance term and the guidance loss (Zhang et al., 2024). In trusted multi-view classification, TMUR is evaluated on fourteen public multi-view benchmarks against fifteen recent trusted and non-trusted methods, and is reported to improve classification accuracy while reducing ECE by approximately $S^{(c)}=\sum_k \alpha_k^{(c)}$0 points on average (Zhang et al., 10 Apr 2026). In challenging-tailed sequence learning, UME reports a performance gain of up to $S^{(c)}=\sum_k \alpha_k^{(c)}$1 over the best baseline on individual categories while reducing trainable parameters by up to $S^{(c)}=\sum_k \alpha_k^{(c)}$2 through Ensemble LoRA (Wang et al., 16 Mar 2026).

For multi-modal object detection on KITTI, Zhang et al. report single-modal clean performance of $S^{(c)}=\sum_k \alpha_k^{(c)}$3 mAP for RGB and $S^{(c)}=\sum_k \alpha_k^{(c)}$4 mAP for Depth. Fusion without selection reaches $S^{(c)}=\sum_k \alpha_k^{(c)}$5 mAP, while UMF fusion reaches $S^{(c)}=\sum_k \alpha_k^{(c)}$6 mAP on clean data. Under single-modality corruption, UMF remains comparatively stable: when only RGB is corrupted, mAP stays around $S^{(c)}=\sum_k \alpha_k^{(c)}$7 across Gaussian noise levels $S^{(c)}=\sum_k \alpha_k^{(c)}$8–$S^{(c)}=\sum_k \alpha_k^{(c)}$9, and when only Depth is corrupted, mAP remains around $u^{(c)}=\frac{K}{S^{(c)}}.$0, far above the corresponding degraded single-modal depth detector (Zhang et al., 2023). The stated interpretation is that uncertainty-aware fusion learns to trust whichever modality is more informative on a per-object basis.

UMoE reports a maximum of $u^{(c)}=\frac{K}{S^{(c)}}.$1, $u^{(c)}=\frac{K}{S^{(c)}}.$2, and $u^{(c)}=\frac{K}{S^{(c)}}.$3 performance gain compared with state-of-the-art proposal-level multi-modal detectors under extreme weather, adversarial, and blinding attack scenarios. The detailed adverse-scene comparisons show gains over a CLOCs baseline in KITTIAdv, KITTIBlind, fog, and snow, with $u^{(c)}=\frac{K}{S^{(c)}}.$4 in all adverse cases over $u^{(c)}=\frac{K}{S^{(c)}}.$5 runs (Lou et al., 2023). In 3D panoptic segmentation, UP-Fuse reports Panoptic Quality on Panoptic nuScenes of $u^{(c)}=\frac{K}{S^{(c)}}.$6 for LiDAR-only, $u^{(c)}=\frac{K}{S^{(c)}}.$7 for IAL, and $u^{(c)}=\frac{K}{S^{(c)}}.$8 for LiDAR-camera UP-Fuse. Under complete camera dropout, IAL drops from $u^{(c)}=\frac{K}{S^{(c)}}.$9 while UP-Fuse drops from $b_k^{(c)}=(\alpha_k^{(c)}-1)/S^{(c)}$0, remaining above the LiDAR-only model (Mohan et al., 22 Feb 2026).

In medical segmentation, UGDD-Net is reported to achieve state-of-the-art performance on ISIC2017, ISIC2018, PH2, and HAM10000, especially on “Hard Samples,” and its uncertainty maps are said to align with expert inter-observer variability (Dai et al., 10 May 2026). In deraining, UMFFNet is reported to achieve significant performance improvements with few parameters and to surpass state-of-the-art image deraining methods, though the summary available here does not include the detailed PSNR/SSIM tables (Tong et al., 2023).

6. Misconceptions, limitations, and methodological tensions

A recurrent misconception in adaptive inference is that the last classifier head always dominates earlier exits. CDM/GCDM explicitly reject this assumption, reporting that earlier classifier heads can outperform the last head for certain classes; this is the direct motivation for collaborative decision fusion rather than simple last-exit selection (Zhang et al., 2024). Another common assumption is that independently trained branches produce uncertainty values on a common numeric scale. TMUR argues that this assumption is fragile because views differ in feature space, noise level, and semantic granularity, and because independently trained branches are optimized for prediction correctness rather than cross-view consistency in evidence strength (Zhang et al., 10 Apr 2026).

Related concerns appear in proposal-level fusion. UMoE states that raw entropy alone can be misleading when modalities have different baseline confidences, and therefore introduces a classification deviation ratio calibrated against true-positive statistics on a held-out clear validation set (Lou et al., 2023). In feature-level fusion, UP-Fuse resolves the reliability problem asymmetrically by fixing $b_k^{(c)}=(\alpha_k^{(c)}-1)/S^{(c)}$1 and only down-weighting the camera branch (Mohan et al., 22 Feb 2026). This suggests that some UMF systems are designed not for symmetric consensus, but for graceful degradation toward a privileged fallback modality.

The surveyed methods also expose a tension between local and global arbitration. DST- or EDL-based systems often let each branch self-report uncertainty and then use that uncertainty directly in fusion. TMUR argues that when reliability is sample-dependent and cross-view context matters, branch-local arbitration is suboptimal and a unified global router is preferable (Zhang et al., 10 Apr 2026). A plausible implication is that future UMF research will increasingly decouple uncertainty estimation from fusion authority, especially in settings where branch scales are incomparable.

Taken together, these works define UMF as a technical program centered on conditional trust. Its core question is not merely how to combine branches, but how to quantify when each branch should be trusted, corrected, ignored, or allowed to dominate. Across deraining, classification, detection, segmentation, and panoptic perception, the answer has converged on uncertainty-aware weighting, gating, routing, or rectification—while the precise semantics and comparability of uncertainty remain the principal unresolved issue.