Uncertainty Modal Modeling in Multimodal AI

• Uncertainty Modal Modeling (UMM) is a framework that explicitly models uncertainty in modalities, latent modes, and parameters to regulate multimodal learning.
• It employs probabilistic representations, evidential prediction, and consistency-guided transfer to mitigate challenges like noisy data, missing channels, and modality misalignment.
• UMM is validated across domains such as remote sensing, medical imaging, and autonomous driving, showing improved calibration, robustness, and safe control under uncertainty.

Uncertainty Modal Modeling (UMM) denotes a family of formulations in which uncertainty tied to modalities, latent modes, regions, samples, dynamics, or modal parameters is modeled explicitly and then used to regulate representation learning, fusion, retrieval, generation, inference, calibration, or control. Across recent work, UMM appears in multimodal learning systems that must withstand noisy data, low-quality labels, heterogeneous modality characteristics, missing or corrupted channels, temporal misalignment, one-to-many target ambiguity, and modality failure. The central recurrent idea is not merely to estimate uncertainty post hoc, but to make uncertainty operational inside the learning or decision process itself, for example by weighting cross-modal transfer through semantic consistency, routing inputs toward low-uncertainty experts, replacing point embeddings with probabilistic representations, or enforcing probabilistic safety constraints (Jang, 18 Nov 2025).

1. Conceptual scope and terminology

In the multimodal learning literature, UMM is motivated by the observation that different sources are rarely equally trustworthy. The thesis on consistency-guided cross-modal transfer frames the problem in settings where uncertainty arises from observation noise in one modality, missing or corrupted channels, incomplete alignment between modalities, and imperfect labels or supervision, with particular emphasis on human-computer interaction and brain-computer interface scenarios (Jang, 18 Nov 2025). Related work in remote sensing describes temporally induced cross-modal inconsistency as aleatoric uncertainty, because optical, LiDAR, or SAR acquisitions may be separated by days or months and therefore no longer depict the same semantic state (Gui et al., 5 Sep 2025). In dynamic fusion, unexpected modality failure is treated as model uncertainty over which observation mechanism is currently valid (Liu, 2021). In conditional generation, uncertainty is associated with one-to-many mappings rather than generic Gaussian noise (Qiu et al., 2020).

Taken together, these works suggest that UMM functions less as a single algorithm than as a recurrent design principle: uncertainty is elevated from an auxiliary confidence score to a first-class structural variable that governs multimodal interaction.

Paradigm	Uncertainty carrier	Representative papers
Latent-mode modeling	Discrete latent code probabilities	(Qiu et al., 2020)
Probabilistic representation learning	Gaussian embeddings, patch Gaussians, region Gaussians	(Pishdad et al., 2022, Wan et al., 7 Jul 2025, Liu et al., 11 Nov 2025)
Reliability-aware fusion	Consistency weights, uncertainty maps, modality gates	(Jang, 18 Nov 2025, Wang et al., 2022, Wu et al., 27 Jan 2026)
Model-set uncertainty	Binary modality usefulness and $2^n$ candidate models	(Liu, 2021)
Evidential prediction	Normal Inverse Gamma and Dirichlet evidence	(Marvi et al., 7 Mar 2025)
Calibration-oriented uncertain inference	Confidence-bin distributions and semantic entropy	(Zhang et al., 9 Apr 2026, Zhang et al., 9 Jun 2025)

A separate but related usage appears in hierarchical Bayesian operational modal analysis, where “modal” refers to vibration modes rather than data modalities. There, uncertainty is attached to modal parameters across datasets through a Gaussian hyper-probability model, separating identification precision from inter-dataset variability (Sedehi et al., 2019). This suggests that the phrase “Uncertainty Modal Modeling” spans both multimodal AI and probabilistic modal identification, united by the explicit modeling of uncertainty over structured modal entities.

2. Mathematical forms of uncertainty representation

A major axis of UMM variation is the object in which uncertainty is encoded. One prominent line represents uncertainty as probability mass over discrete latent modes. “Modal uncertainty estimation” models one-to-many conditional prediction through a discrete latent variable $\mathbf{c}\in\mathcal{C}$, with

$$p_\theta(\mathbf{y}\mid \mathbf{x})=\sum_{\mathbf{c}\in\mathcal{C}} p_\theta(\mathbf{y}\mid \mathbf{c},\mathbf{x})\,p_\theta(\mathbf{c}\mid \mathbf{x}),$$

so that uncertainty is literally the distribution over latent mode hypotheses $p_\theta(\mathbf{c}\mid\mathbf{x})$. This formulation is motivated by the multimodal posterior collapse problem of Gaussian conditional VAEs and is trained with a deterministic posterior encoder plus vector-quantization-style code assignment (Qiu et al., 2020).

A second line embeds uncertainty directly into representations. In cross-modal retrieval, images and captions are mapped to Gaussian distributions

$$p(i)=\mathcal{N}(\mu_i,\Sigma_i), \qquad p(c)=\mathcal{N}(\mu_c,\Sigma_c),$$

and matched using negative KL divergence, negative minimum KL divergence, or negative 2-Wasserstein distance. The per-instance uncertainty score is defined as $U(x)=\log\det(\Sigma_x)$, turning ambiguity into a geometric property of the shared latent space (Pishdad et al., 2022). Region-centric variants refine this principle. GRM models each prompted visual region as

$$\mathbf{r}_k \sim \mathcal{N}(\boldsymbol{\mu}_k,\operatorname{diag}(\boldsymbol{\sigma}_k^2)),$$

then samples patch-aware uncertainty-aware region embeddings for fine-grained image-text alignment (Liu et al., 11 Nov 2025). UGG-ReID similarly maps each class or patch token to $\mathcal{N}(\mu_i^m,(\sigma_i^m)^2)$, propagates mean and variance separately over a Gaussian patch graph, and later lifts the full sample embedding to a second Gaussian for sample-level uncertainty (Wan et al., 7 Jul 2025).

A third line uses probabilistic latent variables for patch or modality repair. MURTreeFormer models each auxiliary or primary patch feature as

$$p(z_i\mid x_i^M)=\mathcal{N}\!\left(\mu_i^M,\operatorname{diag}(\sigma_i^M)\right),$$

scores cross-modal inconsistency via

$$v_i=\log_2 \frac{|\operatorname{diag}(\sigma_i^A)|}{|\operatorname{diag}(\sigma_i^P)|},$$

and reconstructs the top-$\mathbf{c}\in\mathcal{C}$0 uncertain auxiliary patches by sampling from the primary modality’s latent distribution (Gui et al., 5 Sep 2025). In multimodal recommendation, SPUMR estimates $\mathbf{c}\in\mathcal{C}$1 for each entity-modality pair and uses reparameterized Gaussian experts in uncertainty-aware preference aggregation (Wu et al., 27 Jan 2026).

A fourth line is evidential. For multi-modal trajectory prediction, positional uncertainty is modeled with a Normal Inverse Gamma distribution over Gaussian parameters, while mode uncertainty is modeled with a Dirichlet distribution over trajectory-mode probabilities. The decomposition

$\mathbf{c}\in\mathcal{C}$2

makes aleatoric and epistemic contributions explicit, and classification uncertainty is summarized by $\mathbf{c}\in\mathcal{C}$3 (Marvi et al., 7 Mar 2025).

A fifth line uses uncertainty over model hypotheses. Robust dynamic multi-modal data fusion introduces a binary usefulness variable $\mathbf{c}\in\mathcal{C}$4 for each modality, generating $\mathbf{c}\in\mathcal{C}$5 hypothetical models whose posterior weights are updated by dynamic model averaging (Liu, 2021). In hierarchical Bayesian operational modal analysis, each dataset-specific modal parameter vector $\mathbf{c}\in\mathcal{C}$6 is treated as a draw from

$\mathbf{c}\in\mathcal{C}$7

so uncertainty resides simultaneously in the dataset-level posterior and the population-level hyperdistribution (Sedehi et al., 2019).

3. How UMM regulates learning, fusion, and inference

In many systems, the distinctive feature of UMM is not the uncertainty representation itself but the way that representation controls information flow. The consistency-guided cross-modal transfer thesis states that a modality should borrow information from another modality only when their semantic predictions or latent representations are sufficiently consistent. A typical objective combines task supervision with a consistency loss

$\mathbf{c}\in\mathcal{C}$8

where the weights $\mathbf{c}\in\mathcal{C}$9 act as reliability terms. This yields an asymmetric transfer rule: reliable modalities teach uncertain ones, but only under sufficiently high consistency to avoid negative transfer (Jang, 18 Nov 2025).

CRNP uses a different mechanism. Its uncertainty signal is the discrepancy between a fixed random network $$p_\theta(\mathbf{y}\mid \mathbf{x})=\sum_{\mathbf{c}\in\mathcal{C}} p_\theta(\mathbf{y}\mid \mathbf{c},\mathbf{x})\,p_\theta(\mathbf{c}\mid \mathbf{x}),$$0 and a low-capacity predictor $$p_\theta(\mathbf{y}\mid \mathbf{x})=\sum_{\mathbf{c}\in\mathcal{C}} p_\theta(\mathbf{y}\mid \mathbf{c},\mathbf{x})\,p_\theta(\mathbf{c}\mid \mathbf{x}),$$1, trained by

$$p_\theta(\mathbf{y}\mid \mathbf{x})=\sum_{\mathbf{c}\in\mathcal{C}} p_\theta(\mathbf{y}\mid \mathbf{c},\mathbf{x})\,p_\theta(\mathbf{c}\mid \mathbf{x}),$$2

The resulting feature-density-based uncertainty is then used cross-modally to reweight features before self-attentive fusion, so that uncertain modalities are down-weighted or complemented by the other modality’s information (Wang et al., 2022).

Several methods implement uncertainty-aware gating or routing. SPUMR first refines modality features using a Modality Similarity Graph and a Collaborative Similarity Graph, then applies a gate

$$p_\theta(\mathbf{y}\mid \mathbf{x})=\sum_{\mathbf{c}\in\mathcal{C}} p_\theta(\mathbf{y}\mid \mathbf{c},\mathbf{x})\,p_\theta(\mathbf{c}\mid \mathbf{x}),$$3

and regularizes it with

$$p_\theta(\mathbf{y}\mid \mathbf{x})=\sum_{\mathbf{c}\in\mathcal{C}} p_\theta(\mathbf{y}\mid \mathbf{c},\mathbf{x})\,p_\theta(\mathbf{c}\mid \mathbf{x}),$$4

so that high-variance modalities receive lower fusion weight (Wu et al., 27 Jan 2026). UGG-ReID routes sample embeddings to low-uncertainty experts with an uncertainty-guided mixture-of-experts strategy and supplements this with an expert-balance regularizer (Wan et al., 7 Jul 2025). In multi-modal healthcare regression, uncertainty-aware boosted ensembling replaces loss-based boosting with predictive-uncertainty-based boosting, and the weighted variant aggregates predictions by inverse uncertainty,

$$p_\theta(\mathbf{y}\mid \mathbf{x})=\sum_{\mathbf{c}\in\mathcal{C}} p_\theta(\mathbf{y}\mid \mathbf{c},\mathbf{x})\,p_\theta(\mathbf{c}\mid \mathbf{x}),$$5

using uncertainty both during boosting and at inference (Sarawgi et al., 2021).

Calibration-oriented formulations use uncertainty to shape outputs rather than fusion. CLUE discretizes $$p_\theta(\mathbf{y}\mid \mathbf{x})=\sum_{\mathbf{c}\in\mathcal{C}} p_\theta(\mathbf{y}\mid \mathbf{c},\mathbf{x})\,p_\theta(\mathbf{c}\mid \mathbf{x}),$$6 into $$p_\theta(\mathbf{y}\mid \mathbf{x})=\sum_{\mathbf{c}\in\mathcal{C}} p_\theta(\mathbf{y}\mid \mathbf{c},\mathbf{x})\,p_\theta(\mathbf{c}\mid \mathbf{x}),$$7 confidence bins, trains on a Gaussian target $$p_\theta(\mathbf{y}\mid \mathbf{x})=\sum_{\mathbf{c}\in\mathcal{C}} p_\theta(\mathbf{y}\mid \mathbf{c},\mathbf{x})\,p_\theta(\mathbf{c}\mid \mathbf{x}),$$8 with $$p_\theta(\mathbf{y}\mid \mathbf{x})=\sum_{\mathbf{c}\in\mathcal{C}} p_\theta(\mathbf{y}\mid \mathbf{c},\mathbf{x})\,p_\theta(\mathbf{c}\mid \mathbf{x}),$$9, and reconstructs the scalar probability by weighted averaging over bin probabilities. The method combines self-consistent teacher calibration, distribution-based confidence probing, and modality-specific batching (Zhang et al., 9 Apr 2026). Uncertainty-o instead perturbs multimodal prompts, clusters semantically equivalent responses, and computes multimodal semantic entropy

$p_\theta(\mathbf{c}\mid\mathbf{x})$0

treating response variability under semantic-preserving perturbations as an uncertainty signal (Zhang et al., 9 Jun 2025).

4. Major application domains

UMM has been instantiated across a wide range of problem classes. In multimodal affect recognition and related HCI/BCI settings, consistency-guided cross-modal transfer is proposed as a unified uncertainty-resilient framework that projects heterogeneous modalities into a shared latent space and uses semantic consistency to stabilize feature learning under noise, incompleteness, and imperfect supervision (Jang, 18 Nov 2025). In remote sensing tree-cover segmentation, MURTreeFormer uses a selective uncertainty-guided reconstruction module, primary/auxiliary modality asymmetry, a VAE-based resampling mechanism, gradient magnitude attention, and a lightweight refinement head to mitigate temporally induced aleatoric uncertainty (Gui et al., 5 Sep 2025).

In medical imaging, UMM-CSGM reformulates missing-modality completion as stochastic conditional score-based generation in the full modality space, using a multi-in multi-out Conditional Score Network to support all missing-modality configurations in one model (Meng et al., 2022). CRNP addresses both 3D multi-modal medical image segmentation and 2D computer-vision classification with the same feature-density uncertainty mechanism (Wang et al., 2022). Evidential trajectory prediction extends UMM to autonomous driving by jointly estimating future trajectories, positional uncertainty, and mode-probability uncertainty in a single forward pass on top of HiVT (Marvi et al., 7 Mar 2025).

In retrieval and identification, probabilistic cross-modal embeddings have been used for image-text matching, region-uncertain modeling has been used for fine-grained image-text alignment, a lightweight CLIP-based UMM framework has been proposed for pedestrian ReID under uncertain or missing modalities, and UGG-ReID has targeted RGB/NIR/TIR person and vehicle ReID with Gaussian patch graphs and uncertainty-guided expert routing (Pishdad et al., 2022, Liu et al., 11 Nov 2025, Li et al., 15 Aug 2025, Wan et al., 7 Jul 2025). Recommendation systems use uncertainty-aware multimodal aggregation after graph-based similarity propagation, explicitly treating blurry images, visually diverse products, and vague text as modality-specific uncertainty sources (Wu et al., 27 Jan 2026).

UMM also extends beyond perception. Unified Multimodal Uncertain Inference formalizes calibrated scalar probability estimation over text, audio, video, and audiovisual premises, rather than binary entailment judgments (Zhang et al., 9 Apr 2026). Robust dynamic multi-modal data fusion treats modality failure as model uncertainty and performs Bayesian model averaging over usefulness configurations (Liu, 2021). Robust safe control treats dynamics uncertainty as a discrete mixture of Gaussian modes and derives least-conservative or locally least-conservative safe control under additive or multiplicative multi-modal uncertainty (Wei et al., 2023). Hierarchical Bayesian operational modal analysis uses a Gaussian hyper-probability model to quantify both identification precision and ensemble variability of modal parameters across repeated experiments (Sedehi et al., 2019).

5. Empirical evaluation and recurring performance patterns

Reported evaluations indicate that UMM methods are typically assessed not only by task accuracy but also by calibration, robustness under corruption or incompleteness, and degradation under adverse conditions. MURTreeFormer reports on Zurich and Shanghai that it achieves mIoU $p_\theta(\mathbf{c}\mid\mathbf{x})$1, tree-cover IoU $p_\theta(\mathbf{c}\mid\mathbf{x})$2, and F1 $p_\theta(\mathbf{c}\mid\mathbf{x})$3 on Zurich, and mIoU $p_\theta(\mathbf{c}\mid\mathbf{x})$4, tree-cover IoU $p_\theta(\mathbf{c}\mid\mathbf{x})$5, and F1 $p_\theta(\mathbf{c}\mid\mathbf{x})$6 on Shanghai, outperforming baselines such as SegFormer, UNetFormer, FTransUNet, FuseNet, and ASANet (Gui et al., 5 Sep 2025). UMM-CSGM on BraTS19 reports, for example, PSNR $p_\theta(\mathbf{c}\mid\mathbf{x})$7, SSIM $p_\theta(\mathbf{c}\mid\mathbf{x})$8, and MAE $p_\theta(\mathbf{c}\mid\mathbf{x})$9 for missing Flair, with similarly strong results for missing T1, T1c, and T2 (Meng et al., 2022).

Calibration-centric methods report explicit uncertainty metrics. CLUE-D achieves UNLI MSE $$p(i)=\mathcal{N}(\mu_i,\Sigma_i), \qquad p(c)=\mathcal{N}(\mu_c,\Sigma_c),$$0, WV-V $$p(i)=\mathcal{N}(\mu_i,\Sigma_i), \qquad p(c)=\mathcal{N}(\mu_c,\Sigma_c),$$1, WV-A $$p(i)=\mathcal{N}(\mu_i,\Sigma_i), \qquad p(c)=\mathcal{N}(\mu_c,\Sigma_c),$$2, and WV-AV $$p(i)=\mathcal{N}(\mu_i,\Sigma_i), \qquad p(c)=\mathcal{N}(\mu_c,\Sigma_c),$$3, and appendix values of UNLI MSE $$p(i)=\mathcal{N}(\mu_i,\Sigma_i), \qquad p(c)=\mathcal{N}(\mu_c,\Sigma_c),$$4, NLL $$p(i)=\mathcal{N}(\mu_i,\Sigma_i), \qquad p(c)=\mathcal{N}(\mu_c,\Sigma_c),$$5, ECE $$p(i)=\mathcal{N}(\mu_i,\Sigma_i), \qquad p(c)=\mathcal{N}(\mu_c,\Sigma_c),$$6, while binary results include CLUE-T $$p(i)=\mathcal{N}(\mu_i,\Sigma_i), \qquad p(c)=\mathcal{N}(\mu_c,\Sigma_c),$$7 on Clotho and CLUE-D $$p(i)=\mathcal{N}(\mu_i,\Sigma_i), \qquad p(c)=\mathcal{N}(\mu_c,\Sigma_c),$$8 on WV-V (Zhang et al., 9 Apr 2026). Uncertainty-o evaluates AUROC, AURAC, and ECE across 18 benchmarks and 10 LMMs, reporting, among other values, AUROC $$p(i)=\mathcal{N}(\mu_i,\Sigma_i), \qquad p(c)=\mathcal{N}(\mu_c,\Sigma_c),$$9 on ClothoV2, $U(x)=\log\det(\Sigma_x)$0 AUROC and $U(x)=\log\det(\Sigma_x)$1 AURAC on MMVet, and AUROC/AURAC/ECE of $U(x)=\log\det(\Sigma_x)$2 on Flickr generation hallucination detection (Zhang et al., 9 Jun 2025).

In retrieval and recommendation, uncertainty-aware representation learning is evaluated with both classical task metrics and ambiguity-sensitive analyses. Probabilistic VSRN on MS-COCO 5K reports image-to-text R@1 $U(x)=\log\det(\Sigma_x)$3 and text-to-image R@1 $U(x)=\log\det(\Sigma_x)$4, while also improving PMRP and performing well on CxC, where plausible matches are more richly annotated (Pishdad et al., 2022). SPUMR reports gains over the best prior methods of $U(x)=\log\det(\Sigma_x)$5 R@10, $U(x)=\log\det(\Sigma_x)$6 R@20, $U(x)=\log\det(\Sigma_x)$7 N@10, and $U(x)=\log\det(\Sigma_x)$8 N@20 on Baby, with corresponding gains on Sports and Clothing (Wu et al., 27 Jan 2026).

In forecasting and dynamic estimation, evidential trajectory prediction on Argoverse 2 reports minADE $U(x)=\log\det(\Sigma_x)$9, minFDE $$\mathbf{r}_k \sim \mathcal{N}(\boldsymbol{\mu}_k,\operatorname{diag}(\boldsymbol{\sigma}_k^2)),$$0, ECE $$\mathbf{r}_k \sim \mathcal{N}(\boldsymbol{\mu}_k,\operatorname{diag}(\boldsymbol{\sigma}_k^2)),$$1, and inference time $$\mathbf{r}_k \sim \mathcal{N}(\boldsymbol{\mu}_k,\operatorname{diag}(\boldsymbol{\sigma}_k^2)),$$2 sec, while the uncertainty-selected $$\mathbf{r}_k \sim \mathcal{N}(\boldsymbol{\mu}_k,\operatorname{diag}(\boldsymbol{\sigma}_k^2)),$$3 training subset attains minADE $$\mathbf{r}_k \sim \mathcal{N}(\boldsymbol{\mu}_k,\operatorname{diag}(\boldsymbol{\sigma}_k^2)),$$4, wADE $$\mathbf{r}_k \sim \mathcal{N}(\boldsymbol{\mu}_k,\operatorname{diag}(\boldsymbol{\sigma}_k^2)),$$5, minFDE $$\mathbf{r}_k \sim \mathcal{N}(\boldsymbol{\mu}_k,\operatorname{diag}(\boldsymbol{\sigma}_k^2)),$$6, wFDE $$\mathbf{r}_k \sim \mathcal{N}(\boldsymbol{\mu}_k,\operatorname{diag}(\boldsymbol{\sigma}_k^2)),$$7, and MR $$\mathbf{r}_k \sim \mathcal{N}(\boldsymbol{\mu}_k,\operatorname{diag}(\boldsymbol{\sigma}_k^2)),$$8, outperforming full training on several metrics (Marvi et al., 7 Mar 2025). Dynamic model averaging for robust multi-modal fusion reports overall averaged RMSE $$\mathbf{r}_k \sim \mathcal{N}(\boldsymbol{\mu}_k,\operatorname{diag}(\boldsymbol{\sigma}_k^2)),$$9, compared with $\mathcal{N}(\mu_i^m,(\sigma_i^m)^2)$0 for PF, $\mathcal{N}(\mu_i^m,(\sigma_i^m)^2)$1 for TS, and $\mathcal{N}(\mu_i^m,(\sigma_i^m)^2)$2 for SMA, with only slight runtime increase relative to PF (Liu, 2021). In healthcare regression, uncertainty-aware boosting reports ADReSS RMSE $\mathcal{N}(\mu_i^m,(\sigma_i^m)^2)$3 for UA Ensemble and $\mathcal{N}(\mu_i^m,(\sigma_i^m)^2)$4 for UA Ensemble (weighted), versus $\mathcal{N}(\mu_i^m,(\sigma_i^m)^2)$5 for the vanilla ensemble, and Parkinson’s RMSE $\mathcal{N}(\mu_i^m,(\sigma_i^m)^2)$6 for UA Ensemble versus $\mathcal{N}(\mu_i^m,(\sigma_i^m)^2)$7 for the vanilla ensemble (Sarawgi et al., 2021).

A recurring empirical pattern is that UMM methods are strongest when evaluation stresses exactly the phenomena they model: ambiguity, modality corruption, missing-modality settings, noisy labels, weak supervision, hallucination, or control infeasibility. This suggests that their principal advantage is not merely higher nominal accuracy, but lower performance collapse under uncertainty-heavy operating regimes.

6. Distinctions, misconceptions, and unresolved issues

One common misconception is that UMM is synonymous with Bayesian inference. The literature does not support that reduction. Some methods are explicitly probabilistic or evidential, such as Normal Inverse Gamma/Dirichlet trajectory prediction, conditional diffusion for image completion, or hierarchical Gaussian hyper-modeling in operational modal analysis (Marvi et al., 7 Mar 2025, Meng et al., 2022, Sedehi et al., 2019). Others emphasize reliability estimation without “heavy Bayesian machinery,” as in consistency-guided cross-modal transfer, or feature-density discrepancy, as in CRNP (Jang, 18 Nov 2025, Wang et al., 2022).

A second misconception is that uncertainty is always epistemic. Several works explicitly target aleatoric mechanisms: temporal cross-modal misalignment in remote sensing, local occlusion and degraded capture in ReID, and input heteroscedasticity in healthcare regression (Gui et al., 5 Sep 2025, Wan et al., 7 Jul 2025, Sarawgi et al., 2021). By contrast, dynamic model averaging over modality usefulness is explicitly a form of model uncertainty over the observation mechanism, and safe control under Gaussian mixture dynamics addresses uncertainty over discrete physical regimes (Liu, 2021, Wei et al., 2023).

A third misconception is that uncertainty is merely diagnostic. In UMM, uncertainty frequently determines the computation itself: it decides whether one modality may teach another, which patches should be reconstructed, which expert should process a sample, how modalities are weighted, which training samples are prioritized, or whether a control action is feasible (Jang, 18 Nov 2025, Gui et al., 5 Sep 2025, Wan et al., 7 Jul 2025, Marvi et al., 7 Mar 2025, Wei et al., 2023).

The literature also records several limitations and balancing conditions. Uncertainty-o reports that semantic-preserving perturbations are better than semantic-altering ones, that perturbing multiple modalities simultaneously is preferable, and that too many perturbation samples can overestimate uncertainty, with five perturbation samples as the default sweet spot (Zhang et al., 9 Jun 2025). GRM notes that overly emphasizing uncertainty-aware similarity can hurt performance and reports best results at $\mathcal{N}(\mu_i^m,(\sigma_i^m)^2)$8, $\mathcal{N}(\mu_i^m,(\sigma_i^m)^2)$9 (Liu et al., 11 Nov 2025). Hierarchical Bayesian operational modal analysis observes that for a small number of datasets the hyper covariance matrix is often unidentifiable, motivating an eigenbasis transformation remedy (Sedehi et al., 2019). Safe control work shows that uni-modal approximations can become overly conservative and, under control limits, may even become infeasible (Wei et al., 2023).

These distinctions indicate that UMM is best understood as a structured response to modality-dependent unreliability. The specific mathematics vary—from discrete codebooks, Gaussian regions, and evidential heads to semantic entropy and model averaging—but the shared objective is stable: to identify what is trustworthy, quantify what is not, and use that quantification to change the behavior of the multimodal system itself.