---
title: Multimodal Uncertainty Propagation Model (MUPM)
url: https://www.emergentmind.com/topics/multimodal-uncertainty-propagation-model-mupm
type: topic
---

# Multimodal Uncertainty Propagation Model (MUPM)

The Multimodal Uncertainty Propagation Model (MUPM) denotes a class of multimodal learning formulations in which uncertainty is treated as an explicit object of representation, transformation, and fusion rather than as a by-product of final prediction scores. In the cited literature, the term has been used for technically distinct systems: graph-based multimodal recommendation with modality-specific Gaussian uncertainty, active multi-agent collaboration with inverse-variance fusion, uncertainty decomposition for multimodal large language models (MLLMs), discounted evidential fusion, dynamic model averaging for sensor failure, uncertainty-aware vision-language pre-training, deterministic feature-level uncertainty fusion in autonomous driving, and missing-modality reconstruction with statistical error propagation [2601.19198][2602.04763][2507.12945]. Across these variants, the common premise is that multimodal inputs are noisy, incomplete, conflicting, or failure-prone, and that robust inference requires propagating modality-level uncertainty through the architecture rather than collapsing all modalities into deterministic point embeddings at the outset.

## 1. Taxonomic scope and recurring design pattern

The term is not tied to a single canonical architecture. Instead, the literature presents a recurring design pattern: estimate uncertainty for each modality, agent, view, or latent reconstruction; propagate that uncertainty through graph operators, attention-like weighting, evidential discounting, model averaging, or first-order error analysis; then fuse modalities in a way that suppresses unreliable inputs and preserves informative ones [2412.18024][2105.06018][2210.05335][2503.20011][2504.13465].

| Instantiation | Uncertainty object | Propagation / fusion mechanism |
|---|---|---|
| SPUMR | Gaussian modality uncertainty for users/items | Similarity graphs + uncertainty-aware preference aggregation |
| A2MAML | Diagonal Gaussian per agent-modality pair | Active selection + Bayesian inverse-variance weighting |
| Tang et al. MUPM | Image-only, text-only, and joint output variances | Three-parameter linear uncertainty propagation model |
| DBF | Subjective-Logic opinions \((b,u,a)\) | Conflict-based discounting + order-invariant belief averaging |
| DMA-PF formulation | Binary modality usefulness vector | Dynamic model averaging over \(2^n\) hypothetical models |
| MAP | Gaussian token and cross-modal distributions | Wasserstein-based alignment + distributional pre-training |
| HyperDUM | Deterministic feature-level epistemic uncertainty | Hyperdimensional projection/bundling + adaptive weighting |
| SURE | Reconstruction variance and output variance | Latent reconstruction + first-order statistical error propagation |

A recurrent misconception is that uncertainty propagation in multimodal systems is equivalent to post hoc calibration at the classifier output. The surveyed formulations show otherwise: uncertainty may be attached to item embeddings, agent-modality features, subjective opinions, latent reconstruction channels, or dynamic sensor usefulness, and it may influence fusion before the downstream decision is computed [2601.19198][2503.20011].

## 2. Mathematical representations of multimodal uncertainty

A major branch of MUPM models each modality-specific feature as a Gaussian random variable. In SPUMR, each refined representation \(h_e^{c,m}\) for entity \(e\in\mathcal U\cup\mathcal I\) and modality \(m\) is assumed to be sampled from \(N(\mu_e^m,(\sigma_e^m)^2)\), where \(\sigma_e^m\) measures aleatoric uncertainty. The model estimates
\[
\mu_e^m=\mathrm{MLP}_\mu(h_e^{c,m}), \qquad \log (\sigma_e^m)^2=\mathrm{MLP}_\sigma(h_e^{c,m}),
\]
and samples
\[
z_e^m=\mu_e^m+\sigma_e^m\odot \epsilon,\qquad \epsilon\sim N(0,I),
\]
with a KL regularizer to \(N(0,I)\) [2601.19198]. A2MAML uses the same latent-Gaussian logic at the agent-modality level:
\[
q(\mathbf f_{i,m}\mid \mathbf x_{i,m})=\mathcal N(\mathbf f_{i,m};\boldsymbol\mu_{i,m},\mathrm{diag}(\boldsymbol\sigma^2_{i,m})),
\]
where \(\boldsymbol\mu_{i,m}\) is the embedding and \(\boldsymbol\sigma^2_{i,m}\) is the learned aleatoric variance. A compact scalar token
\[
\rho_{i,m}=\frac1d\sum_{j=1}^d \sigma^2_{i,m,j}
\]
is then used in active selection [2602.04763]. MAP likewise projects image and language embeddings into diagonal Gaussians \(q_m(z_m\|x_m)=\mathcal N(\mu_m,\Sigma_m)\), where \(\Sigma_m=\mathrm{diag}(\sigma_m^2)\), and places such distributions both on unimodal encoders and after the cross-modal transformer [2210.05335].

A second branch adopts evidential rather than Gaussian uncertainty. DBF represents each modality \(v\) by a Subjective-Logic opinion
\[
\omega^v=(b^v,u^v,a^v),
\]
where \(b^v\) is the belief mass vector, \(u^v\) is the uncertainty mass, and \(a^v\) is the base rate, subject to
\[
\sum_{k=1}^K b_k^v+u^v=1.
\]
The same information can be expressed with Dirichlet parameters \(\alpha_k^v=e_k^v+1\), with
\[
b_k^v=\frac{e_k^v}{S^v},\qquad u^v=\frac{K}{S^v},\qquad P_k^v=b_k^v+a_k^v u^v=\frac{\alpha_k^v}{S^v}
\]
[2412.18024].

A third branch treats uncertainty as latent model structure, not merely feature dispersion. In the DMA-particle-filter formulation, each modality has a binary usefulness indicator \(u_{i,t}\in\{0,1\}\), and the full usefulness vector \(u_t\in\{0,1\}^n\) induces \(J=2^n\) hypothetical generative models \(M_j\). Under \(M_j\), trustworthy modalities contribute ordinary likelihoods \(p(y_{i,t}\mid x_t)\), while useless modalities use null likelihoods \(L_i^0(y_{i,t})\) [2105.06018].

Tang et al. introduce a different MUPM for MLLMs: rather than placing a distribution directly on internal features, they define empirical image-only, text-only, and joint output variances under augmentation and fit the linear relation
\[
s_{F_{I,T}}^2=\beta_1 s_{F_I}^2+\beta_2 s_{F_T}^2+\beta_3 s_{F_I}s_{F_T}+\epsilon.
\]
This is derived from first-order uncertainty propagation and then implemented as a three-parameter regression because direct finite-difference sensitivity estimation in large MLLMs is numerically unstable [2507.12945].

SURE places uncertainty on reconstructed missing modalities and on the downstream head. Its central decomposition is
\[
\sigma_Y^2=\sigma_\omega^2+\sigma_{\text{input}}^2,
\]
where \(\sigma_\omega^2\) is model-intrinsic variance from a two-headed predictor and \(\sigma_{\text{input}}^2\) is obtained by first-order Taylor propagation of reconstruction variances through the frozen fusion module [2504.13465].

## 3. Propagation operators and uncertainty-aware fusion

Propagation in MUPM is operationalized in multiple ways. SPUMR performs propagation over two graph families. For each modality \(m\), the Modality Similarity Graph (MSG) builds user-user and item-item KNN graphs using cosine similarity,
\[
\mathrm{sim}(a,b)=\frac{a^\top b}{\|a\|_2\|b\|_2},
\]
and applies symmetrical GCN layers,
\[
H_{t+1}^m=\sigma\!\bigl(\tilde D^{-1/2}A^m\tilde D^{-1/2}H_t^mW^m\bigr).
\]
A Collaborative Similarity Graph (CSG) then injects behavioral smoothing using Jaccard similarities \(w_{uv}=|\mathcal I_u\cap \mathcal I_v|/|\mathcal I_u\cup \mathcal I_v|\) and weighted neighborhood aggregation
\[
h_e^{c,m}=\sum_{f\in\mathcal N_e^c\cup\{e\}} \tilde w_{ef}\,h_f^m.
\]
Fusion is performed by uncertainty-aware preference aggregation, either with inverse-uncertainty weights
\[
w_e^m=\frac{1/\sigma_e^m}{\sum_{k\in\mathcal M}1/\sigma_e^k},\qquad z_e=\sum_{m\in\mathcal M}w_e^m z_e^m,
\]
or, in SPUMR itself, with a small gating network and Top-K sparsification [2601.19198].

A2MAML also uses uncertainty as a control signal for selection and fusion, but at the granularity of agent-modality pairs. The active selector receives handshake tokens \((\rho_{0,m},\rho_{i,m})\), predicts accept/reject logits with an MLP policy \(\pi_\theta\), and uses Gumbel-softmax to produce binary mask variables \(Z_{i,m}\). Accepted features are fused by precision weighting:
\[
\hat{\mathbf f}=
\frac{\sum_{i,m} Z_{i,m}\mathbf W_{i,m}\boldsymbol\mu_{i,m}}
{\sum_{i,m} Z_{i,m}\mathbf W_{i,m}},
\qquad
\mathbf W_{i,m}=\bigl(\mathrm{diag}(\boldsymbol\sigma^2_{i,m})\bigr)^{-1}.
\]
The scalar shorthand is \(w_{i,m}=1/\bar\sigma_{i,m}^2\), with fusion by the corresponding weighted average [2602.04763].

DBF propagates uncertainty by conflict-sensitive discounting. Pairwise disagreement is quantified by
\[
\mathrm{PD}(\omega^i,\omega^j)=\frac12\sum_{k=1}^K|P_k^i-P_k^j|,
\qquad
\mathrm{CC}(\omega^i,\omega^j)=(1-u^i)(1-u^j),
\]
and
\[
\mathrm{DC}(\omega^i,\omega^j)=\mathrm{PD}\times \mathrm{CC}.
\]
The conflict matrix is converted into agreement scores
\[
A_{ij}=\bigl(1-C_{ij}^{\,\lambda}\bigr)^{1/\lambda},
\qquad
\eta^v=\prod_{i=1}^V A_{vi},
\]
after which each opinion is discounted by
\[
b_k^{v\prime}=\eta^v b_k^v,\qquad
u^{v\prime}=1-\eta^v+\eta^v u^v.
\]
All views are then fused simultaneously by generalized belief averaging, which is order-invariant because the formulas are symmetric in all modalities [2412.18024].

Other MUPM variants instantiate propagation without explicit probabilistic message passing. MAP aligns diagonal Gaussians from image and text encoders via the squared \(2\)-Wasserstein distance
\[
D_{2W}^2=\|\mu_1-\mu_2\|_2^2+\|\sigma_1-\sigma_2\|_2^2,
\]
converted into similarity \(s(I,T)=a\cdot D_{2W}^2+b\) [2210.05335]. HyperDUM projects modality features into hypervectors, builds context prototypes by bundling, computes similarity-based uncertainty sets \(\mathbb U_m^C\) and \(\mathbb U_m^P\), and feeds them into a \(1\times1\) uncertainty-weighting module \(\Omega\) to obtain modality weights \(w_m\) for re-scaling the original features [2503.20011]. SURE propagates uncertainty through a frozen deep fusion module by Jacobian-based first-order approximation,
\[
\sigma_{\text{input}}^2\approx \sum_{i\in\mathcal J}\sum_{d=1}^D
\left(\frac{\partial \omega}{\partial \widetilde Z_d^i}\right)^2 \tilde\sigma_{z^i}^2,
\]
which is then added to the head variance \(\sigma_\omega^2\) [2504.13465]. In the DMA formulation, propagation occurs at the model level through
\[
p(x_t\mid y_{1:t})\simeq \sum_{j=1}^J p(x_t\mid u_t=j,y_{1:t})\,\pi_{j,t},
\qquad
\pi_{j,t}=p(u_t=j\mid y_{1:t}),
\]
so uncertainty about modality usefulness becomes part of the filtering recursion itself [2105.06018].

Taken together, the literature suggests that “propagation” in MUPM is not a single operator. It may mean graph diffusion, precision-weighted aggregation, evidential discounting, model posterior recursion, Wasserstein-space alignment, prototype-similarity weighting, or Jacobian-based error transfer.

## 4. Objectives, estimation procedures, and training regimes

SPUMR uses Bayesian Personalized Ranking as its main objective,
\[
L_{\mathrm{BPR}}=-\sum_{u,i^+,i^-}\log \sigma\bigl(z_u^\top z_{i^+}-z_u^\top z_{i^-}\bigr),
\]
optionally adds a contrastive loss over positive items, and regularizes with both KL divergence and an uncertainty penalty
\[
L=L_{\mathrm{BPR}}+\lambda_{CL}L_{CL}+\lambda_{KL}L_{KL}+\lambda_U L_U,
\qquad
L_U=\sum_{e,m} w_e^m\|\sigma_e^m\|_2^2.
\]
Its algorithm initializes modality projections and uncertainty heads, propagates through modality and collaborative graphs, samples latent variables via reparameterization, computes fusion weights, samples negatives, and repeats until convergence, for example with early stopping on Recall@20 [2601.19198].

A2MAML combines learned variance estimation, active selection, and prediction. Each modality encoder has a mean head and a variance head, with positivity enforced by
\[
\boldsymbol\sigma_{i,m}^2=\mathrm{softplus}(\mathbf a_{i,m})=\ln(1+\exp(\mathbf a_{i,m})).
\]
The binary accept/reject mask is trained with a Gumbel-softmax reparameterization over logits from \(\pi_\theta\), and rejected pairs are removed entirely from the fusion sums [2602.04763].

MAP turns uncertainty-aware representation learning into a pre-training problem. Its full loss is
\[
\mathcal L_{\mathrm{pre}}=\mathcal L_{D\text{-}VLC}+\mathcal L_{D\text{-}MLM}+\mathcal L_{D\text{-}ITM}+\alpha\cdot \mathcal L_{\mathrm{reg}}.
\]
The three principal terms are Distribution-based Vision-Language Contrastive learning, Distribution-based Masked Language Modeling, and Distribution-based Image-Text Matching, all defined over Gaussian embeddings or samples from them. A variance-preserving regularizer
\[
\mathcal L_{\mathrm{reg}}=\max(0,\gamma-h(\mathcal N(\mu,\sigma^2)))
\]
prevents variance collapse [2210.05335].

SURE uses a two-phase procedure. In phase 1, latent reconstructors \(r^i\) are trained to produce both reconstructed latent codes \(\widetilde Z^i\) and reconstruction variances \(\tilde\sigma_{z^i}^2\). The loss combines MSE with a Pearson-correlation-based term
\[
\mathcal L_{\mathrm{PCC}}(v,e)=1-r(v,e),
\]
so that predicted variance correlates with the actual squared reconstruction error. In phase 2, reconstructors are frozen, a two-headed downstream head is trained, and the final uncertainty is computed as \(\sigma_Y^2=\sigma_\omega^2+\sigma_{\text{input}}^2\) [2504.13465].

Tang et al. use a markedly simpler estimation regime. For \(M\) held-out image-text pairs, they compute sample variances under image-only, text-only, and joint augmentations, form the design matrix with rows \([s_{F_I,i}^2,s_{F_T,i}^2,s_{F_I,i}s_{F_T,i}]\), and solve the ordinary least-squares problem
\[
\hat\beta=\arg\min_{\beta\in\mathbb R^3}\sum_{i=1}^M
\bigl(s_{F_{I,T},i}^2-\beta_1 s_{F_I,i}^2-\beta_2 s_{F_T,i}^2-\beta_3 s_{F_I,i}s_{F_T,i}\bigr)^2.
\]
No explicit regularization was needed in practice because the model has only three parameters and \(M\) can be as small as \(60\)–\(100\) [2507.12945].

## 5. Empirical domains and reported performance

The empirical record for MUPM is heterogeneous because the models target different tasks. In multimodal recommendation, SPUMR was evaluated on the Amazon-5core subsets “Baby,” “Sports,” and “Clothing,” using Recall@10/20 and NDCG@10/20. Against baselines including MF-BPR, LightGCN, VBPR, MMGCN, and FREEDOM, it reported approximately \(4\)–\(7\%\) relative improvement in NDCG@10 and NDCG@20 across all three datasets. Ablation studies showed that removing MSG, CSG, or UAPA degraded performance by \(1\)–\(2\%\), and hyper-parameter analysis identified balanced \(\lambda_{KL},\lambda_U\) in \(10^{-2}\ldots 10^{-4}\), depth \(L=3\), and \(k=10\) [2601.19198].

In connected autonomous driving for collaborative accident detection, A2MAML reported accident detection rate gains up to \(18.7\%\) higher than single-agent baselines. Relative to V2VNet, Who2Com, When2Com, and V2X-ViT, mean ADR improved by \(13\)–\(16\) percentage points under \(p=0.3\) corruption. The ablation “w/o active selection” dropped ADR by \(2\)–\(3\) percentage points, “w/o Bayesian fusion” by \(7\)–\(9\) percentage points, and removing both lost up to \(17\) percentage points. Under higher corruption levels \(p=0.5\) and \(p=0.7\), ADR degraded more gracefully than transformer-based baselines [2602.04763].

In multimodal uncertainty quantification based on discounted belief fusion, the principal evaluation criterion was AUC for conflictive-sample detection by fused uncertainty. On CUB, CalTech, HandWritten, PIE, and Scene15, DBF achieved \(0.57\pm0.06\), \(1.00\pm0.00\), \(0.80\pm0.02\), \(0.71\pm0.04\), and \(0.53\pm0.01\), respectively, outperforming BCF, CBF, BAF, and GBAF on all datasets except the Scene15 tie at \(0.53\pm0.01\). Classification accuracy remained essentially unchanged across fusion methods [2412.18024].

Tang et al. studied MUPM in cardiac MR and digital health record settings. Under distribution shifts involving higher-quality images, higher-quality text, or both, the fitted coefficients remained within \(0.20\)–\(0.24\), \(0.88\)–\(0.93\), and \(-0.22\pm0.1\), with \(R^2\in[0.76,0.84]\), expected calibration error approximately \(0.06\)–\(0.12\), and ANOVA \(p\)-values greater than \(0.3\). Across 1-year, 3-year, and 5-year cardiac disease prediction tasks, \(\hat\beta\) varied by less than \(0.03\), with \(R^2\in[0.78,0.82]\), ECE approximately \(0.07\)–\(0.10\), and ANOVA \(p>0.70\). The fitted model could be used for guided data-quality improvement because \(\hat\beta_2\approx 0.9\) was much larger than \(\hat\beta_1\approx 0.22\), and for efficient uncertainty estimation because \(n\ge 20\) augmentations achieved \(\|\sigma_{F_{I,T}}^{(n)}-\sigma_{F_{I,T}}^{(100)}\|<10^{-3}\), reducing GPU time by more than \(80\%\) relative to \(n=100\) [2507.12945].

The dynamic-model-averaging particle-filter formulation was tested on 2D target tracking with angle and range modalities. Over four failure scenarios, the overall average RMSE over \(100\) runs was \(26.34\) for DMA, compared with \(79.46\) for PF, \(116.78\) for TS, and \(117.36\) for SMA. Runtime was reported as approximately \(27.7\) s for DMA versus \(25.7\) s for PF per \(300\)-step run, and model-weight traces showed rapid identification of failed modalities [2105.06018].

MAP reported state-of-the-art downstream results in several vision-language tasks: \(60.9\%\) IR@1 and \(79.3\%\) TR@1 on MSCOCO 5K image-text retrieval; \(78.03\%\) on VQA2.0 test-dev; \(83.30\%\) on NLVR2 dev; and \(81.40\%\) on SNLI-VE val. Ablations showed that removing PDE or omitting any of the three distributional losses degraded downstream accuracy, often by \(1\)–\(2+\) points [2210.05335].

HyperDUM reported improvement of mean all-point AP by \(+2.01\%\) over LDU and \(+1.91\%\) over BEVFusion on aiMotive 3D object detection, and mean mIoU gains of \(+1.29\%\) over CMNeXt and \(+0.69\%\) over GeminiFusion on DeLiVER semantic segmentation. It also reduced expected calibration error by \(0.33\%\), required \(2.36\times\) fewer FLOPs and up to \(38.3\times\) fewer trainable parameters than LDU, and degraded by only approximately \(0.5\%\) under unseen corner cases where other methods dropped by more than \(2\%\) [2503.20011].

SURE was evaluated on sentiment analysis, genre classification, and action recognition, and was reported to achieve state-of-the-art performance while providing uncertainty estimates for both reconstructed modalities and downstream predictions in settings with missing modalities [2504.13465].

## 6. Interpretive issues, limitations, and boundary conditions

A first interpretive issue is definitional. The literature does not support a single narrow definition of MUPM; rather, it supports a family resemblance across methods that explicitly carry uncertainty through multimodal processing. This suggests that the central criterion is architectural role, not representational form: what makes a model “MUPM-like” is that uncertainty influences intermediate fusion or inference, not merely that a confidence score is produced at the end [2601.19198][2507.12945].

A second issue concerns the type of uncertainty being modeled. Several systems focus on aleatoric variance. SPUMR defines \(\sigma_e^m\) as aleatoric uncertainty in modality \(m\), and A2MAML states that its core formulation focuses on learned aleatoric variance, with epistemic uncertainty left optional through a parameter distribution or Monte-Carlo dropout [2602.04763]. By contrast, HyperDUM is explicitly positioned as a deterministic uncertainty method that quantifies feature-level epistemic uncertainty without Bayesian approximations [2503.20011]. DBF addresses conflict between confident modalities rather than Gaussian variance, and the DMA formulation addresses uncertainty about modality usefulness rather than uncertainty internal to a continuous embedding [2412.18024][2105.06018].

A third issue is the scope of generalization claims. Tang et al. report transferability across related input distributions and downstream cardiac disease prediction tasks, and explain that stability by shared pretraining, comparatively light MLLM fine-tuning, and the low-dimensional nature of the three-parameter MUPM [2507.12945]. A plausible implication is that some MUPM formulations can function as compact post hoc uncertainty surrogates rather than large end-to-end replacements, but the evidence given is domain-specific.

Several limitations are explicit. DBF cannot amplify uncertainty further when all uni-modal uncertainties are already high, because conflict remains low in that regime [2412.18024]. The DMA formulation has per-step complexity \(O(2^n\cdot N)\), which is described as affordable when \(n\) is small, for example \(n\le 4\), but this scaling is still exponential in the number of modalities [2105.06018]. SURE adds output variances under an independence assumption, and its propagated input variance relies on a first-order Taylor approximation through the fusion module [2504.13465]. SPUMR, A2MAML, and MAP all adopt diagonal or modality-wise Gaussian parameterizations, which simplify estimation and fusion but constrain the covariance structure they can express [2601.19198][2210.05335].

A final misconception is that uncertainty propagation necessarily requires expensive Bayesian training or repeated stochastic inference. The surveyed literature includes deterministic single-pass or low-parameter alternatives: HyperDUM uses hyperdimensional projections and prototype similarity; Tang et al. fit a three-parameter regression; SURE freezes the pretrained backbone and retrains only reconstructors and a final head; and the DMA particle filter shares a single particle set across all models to keep overhead modest [2503.20011][2507.12945][2504.13465][2105.06018]. The literature therefore supports a broader view in which MUPM is less a single model family than a methodological principle: uncertainty should be estimated at the modality level, propagated through the multimodal computation graph, and used to govern fusion, discounting, or model selection under noise, conflict, incompleteness, or failure.

Source: https://www.emergentmind.com/topics/multimodal-uncertainty-propagation-model-mupm