Robult: Scalable Multimodal Robustness

• Robult is a scalable multimodal framework that preserves both redundant and unique modality-specific features to enable robust predictions in incomplete input scenarios.
• It leverages a soft Positive-Unlabeled contrastive loss alongside a latent reconstruction objective to align unimodal and fused representations, improving performance with sparse labeling.
• Empirical results on sentiment analysis and vision tasks demonstrate that Robult outperforms baseline models in missing-modality inferences and semi-supervised settings.

Searching arXiv for the exact topic name and closely related multimodal robustness work to ground the article. Robult is a scalable multimodal learning framework designed to address two coupled failure modes in practical multimodal systems: missing modalities at inference and limited labeled data during training. Its central design premise is that robust multimodal prediction requires preserving both task-relevant redundancy across modalities and modality-specific information that remains useful when other inputs are absent. To operationalize that premise, Robult combines a soft Positive-Unlabeled (PU) contrastive objective for aligning unimodal and fused latent representations with a latent reconstruction objective that prevents unimodal pathways from collapsing into a purely shared space (Nguyen et al., 3 Sep 2025).

1. Conceptual basis and problem setting

Robult is formulated for a training regime in which all modalities are available for each sample, but only a subset of samples is labeled. The training dataset is written as

$$\mathcal{D}_{train} = \{(x_1, y_1), \dots, (x_k, y_k), x_{k+1}, \dots, x_n\},$$

where each $x_j = (x_j^1,\dots,x_j^M)$ contains all $M$ modalities, and only the first $k$ samples are labeled. Evaluation is defined over

$$\mathcal{X}_{test} = \{x_1,\dots,x_m\},\quad x_j = (x_j^a)_{a\in\mathcal{A}_j^M},\ \mathcal{A}_j^M \subseteq \{1,\dots,M\},$$

so modalities may be missing on a per-sample basis at test time (Nguyen et al., 3 Sep 2025).

The framework is explicitly motivated by Partial Information Decomposition (PID). In the notation used for $M$ modalities $X^1,\dots,X^M$ and target $Y$,

$$\mathcal{I}\big(\{X^1,\dots,X^M\}; Y\big) = \mathcal{R}(\{X^1,\dots,X^M\}; Y) + \sum_{i=1}^M \mathcal{U}(X^i; Y) + \mathcal{S}(\{X^1,\dots,X^M\}; Y),$$

where $\mathcal{R}$ denotes redundant task-relevant information, $x_j = (x_j^1,\dots,x_j^M)$0 denotes unique modality-specific information, and $x_j = (x_j^1,\dots,x_j^M)$1 denotes synergy available only from joint multimodal composition. Robult’s two design principles follow directly from this decomposition: maximize redundant task-relevant information in unimodal latent spaces by aligning them with a fused representation, and preserve unique modality-specific information so unimodal branches remain useful when other modalities are missing (Nguyen et al., 3 Sep 2025).

A common misconception is to treat robustness here as a generative missing-data problem. Robult does not perform explicit imputation of missing modalities. Instead, it trains unimodal pathways so that they retain both aligned redundant features and modality-specific features, then uses those pathways directly at inference when only a subset of modalities is available (Nguyen et al., 3 Sep 2025).

2. Architecture and latent decomposition

Robult is a modular layer placed on top of existing encoders. Its core components are modality-specific encoders $x_j = (x_j^1,\dots,x_j^M)$2, a fusion encoder $x_j = (x_j^1,\dots,x_j^M)$3, a shared branch $x_j = (x_j^1,\dots,x_j^M)$4, modality-specific unique branches $x_j = (x_j^1,\dots,x_j^M)$5, reconstruction modules $x_j = (x_j^1,\dots,x_j^M)$6, and a shared predictor $x_j = (x_j^1,\dots,x_j^M)$7 (Nguyen et al., 3 Sep 2025).

The initial projection stage maps each modality and the fused multimodal input into a common latent space: $x_j = (x_j^1,\dots,x_j^M)$8 The representation is then split into a shared or redundant part and a unique part. The shared branch applies the same map $x_j = (x_j^1,\dots,x_j^M)$9 to both unimodal and fused latents: $M$0 where $M$1 is the unimodal redundant representation and $M$2 is the fused redundant representation. In parallel, each modality-specific branch produces

$M$3

During training, a reconstruction module attempts to recover the original unimodal latent from both components,

$M$4

These reconstruction modules are used only in training and are not part of inference-time execution (Nguyen et al., 3 Sep 2025).

This architecture distinguishes two prediction regimes. When all modalities are available, prediction uses only the fused branch,

$M$5

which is intended to exploit both redundancy and synergy. When modalities are missing, each available modality $M$6 produces

$M$7

followed by a unimodal prediction

$M$8

Final prediction is obtained by late fusion across available modalities,

$M$9

In the reported experiments, this aggregation is often a simple average when multiple modalities are present (Nguyen et al., 3 Sep 2025).

3. Core objectives and information-theoretic formulation

Robult optimizes three losses: a task-specific supervised loss $k$0, a soft PU contrastive loss $k$1, and a latent reconstruction loss $k$2. The distinctive contribution lies in the latter two (Nguyen et al., 3 Sep 2025).

The contrastive component is designed to maximize $k$3 for each modality. The paper introduces a lower bound based on a binary indicator $k$4 that marks whether a pair $k$5 comes from the joint or from the product of marginals: $k$6 and

$k$7

where $k$8 is a non-parametric approximation to $k$9. In a batch of size $$\mathcal{X}_{test} = \{x_1,\dots,x_m\},\quad x_j = (x_j^a)_{a\in\mathcal{A}_j^M},\ \mathcal{A}_j^M \subseteq \{1,\dots,M\},$$0, Robult uses the similarity

$$\mathcal{X}_{test} = \{x_1,\dots,x_m\},\quad x_j = (x_j^a)_{a\in\mathcal{A}_j^M},\ \mathcal{A}_j^M \subseteq \{1,\dots,M\},$$1

which is structurally similar to NT-Xent or InfoNCE (Nguyen et al., 3 Sep 2025).

What differentiates Robult from standard supervised contrastive learning is its Positive-Unlabeled decomposition. The labeled term $$\mathcal{X}_{test} = \{x_1,\dots,x_m\},\quad x_j = (x_j^a)_{a\in\mathcal{A}_j^M},\ \mathcal{A}_j^M \subseteq \{1,\dots,M\},$$2 uses all same-class pairs inside the labeled subset as positives. For unlabeled data, Robult does not assume non-matching samples are negatives. Instead, it builds a soft positive set from pseudo-labels generated by the current classifier, then reweights these candidate positives with an adaptive RBF weight $$\mathcal{X}_{test} = \{x_1,\dots,x_m\},\quad x_j = (x_j^a)_{a\in\mathcal{A}_j^M},\ \mathcal{A}_j^M \subseteq \{1,\dots,M\},$$3 based on how compatible their similarity is with the similarity profile of true labeled positives. The combined objective is

$$\mathcal{X}_{test} = \{x_1,\dots,x_m\},\quad x_j = (x_j^a)_{a\in\mathcal{A}_j^M},\ \mathcal{A}_j^M \subseteq \{1,\dots,M\},$$4

The intended effect is to exploit unlabeled data without introducing the large number of false negatives characteristic of instance-level contrastive learning (Nguyen et al., 3 Sep 2025).

The reconstruction objective targets preservation of unique modality-specific information by minimizing

$$\mathcal{X}_{test} = \{x_1,\dots,x_m\},\quad x_j = (x_j^a)_{a\in\mathcal{A}_j^M},\ \mathcal{A}_j^M \subseteq \{1,\dots,M\},$$5

Using an approximate conditional $$\mathcal{X}_{test} = \{x_1,\dots,x_m\},\quad x_j = (x_j^a)_{a\in\mathcal{A}_j^M},\ \mathcal{A}_j^M \subseteq \{1,\dots,M\},$$6, the paper derives the upper bound

$$\mathcal{X}_{test} = \{x_1,\dots,x_m\},\quad x_j = (x_j^a)_{a\in\mathcal{A}_j^M},\ \mathcal{A}_j^M \subseteq \{1,\dots,M\},$$7

In practice, the reconstruction network $$\mathcal{X}_{test} = \{x_1,\dots,x_m\},\quad x_j = (x_j^a)_{a\in\mathcal{A}_j^M},\ \mathcal{A}_j^M \subseteq \{1,\dots,M\},$$8 is deterministic, and the implemented loss is

$$\mathcal{X}_{test} = \{x_1,\dots,x_m\},\quad x_j = (x_j^a)_{a\in\mathcal{A}_j^M},\ \mathcal{A}_j^M \subseteq \{1,\dots,M\},$$9

with $M$0 the $M$1-normalized dot product. This makes $M$2 and $M$3 jointly sufficient for recovering $M$4, which in turn resists over-compression of the unimodal pathway into only shared information (Nguyen et al., 3 Sep 2025).

4. Training procedure and robustness mechanism

The paper conceptually treats the batch objective as

$M$5

with the three terms routed to different parameter subsets through gradient toggling. For $M$6, gradients are enabled only for $M$7, $M$8, and $M$9. For $X^1,\dots,X^M$0 and $X^1,\dots,X^M$1, gradients are enabled for $X^1,\dots,X^M$2, $X^1,\dots,X^M$3, and $X^1,\dots,X^M$4. For $X^1,\dots,X^M$5, gradients are enabled for the full model. This parameter routing is used to separate the functions of redundancy extraction, unique-information preservation, and end-task supervision (Nguyen et al., 3 Sep 2025).

The framework’s robustness to missing modalities follows from this division of labor. The soft PU loss drives each unimodal redundant representation $X^1,\dots,X^M$6 toward the fused representation $X^1,\dots,X^M$7, so each available modality approximates the fused teacher in the task-relevant shared subspace. The reconstruction loss, by contrast, preserves $X^1,\dots,X^M$8, which supplies details lost by a purely shared representation. A plausible implication is that Robult’s robustness is not reducible to alignment alone: its unimodal behavior depends on the coexistence of shared alignment and explicit unique-information retention (Nguyen et al., 3 Sep 2025).

The paper also states several operational constraints that define the scope of the robustness claim. Training assumes all modalities are present. Synergy is modeled indirectly through the fused encoder and predictor rather than through a dedicated objective. The RBF weighting used for soft positives assumes a Gaussian-like similarity distribution, which is empirically motivated rather than theoretically proved (Nguyen et al., 3 Sep 2025).

5. Empirical evaluation

Robult is evaluated on sentiment analysis and classification benchmarks with both missing-modality inference and semi-supervised supervision ratios. The datasets are CMU-MOSI and CMU-MOSEI for three-modality sentiment regression, MM-IMDb for text-image multi-label classification, UPMC Food-101 for text-image classification, and Hateful Memes for text-image binary classification (Nguyen et al., 3 Sep 2025).

Dataset	Task	Reported metrics
CMU-MOSI / CMU-MOSEI	Sentiment regression	MAE, correlation, binary accuracy, F1
MM-IMDb	Multi-label genre classification	F1-Macro
UPMC Food-101	Food category classification	Accuracy
Hateful Memes	Hate vs benign classification	AUROC, accuracy

In the 5% labeled setting, the reported results show consistent improvements over unimodal baselines, GMC, ActionMAE, and, on two-modality datasets, Prompt-Trans. On CMU-MOSI full-modality evaluation, Robult reports MAE 1.392, Corr 0.247, and F1 0.657, compared with GMC at MAE 1.47, Corr 0.101, and F1 0.497. On CMU-MOSEI full-modality evaluation, Robult reports MAE 0.779 and Corr 0.504, compared with unimodal text at MAE 0.783 and Corr 0.364. For unimodal missing-modality cases, the paper states that Robult consistently attains the best or tied-best MAE, Corr, F1, and Acc across text, audio, and vision on both datasets; on MOSEI, correlation improves by up to 19.8% compared to the best baseline (Nguyen et al., 3 Sep 2025).

The same pattern appears on text-image benchmarks. On MM-IMDb full-modality evaluation, Robult reports F1-Macro 0.332 versus 0.307 for GMC and 0.268 for Prompt-Trans. On Food-101 full-modality evaluation, Robult reports accuracy 0.446 versus 0.432 for Prompt-Trans and 0.41 for GMC. On Hateful Memes full-modality evaluation, Robult reports AUROC 0.632, slightly below Prompt-Trans at 0.635, but with better accuracy. The paper further states that Critical Difference diagrams over F1-Macro across datasets assign Robult the best average rank, and that ActionMAE and Prompt-Trans are not statistically distinguishable from each other but are significantly worse than Robult (Nguyen et al., 3 Sep 2025).

Ablation results support the architectural decomposition. Removing $X^1,\dots,X^M$9 produces the largest degradation in unimodal performance, indicating that modality-specific retention is central to missing-modality robustness. Removing either $Y$0 or $Y$1 also degrades performance, showing that both labeled and pseudo-labeled positives contribute materially in low-label regimes. Removing unique branches $Y$2 harms especially weaker modalities. Variants without RBF weighting or without pseudo-labeling also underperform the full method (Nguyen et al., 3 Sep 2025).

6. Efficiency, limitations, and significance

Robult is designed as a lightweight latent-space extension rather than a backbone replacement. The added modules $Y$3, $Y$4, and $Y$5 are small MLPs, and the paper states that additional complexity scales linearly in the number of modalities $Y$6 in both time and memory. For latent dimension $Y$7, space complexity is given as $Y$8, with $Y$9 the number of fully connected layers (Nguyen et al., 3 Sep 2025).

The reported computational comparison illustrates that the overhead is modest. On MOSI, Robult reports 1.08 GFLOPs and 1.46M parameters, compared with GMC at 1.05 GFLOPs and 1.16M parameters, and ActionMAE at 1.13 GFLOPs and 11.55M parameters. This supports the paper’s characterization of Robult as lightweight relative to decoder-heavy alternatives (Nguyen et al., 3 Sep 2025).

Its limitations define the boundaries of its present interpretation. The training regime assumes complete modalities and therefore does not directly solve missing-modality training. Synergy $$\mathcal{I}\big(\{X^1,\dots,X^M\}; Y\big) = \mathcal{R}(\{X^1,\dots,X^M\}; Y) + \sum_{i=1}^M \mathcal{U}(X^i; Y) + \mathcal{S}(\{X^1,\dots,X^M\}; Y),$$0 is exploited through the fused encoder and predictor rather than a dedicated synergy objective. The method does not generate missing modalities. These constraints matter because they locate Robult within a specific family of robust multimodal methods: it is a representation-learning framework for inference-time incompleteness and semi-supervised supervision scarcity, not a general-purpose missing-data model or multimodal generator (Nguyen et al., 3 Sep 2025).

Taken together, Robult represents an information-theoretic approach to robust multimodal learning in which redundancy and modality-specific structure are treated as separate but complementary objects. Its empirical results suggest that robustness to missing modalities is improved not by collapsing modalities into a single shared space, but by jointly aligning them to a fused teacher and preserving what remains irreducibly specific to each modality (Nguyen et al., 3 Sep 2025).