Missing Modality Agnostic (MMA) in Multimodal Systems

• Missing Modality Agnostic (MMA) is a multimodal framework designed to maintain robust performance even when one or more modalities are absent.
• It employs techniques such as circuit isolation, masked projection, and dynamic evidence routing to neutralize corrupted inputs instead of reconstructing missing data.
• Applied in both quantum federated learning and classical architectures, MMA systems demonstrate significant improvements in handling incomplete modality inputs without retraining multiple models.

Searching arXiv for the cited works and closely related missing-modality-agnostic literature. arXiv search query: "Missing Modality Agnostic multimodal" arXiv search query: "(Pokharel et al., 10 Jul 2025) Quantum Federated Learning for Multimodal Data modality agnostic" Missing Modality Agnostic (MMA) denotes a class of multimodal learning designs that remain operational when one or more modalities are absent. In recent arXiv literature, the expression is used both narrowly, as the name of a specific mechanism in multimodal quantum federated learning, and more broadly, as a design objective for systems that must tolerate changing modality availability without retraining a separate model for each missing-pattern configuration. Across domains, MMA systems achieve this through circuit isolation, masked projection, reconstruction, prompt adaptation, evidence routing, or subset-stable fusion, but they share a common requirement: the model must continue to produce useful outputs from whichever modalities survive (Pokharel et al., 10 Jul 2025, Ke et al., 27 Feb 2025, Chen et al., 7 Apr 2026, Wang et al., 24 May 2026).

1. Conceptual scope and terminological usage

In the strictest sense, “Missing Modality Agnostic” is the name of the mechanism introduced in “Quantum Federated Learning for Multimodal Data: A Modality-Agnostic Approach” (Pokharel et al., 10 Jul 2025). There, MMA refers to a circuit-level design in which missing-modality subcircuits are isolated with no-op gates and fixed $\lvert 0 \rangle$ states so that entangling fusion does not mix in corrupted information. In a broader current usage, however, MMA functions as an umbrella label for methods that are robust to absent modalities by construction, whether they neutralize the missing branch, infer a replacement representation, or route computation only through surviving evidence (Pokharel et al., 10 Jul 2025, Nezakati et al., 2024).

A recurring distinction in this literature is between being agnostic to which known modality is missing and being agnostic to unseen modality types. Several systems are robust over the former but not the latter. ProMMA, for example, is evaluated over six missing patterns for text, audio, and visual inputs, but “cannot handle modalities unseen in training” (Chen et al., 7 Apr 2026). Likewise, “Fine-Grained Scene Image Classification with Modality-Agnostic Adapter” presents a modality-agnostic fusion architecture, yet it “does not explicitly study missing modalities” and instead evaluates fixed subsets of available modalities (Wang et al., 2024). This suggests that MMA is best understood as a robustness property over modality availability, not as a guarantee of zero-shot extensibility to arbitrary new sensing channels.

Formulation	Domain	Characteristic MMA mechanism
MMA in mmQFL (Pokharel et al., 10 Jul 2025)	Quantum federated multimodal learning	Context vector, no-op gates, fixed $\lvert 0 \rangle$ states
Knowledge Bridger (Ke et al., 27 Feb 2025)	Missing modality completion	Knowledge graphs, candidate generation, ranking
ProMMA (Chen et al., 7 Apr 2026)	Multimodal sentiment analysis	Evaluation before generation, prompt disentanglement, dynamic weighting
ActionMAE (Woo et al., 2022)	Action recognition	Random modality dropping and reconstruction
GMD + DS (Wang et al., 2024)	Segmentation and sentiment	Gradient-guided decoupling and dynamic sharing
MMP (Nezakati et al., 2024)	Generic multimodal learning	Random masking and masked modality projection
SimMLM (Li et al., 25 Jul 2025)	Segmentation and classification	Dynamic mixture of modality experts and MoFe ranking loss
Meta-Modal Agent (Wang et al., 24 May 2026)	Recommendation reranking	Sequential evidence routing over modality-specific tools

2. Main technical patterns

One major MMA pattern is neutralization rather than reconstruction. In the quantum setting, a missing branch is not synthesized; it is held in a controlled zero state and excluded from data-dependent evolution, so gradients for its parameters vanish naturally on samples where that modality is absent (Pokharel et al., 10 Jul 2025). A related principle appears in systems that question whether generation is needed at all. ProMMA introduces a Missing Modality Evaluator that estimates whether a missing modality is “worth generating” before any imputation, and the Meta-Modal Agent treats missingness as a sequential evidence-routing problem in which tool calls may return Null, after which the policy learns to route elsewhere rather than invent unsupported evidence (Chen et al., 7 Apr 2026, Wang et al., 24 May 2026).

A second pattern is representation completion or projection. Knowledge Bridger is explicitly training-free and reconstructs missing modalities by converting available inputs into knowledge graphs, generating multiple candidates, and ranking them with a combined graph-and-embedding score,

$$QS(x_a, x_m) = \cos_{graph}(f_a(x_a), f_a(x_m)) + \Big[\cos(f_c(x_a), f_c(x_m)) + \cos(f_b(x_a), f_b(x_m))\Big].$$

This is modality-agnostic at the level of reasoning and completion logic because the same pipeline—knowledge extraction, graph construction, candidate generation, and ranking—applies regardless of which modality is absent (Ke et al., 27 Feb 2025). Related projection-style approaches include MMP, which “randomly mask[s] a subset of modalities during training and learn[s] to project available input modalities to estimate the tokens for the masked modalities,” and MUST, which decomposes each modality into shared and modality-specific components and uses conditional latent diffusion only for the truly modality-specific residuals that cannot be inferred from the other modality (Nezakati et al., 2024, Kim et al., 27 Mar 2026).

A third pattern is subset-robust training without full data generation. ActionMAE randomly drops modality tokens, reconstructs missing tokens with a memory-token autoencoder, and couples reconstruction with classification so that one model can operate over arbitrary subsets at inference (Woo et al., 2022). GMD and its Dynamic Sharing framework instead target optimization pathologies: missing-modality fragility is traced to modality dominance and conflicting gradients, so the method removes conflicted gradient components and switches parameters on or off according to availability (Wang et al., 2024). Meta-learning and late-fusion variants pursue the same goal at the training-regime level. M3S treats sampled missing-pattern configurations as tasks within MAML, and UME-MMA argues from an information-theoretic perspective that robustness approaches the missing-modality ceiling when the non-missing modality is encoded strongly and fused in a way that does not inject avoidable noise (Chi et al., 2022, Li et al., 2023).

3. MMA in multimodal quantum federated learning

The quantum MMA mechanism is defined inside a multimodal quantum federated learning framework with $M$ modalities, modality-specific parameterized quantum circuits (PQCs), and an entangling fusion PQC (Pokharel et al., 10 Jul 2025). For client $k$, each modality $m$ is amplitude-encoded as

$\mathbf{x}_m^k \mapsto \lvert{\psi_{m,\text{enc}^k}\rangle = \sum_{i=1}^{N_m} \bigl( x^k_{m,i} \bigr) \lvert{i}_m\rangle, \quad N_m = 2^{n_m},$

then processed by an $L$-layer PQC and fused by a global unitary $U_{\text{fusion}^k}$ over

$n_q = \sum_{m=1}^M n_m$

qubits. MMA introduces a context vector $\lvert 0 \rangle$0 with $\lvert 0 \rangle$1 indicating a missing modality. If $\lvert 0 \rangle$2, encoding is skipped, the modality PQC is effectively set to identity,

$\lvert 0 \rangle$3

and the corresponding qubits remain in $\lvert 0 \rangle$4 under no-op gates (Pokharel et al., 10 Jul 2025).

This design isolates untrained quantum circuits. The fusion unitary still acts on all qubits, but missing-modality qubits enter entanglement from a fixed, known state rather than from random initialization or faulty sensor values. The paper’s central claim is that, without MMA, a missing branch can introduce “corrupted quantum states” that contaminate the full joint state during entangling fusion; with MMA, the system remains functional under partial modality availability because benign zero states replace garbage inputs (Pokharel et al., 10 Jul 2025).

The federated loop itself remains standard. Global rounds broadcast parameters, clients perform local training, gradients are computed with the parameter-shift rule, and global aggregation uses FedAvg-like averaging. MMA does not require explicit masking in the loss: because the missing branch is not activated, the loss becomes independent of its parameters for that sample, so their gradients are effectively zero (Pokharel et al., 10 Jul 2025).

The empirical evidence is reported on CMU-MOSEI with image, audio, and text modalities. Missingness is simulated at levels of $\lvert 0 \rangle$5, $\lvert 0 \rangle$6, and $\lvert 0 \rangle$7. Under IID data, no missingness yields 96.36% without MMA and 96.41% with MMA, while “Image missing 20%” improves from 72.18% to 80.02%, “Audio missing 20%” from 82.47% to 86.52%, and “Text missing 20%” from 64.64% to 75.30%. Under non-IID data, the corresponding no-missing case is 82.36% vs. 82.41%, and the 20% missing cases improve from 69.22% to 74.71% for image, 72.01% to 75.45% for audio, and 63.58% to 70.30% for text. With 10% missing on all modalities, the full mmQFL model with fusion and MMA achieves 85.61% in IID and 79.27% in non-IID settings, and the paper states an improvement of 6.84% in IID and 7.25% in non-IID over the state of the art (Pokharel et al., 10 Jul 2025).

A common misconception is that this quantum MMA “fills in” missing data. The paper states the opposite: it does not restore absent information; it prevents the absence of a modality from actively damaging the quantum state of the available modalities. That distinction is central to many later MMA formulations as well (Pokharel et al., 10 Jul 2025).

4. Classical architectural and optimization strategies

A large branch of MMA research uses architecture-level subset robustness. ActionMAE is exemplary: it learns modality-token predictive coding by randomly dropping modality features, reconstructing them with an encoder-decoder equipped with a memory token, and training with

$\lvert 0 \rangle$8

On NTU60, an ActionMAE model trained with RGB+Depth+IR reports 93.0% with all modalities, 92.6% with RGB+Depth, 90.1% with Depth only, and 83.4% with RGB only, reducing the discrepancy between complete and missing-modality inference without requiring separate models per subset (Woo et al., 2022). SimMLM generalizes the same intuition with a Dynamic Mixture of Modality Experts and a “More vs. Fewer” ranking loss enforcing the principle that “removing one or more modalities should not increase accuracy” (Li et al., 25 Jul 2025).

A second line targets the optimization dynamics that cause missing-modality collapse. “Gradient-Guided Modality Decoupling for Missing-Modality Robustness” argues that modality dominance is reflected in conflicting gradient components from different modal-incomplete cases, and removes those conflicted components while using a Dynamic Sharing framework that switches network parameters on or off based on modality availability (Wang et al., 2024). A segmentation-oriented variant based on a shared 3D U-Net backbone instead uses mutual information and Hölder divergence so that each MRI modality is processed independently, fused dynamically, and aligned with full-modality representations. On BraTS 2018, the combination of Dice, mutual information, and Hölder divergence improves the average score from 70.7 with Dice only to 80.1, with especially large gains in the “3 missing” regime (Cheng et al., 2024).

Projection-based systems occupy a middle ground between neutralization and full reconstruction. MMP trains a single model “robust to any missing modality scenario” by randomly masking modalities and learning to project available modality tokens to estimate the masked ones, then supervising those projected tokens with an alignment loss (Nezakati et al., 2024). MUST provides a more structured version of the same idea in multimodal survival prediction: each modality is decomposed into a low-rank shared component and an orthogonal modality-specific component, and only the modality-specific residual is generated via conditional latent diffusion when pathology or genomics is missing. On five TCGA datasets, MUST reports an overall C-index of 0.742 with complete data, 0.716 with missing genomics, and 0.739 with missing pathology (Kim et al., 27 Mar 2026).

Training-regime designs offer another route to MMA. UME-MMA argues that missing-modality robustness is governed by the complementary information $\lvert 0 \rangle$9 in the missing modality and proposes uni-modal pre-trained weights together with missing-modality data augmentation inside a late-fusion framework (Li et al., 2023). M3S, by contrast, treats missing-pattern mixtures as tasks in a MAML-style loop and reports improved robustness on IEMOCAP, SIMS, and CMU-MOSI without changing the base model architecture (Chi et al., 2022). A related but weaker precursor is the shared-representation literature: “Exploring modality-agnostic representations for music classification” learns a 128-dimensional shared space from audio-image pairs and shows that a single classifier can operate on either modality, reaching almost 70% of the best-performing system in a zero-shot setting, although this work is about unified modality representations rather than explicit missing-modality training (Wu et al., 2021).

5. Prompt-, knowledge-, and agent-based MMA systems

Knowledge-guided systems treat MMA as an inference-time reasoning problem. Knowledge Bridger is explicitly training-free and uses three stages—Knowledge Graph Modeling, Knowledge-driven Generation, and Knowledge-based Ranking—to complete missing modalities with off-the-shelf generators and large multimodal models. It defines domain-specific priors, extracts structured information and triples, generates multiple candidates, and ranks them with the quality score

$$QS(x_a, x_m) = \cos_{graph}(f_a(x_a), f_a(x_m)) + \Big[\cos(f_c(x_a), f_c(x_m)) + \cos(f_b(x_a), f_b(x_m))\Big].$$0

On COCO-2014, MM-IMDb, and IU X-ray, the same pipeline is used in both general and medical domains, and on IU X-ray at missing rate $$QS(x_a, x_m) = \cos_{graph}(f_a(x_a), f_a(x_m)) + \Big[\cos(f_c(x_a), f_c(x_m)) + \cos(f_b(x_a), f_b(x_m))\Big].$$1 it reports F1 46.3, mAP 70.5, and SS 19.8, compared with best baselines at 36.8 F1, 61.9 mAP, and 13.3 SS (Ke et al., 27 Feb 2025).

Prompt-based sentiment systems emphasize evaluation and suppression of unreliable generation. ProMMA introduces a Missing Modality Evaluator, Modality-invariant Prompt Disentanglement, Dynamic Prompt Weighting, and Multi-level Prompt Dynamic Connection around a pre-trained MULT backbone. Its Missing Modality Evaluator constructs a pseudo-label

$$QS(x_a, x_m) = \cos_{graph}(f_a(x_a), f_a(x_m)) + \Big[\cos(f_c(x_a), f_c(x_m)) + \cos(f_b(x_a), f_b(x_m))\Big].$$2

with $$QS(x_a, x_m) = \cos_{graph}(f_a(x_a), f_a(x_m)) + \Big[\cos(f_c(x_a), f_c(x_m)) + \cos(f_b(x_a), f_b(x_m))\Big].$$3, so the system decides whether missing-modality generation is justified before it is performed. At 30% missing rate, ProMMA reports 78.04% ACC and 77.50% F1 on MOSI, 81.75% ACC and 81.32% F1 on MOSEI, and 74.89% ACC and 77.74% F1 on CH-SIMS, outperforming strong prompt-based baselines across six missing patterns (Chen et al., 7 Apr 2026).

A more asymmetric but highly effective design appears in “Enhancing Multimodal Sentiment Analysis for Missing Modality through Self-Distillation and Unified Modality Cross-Attention.” That framework targets missing text specifically. It uses a Double-Flow Self-Distillation setup with a complete-modality teacher, a missing-text student, Unified Modality Cross-Attention (UMCA), a Modality Imagination Autoencoder (MIA), and Rank-N Contrast. Real text embeddings from Vicuna supervise LLM-generated text representations obtained from audio via a projector and MIA refinement. On CMU-MOSEI, it reports 0.506 MAE and 87.6 ACC with text, and 0.550 MAE and 84.2 ACC without text, substantially narrowing the gap between full and missing-text regimes (Weng et al., 2024).

Agentic systems move further away from direct completion and toward adaptive evidence use. The Meta-Modal Agent treats missingness in multimodal recommendation as a sequential evidence-routing problem over text, image, graph, and optional clarification tools. In oracle-free one-observed-modality availability, MMA-Auto improves target-positive OOMA NDCG@10 by 4.0% over the strongest non-interactive baseline and improves fixed-pool full-catalog reranking NDCG@10 by 12.7%; the deterministic control RuleRouter-Fuse underperforms MMA-Auto, and MMA-Interactive adds a 4.1% upper-bound gain when clarification is available (Wang et al., 24 May 2026). In generative modeling, MAGE applies the same missing-modality principle to music generation and editing: a single flow-based Transformer is trained with a dynamic modality-masking curriculum over text-only, visual-only, joint multimodal, and mixture-guided settings so that it can operate under any available subset of conditions (Saleem et al., 10 Apr 2026).

6. Empirical themes, misconceptions, and open problems

A first empirical theme is that full-modality accuracy and missing-modality robustness are related but not identical objectives. Several methods improve both, but many papers explicitly expose a trade-off. MMP improves average robustness across missing patterns while giving up some full-modality peak performance relative to a fully observed baseline, and SimMLM introduces MoFe precisely because “removing one or more modalities should not increase accuracy,” indicating that monotonic behavior across subsets is not automatically guaranteed by standard multimodal training (Nezakati et al., 2024, Li et al., 25 Jul 2025). A closely related issue appears in the counterintuitive-rate analysis of SimMLM and in the stability arguments of the quantum MMA mechanism: more modalities can hurt if the added branch is noisy, misaligned, or untrained (Li et al., 25 Jul 2025, Pokharel et al., 10 Jul 2025).

A second misconception is that MMA always means imputation. The literature is divided. Knowledge Bridger, MMP, MUST, and the missing-text UMCA framework do synthesize missing representations or modalities (Ke et al., 27 Feb 2025, Nezakati et al., 2024, Kim et al., 27 Mar 2026, Weng et al., 2024). By contrast, quantum MMA, Meta-Modal Agent, and ProMMA all contain mechanisms whose purpose is to avoid unnecessary or harmful generation: no-op isolation, evidence routing after Null, or “evaluation before generation” (Pokharel et al., 10 Jul 2025, Wang et al., 24 May 2026, Chen et al., 7 Apr 2026). This suggests that MMA is best characterized by robustness under missingness, not by any single stance on reconstruction.

A third theme is that most current “modality-agnostic” claims remain bounded by a fixed modality set. ProMMA explicitly notes that it cannot handle modalities unseen in training (Chen et al., 7 Apr 2026). MUST’s main method assumes paired pathology-genomics training data and currently instantiates only two modalities, even though the paper argues that its algebraic decomposition extends to $$QS(x_a, x_m) = \cos_{graph}(f_a(x_a), f_a(x_m)) + \Big[\cos(f_c(x_a), f_c(x_m)) + \cos(f_b(x_a), f_b(x_m))\Big].$$4 modalities (Kim et al., 27 Mar 2026). Knowledge Bridger reduces dependence on task-specific training, but still requires domain-specific priors, modality-specific generators, and shared embedding spaces such as CLIP or BLIP (Ke et al., 27 Feb 2025). In other words, present MMA systems are typically agnostic to subset patterns of known modalities, not universally agnostic to arbitrary new modalities.

Open problems therefore recur across domains. The quantum literature identifies the absence of theoretical convergence analysis and suggests conditional fusion that skips entangling operations rather than merely feeding dummy qubits (Pokharel et al., 10 Jul 2025). Knowledge Bridger highlights hallucination in LMM-based knowledge extraction, the need for Retrieval-Augmented Generation, and extension beyond image-text pairs (Ke et al., 27 Feb 2025). ProMMA points to improved evaluator design, dynamic thresholding, and broader incomplete multimodal learning settings (Chen et al., 7 Apr 2026). Meta-Modal Agent makes clear that routing-based MMA can improve robustness but pays a latency cost, with agentic inference around 1.25 s per query versus approximately 0.01–0.02 s for static baselines (Wang et al., 24 May 2026). A plausible synthesis is that future MMA systems will combine three ingredients now usually separated in the literature: strong per-modality encoders, explicit subset-aware training, and a learned decision rule for when to neutralize, reconstruct, or ignore a missing branch.

Across these formulations, MMA is no longer a single mechanism but a research program. Its central question is not whether modalities can be fused, but whether multimodal systems can remain reliable when fusion conditions are imperfect, asymmetric, or changing. The answer emerging from current work is affirmative, but only when the model is trained or structured to make modality absence an explicit part of its operating regime rather than an afterthought.