Modality Dominance Index (MDI)

• Modality Dominance Index (MDI) is a set of metrics that quantify the imbalance in influence among multiple modalities in multimodal systems.
• Various formulations capture modality imbalance via attention ratios, optimization dynamics, information decomposition, and behavioral arbitration.
• MDI diagnostics help identify overreliance on specific modalities and inform interventions such as token compression and adaptive weighting.

Modality Dominance Index (MDI) denotes a family of quantitative measures for modality imbalance in multimodal systems. In contemporary multimodal learning, the term is used to characterize whether model behavior is disproportionately driven by one modality rather than others, but the literature does not treat it as a single standardized quantity. Instead, distinct papers define MDI through attention allocation during generation, optimization dynamics, information decomposition, behavioral arbitration under cross-modal conflict, or normalized information-contribution scores. A separate and older use of the same acronym denotes Mean Decrease of Impurity in random forests, which is unrelated to modality dominance in multimodal learning (Wu et al., 14 Aug 2025, Liu et al., 2 Jan 2026, Billa, 12 Feb 2026, Sutera et al., 2021).

1. Terminological scope and definitional variability

The multimodal literature uses MDI to answer different questions. In multimodal LLMs, it is formulated as a per-token attention ratio between text and non-text tokens during autoregressive generation (Wu et al., 14 Aug 2025). In RGB-Infrared perception, it is a per-modality score combining feature entropy and gradient contribution to quantify optimization imbalance (Liu et al., 2 Jan 2026). In representation-level information analysis, it is defined from normalized unique information obtained through Partial Information Decomposition (PID) (Amit et al., 22 Nov 2025). In audio-text arbitration, the operational index is a behavioral dominance ratio measuring how often the model follows text rather than audio under controlled conflict (Billa, 12 Feb 2026). In MODIX, unified contribution scores derived from intra-modal entropy and inter-modal alignment serve as the modality dominance index that drives positional index scaling (Huang et al., 14 Apr 2026).

A recurrent source of confusion is that the scale and preferred operating point of MDI depend on the formulation. In the attention-ratio definition, $\mathrm{MDI} > 1$ indicates text dominance, $\mathrm{MDI} \approx 1$ balanced attention, and $\mathrm{MDI} < 1$ non-text dominance (Wu et al., 14 Aug 2025). In the PID-based definition, larger values indicate greater imbalance because $\mathrm{MDI} = |C_1 - C_2|$ (Amit et al., 22 Nov 2025). In the audio-text arbitration setting, the behavioral index ranges from $0$ to $1$, with $0.5$ denoting no systematic preference (Billa, 12 Feb 2026). In the RGB-IR setting, the paper uses per-modality scores $S_i$, where a higher score means modality $i$ is more dominant in the current training step (Liu et al., 2 Jan 2026).

Variant	Core expression	Reading
Attention-ratio MDI	$$\mathrm{MDI} = \left( \frac{A_T}{|\mathcal{T}|} \right)\left( \frac{A_O}{|\mathcal{O}|} \right)^{-1}$$	$\mathrm{MDI} \approx 1$0 text-dominant
RGB-IR MDI	$\mathrm{MDI} \approx 1$1	Higher $\mathrm{MDI} \approx 1$2 = more dominant modality
PID-based MDI	$\mathrm{MDI} \approx 1$3	Larger = stronger imbalance
Audio-text dominance ratio	$\mathrm{MDI} \approx 1$4	Larger = stronger text dominance
MODIX contribution score	$\mathrm{MDI} \approx 1$5	Higher $\mathrm{MDI} \approx 1$6 = stronger modality contribution

This definitional variability suggests that MDI is best understood as a problem family rather than a single canonical metric. The common objective is to render modality imbalance observable and actionable, but the measured object differs: attention, optimization, information, behavior, or representational contribution.

2. Attention-based MDI in multimodal LLMs

A widely cited formulation defines MDI as a ratio of average attention per text token to average attention per non-text token during autoregressive generation (Wu et al., 14 Aug 2025): $\mathrm{MDI} \approx 1$7 where $\mathrm{MDI} \approx 1$8 is the set of text tokens, $\mathrm{MDI} \approx 1$9 the set of non-text tokens, $\mathrm{MDI} < 1$0 the total attention from all output tokens toward text, and $\mathrm{MDI} < 1$1 the total attention toward non-text tokens, with $\mathrm{MDI} < 1$2. The metric is designed to measure reliance on text relative to non-text modalities on a per-token basis. Because non-text inputs are often represented by many tokens, the normalization by token count is central to the interpretation.

Under this definition, the reported empirical pattern is pervasive text dominance across images, videos, audio, time-series, and graphs, with dominance intensifying in deeper layers and under higher non-text token redundancy (Wu et al., 14 Aug 2025). Representative values include late-layer MDI $\mathrm{MDI} < 1$3 for Qwen2.5-VL-7B on images, $\mathrm{MDI} < 1$4 for LLaVA-1.5-7B on images, and $\mathrm{MDI} < 1$5 for VideoLLaMA3-7B on video. Replication of redundant non-text tokens increases MDI in audio from $\mathrm{MDI} < 1$6 to $\mathrm{MDI} < 1$7, in time-series from $\mathrm{MDI} < 1$8 to $\mathrm{MDI} < 1$9, and in graphs from $\mathrm{MDI} = |C_1 - C_2|$0 to $\mathrm{MDI} = |C_1 - C_2|$1, illustrating the reported mechanism of attention dilution from token redundancy (Wu et al., 14 Aug 2025).

The same work attributes text dominance to three causes: attention dilution from severe token redundancy in non-textual modalities, the influence of fusion architecture design, and task formulations that implicitly favor textual inputs (Wu et al., 14 Aug 2025). The proposed mitigation is CLS-guided token compression, which ranks non-text tokens by their attention from the [CLS] token and retains only the most important ones. Quantitatively, for LLaVA-7B, late-layer MDI drops from $\mathrm{MDI} = |C_1 - C_2|$2 without compression to $\mathrm{MDI} = |C_1 - C_2|$3 at $\mathrm{MDI} = |C_1 - C_2|$4 compression, and the paper states that in the middle layers MDI drops from $\mathrm{MDI} = |C_1 - C_2|$5 to $\mathrm{MDI} = |C_1 - C_2|$6 under $\mathrm{MDI} = |C_1 - C_2|$7 token reduction (Wu et al., 14 Aug 2025). Within this formulation, a value near $\mathrm{MDI} = |C_1 - C_2|$8 is the target condition for balanced attention rather than a minimization objective in the abstract.

3. Optimization- and information-level formulations

In RGB-Infrared embodied perception, MDI is defined as a modality-level score that jointly models feature entropy and gradient contribution (Liu et al., 2 Jan 2026). For modality $\mathrm{MDI} = |C_1 - C_2|$9 with feature map $0$0, the paper defines representational diversity

$0$1

an auxiliary loss

$0$2

task-response sensitivity

$0$3

and the combined dominance score

$0$4

After normalization of the diversity and sensitivity terms across modalities, a higher $0$5 indicates that modality $0$6 is more dominant at the current training step (Liu et al., 2 Jan 2026).

This formulation is not an attention metric; it is explicitly an optimization-imbalance metric. It is used inside the Modality Dominance-Aware Cross-modal Learning (MDACL) framework, where MDI identifies dominant and non-dominant modalities for Hierarchical Cross-modal Guidance and supports inverse fusion weighting in Adversarial Equilibrium Regularization. The paper writes

$0$7

so that the non-dominant modality receives relatively stronger support during fusion, counteracting optimization bias (Liu et al., 2 Jan 2026). The reported claim is that this reduces gradient bias and improves performance on M3FD, LLVIP, and FLIR.

A different information-level perspective appears in the MoIR framework for vision-LLMs. MoIR does not redefine MDI; it reports MDI and AEI following prior work and treats lower MDI as more balanced modality use (Kim et al., 17 Apr 2026). The method’s central claim is that attention steering alone assumes all modalities already contain sufficient information, whereas real inputs differ in information density and signal-to-noise ratio. MoIR therefore modifies information availability before fusion by routing complementary information from a stronger modality into less informative tokens. On ScienceQA with LLaVA-1.5-7B at the attention layer, the reported MDI decreases from $0$8 under fine-tuning to $0$9 with MoIR; on VizWiz it decreases from $1$0 to $1$1; on MMBench-Video with Qwen2.5-VL it changes from $1$2 to $1$3 (Kim et al., 17 Apr 2026). The same study reports that under visual corruption, unchanged prediction rate drops markedly with MoIR, which is interpreted as reduced language dominance.

These formulations indicate that modality dominance can be targeted either by monitoring optimization asymmetry directly or by modifying information disparity before attention operates. This suggests that MDI is not tied to a single architectural locus.

4. Representation-level and behavioral formulations

A representation-level MDI is proposed in a PID-based framework for quantifying modality contributions by disentangling unique, redundant, and synergistic predictive information in internal embeddings (Amit et al., 22 Nov 2025). Given modality-specific embeddings $1$4 and $1$5 and target $1$6, the total mutual information is decomposed as

$1$7

Normalized modality contributions are then defined as

$1$8

and modality dominance is measured by

$1$9

The rationale is that unique information isolates what each modality contributes independently, while redundancy and synergy characterize overlap and interaction. The framework uses an Iterative Proportional Fitting Procedure to compute layer- and dataset-level contributions without retraining (Amit et al., 22 Nov 2025).

A different behavioral formulation appears in audio-text LLMs under controlled conflict (Billa, 12 Feb 2026). There the principal quantity is the Text Dominance Ratio (TDR),

$0.5$0

which the paper treats as a modality dominance index. It is measured on ALME, a benchmark of $0.5$1 controlled audio-text conflict stimuli across $0.5$2 languages. Gemini 2.0 Flash exhibits $0.5$3 text dominance under audio-text conflict versus $0.5$4 under text-text conflict with identical reliability cues, while audio-only accuracy is $0.5$5 and cascade accuracy is $0.5$6 (Billa, 12 Feb 2026). The paper interprets this as evidence that dominance is not explained by inferior audio information content; instead, it introduces the notion of arbitration accessibility, defined as how easily the model can reason over competing representations.

The same benchmark reports substantial cross-model variation, with TDR values of $0.5$7 for Gemini 2.0 Flash, $0.5$8 for GPT-4o, $0.5$9 for Ultravox, and $S_i$0 for Qwen2-Audio (Billa, 12 Feb 2026). Prompt framing also changes the ratio: labeling the transcript as “deliberately corrupted” reduces text dominance by $S_i$1, while forcing transcription before answering increases TDR from $S_i$2 to $S_i$3. A fine-tuning ablation shows that training only the audio projection layer increases text dominance by $S_i$4, while LoRA on the LLM reduces it by $S_i$5 (Billa, 12 Feb 2026). In this setting, MDI is neither an attention statistic nor an information decomposition; it is a behavioral observable derived from forced-choice outputs.

5. Related measures, precursors, and neighboring concepts

Several multimodal papers study modality importance without introducing a scalar MDI. In the Modal-Domain Attention model for medical multimodal fusion, the paper states that it “does not introduce a scalar metric named Modality Dominance Index per se” (Fan et al., 2024). Instead, modality contribution is quantified through inter-modal attention weights produced by softmax-normalized continuous attention, with interpretability based on mean attention weights, standard deviation of attention, and weight-shift analysis under missing or noisy modalities. Table 3 reports, for example, that in GIST, when WLE is missing, the WLE weight decreases from $S_i$6 to $S_i$7 while the report weight increases from $S_i$8 to $S_i$9 (Fan et al., 2024). This is functionally close to dominance analysis, but the unit of analysis is disease- and sample-specific attention weighting rather than a universal scalar index.

Gradient-Guided Modality Decoupling likewise analyzes modality dominance without an explicit scalar MDI (Wang et al., 2024). The paper uses gradient statistics as the operative indicator. For modal-incomplete case $i$0, with gradient $i$1, dominance is diagnosed by pairwise cosine similarity

$i$2

together with gradient-norm imbalance. Negative cosine similarity indicates conflict, and large norm disparities imply that a stronger modality will dominate updates. The proposed intervention removes conflicting gradient projections to decouple dependence on dominating modalities (Wang et al., 2024). Here, modality dominance is monitored in gradient space rather than summarized by an index.

At the feature level, “Multi-Faceted Multimodal Monosemanticity” introduces the Modality Dominance Score (MDS), not MDI, for CLIP features (Yan et al., 16 Feb 2025): $i$3 Features are categorized as TextD, CrossD, or ImgD using thresholds based on $i$4 over the distribution of $i$5 (Yan et al., 16 Feb 2025). This measure attributes dominance at the neuron or feature dimension level, rather than at the sample, output-token, or training-step level.

Outcome-based masking studies provide another nearby family of measures. “Modality Influence in Multimodal Machine Learning” trains models on all non-empty modality subsets and quantifies influence via performance differences and a reported $i$6 improvement metric, while explicitly noting that it does not define a formal MDI (Haouhat et al., 2023). The later PID-based framework criticizes such outcome-driven approaches for conflating whether a modality is inherently informative with whether its contribution arises through interaction (Amit et al., 22 Nov 2025). Taken together, these works mark a transition from ablation-based influence estimation toward representation-level or process-level dominance quantification.

6. Applications, interpretation, and acronym collision

Across formulations, MDI is used as a diagnostic and control variable. In attention-based MLLM studies, it diagnoses text overreliance and evaluates token compression as a balancing intervention (Wu et al., 14 Aug 2025). In RGB-IR perception, it governs dominant-versus-non-dominant role assignment for guidance and inverse weighting during fusion (Liu et al., 2 Jan 2026). In vision-language robustness, it serves as an evaluation target for information-level routing methods such as MoIR (Kim et al., 17 Apr 2026). In audio-text systems, the dominance ratio exposes failures of modality arbitration that are invisible under aligned inputs and can be shifted by prompt framing or selective fine-tuning (Billa, 12 Feb 2026). In MODIX, normalized contribution scores determine adaptive positional strides, with document QA reported as text-dominant at $i$7 and RealWorldQA as vision-dominant at $i$8 (Huang et al., 14 Apr 2026).

These use cases show that “balanced” modality use is not synonymous with equal raw token count, equal attention mass, or equal predictive necessity. In some tasks, a modality is expected to dominate because it carries more unique or reliable information. The PID literature therefore separates unique, redundant, and synergistic information rather than collapsing all modality effects into one scalar (Liang et al., 2023, Amit et al., 22 Nov 2025). A plausible implication is that MDI is most informative when interpreted together with the object it measures—attention, gradients, information, or behavior—rather than as a standalone universal score.

The acronym collision with random-forest interpretability is historically important. In tree ensembles, MDI conventionally means Mean Decrease of Impurity, defined for feature $i$9 in a tree as a weighted sum of impurity decreases over splits using that variable, and averaged across trees for a forest (Sutera et al., 2021, Li et al., 2019). That literature studies feature relevance, bias, debiasing, and Shapley-value connections; it is unrelated to modality dominance in multimodal models. Because both uses are current, disambiguation is necessary in technical writing, especially in cross-disciplinary venues.

In present multimodal research, the term Modality Dominance Index therefore denotes not a single settled metric, but a class of formal devices for quantifying disproportionate modality influence. The specific formulation determines the interpretation: per-token attention preference in multimodal generation, optimization imbalance in cross-modal training, unique-information asymmetry in internal representations, behavioral preference under conflicting evidence, or adaptive contribution scores used to redistribute modeling resources.