M3P-Score in Visual Instruction Following

• M3P-Score is a pair-selection metric that defines hard negatives by combining a black-box Multimodal Alignment Score (MAS) with the model’s sequence log-probability.
• It identifies a preferred response by MAS and selects a challenging dispreferred response when the confidence gap exceeds a margin, optimizing contrastive supervision.
• Empirical results demonstrate that using M3P-Score in the M3PO pipeline improves performance on benchmarks like MME-Bench, POPE, IFT, and Human Pref Score.

M3P-Score is the core selection metric in M3PO, a preference-data construction method for visual instruction following in Large Vision-LLMs (LVLMs). In "M3PO: Multimodal-Model-Guided Preference Optimization for Visual Instruction Following" (Gao et al., 17 Aug 2025), it fuses an external quality signal—the Multimodal Alignment Score (MAS)—with the LVLM’s internal belief, captured as sequence log-probability. Its purpose is to surface “learning-valuable” preference pairs by selecting, for each image-instruction pair, a highly aligned preferred response and a hard negative that the model is confident about despite being wrong or misaligned. The resulting tuples are used for Direct Preference Optimization (DPO) fine-tuning of base LVLMs such as LLaVA-1.5 7B and 13B with LoRA (Gao et al., 17 Aug 2025).

1. Terminological scope

Within the literature provided here, the term M3P-Score is formally defined only in the M3PO work on visual instruction following (Gao et al., 17 Aug 2025). The term should not be conflated with several unrelated uses of the abbreviation M3P. In "M3P: Learning Universal Representations via Multitask Multilingual Multimodal Pre-training" (Ni et al., 2020), there is no metric explicitly called “M3P-Score”; the operational image-text alignment score is instead the MC-VLM matching logit from the joint Transformer’s $[CLS]$ representation. In "M3-BENCH: Process-Aware Evaluation of LLM Agents Social Behaviors in Mixed-Motive Games" (Xie et al., 13 Jan 2026), the canonical quantities are BTA, RPA, CCA, the composite task score $S_\tau$, and the three-view consistency score $\sigma$, not a metric called M3P-Score. In "Motion Planning in Non-Gaussian Belief Spaces (M3P): The Case of a Kidnapped Robot" (Agarwal et al., 2015), the paper does not introduce a metric explicitly named “M3P-Score”; candidate policies are evaluated by expected information gain. In "Lean 4 Machine-Verified Proof of P = NP via the Pedigree Polytope Membership Problem" (Arthanari, 2 Jun 2026), “M3P” denotes the Membership Problem for Pedigree Polytope, and the ratio $z^*(X)/z_{\max}(X)$ is presented only as a natural score consistent with the membership criterion. In "M3-Bench: Multi-Modal, Multi-Hop, Multi-Threaded Tool-Using MLLM Agent Benchmark" (Zhou et al., 21 Nov 2025), the paper reports “Average Score,” not “M3P-Score.”

This terminological separation matters because the M3PO definition is not a generic multimodal alignment score, nor a benchmark aggregate, nor a retrieval logit. It is a pair-selection criterion for preference optimization in LVLM fine-tuning.

2. Formal definition

For each image-instruction pair $(I, Q)$, M3PO first generates $N = 32$ candidate responses $\{R_1, \dots, R_N\}$ using the base LVLM with sampling-based decoding, specifically top-p/nucleus sampling with a moderate temperature to encourage diversity (Gao et al., 17 Aug 2025). Two signals are then computed for every candidate $R_i$.

The first signal is the Multimodal Alignment Score: $\mathrm{MAS}(R_i \mid I, Q),$ produced by an external visual-language assessment model such as CLIP ViT-L/14 or BLIP-2 plus GPT-4V. MAS is treated as a scalar alignment measure reflecting visual relevance, semantic accuracy, and instruction adherence. The paper does not specify an explicit formula, normalization procedure, or aggregation weights for MAS (Gao et al., 17 Aug 2025).

The second signal is the model’s Self-Consistency / Confidence, defined as the sequence-level log-probability of the base LVLM generating $R_i$ given $S_\tau$0: $S_\tau$1 The paper uses the full sequence log-likelihood, with no length-normalization, token-average, length penalties, or additional calibration (Gao et al., 17 Aug 2025).

The preferred response is chosen solely by MAS: $S_\tau$2

For each remaining candidate $S_\tau$3, the M3P-Score is then defined relative to $S_\tau$4: $S_\tau$5

The dispreferred response is selected by maximizing this score: $S_\tau$6

The hyperparameters reported for M3P-Score are $S_\tau$7 and $S_\tau$8 (Gao et al., 17 Aug 2025). No top-$S_\tau$9 or thresholding is used; one $\sigma$0 pair is selected per $\sigma$1.

A recurrent misconception is that both the preferred and dispreferred responses are ranked by M3P-Score. The paper states the opposite: the preferred response is chosen solely by MAS, whereas the dispreferred response is chosen by maximizing M3P-Score (Gao et al., 17 Aug 2025).

3. Signals, ranking logic, and hard negative mining

The first term in M3P-Score,

$\sigma$2

prioritizes negatives that are clearly worse than the preferred response on external alignment. The second term penalizes easy negatives whose log-probability is much lower than the preferred response’s, but only when the confidence gap exceeds the margin $\sigma$3. If $\sigma$4’s confidence is close to or higher than $\sigma$5’s, within the margin, the penalty is zero. This favors “confident yet wrong” hard negatives (Gao et al., 17 Aug 2025).

The scoring rule therefore enforces two principles. First, preferred candidates must be externally superior, because $\sigma$6 is the candidate with the highest MAS. Second, dispreferred candidates should be externally inferior but internally plausible or confident, because $\sigma$7 is chosen to maximize the M3P-Score after accounting for the confidence-difference penalty (Gao et al., 17 Aug 2025).

The paper’s stated intuition is that MAS acts like an external reward or quality estimator, while the confidence term identifies non-trivial negatives for the current policy. This addresses a failure mode in preference optimization in which models are trained mostly on “easy” negatives they already strongly disfavor, limiting the learning signal (Gao et al., 17 Aug 2025). A plausible implication is that M3P-Score is not primarily a quality-estimation metric; it is a selection metric for contrastive supervision.

The MAS signal itself remains evaluator-dependent. It is computed by an external visual-language assessment model distinct from the LVLM being fine-tuned, and it reflects visual relevance, semantic accuracy, and instruction adherence. Because the paper leaves MAS as a black-box scalar without a closed-form formula, the discriminative behavior of M3P-Score depends in part on the external evaluator design (Gao et al., 17 Aug 2025).

4. Integration into the M3PO pipeline and DPO objective

M3P-Score is embedded in a five-step data construction pipeline. For each $\sigma$8 in the base dataset $\sigma$9, the base LVLM samples $z^*(X)/z_{\max}(X)$0 responses; MAS and sequence log-probability are computed for every response; the preferred response $z^*(X)/z_{\max}(X)$1 is chosen by $z^*(X)/z_{\max}(X)$2; the dispreferred response $z^*(X)/z_{\max}(X)$3 is chosen by maximizing $z^*(X)/z_{\max}(X)$4; and the tuple $z^*(X)/z_{\max}(X)$5 is collected into $z^*(X)/z_{\max}(X)$6 (Gao et al., 17 Aug 2025).

The resulting preference dataset is used to fine-tune the LVLM with the standard DPO objective against a frozen reference model $z^*(X)/z_{\max}(X)$7, which is the base LVLM: $z^*(X)/z_{\max}(X)$8

The paper does not specify a numeric value for $z^*(X)/z_{\max}(X)$9, and it does not apply loss weighting by M3P-Score; all selected pairs are treated uniformly (Gao et al., 17 Aug 2025). Fine-tuning uses LoRA adapters only, with batch size $(I, Q)$0, learning rate $(I, Q)$1, warm-up $(I, Q)$2 of steps, and convergence typically within approximately one epoch (Gao et al., 17 Aug 2025).

This design fixes the role of M3P-Score quite precisely. It does not enter the training loss directly. It governs the composition of the preference dataset on which DPO is subsequently trained.

5. Empirical results and ablations

The paper reports benchmark results on MME-Bench, POPE, IFT, and Human Pref. Score for LLaVA-1.5 7B and 13B, comparing Base, SFT, vanilla DPO, RM-DPO, and M3PO (Gao et al., 17 Aug 2025).

Backbone	Method	MME / POPE / IFT / Human Pref
LLaVA-1.5 7B	Base	1345.2 / 85.12% / 68.30 / 2.85
LLaVA-1.5 7B	SFT	1378.1 / 86.55% / 70.15 / 3.10
LLaVA-1.5 7B	DPO (vanilla)	1388.9 / 86.88% / 70.95 / 3.20
LLaVA-1.5 7B	RM-DPO	1398.7 / 87.21% / 71.55 / 3.30
LLaVA-1.5 7B	M3PO	1402.3 / 87.35% / 71.80 / 3.38
LLaVA-1.5 13B	Base	1489.1 / 87.20% / 72.10 / 3.10
LLaVA-1.5 13B	SFT	1512.4 / 88.50% / 73.45 / 3.40
LLaVA-1.5 13B	DPO (vanilla)	1524.6 / 88.90% / 73.90 / 3.50
LLaVA-1.5 13B	RM-DPO	1533.2 / 89.18% / 74.45 / 3.60
LLaVA-1.5 13B	M3PO	1537.8 / 89.30% / 74.70 / 3.65

The ablation that removes the confidence term by setting $(I, Q)$3 is central to the interpretation of M3P-Score. On LLaVA-1.5 7B, full M3PO yields MME $(I, Q)$4, POPE $(I, Q)$5, IFT $(I, Q)$6, and Human Pref $(I, Q)$7, whereas M3PO without confidence yields MME $(I, Q)$8, POPE $(I, Q)$9, IFT $N = 32$0, and Human Pref $N = 32$1 (Gao et al., 17 Aug 2025). The paper’s interpretation is explicit: selecting negatives solely by MAS difference misses hard, challenging examples.

Sensitivity analyses further locate the reported optimum at $N = 32$2 when $N = 32$3, and at $N = 32$4 when $N = 32$5. Too small an $N = 32$6 approaches MAS-only behavior, while too large an $N = 32$7 over-emphasizes confidence and slightly degrades performance; a small margin $N = 32$8 is effective for mining confident hard negatives (Gao et al., 17 Aug 2025).

These results support the paper’s narrower claim about M3P-Score: the utility of the metric lies in selecting difficult, informative preference pairs rather than in merely ranking outputs by external alignment.

6. Efficiency, limitations, and boundaries of the metric

The implementation details reported for LLaVA-1.5 7B indicate that M3PO fine-tuning only requires approximately $N = 32$9 GPU hours with peak approximately $\{R_1, \dots, R_N\}$0 GB, while M3PO data generation plus fine-tuning requires approximately $\{R_1, \dots, R_N\}$1 GPU hours total, with peak approximately $\{R_1, \dots, R_N\}$2 GB during generation and approximately $\{R_1, \dots, R_N\}$3 GB during fine-tuning. The same source reports vanilla DPO at approximately $\{R_1, \dots, R_N\}$4 GPU hours, RM-DPO at approximately $\{R_1, \dots, R_N\}$5, SFT at approximately $\{R_1, \dots, R_N\}$6, and simulated RLHF at approximately $\{R_1, \dots, R_N\}$7 (Gao et al., 17 Aug 2025).

The limits of M3P-Score are also explicitly stated. First, it has MAS dependency: accuracy depends on external evaluators such as CLIP/BLIP-2 plus GPT-4V, and evaluator misjudgments can inject noise into pair selection. The paper lists evaluator ensembles or self-correction loops as mitigations to stabilize MAS (Gao et al., 17 Aug 2025). Second, it uses a fixed candidate pool of $\{R_1, \dots, R_N\}$8, which may under-sample hard cases for complex instructions; dynamic candidate counts conditioned on task complexity are suggested as a possible improvement (Gao et al., 17 Aug 2025). Third, the reported system has a single-turn focus. M3PO is demonstrated for single-turn visual instruction following, and extension to multi-turn dialogue would require MAS and confidence definitions that incorporate conversational coherence and context across turns (Gao et al., 17 Aug 2025). Fourth, although LoRA keeps fine-tuning efficient, scaling to much larger LVLMs or continuous pre-training pipelines may require more advanced distributed and parameter-efficient methods (Gao et al., 17 Aug 2025).

A further misconception is that M3P-Score itself is a general-purpose evaluation benchmark. The paper does not use it that way. It is a data-selection mechanism inside a specific training pipeline. Benchmark outcomes are reported separately on MME-Bench, POPE, IFT, and Human Pref. Score after DPO training on the tuples selected by M3P-Score (Gao et al., 17 Aug 2025).

In summary, M3P-Score in the strict technical sense denotes a multimodal preference-pair mining rule for LVLM alignment. It combines a black-box external alignment score with the model’s own sequence log-likelihood to select a preferred response of highest MAS and a dispreferred response that is worse by MAS yet remains internally plausible. Its empirical value, as reported, lies in hard negative mining for DPO-based visual instruction following (Gao et al., 17 Aug 2025).