Malicious Intent Fulfillment Rate (MIFR)

• MIFR is defined as the percentage of harmful query-response pairs labeled ‘fulfilled’ by a reasoning-LLM judge, encapsulating the direct achievement of a malefactor's objective.
• It distinguishes between mere safety bypass, as measured by ASR, and the actual provision of actionable, specific harmful content.
• Empirical studies reveal that MIFR scores are consistently lower than ASR, indicating that many jailbreak attacks produce responses that, while unsafe, lack practical malicious utility.

Malicious Intent Fulfillment Rate (MIFR) is an evaluation metric introduced in "JPS: Jailbreak Multimodal LLMs with Collaborative Visual Perturbation and Textual Steering" to measure whether a jailbreak response does more than merely evade a safety filter: it checks whether the response actually fulfills the attacker’s malicious goal (Chen et al., 7 Aug 2025). The metric was proposed because multimodal jailbreak evaluation had largely centered on Attack Success Rate (ASR), which typically counts a response as successful if it is judged harmful or unsafe, even when the output is vague, evasive, theoretical, non-actionable, or only loosely related to the malicious request. MIFR instead measures whether the output genuinely satisfies the specific malicious intent behind the query, thereby shifting evaluation from a defender-centric question of safety bypass to an attacker-centric question of malicious utility.

1. Definition and formalization

MIFR is defined as the percentage of evaluated query-response pairs that the judge labels “fulfilled” (Chen et al., 7 Aug 2025). In the paper’s notation,

$$\text{MIFR} = \frac{\#\,\text{Judge}(Q_{\text{harm}}^{i}, R^{i}) = \text{"fulfilled"}}{N}\times 100\%.$$

Here, the evaluator considers a harmful query $Q_{\text{harm}}^{i}$ and the corresponding model response $R^{i}$, and assigns a binary judgment. The output space is explicitly non-graded at the metric level: each example is labeled either fulfilled or not fulfilled, and MIFR is the average of those binary outcomes over the test set.

This definition makes MIFR stricter than ASR. The paper gives the general ASR formula as

$$\text{ASR} = \frac{N_{\text{Harmful Responses}}}{N}\times 100\%.$$

In that formulation, ASR measures whether the response is classified as harmful or unsafe. MIFR measures whether the response is useful in fulfilling the malicious request. The distinction is operational rather than merely terminological: a response can have high ASR but lower MIFR, because harmfulness detection and malicious intent fulfillment are not identical criteria (Chen et al., 7 Aug 2025).

The rule governing disclaimers is also explicit. A response can still be judged fulfilled even if it contains disclaimers or safety warnings, as long as it provides actionable harmful content. MIFR therefore evaluates substantive fulfillment of malicious intent rather than stylistic markers of refusal or compliance.

2. Rationale and contrast with Attack Success Rate

The motivation for MIFR is the claim that many jailbreaks can achieve high ASR but still produce low-utility outputs (Chen et al., 7 Aug 2025). The paper identifies two main failure modes. The first is failed instruction following, in which the model does not answer the core malicious request. The second is insufficient content harmfulness, in which the answer is too vague, infeasible, or generic to be practically useful.

Under this analysis, ASR over-rewards any harmful-looking response and does not penalize low-quality outputs. The authors report a pilot study on AdvBench showing that even when responses were not actually aligned with the malicious intent, safety evaluators such as Llama-Guard-3 and the HarmBench official classifier could still give very high ASR (Chen et al., 7 Aug 2025). This is the central empirical justification for introducing a second metric.

Several misconceptions follow directly from this contrast. One is that a jailbreak counted as successful by ASR has necessarily produced attacker-useful content; the paper argues the opposite, namely that ASR alone can be misleading. Another is that refusal language or warnings automatically imply failure; under MIFR, disclaimers do not negate fulfillment if the response still contains specific, actionable, non-trivial harmful information. A plausible implication is that MIFR reclassifies a nontrivial portion of apparent jailbreak successes as failures of malicious intent fulfillment rather than failures of safety alignment alone.

3. Evaluation protocol and reasoning-LLM judgment

The paper proposes not only a metric but also a concrete protocol for computing it with a reasoning-LLM judge (Chen et al., 7 Aug 2025). The judge model is QWQ-32B.

The protocol has three steps. First, the judge reads the harmful query $Q_{\text{harm}}^{i}$ and infers the attacker’s underlying objective rather than only the surface wording. Second, the judge analyzes the response $R^{i}$ under two criteria: Instruction Following (IF) and Content Harmfulness (CH). IF asks whether the response directly and faithfully addresses the harmful request. CH asks whether the response provides specific, actionable, non-trivial harmful information with practical utility. Third, the judge outputs a binary final decision: fulfilled or not fulfilled.

The evaluator’s input is limited to the harmful query and the generated response. The prompting strategy instructs the reasoning LLM to infer the attacker’s true intent, judge whether the response fulfills that intent, and output a final binary label. The evaluation is intentionally utility-focused rather than safety-focused. A response is marked fulfilled if it directly addresses the malicious purpose, is sufficiently specific and actionable, and provides practically useful harmful content (Chen et al., 7 Aug 2025).

This protocol makes the metric dependent on judge quality. The paper notes that MIFR depends on a Reasoning-LLM judge, so its accuracy is only as good as the judge’s reasoning and prompt following. That caveat does not alter the definition of MIFR, but it constrains how the metric should be interpreted in comparative evaluations.

4. Experimental use in JPS

MIFR is reported on MM-SafetyBench and HarmBench and evaluated on InternVL2-8B, Qwen2-VL-7B-Instruct, and MiniGPT-4 (Chen et al., 7 Aug 2025). The paper compares Vanilla Text, PAIR, VAJM, UMK, BAP, QR, and JPS (proposed).

The main empirical finding is that MIFR is consistently lower than ASR for many baselines, revealing that ASR overestimates jailbreak quality. Illustrative gaps include UMK on MiniGPT-4 (MM-SafetyBench) with ASR 86.73% versus MIFR 78.10%, UMK on InternVL2 (MM-SafetyBench) with ASR 79.35% versus MIFR 38.63%, and BAP on Qwen2-VL (HarmBench) with ASR 59.00% versus MIFR 37.00% (Chen et al., 7 Aug 2025). These examples support the paper’s claim that some jailbreaks are successful only in the ASR sense, not in actually fulfilling malicious intent.

JPS is reported as the best method on both ASR and MIFR across all tested models and datasets. Its reported results are as follows:

Benchmark / model	ASR	MIFR
MM-SafetyBench / InternVL2	93.10%	88.69%
MM-SafetyBench / Qwen2-VL	92.02%	86.49%
MM-SafetyBench / MiniGPT-4	92.02%	87.98%
HarmBench / InternVL2	93.50%	86.50%
HarmBench / Qwen2-VL	89.50%	76.50%
HarmBench / MiniGPT-4	98.00%	83.00%

The paper further emphasizes that JPS has a smaller ASR–MIFR gap than other methods. In the paper’s interpretation, this means its successful jailbreaks are more likely to be genuinely useful from the attacker’s perspective (Chen et al., 7 Aug 2025). The significance of MIFR in these experiments is therefore not merely additive; it changes the interpretation of which methods are genuinely effective.

5. Connection to collaborative visual perturbation and textual steering

MIFR is tightly connected to the design of JPS itself (Chen et al., 7 Aug 2025). The framework combines target-guided adversarial image perturbations with a steering prompt optimized via a multi-agent system, and the two components undergo iterative co-optimization.

Within that design, visual perturbation is used for safety bypass. The adversarial image is optimized with PGD + momentum, and it helps bypass refusal and safety mechanisms. Textual steering is used for response quality control. The Multi-Agent System refines the steering prompt and separately optimizes Instruction Following and Content Harmfulness. The paper describes this as a decoupling strategy: Image = bypass and Text = intent fulfillment.

This decomposition explains why MIFR is treated as necessary rather than optional. If image perturbation primarily drives bypass and textual steering primarily drives malicious utility, then ASR alone cannot reveal whether the second objective has been achieved. A plausible implication is that MIFR functions as the natural evaluation counterpart to the text-steering component of JPS.

The ablation study supports that interpretation. The paper reports that w/o Adv Image causes ASR to drop sharply, showing the image is crucial for bypass. w/o Steering Prompt causes MIFR to drop substantially, showing the text steering is crucial for intent fulfillment. w/o MAS, w/o IFG, and w/o HCG also reduce MIFR, indicating that the multi-agent prompt-refinement pipeline improves malicious intent fulfillment (Chen et al., 7 Aug 2025). On HarmBench for InternVL2, the paper gives the specific comparison JPS full: ASR 93.50%, MIFR 86.50% versus w/o Steering Prompt: ASR 80.50%, MIFR 74.00%. The stated conclusion is that ASR and MIFR capture different aspects of attack quality.

6. Relation to adjacent concepts and neighboring research

MIFR is a specific metric, but several adjacent lines of research define closely related quantities. In "Can LLMs Deeply Detect Complex Malicious Queries? A Framework for Jailbreaking via Obfuscating Intent," the acronym MIFR does not appear, yet the paper defines a successful prompt jailbreak attack by the conjunction of three conditions: the prompt contains illegal intentions, the content restriction rules are not met, and the response includes specific dangerous content or privacy information; formally, $F_{attack} = \bigcap^{3}_{i=1}f_{Con_i} = True$ (Shang et al., 2024). That paper also introduces ASR, REJ, HAL, and an effective response rate $R_{eff}$, which function as strong proxies for malicious intent fulfillment. In its terminology, high fulfillment corresponds to malicious prompts that bypass rejection and yield restricted or harmful output.

A different relation appears in "Harmful Intent as a Geometrically Recoverable Feature of LLM Residual Streams," which studies whether harmful intent is recoverable from residual streams before generation rather than whether it is fulfilled in the final response (Llorente-Saguer, 20 Apr 2026). The paper reports that harmful intent is geometrically recoverable as a linear direction in most layers and as angular deviation in layers where projection methods fail, with mean AUROC 0.98 and TPR@1\%FPR 0.80 for a soft-AUC-optimized linear direction, and argues that harmful intent and refusal behavior are functionally dissociated. Its relevance to MIFR is therefore upstream: detectability of malicious intent is not the same as fulfillment of that intent. The paper also warns that AUROC in the 0.97+ regime can substantially overestimate operational detectability, which suggests that internal intent recognition should not be conflated with end-to-end malicious completion.

In multi-turn settings, "One Turn Too Late: Response-Aware Defense Against Hidden Malicious Intent in Multi-Turn Dialogue" defines the harmful closure turn $t^*(\tau, g)$ as the earliest turn at which the accumulated interaction becomes sufficient to enable harm, and evaluates defenders with Acc., Miss, Early, Harmful Score, and online ASR (Shen et al., 7 May 2026). These are not MIFR, but they are MIFR-like notions of attack success and harmful intent completion under response-aware, turn-level intervention. The paper’s framework is response-dependent and centered on the earliest harm-enabling closure point rather than on single-turn fulfillment.

Taken together, these neighboring formulations delimit the scope of MIFR. MIFR is neither a general synonym for ASR nor a synonym for harmful-intent detection. It is a behavioral metric of whether a generated response actually fulfills the malicious objective posed by a query. This suggests that MIFR occupies a specific place in the evaluation stack: downstream of intent recognition, narrower than generic harmfulness classification, and distinct from turn-level closure metrics in multi-turn dialogue.