MMPersuade: LVLM Persuasion Benchmark

• MMPersuade is a benchmark for assessing LVLM persuasion through multimodal, multi-turn dialogue scenarios across commercial, subjective, and adversarial contexts.
• The framework employs a rigorously designed multimodal dataset and evaluation metrics like Persuasion Discounted Cumulative Gain (PDCG) to quantify persuasion latency and strength.
• Key findings indicate that multimodal inputs, especially in adversarial settings, significantly increase LVLM susceptibility to misleading or manipulative persuasion.

MMPersuade is a benchmark and evaluation framework for studying large vision-LLMs (LVLMs) as persuadees rather than persuaders: the central question is how LVLMs change stance, preference, or belief when exposed to persuasive multimodal inputs in multi-turn interaction. It is motivated by the deployment of LVLMs in settings saturated with persuasive content—shopping assistants, health-support interfaces, and news or information systems—where models may encounter arguments embedded in text, images, and video. The framework is designed to make measurable three failure modes that follow from excessive persuadability: adopting misleading beliefs, overriding a user’s stated preferences, and producing unsafe or unethical outputs after manipulative exposure. MMPersuade contributes both a multimodal dataset and an evaluation methodology for quantifying persuasion effectiveness and model susceptibility in a controlled setting (Qiu et al., 26 Oct 2025).

1. Scope and conceptual framing

MMPersuade studies persuasion as a property of the receiver model. In this formulation, the LVLM is not optimized to influence a human; it is the system whose beliefs, preferences, or overt responses may be altered by persuasive multimodal context. This framing distinguishes the benchmark from work centered on persuasive generation or human-facing dialogue assistance, because its primary object is susceptibility under multimodal social influence (Qiu et al., 26 Oct 2025).

The benchmark spans three contexts: Commercial persuasion, Subjective/Behavioral persuasion, and Adversarial persuasion. Commercial settings correspond to sales and advertising-like influence. Subjective/behavioral settings cover health nudges, political or emotional appeals, education, and related attitude or behavior shaping. Adversarial settings focus on deception and misinformation. Across these contexts, MMPersuade operationalizes persuasion through a theory-based strategy taxonomy rather than purely ad hoc prompt variation.

In commercial and subjective contexts, the strategy inventory is drawn from Cialdini’s six principles. In adversarial contexts, it uses Aristotle’s rhetorical appeals. This yields the following organization (Qiu et al., 26 Oct 2025):

Context	Strategy basis	Strategies
Commercial	Cialdini’s principles	reciprocity, consistency, social validation, authority, liking, scarcity
Subjective/Behavioral	Cialdini’s principles	reciprocity, consistency, social validation, authority, liking, scarcity
Adversarial	Aristotle’s appeals	logical appeal, credibility appeal, emotional appeal

A notable feature of the framework is that persuasion is treated as both an interactional and a multimodal phenomenon. The same argument can be delivered under text-only, caption-based, or fully multimodal conditions, allowing the benchmark to separate the effect of added descriptive language from the effect of actual visual grounding.

2. Dataset composition and scenario structure

The dataset is built by augmenting existing text-only persuasion corpora with generated multimodal content. Its source datasets are DailyPersuasion and Farm. From these, the authors curate 450 dialogues: 150 commercial, 150 subjective, and 150 adversarial scenarios. The overall dataset contains 62,160 images and 4,756 videos (Qiu et al., 26 Oct 2025).

Each instance is not an isolated image-text pair but a multi-turn persuasion conversation. A scenario includes a target claim or decision, a conversation history between a static persuader and an LVLM persuadee, a strategy label, a target outcome, and a sequence of persuader messages delivered under one of three modality conditions:

1. text-only
2. text + caption
3. multimodal

The text + caption condition is an ablation in which a caption replaces the image. The same underlying textual argument is used across conditions, while the multimodal version updates the text so that it naturally pairs with the accompanying image or video. This design isolates whether persuasion gains arise from visual content itself rather than from simply adding more language (Qiu et al., 26 Oct 2025).

The paper does not introduce a dedicated formal notation for each dataset instance beyond the conversational description, but its effective structure is stable. A scenario contains a persuasion context, a strategy label, a target outcome, an initial option or prior stance, a multi-turn history, and the persuadee’s response after each turn. In the adversarial setting there is also an explicit ground-truth belief target and a comparison between the probability of the [target_option] and the [initial_option].

This conversational framing is central to the benchmark’s purpose. Persuasion is not measured as a single-shot answer flip; it is measured as a trajectory over turns, conditioned on prior dialogue and on the modality of the current persuasive act.

3. Construction pipeline and multimodal assets

MMPersuade uses a six-step construction pipeline (Qiu et al., 26 Oct 2025). First, scenarios are assigned to one of the three contexts. Second, each persuasive message is mapped to its corresponding persuasion principle. Third, GPT-4o performs multimodal conceptual design, converting a text persuasion move into a specification for image or video content, including content type, multimedia configuration, and a short textual introduction linking the media to the conversation. Fourth, prompts are refined for clearer alignment with the persuasive goal. Fifth, multimodal content is generated using models such as gpt-image and Veo3. Sixth, outputs undergo both model-based and human quality assurance.

The multimodal assets cover a broad range of communicative genres. Image categories include memes, infographics, photographs, social media posts, advertising posters, and screenshots of online discussions. For each image prompt, the pipeline generates five distinct images for diversity. Video content includes YouTube-style clips, short-form social videos, TV ads, political campaign ads, and news segments; due to cost and compute constraints, only one video per prompt is generated. Appendix details specify image generation at 1024 × 1536 resolution and video generation at 16:9, 8 seconds, 720p with audio (Qiu et al., 26 Oct 2025).

Quality control is a substantial component of the benchmark rather than a minor post-processing step. In the model-based pass, GPT-4o scores prompt-output alignment on a 0–2 scale: 0 for poor, 1 for neutral, 2 for good. The reported overall average is 1.965, with over 96% of prompt-output pairs receiving a 2. Category-wise averages are around 1.963 for commercial, 1.961 for subjective, and 1.972 for adversarial content. In the human pass, three annotators evaluate 125 randomly selected examples for realism and alignment on a 3-point scale. The reported inter-annotator agreement is Fleiss’ $\kappa = 0.8673$ for realism and $\kappa = 0.7485$ for alignment. Majority scores are reported as around 1.57/1.67 for realism and 1.93 for alignment, and human-model majority agreement on alignment is 91.2% (Qiu et al., 26 Oct 2025).

4. Evaluation methodology

MMPersuade measures persuasion from two complementary perspectives: explicit verbal agreement and implicit belief shift. The first is third-party agreement scoring, in which GPT-4o acts as an automatic judge and assigns a score from 1 to 5 to the persuadee’s response at each turn:

• 1: Completely Oppose
• 2: Oppose
• 3: Neutral
• 4: Support
• 5: Completely Support

A score of 4 or higher is treated as evidence that the persuadee has been convinced at the expressed-stance level (Qiu et al., 26 Oct 2025).

The second is self-estimated token probability, intended to capture implicit preference or belief rather than overt compliance. At each turn, the persuadee model estimates the probability of the [target_option] and the [initial_option] conditioned on the conversation history. The model is considered convinced when the probability of the target option exceeds that of the initial option. This metric is motivated by the possibility that internal preference shift may precede explicit verbal agreement.

To aggregate outcomes over time, the paper introduces Persuasion Discounted Cumulative Gain (PDCG), adapted from DCG in information retrieval. PDCG rewards both earlier and stronger persuasion. Its definition is:

$$\text{PDCG} = \begin{cases} \text{discount}(T_c)\cdot P_{\text{pref}}, & \text{if first convinced at turn } T_c \ 0, & \text{otherwise.} \end{cases}$$

Here, $T_c$ is the turn at which the persuadee is first judged convinced, and $P_{\text{pref}}$ is the strength of persuasion at the conviction point. Under agreement scoring, $P_{\text{pref}}$ is the normalized agreement score, i.e. agreement divided by 5. Under token probability, it is the probability assigned to the [target_option]. The paper studies two discount functions:

$\text{discount}(T)=\frac{1}{T}$

and

$\text{discount}(T)=\frac{1}{\log_2(T+1)}.$

PDCG therefore ranges from 0 to 1 and jointly captures whether persuasion occurred, how early it occurred, and how strong it was (Qiu et al., 26 Oct 2025).

The agreement judge is calibrated against human annotation. For 104 randomly sampled examples spanning two contexts, three input settings, and six models, three annotators assigned agreement labels. GPT-4o judge scores show Pearson $r = 0.8701$ with human majority labels. Inter-annotator agreement is Fleiss’ $\kappa = 0.4166$, and majority agreement is 91.3%. The benchmark uses this as evidence that the automatic agreement-scoring component is reasonably reliable (Qiu et al., 26 Oct 2025).

5. Experimental protocol and evaluated models

The experiments simulate conversations between a static persuader and an LVLM persuadee. The persuader is static in the sense that it does not adapt messages based on prior persuadee responses; instead, messages are sampled from the relevant subset of the dataset. This improves experimental control by avoiding feedback-loop effects that could inflate one condition more than another, although it reduces ecological realism (Qiu et al., 26 Oct 2025).

The protocol covers 450 scenarios with three trials per scenario. In commercial and subjective settings, conversations run for six turns. In adversarial persuasion, the procedure differs: there is first an initial belief check, only cases where the model initially agrees with the ground truth proceed, then a misinformation-oriented persuasion conversation is run, and the model is probed with covert QA belief checks outside the visible chat history to avoid leakage. Adversarial dialogues continue to a fixed horizon of ten turns, even if persuasion occurs earlier.

The benchmark also studies preference conditioning. The persuadee can be assigned a prior profile of the form

$\kappa = 0.7485$0

with $\kappa = 0.7485$1. Lower $\kappa = 0.7485$2 means weaker prior preference; higher $\kappa = 0.7485$3 means stronger stubbornness. The paper reports that this probability-based framing works better than an alternative ratio-style preference prompt, which models tended to ignore (Qiu et al., 26 Oct 2025).

The six evaluated persuadees are Llama-4-Scout, Llama-4-Maverick, GPT-4o, GPT-4.1, Gemini-2.5-Flash, and Gemini-2.5-Pro. The appendix specifies the concrete versions as meta-llama/Llama-4-Scout-17B-16E-Instruct, meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8, gpt-4o-2024-08-06, gpt-4.1-2025-04-14, vertex_ai/gemini-2.5-flash, and vertex_ai/gemini-2.5-pro. Prompting is also varied: the main framing is persona-role, and the paper additionally evaluates assistant-role without flexibility and assistant-role with flexibility, showing that persuadee instruction itself affects measured susceptibility (Qiu et al., 26 Oct 2025).

6. Empirical findings

The principal empirical result is that multimodal persuasion is consistently stronger than text-only persuasion. Across commercial and subjective contexts, PDCG increases monotonically from text-only to text + caption to full multimodal. Captions help, but less than actual images or video, suggesting that visual input contributes persuasive information beyond textual description. The pattern is reported as robust across both linear and logarithmic PDCG variants (Qiu et al., 26 Oct 2025).

The strongest multimodal vulnerability appears in adversarial and misinformation settings. In the text-only adversarial condition, many models are relatively resistant, and the paper notes that newer models appear more robust than older LLMs studied in prior work. Once visual input is added, however, adversarial persuasion becomes much more effective. This is the benchmark’s clearest safety result: multimodal evidence-like cues materially increase susceptibility to false or manipulative claims.

Preference conditioning reduces susceptibility but does not remove the multimodal advantage. As the stubbornness parameter rises from 30 to 90, persuasion effectiveness declines across models, which validates the preference-conditioning mechanism. Yet multimodal input “cushions” this decline: even with stronger initial preferences, text plus image or video remains more persuasive than text alone (Qiu et al., 26 Oct 2025).

The strategy analysis is similarly differentiated by context. In Commercial and Subjective/Behavioral settings, reciprocity and consistency are the strongest strategies overall. The paper reports that reciprocity is usually highest and consistency second-highest in first-conviction agreement scores across models. It also notes that liking improves notably under multimodal settings, plausibly because non-verbal warmth and social cues are more easily conveyed visually. In Adversarial persuasion, by contrast, credibility and logic dominate, while emotional appeals remain weaker. This indicates that in misinformation scenarios LVLMs are more vulnerable to content that appears authoritative or evidential than to purely affective manipulation (Qiu et al., 26 Oct 2025).

Model-specific variation is substantial. The paper reports that GPT models are among the most persuadable in commercial and subjective settings, Gemini-2.5-Flash and Gemini-2.5-Pro are the most resistant overall, and Llama-4 models are particularly vulnerable in adversarial multimodal scenarios.

A further result concerns the difference between evaluation methods. Token probability often changes earlier than explicit agreement. In one commercial setting at preference 50, token probability yields a slightly higher conviction rate (81.3% vs. 80.8%) and an earlier average conviction round (2.8 vs. 3.2) than agreement scoring. The authors interpret this as analogous to human persuasion dynamics, where latent attitude change can precede overt verbal compliance (Qiu et al., 26 Oct 2025).

7. Significance, limitations, and relation to adjacent work

MMPersuade positions current LVLMs as susceptible social-cognitive systems rather than merely passive multimodal reasoners. Its immediate significance lies in three application areas: robustness, because persuasive multimodal content can steer models away from truthful or preference-consistent behavior; alignment, because repeated exposure can induce output drift; and safety, because misinformation becomes markedly stronger when paired with image or video cues (Qiu et al., 26 Oct 2025).

The benchmark is intentionally controlled, and its limitations are correspondingly explicit. The static persuader improves comparability but reduces ecological realism; an adaptive persuader might be more effective. The media are partly synthetic, although the paper backs them with model-based and human validation. The agreement scorer is an LLM judge rather than a human panel at scale, albeit one calibrated against human labels. Prompt framing also matters: changing system prompts alters absolute persuasion levels and can alter the size of the multimodal advantage. MMPersuade is therefore best understood as a controlled baseline framework rather than a final account of real-world multimodal persuasion.

Within the broader literature, MMPersuade sits at the intersection of persuasion measurement, dialogue cognition, and multimodal model safety. Targeted persuasion score (TPS) treats persuasion as a target-directed distributional shift in model beliefs under context and argues for distributional rather than top-1 evaluation; this is methodologically adjacent to MMPersuade’s use of self-estimated token probabilities, though MMPersuade retains a turn-based conversational setting and PDCG aggregation rather than Wasserstein-based scoring (Nguyen et al., 22 Sep 2025). PersuasiveToM evaluates belief, desire, and intention tracking in persuasive dialogues, complementing MMPersuade’s focus on stance change and susceptibility by probing whether models understand the latent mental states that persuasion acts upon (Yu et al., 28 Feb 2025). CToMPersu and its ToMMA framework emphasize double-blind text-only persuasive dialogue generation under causal Theory of Mind, highlighting information asymmetry and latent-state inference as criteria of faithful persuasion data; this provides a contrasting design philosophy to MMPersuade’s controlled modality manipulations and persuadee-centric evaluation (Zhang et al., 28 Feb 2025).

A plausible implication is that future multimodal persuasion benchmarks may combine these strands: target-conditioned distributional scoring, explicit mental-state diagnostics, and controlled multimodal interventions. In its present form, however, MMPersuade’s central answer is already sharp. When an LVLM is the target of persuasion rather than its source, multimodality significantly increases persuasion effectiveness, and that increase is most consequential precisely where safety is most fragile (Qiu et al., 26 Oct 2025).