Targeted Persuasion Score (TPS)

• Targeted Persuasion Score (TPS) is a target-sensitive distributional metric that measures how context shifts a model's answer distribution toward a specified target.
• It employs optimal transport and cost functions—such as basic match, ordinal, or embedding-based—to capture nuanced persuasion beyond binary answer changes.
• Empirical studies reveal that TPS can detect persuasive shifts undetected by greedy decoding, highlighting its practical value in evaluating context effects.

Searching arXiv for Targeted Persuasion Score and closely related persuasion metrics. Targeted Persuasion Score (TPS) is a target-sensitive distributional persuasion metric for LLMs that quantifies how persuasive a given context is by measuring how much the context shifts the model’s answer distribution toward a user-specified target answer distribution. It was introduced to address a limitation of coarse persuasion measures that inspect only the greedily decoded answer or binary accuracy. In TPS, persuasion is operationalized not as generic change, but as directed movement toward a specified target, with the additional ability to account for similarity among alternative answers through a task-specific cost function. This makes TPS a distributional, target-relative measure rather than a universal score of correctness, truth, or usefulness (Nguyen et al., 22 Sep 2025).

1. Conceptual basis and motivation

TPS was proposed in the setting where a LLM has both prior knowledge and the capacity to adapt to information supplied in context. The motivating problem is that common evaluation methods often fail to capture whether a context is genuinely persuasive in the intended direction. A greedy-decoding metric records only whether the top answer changed; accuracy treats persuasion as a binary correct/incorrect event; and a prior KL-based persuasion score measures distributional change without encoding whether the change moved the model toward a specified target answer (Nguyen et al., 22 Sep 2025).

The central conceptual distinction is therefore between change and targeted change. A context may substantially alter answer probabilities without flipping the argmax, and it may move mass toward an answer that is close to, but not identical with, the intended target. TPS was designed to capture both effects. Its object of measurement is not merely whether the model changed, but whether it changed toward the intended answer, and by how much.

This target sensitivity also distinguishes TPS from broader work on persuasive language. For example, work on LLM rewriting benchmarks measures relative persuasive language between text pairs rather than directed movement of an answer distribution toward a target (Pauli et al., 2024). Likewise, phishing research analyzes persuasion principles such as authority, scarcity, reciprocity, and social proof in message content, but does not define a single formal target-sensitive persuasion metric (Khadka et al., 2024). TPS is therefore best understood as a formal metric for in-context answer steering rather than a generic index of rhetorical persuasiveness.

2. Formal definition

The formalism begins with a finite prefix-free cover $P \subset \Sigma^*$, so that probabilities over possible answers are well-defined even for multi-token answers. The model’s prior answer distribution given a query $q$ is denoted $p(\cdot \mid q)$, the context-conditional distribution given context $c$ followed by query $q$ is $p(\cdot \mid cq)$, and the user-specified target distribution is $p^\star$ (Nguyen et al., 22 Sep 2025).

TPS is defined as the reduction in Wasserstein distance from the model’s answer distribution to the target distribution after adding the context:

$$\mathrm{TPS}(c,q,p^\star)=W\!\big(p(\cdot\mid q),p^\star\big)-W\!\big(p(\cdot\mid cq),p^\star\big).$$

Its interpretation is direct. If the context makes the model closer to the target, TPS is positive. If the context moves the model away from the target, TPS is negative. Because the score is defined relative to both a target distribution and a transport cost, it is inherently task-relative.

The Wasserstein term is defined using a cost function $c(x,y)$ over answers:

$$W(q_1,q_2)=\inf_{\gamma\in\Pi(q_1,q_2)} \sum_{(x,y)\in\mathcal{X}\times\mathcal{X}} \gamma(x,y)c(x,y),$$

where $q$0 is a transportation plan with the correct marginals. The operational meaning is that TPS measures how much probability mass must be moved, and how costly that movement is, to transform the model’s distribution into the target distribution. The crucial consequence is that answer similarity can be encoded explicitly rather than collapsed into exact match.

This formalization makes TPS a directional distributional metric. It differs from a raw divergence between prior and posterior because it compares both distributions to the target and subtracts those distances. That subtraction is what turns generic shift into targeted persuasion.

3. Variants, cost design, and practical computation

The simplest variant is BasicTPS, defined for a single distinguished target answer $q$1. The paper specifies the cost function

$q$2

Under this choice, BasicTPS lies in $q$3 and effectively measures how much probability mass is moved onto or off the target answer (Nguyen et al., 22 Sep 2025).

The paper gives two limiting intuitions. BasicTPS $q$4 corresponds to a context that moves all probability mass from a non-target prior to the target answer. BasicTPS $q$5 corresponds to a context that moves all mass from the target prior to a non-target answer. In this form, TPS is closely related to target-probability transfer, but remains distributional because it is derived from optimal transport.

For ordinal tasks, the cost function can encode graded distance. In the movie-rating experiments, answers lie on a $q$6–$q$7 scale and the paper defines

$q$8

This version makes shifts to nearby ratings cheaper than shifts to distant ratings.

For semantically related answers, the paper uses embedding cosine distance:

$q$9

This makes semantically closer answers cheaper to move probability mass between. The result is a distance-based TPS that can treat near-miss answers differently from unrelated alternatives.

The practical computation of TPS depends on access to the model’s answer distribution. The paper states a major implementation caveat: although the theory is over all outputs $p(\cdot \mid q)$0, in practice it is expensive to estimate the full output distribution, so the next-token distribution is often used as a noisy proxy. This is especially problematic when the context-suggested and prior answers share the same first token. TPS is therefore theoretically distributional over full answers, but often operationalized through an approximation that introduces tokenization sensitivity and underrepresentation of multi-token distinctions.

4. Empirical validation and observed behavior

The paper validates TPS through four studies: an official-language toy case, a word-sense disambiguation case, a movie-review rating study, and a political text coding task (Nguyen et al., 22 Sep 2025). Together, these experiments are intended to show that TPS captures persuasion effects that greedy answer change and aggregate accuracy may miss.

Study	Setup	Main observation
Official language	500 YAGO officialLanguage queries; contradictory context; Qwen-2.5 7B Instruct	TPS can be high even when greedy decoding does not change
Word sense disambiguation	100 words, four WordNet senses each, 100 GPT-generated contexts per sense	TPS is positive for target senses and negative for non-target senses
Movie review ratings	1,000 IMDb movies; synthetic positive/negative reviews; target ratings on a $p(\cdot \mid q)$1–$p(\cdot \mid q)$2 scale	TPS reveals asymmetries, noise effects, and lost-in-the-middle behavior
Political text coding	British manifesto sentences coded on a 1–5 left-right scale	Detailed prompts can help slightly, but can also increase variation or move away from expert opinion

In the official-language case, the paper’s primary point is that persuasion need not coincide with a greedy answer flip. A contradictory context can substantially shift probability mass toward the context’s claim even if the most likely answer remains unchanged. TPS records that as persuasion; greedy decoding does not.

In the word-sense disambiguation study, both BasicTPS and semantic distance-based TPS behave in the expected direction: target senses receive positive scores and non-target senses negative scores. The semantic variant is more selective than BasicTPS because semantically closer senses receive lower scores and farther senses higher scores. Across 100 words, the paper reports a Spearman correlation of $p(\cdot \mid q)$3 between cosine distance of senses and the difference in BasicTPS versus distance-based TPS, indicating that semantic cost materially changes the score (Nguyen et al., 22 Sep 2025).

In the movie-rating study, several phenomena appear that would be difficult to characterize with binary correctness alone. For small numbers of reviews, negative contexts are more persuasive than positive ones; this asymmetry weakens as the number of reviews increases; noisy contexts are less persuasive than uniform ones; contradictory reviews have less effect when embedded in longer concatenations than when evaluated individually; and a lost-in-the-middle effect emerges, with contradictory reviews at the beginning or end of the context exerting a larger effect than those placed in the middle. The paper emphasizes that this last effect is visible in distance-based TPS but not in greedy-decoded ratings.

In political text coding, TPS is used to examine whether prompt engineering moves model outputs toward expert labels. Technical definitions shift the model only slightly toward expert annotations. Five-shot prompts can increase variation and can sometimes move the model away from expert opinion for economic sentences. The significance of this result lies less in the average movement than in the heterogeneity: some prompts are influential in some examples and unhelpful or counterproductive in others. TPS exposes this heterogeneity more directly than aggregate accuracy.

5. Relation to neighboring persuasion measures

TPS belongs to a broader family of persuasion measurements, but it occupies a specific niche: target-conditioned distributional movement in model outputs. Several adjacent literatures define quantities that are close in spirit yet formally different.

Work	Closest quantity	Relation to TPS
“Measuring and Benchmarking LLMs’ Capabilities to Generate Persuasive Language” (Pauli et al., 2024)	Pairwise relative persuasion score $p(\cdot \mid q)$4	Measures relative persuasive language between rewritten texts, not target-conditioned answer-distribution shift
“A Survey on the Principles of Persuasion as a Social Engineering Strategy in Phishing” (Khadka et al., 2024)	Content-analysis and susceptibility frameworks	Provides cue taxonomies and targeting dimensions, but no formal TPS
“A Framework for Personalized Persuasiveness Prediction via Context-Aware User Profiling” (Park et al., 9 Jan 2026)	$p(\cdot \mid q)$5	Predicts view change for a user-message pair; personalized persuasion prediction rather than distributional answer steering
“Persuasion Effects in Regression Discontinuity Designs” (Jun et al., 30 Sep 2025)	RD persuasion rate $p(\cdot \mid q)$6	Causal local conversion fraction at a cutoff, not an LM distributional metric
“Persuasion Index: A Theory-Guided Framework for Persuasion Analysis” (Gong et al., 12 Jun 2026)	15-dimension feature space	Interpretable rhetorical substrate that could support future TPS-like models

The pairwise metric in (Pauli et al., 2024) is especially close operationally, but it measures relative persuasive language in rewritten text pairs. Its score

$p(\cdot \mid q)$7

is derived from human ordinal judgments about which text is more persuasive and by how much. That construct is a proxy for persuasive style, not a measure of how a context steers a model’s answer distribution toward a target.

By contrast, personalized view-change prediction in (Park et al., 9 Jan 2026) models persuasion success as a binary or probabilistic outcome for a specific persuadee-message pair on ChangeMyView. Its target is the persuadee, not an answer distribution; its outcome is delta versus no-delta, not optimal-transport movement toward $p(\cdot \mid q)$8. This suggests that TPS and personalized persuasion prediction address different layers of the persuasion problem: one models model-internal answer steering, the other models user-level persuasion success.

The phishing survey (Khadka et al., 2024) and the Persuasion Index framework (Gong et al., 12 Jun 2026) are relevant in a different way. They do not define TPS, but they provide structured accounts of persuasive cues, contextual tailoring, and context-sensitive variation. A plausible implication is that such feature spaces could complement TPS in future work when the aim is to relate distributional steering to identifiable rhetorical mechanisms.

6. Interpretation, limitations, and future extensions

TPS has several explicit limitations. It requires a finite set of answers, which fits multiple choice, ratings, and sense selection, but does not directly apply to open-ended free-form generation unless outputs are clustered into a finite semantic space (Nguyen et al., 22 Sep 2025). It formally assumes access to the model’s answer distribution, but in practice often relies on next-token probabilities. It is sensitive to answer tokenization when competing answers share the same first token. It also depends on the user-chosen cost function: exact-match cost yields BasicTPS, ordinal cost presumes an ordering, and embedding-based cost assumes that the embedding geometry reflects meaningful answer similarity.

The most important interpretive caveat is that TPS is target-relative. A high TPS means that the context moved the model toward a specified target distribution; it does not mean that the target is true, desirable, or epistemically justified. This corrects a common misconception that persuasion metrics are automatically aligned with truth. In TPS, persuasion is directional movement toward $p(\cdot \mid q)$9, not a normative endorsement of $c$0.

Several neighboring literatures indicate where extensions may plausibly develop. Research on phishing emphasizes lack of standardization in coding persuasion principles, limited understanding of combination effects, and the importance of contextual tailoring and target characteristics (Khadka et al., 2024). Personalized persuasion prediction shows that useful user profiles are context-dependent and predictor-specific rather than reducible to static attributes (Park et al., 9 Jan 2026). Persuasion Index demonstrates that persuasion-related outcomes can be represented as an interpretable vector of rhetorical dimensions whose importance varies by dataset, topic, and stance (Gong et al., 12 Jun 2026). This suggests that future TPS variants may combine target-conditioned distributional movement with richer models of audience specificity, rhetorical mechanism, and contextual calibration.

Taken together, the current literature positions TPS as a precise metric for one problem—how much a context moves a LLM’s answer distribution toward a designated target—within a larger ecosystem of persuasion research that includes rhetorical feature analysis, personalized outcome prediction, causal persuasion-rate estimation, and adversarial evaluation. Its distinctive contribution is to make persuasion measurable as directed distributional transport rather than as a top-answer flip, a raw divergence, or an undifferentiated style judgment (Nguyen et al., 22 Sep 2025).