Persuasion Index: A Theory-Guided Framework for Persuasion Analysis

Abstract: Identifying persuasive rhetorical cues is critical across domains, from detecting information manipulation and improving AI safety to advancing public health communication. We propose Persuasion Index (PI), a taxonomy of 15 dimensions grounded in persuasion theories from psychology and communication, and one transparent implementation using 55 sub-features built from lexicons and rule-based detectors. The taxonomy is modular: individual detectors can be replaced while preserving the theoretical structure. By evaluating PI on four public datasets varying in domain, style, and outcome measures, we show that PI provides a shared feature space for interpreting rhetorical patterns associated with persuasion-related outcomes. Linear models show that PI features carry meaningful predictive signal while remaining computationally lightweight. Dimension-level analyses reveal recurring associations between PI dimensions and persuasion outcomes across datasets, while also highlighting topic- and stance-specific variation. We release PI as an open-source package and web interface for principled and auditable analysis of human and AI-mediated communication.

• The paper introduces a transparent, theory-grounded framework that quantifies persuasive rhetoric using a 15-dimension taxonomy and 55 sub-feature detectors.
• It leverages established lexicons and rule-based methods to capture Logos, Ethos, and Pathos, facilitating interpretable and modular analysis of text.
• Empirical validation across multiple datasets demonstrates that its fine-grained features rival black-box models like RoBERTa and GPT-4o in performance.

Persuasion Index: A Formal, Theory-Grounded Architecture for Persuasion Analysis

The "Persuasion Index: A Theory-Guided Framework for Persuasion Analysis" (2606.14580) introduces the Persuasion Index (PI), a transparent, interpretable, and extensible taxonomy for characterizing persuasive rhetoric in natural language. Leveraging enduring theories of persuasion in psychology and communication studies, PI consolidates prior fragmented approaches into a 15-dimension framework, supported by 55 sub-feature detectors based on lexicons, rule-based methods, and validated resources. This essay provides an expert-level summary of the architecture, evaluation methodology, main results, and broader implications.

Theoretical Foundations and Taxonomy

PI is constructed atop the Aristotelian triad—Logos, Ethos, Pathos—to provide comprehensive rhetorical coverage. Under each high-level category, relevant dimensions are defined according to empirical and meta-analytic results in persuasion research.

• Logos (reason-based appeals): Includes Evidence (facts/statistics), Logic and Cohesion (explicit connective structure), Argumentation (claim explicitness and premise density), Specificity (concreteness/abstraction anchored in construal-level theory), and Opponent's View (acknowledgment/refutation).
• Ethos (credibility-based appeals): Authority and Credibility (trust/expertise), Politeness (reactance mitigation), Commitment (speaker's expressed resolve), Style (linguistic fluency and surface features).
• Pathos (emotion-based appeals): Sentiment/Emotion (affective charge), Impact (future-oriented consequences/gain-loss framing), Engagement (psychological immersion/self-reference), Reciprocity, Scarcity and Urgency, Propaganda (heuristically-driven compliance and distortion).

The 15-dimension taxonomy (Figure 1) supports modular sub-feature design, allowing replacement or augmentation of detectors without disrupting theoretical integrity.

Figure 1: The PI taxonomy illustrates 15 dimensions, mapping sub-features onto Logos, Ethos, and Pathos.

Implementation and Scoring

The paper's canonical implementation couples theory-driven lexica and rule-based detectors. Each text is encoded as a 55-dimensional vector based on sub-feature frequency/density, saturating as a nonlinear function of match rate to accommodate both sparse and dense cues. Sub-features leverage established resources (e.g., NRC-VAD, LIWC, VADER) and are expanded using LLM-augmented prompt-based generation and human curation to ensure coverage and validity.

Dimension scores are the mean of their sub-features, producing outputs interpretable at both granular and aggregate levels. The design is strictly modular: more advanced computational detectors (e.g., fine-tuned neural models) can supplant lexicon/rule-based features as the field progresses.

Empirical Validation Across Diverse Datasets

PI is empirically validated on four major persuasion datasets: UKPConvArg1 (UKP) (pairwise judgment task), IBM Argument Quality (expert pairwise rankings), ChangeMyView (CMV, Δ-based forum), and Anthropic Persuasion (experimentally measured attitude shifts). For each, a consistent binary persuasion target is established, and logistic regression models are trained on PI features (both 15-dimension mean and full 55-dimension vector); baselines include fine-tuned RoBERTa and zero-shot GPT-4o.

Key findings:

• PI-sub (55 sub-features) consistently matches or outperforms RoBERTa on all domains, and on CMV/IBM achieves the highest F1.
• PI-mean (dimensional mean features) performs less strongly than PI-sub, confirming the utility of fine-grained features.
• PI-sub and GPT-4o are competitive: each leads on two datasets, but PI-sub does so with a much more interpretable, transparent pipeline.

Statistical evaluation (McNemar's test, bootstrapping, equivalence testing) confirms that PI-sub's predictive performance is on par with black-box models, despite its transparency and computational lightness.

Systematic Feature Analysis: Topic, Domain, and Stance

Dimension-level analyses reveal nuanced regularities and contextual shifts:

• Domain Generality: Evidence, Logic/Cohesion, and Sentiment are broadly positive predictors, but with domain-contingent weights. Structured debate tasks (UKP) amplify Logic/Cohesion, while informal or social domains (CMV, Anthropic) see greater influence from affective/authority-based dimensions.
• Topic Sensitivity: Within-topic analyses (Figure 2) show Evidence and Logic/Cohesion consistently high for policy/framing topics, but affective/relational cues (Sentiment, Impact, Commitment) emerge as dominant in interpersonal or identity topics.
• Stance Effects: In the IBM corpus (Figure 3), PRO arguments (in favor) favor Argumentation, Authority, and Scarcity/Urgency, while CON arguments (opposed) weight Evidence and Reciprocity, confirming strategic adaptation by stance.

  Figure 2: Top PI-mean dimensions across UKP topics reveal that dominant persuasive dimensions vary systematically with thematic context.

  

  Figure 3: Stance-conditioned coefficients on IBM: Sentiment stands out for both PRO/CON stances, but other dimensions show stance-contingent divergence.

These findings demonstrate PI's value as an analytical tool to reveal consistent rhetorical mechanisms and context-dependent variance, closely mapping to predictions of dual-process persuasion models (ELM, HSM).

Interpretability and Auditing Utility

Unlike latent neural representations, every PI score is directly traceable to explicit lexical/structural patterns, enabling auditability for both human and LLM-generated content. The clear separation between the taxonomy (theory-grounded categories) and implementation (detectors) allows for cumulative, community-driven enhancement of the framework.

PI is further released as an open-source package and interactive web application, supporting real-time, explainable argument analysis for both policy research and AI safety audit workflows.

Numerical and Contradictory Results

Strong numerical findings include:

• On UKP, PI-sub F1 = 0.768, RoBERTa = 0.565, GPT-4o = 0.842.
• On CMV, PI-sub F1 = 0.588, outperforming both RoBERTa and GPT-4o.

A notable contradictory result is that Engagement has negative coefficients across domains—surface-level proxies for psychological involvement do not correlate reliably with persuasiveness, challenging the assumption of their universal benefit for persuasion.

Limitations

PI models rhetorical strategies visible in text, but cannot capture audience-state, source reputation, or extra-textual dynamics (e.g., prior beliefs, relational context). The current lexicon/rule-based implementation only partially operationalizes pragmatics, irony, and discourse-level phenomena. While modular, extending PI to multimodal/multilingual or spoken language settings will demand substantial development. Variability in some dimensions, especially for sparse signals, illustrates the need for ongoing lexicon refinement and larger annotated datasets.

Broader Implications and Future Directions

PI addresses longstanding calls in computational persuasion for interpretable, theory-driven explanatory frameworks [bassiDecodingPersuasionSurvey2024]. It facilitates interdisciplinary research, content analysis, and manipulation checks in social science and AI safety. By offering a modular, extensible foundation, PI opens the door to cumulative feature development—including hybrid rule-based/LLM-aided sub-feature detection and potentially cross-lingual expansion.

As LLMs increasingly generate, personalize, and evaluate persuasive rhetoric, robust auditing and systematic explication of deployed persuasive strategies become ever more crucial [hackenburgLeversPoliticalPersuasion2025], [salviConversationalPersuasivenessGPT42025]. PI's transparent, modular structure provides an essential toolkit for these challenges, and helps bridge the gap between predictive power and interpretability that has hampered progress in computational persuasion research.

Figure 4: The seven most impactful PI-mean dimensions on persuasion outcomes, highlighting regular cross-domain but context-sensitive coefficients.

Conclusion

The Persuasion Index sets a new standard for computational persuasion analysis: grounded in established theoretical constructs, operationalized with interpretable features, and empirically validated across varied settings. Its open-source release ensures reproducibility and community extensibility, offering a principled scaffold for future study of both human and AI-mediated persuasion. PI’s modularity and interpretability position it as an essential tool—not only for academic research but for practical auditing, comparative analysis, and safe deployment of persuasive technology.