Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions (1602.01103v2)

Abstract: Changing someone's opinion is arguably one of the most important challenges of social interaction. The underlying process proves difficult to study: it is hard to know how someone's opinions are formed and whether and how someone's views shift. Fortunately, ChangeMyView, an active community on Reddit, provides a platform where users present their own opinions and reasoning, invite others to contest them, and acknowledge when the ensuing discussions change their original views. In this work, we study these interactions to understand the mechanisms behind persuasion. We find that persuasive arguments are characterized by interesting patterns of interaction dynamics, such as participant entry-order and degree of back-and-forth exchange. Furthermore, by comparing similar counterarguments to the same opinion, we show that language factors play an essential role. In particular, the interplay between the language of the opinion holder and that of the counterargument provides highly predictive cues of persuasiveness. Finally, since even in this favorable setting people may not be persuaded, we investigate the problem of determining whether someone's opinion is susceptible to being changed at all. For this more difficult task, we show that stylistic choices in how the opinion is expressed carry predictive power.

• The paper shows that early posting and controlled back-and-forth exchanges significantly boost the likelihood of persuasion in online debates.
• The paper demonstrates that distinct linguistic interplay—using different content words while matching stylistic stopwords—predicts argument success.
• The paper reveals that initial post malleability, indicated by personal language and clear formatting, correlates with higher susceptibility to persuasion.

This paper investigates the factors contributing to successful persuasion in online discussions, using data from the Reddit forum /r/ChangeMyView (CMV) (1602.01103). CMV provides a unique environment where users (Original Posters or OPs) state their views and reasoning, invite challenges, and explicitly award a "delta" (Δ) to comments that change their view. The research explores persuasion through three main lenses: interaction dynamics, linguistic features of arguments, and characteristics of the OP's initial post.

Dataset and Methodology

The paper utilizes a dataset collected from CMV between January 2013 and August 2015, separating the last four months as a heldout test set. The dataset contains thousands of discussion trees, millions of comments, and tens of thousands of participants. The explicit delta mechanism provides reliable labels for successful persuasion attempts.

Interaction Dynamics

The analysis reveals several patterns related to how interactions unfold and their correlation with persuasion success:

1. Entry Order: Challengers who post their arguments earlier in a discussion thread have a significantly higher probability of receiving a Δ. This holds true even when controlling for user experience (i.e., considering only first-time challengers), suggesting a potential first-mover advantage or benefit from capturing the OP's initial attention.
2. Back-and-Forth Engagement: Engaging in a limited back-and-forth exchange with the OP (e.g., 2-4 replies) is associated with a higher chance of success compared to a single comment. However, excessively long exchanges (5+ replies) rarely result in the OP changing their view, potentially indicating entrenchment or fruitless debate.
3. Number of Participants: The overall likelihood of an OP changing their view increases with the number of unique challengers participating in the discussion. However, this relationship shows diminishing returns (is sublinear).
4. Argument Diversity vs. Volume: When examining subtrees focused on a specific counterargument, those involving a single challenger are more likely to be successful than those where multiple challengers contribute to the same argument thread (controlling for the number of comments). This suggests that the diversity of arguments presented by multiple unique root challengers might be more impactful than multiple people reinforcing the exact same point.

Language Features of Persuasive Arguments

To isolate the effect of language style, the paper sets up a paired prediction task: given two arguments (rooted path-units) responding to the same OP and having high lexical similarity (Jaccard similarity), predict which one successfully received a Δ. Key findings include:

1. Interplay with OP's Language: Features measuring the linguistic relationship between the challenger's argument and the OP's original post are highly predictive.
   • Dissimilarity in Content Words: Successful arguments tend to use different content words than the OP's post.
   • Similarity in Stopwords: Conversely, successful arguments show higher similarity in stopword usage, potentially indicating stylistic coordination.
   • Predictive Power: These interplay features alone achieve high accuracy (65.1% on heldout data for the root-reply task), significantly outperforming length and bag-of-words baselines.
2. Argument-Only Features (Style):
   • Length: Longer arguments are generally more successful, but this is less predictive than interplay features.
   • Word Choice & Formatting: Successful arguments tend to use more definite articles ("the"), include links as evidence (contrary to potential "backfire effect" concerns), use calmer language (lower arousal), employ formatting like bolding, italics, and bullet points, and show greater lexical diversity (word entropy).
   • Structure: Successful arguments often start with calmer language and maintain lower content-word similarity to the OP throughout different sections (quarters) of the text.

Predicting OP's "Resistance" to Persuasion

The paper also investigates whether the way an OP initially states their view predicts their likelihood of being persuaded later (malleability). This is framed as an imbalanced classification task.

1. Task Difficulty: Predicting malleability is challenging; human annotators performed at chance level in a pilot paper. The models achieve modest performance (AUC ~0.57).
2. Features of Malleable Posts:
   • Pronouns: More first-person singular pronouns ("I", "me") are associated with malleability (linking to self-affirmation theory), while more first-person plural pronouns ("we", "us") correlate with resistance.
   • Formatting & Style: Malleable views tend to be expressed with better organization (more paragraphs, bolding, lists), higher dominance and valence (more confident/positive language), and lower arousal (calmer tone).
   • Word Choice: Resistant posts often use more decisive words ("certain", "nothing", "wrong") and superlative adjectives/adverbs, whereas malleable posts might use words like "help", "please" and comparative adjectives/adverbs.

Practical Implications and Implementation

• For Persuaders: Commenting early, engaging moderately (avoiding very long back-and-forths), using different content vocabulary than the OP while potentially matching style (stopwords), providing evidence (links), using clear formatting, and maintaining a calm tone seem beneficial.
• For Platform Designers/Analysts: The linguistic features identified (especially interplay metrics) can be implemented to build models predicting argument persuasiveness or identifying potentially malleable viewpoints. Interplay features require comparing the argument text ($A$) with the original post text ($O$).
  • Implementation: Calculate word sets (all, content, stopwords) for $A$ and $O$. Compute metrics like Jaccard similarity ($\frac{|A \cap O|}{|A \cup O|}$), comment fraction ($\frac{|A \cap O|}{|A|}$), OP fraction ($\frac{|A \cap O|}{|O|}$), and raw counts ($|A \cap O|$) for each word set type. These features can be fed into standard classifiers (e.g., Logistic Regression with L1/L2 regularization).
  • Lexicons/Tools: Utilize standard stopword lists, part-of-speech taggers, and potentially lexicons for arousal, valence, dominance, and concreteness (like those by Warriner et al. or Brysbaert et al.). Word embeddings can be used to extrapolate lexicon scores to unseen words.
• Dataset: The released CMV dataset is a valuable resource for further research into argumentation and persuasion.

The paper highlights the importance of both interaction patterns and linguistic style, particularly the interplay between the challenger's language and the OP's original expression, in achieving persuasion in online good-faith discussions.