Troll Behavior and Detection Strategies
- Trolls are online actors who post provocative or menacing content to disrupt discussions, manipulate opinions, or trigger conflicts.
- Research employs NLP, contextual encoders, and behavioral analysis to quantify trolling dynamics and differentiate it from hate speech.
- Advanced methods like TROLL RL use trust-region optimization in reinforcement learning to enhance training speed, stability, and model performance.
Searching arXiv for recent and foundational papers on trolling, troll detection, and the TROLL RL method. “Troll” denotes a family of phenomena rather than a single stable object of study. The literature explicitly notes that there are different definitions of what a troll is: a participant who posts provocative, offensive, or menacing messages in conversation; content intended to cause disruption, trigger conflict, or insult for amusement; and actors trying to manipulate public opinion, sometimes for money and sometimes as part of state-sponsored influence campaigns (Mojica, 2016, Hitkul et al., 2020, Mihaylov et al., 2019). The term therefore spans comment-level pragmatics, user-level behavior, coordinated information operations, and moderation-oriented NLP. In a separate reinforcement-learning literature, “TROLL” also denotes “Trust Region Optimization for LLMs,” an acronym unrelated to online antisocial behavior (Becker et al., 4 Oct 2025).
1. Definitions, boundaries, and taxonomies
A central definitional move in recent offensive-language research is to treat trolling as a subtype of offensive content rather than as a synonym for hate speech. “Trawling for Trolling: A Dataset” defines trolling as “content intended to cause disruption, trigger conflict, or insult for amusement,” and notes that trolling may include humor, sarcasm, or mockery, can be directed at individuals or groups, and is set apart from more severe or aggressive forms such as hate speech or derogatory content (Hitkul et al., 2020). In that formulation, the distinction is not merely lexical; it is tied to intent, severity, and whether the utterance is primarily disruptive or identity-directed.
Conversation modeling work makes that boundary more explicit by separating authorial intent from audience uptake. “Modeling Trolling in Social Media Conversations” proposes four aspects for analysis: the troll’s intention, intention disclosure, the responder’s interpretation of the troll’s intention, and the responder’s response strategy (Mojica, 2016). This framework treats trolling as a pragmatic relation within a thread, not just a property of a single utterance. It also accommodates false positives, misunderstandings, mock trolling or playing, and counter-trolling.
Political-communication work uses a narrower and different sense. “Hunting for Troll Comments in News Community Forums” states that a troll can be someone who teases to provoke anger, offends others, seeks to dominate discussion, or attempts to manipulate public opinion, sometimes for money, and then focuses on the last definition as dominant in Bulgarian and Eastern European public discourse (Mihaylov et al., 2019). In this line of work, a “troll” is closer to an opinion manipulator than to a merely rude participant.
A recurring misconception is that trolling is confined to a fixed antisocial minority. “Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions” argues the opposite: ordinary individuals can and do engage in trolling behavior when triggered by situational factors such as mood and discussion context (Cheng et al., 2017). This suggests that the term has to be interpreted at multiple granularities: as content type, as interactional role, and as account-level strategy.
2. Situational dynamics in conversations
Experimental and longitudinal evidence ties trolling behavior to transient conditions rather than only to stable traits. In a two-by-two between-subjects experiment simulating an online news discussion, the proportion of user-written posts labeled as trolling was 35% in PosContext/PosMood, 49% in PosContext/NegMood, 47% in NegContext/PosMood, and 68% in NegContext/NegMood (Cheng et al., 2017). Negative mood increased odds of trolling by 89%, negative context increased odds by 68%, and the combined condition nearly doubled the baseline. Mixed-effects logistic regression found these fixed effects statistically significant.
The same paper extends the argument to field data from over 16 million posts from CNN.com news discussions over about nine months. Trolling peaked late at night and at the start of the week; if a user’s previous post in another discussion was flagged, the next post was twice as likely to be flagged, 4.6% versus 2.1%; if the first post in a discussion was flagged, subsequent posts by new users were almost twice as likely to be flagged, 3.1% versus 1.7%; and if all first four posts in a sub-discussion were flagged, the fifth post by a new user had nearly 50% odds of being flagged (Cheng et al., 2017). In the associated predictive model, context features were the strongest single block with AUC 0.74, while all features together reached AUC 0.78.
Conversation-level taxonomies describe what participants do once trolling is perceived. In Mojica’s framework, response strategies include Engage, Praise, Troll, Follow, Frustrate, Neutralize, and Normal (Mojica, 2016). This matters because the observable trajectory of a thread depends on both the suspected troll and the responders. A thread may escalate because users “take the bait,” or it may flatten if the dominant strategy is neutralization.
The situational account also constrains moderation theory. If negative context and exposure to prior trolling increase subsequent trolling, then hiding or downranking troll posts is not only a content-filtering decision but a possible intervention on contagion dynamics (Cheng et al., 2017). That implication is directly aligned with later work on counter-responses rather than pure deletion.
3. Datasets and annotation infrastructures
The empirical study of trolling has been shaped by a sequence of datasets that differ in unit of analysis, scope, and annotation target.
| Resource | Scope | Key structure |
|---|---|---|
| “Trawling for Trolling: A Dataset” (Hitkul et al., 2020) | 12,490 samples from Twitter, Reddit, and Wikipedia Talk Pages | 5 classes: Normal, Profanity, Trolling, Derogatory, Hate Speech |
| “Modeling Trolling in Social Media Conversations” (Mojica, 2016) | 1000 Reddit snippets; 6833 sentences; 88,047 tokens | 4 aspects: intention, disclosure, interpretation, response strategy |
| “ELF22” (Lee et al., 2022) | 5,535 labeled troll/response pairs plus 1,151 authored counter-responses | Overt/Covert troll type and 7 response strategies |
| “TrollsWithOpinion” (Suryawanshi et al., 2021) | 8,881 English image-with-text memes | 8 classes spanning troll/not-troll, opinion manipulation, and domain |
The five-class dataset of Kumar et al. is notable because it was, to the authors’ knowledge, the first dataset to explicitly distinguish trolling from hate speech and other types of offensive content. It was created by sourcing and reannotating samples from Davidson et al., Founta et al., Jigsaw, Why Swear, #MeToo, plus original Reddit RoastMe data; ambiguous, duplicate, and non-English samples were removed; and only samples with at least two annotators agreeing were kept (Hitkul et al., 2020). The final distribution was Normal 5,053, Trolling 4,537, Profanity 1,582, Derogatory 862, and Hate Speech 456, with Randolph kappa 0.65 and Fleiss’ kappa 0.59. The most common confusion was trolling versus derogatory, with 52% of derogatory samples having a dissenting annotator choosing trolling.
Mojica’s Reddit dataset was designed around suspected trolling attempts and direct responses rather than isolated posts. It contains 1000 annotated snippets, with Cohen’s kappa 0.788 for intention, 0.780 for intention disclosure, 0.797 for interpretation, and 0.776 for response strategy (Mojica, 2016). The dataset is therefore suited to pragmatic modeling and structured prediction rather than simple offensive-language classification.
ELF22 shifts the target from detection to intervention. Constructed from Reddit using root-level troll comments with scores between -2 and -15 and the highest-upvoted direct counter-response, it labels troll comments as overt or covert and counter-responses using seven strategies: Engage, Ignore, Expose, Challenge, Critique, Mock, and Reciprocate (Lee et al., 2022). Fleiss’ kappa was 0.539 for troll type and 0.465 for strategy. This is a moderation-oriented resource rather than a pure detection corpus.
The multimodal case remains difficult. TrollsWithOpinion enhances an existing meme dataset with hierarchical labels for troll or not-troll, presence or absence of opinion manipulation, and three domains—political, product, and other—yielding 8,881 English memes (Suryawanshi et al., 2021). Krippendorff’s alpha was 0.699 nominal and 0.775 interval, indicating substantial agreement, but baseline performance remained low.
4. Detection methodologies
Comment-level and tweet-level detection has progressed from feature engineering to contextual encoders, but the task remains definition-sensitive. On the five-class dataset of Kumar et al., Naive Bayes, Character-level CNN, N-gram CNN, and BERT-base-cased were evaluated, with BERT achieving Accuracy 75.3% and Weighted F1 0.73 under minimal hyperparameter tuning (Hitkul et al., 2020). Quality-control analyses are as important as the score itself: shuffling labels dropped F1 from 0.73 to 0.23, scrambling word order reduced it to 0.58, and removing random words reduced it to 0.56. The authors interpret this as evidence that the dataset is largely devoid of spurious statistical artefacts and that the models rely on sequence and context rather than trivial cues.
Contextual embeddings generally outperform static ones in balanced troll/non-troll tweet classification. In “A Context-Sensitive Word Embedding Approach for The Detection of Troll Tweets,” the dataset contained 18,514 tweets after duplicate and retweet filtering, split into 9,257 troll and 9,257 non-troll tweets (Yilmaz et al., 2022). Across nine embedding-encoder combinations, the best model was ELMo with a GRU classifier, reaching F1 0.859 and AUC 0.929, while BERT and ELMo consistently outperformed GloVe. A related linguistic study, “Understanding Troll Writing as a Linguistic Phenomenon,” reported 91% accuracy for a BLSTM with custom pretrained Word2Vec embeddings in classifying troll and genuine tweets, and then used regression and distributional analysis to argue that troll writing combines speaking with a purpose and trying to mask the purpose of speaking (Monakhov, 2019).
Other lines of work model trolling without relying on standard supervised text classification. “Trolls Identification within an Uncertain Framework” represents messages as basic belief assignments over categories such as off-topic and senseless, computes conflict between messages with an inclusion-based conflict measure and Jousselme distance, aggregates conflict at the user level, and then clusters users with k-means using (Dlala et al., 2015). In a 16-message simulated thread, the troll’s conflict score was 0.203 versus about 0.05–0.06 for regular users, and in a larger 8-user simulation both trolls formed the high-conflict cluster.
Older unsupervised neural approaches focused on user style and activity. “Application of neural networks to identify trolls in social networks” used Kohonen self-organized maps over features such as number of comments, average comment length, and frequencies of certain characters, exclamation marks, and question marks (Filimonov et al., 2015). In the reported experiment, both known trolls were grouped into one cluster, two additional users in that cluster were identified post hoc as trolls, and in a troll-free test set no false positive detections were observed. By contrast, multimodal troll-meme classification remains markedly harder: on TrollsWithOpinion, the best weighted-average F1-score among reported baselines was 0.37, achieved by Random Forest (Suryawanshi et al., 2021).
5. State-sponsored trolls and influence operations
A major branch of the literature studies trolls as instrumented accounts embedded in disinformation campaigns. “TexTrolls: Identifying Russian Trolls on Twitter from a Textual Perspective” analyzed 2,023 Russian troll accounts against 94,643 regular US accounts, totaling nearly 3.8 million tweets (Ghanem et al., 2019). Using thematic information from unsupervised LDA together with emotions, stance, bias cues, LIWC, morality, native language identification, and stylistic features, logistic regression achieved macro-F1 0.94 when all features were combined. The best single block was native language identification at macro-F1 0.91, and the study argues that thematic variation helps reveal “flip-flop” behavior across topics.
“Still out there: Modeling and Identifying Russian Troll Accounts on Twitter” took a broader account-classification view, using 2,286 English-language troll accounts and 171,291 US-based control accounts (Im et al., 2019). Logistic regression over profile, behavioral, stop-word, language-distribution, and bag-of-words features reached precision 78.5%, recall 0.835, F1 0.809, AUC 0.989, and accuracy 0.994 under cross-validation. Applied to 39,103 accounts mentioning 57 high-profile political journalists, the model flagged 1,466 accounts, and the authors estimated that up to 2.6% of top journalists’ mentions are occupied by Russian trolls. They also report that these accounts are not merely software-controlled bots.
Behavior-only approaches seek invariants that are harder to mimic than text. “Detecting Troll Behavior via Inverse Reinforcement Learning” models Twitter as a Markov decision process with states defined by feedback and actions defined by tweeting, retweeting, replying or mentioning, and doing nothing (Luceri et al., 2020). Maximum Entropy IRL plus AdaBoost achieved AUC 89.1% and was interpreted as showing that trolls are more content-oriented and less sensitive to social feedback. “Exposing Influence Campaigns in the Age of LLMs” replaces text altogether with sequences of state-action pairs and classifies trajectories using an LSTM. The account-level decision is summarized by the Troll Score,
and the reported results are AUC close to 99% for sequence classification and AUC of 91% for account discrimination (Ezzeddine et al., 2022).
Coordination-centered systems extend this logic to network structure. TROLLMAGNIFIER was trained on 335 known troll accounts on Reddit and identified 1,248 additional potential troll accounts from a pre-filtered set of 53,763 accounts (Saeed et al., 2021). Random Forest achieved 97.8% precision, 97.7% recall, 97.8% accuracy, and 97.8% F1-score on labeled data, and 66% of detected accounts showed at least one sign of being instrumented by malicious actors. At larger temporal scale, “Cross-National Information Attacks: A Two-Decade Analysis of Troll Behavior in Korea” applied a hierarchical explainable framework to 112M South Korean news comments by 4M users over nearly 20 years, identifying 23,998 accounts exhibiting behavior consistent with coordinated manipulation; the best user-level classifier reached F1 approximately 0.94 (Kim et al., 22 Jun 2026).
Detection, however, should not be conflated with dominant causal influence. In “On the Influence of Twitter Trolls during the 2016 US Presidential Election,” analysis of 152.5 million tweets from 9.9 million users, including 822 troll accounts present in the crawled dataset, found that authentic users were the source of the viral cascades (Salamanos et al., 2019). Ego-net users appeared before trolls in 85% of diffusion events, 54,111 cascades were initiated by ego-net users versus 647 by trolls, only 31 trolls had non-zero influence-degree, and only four troll accounts were truly influential. Removing trolls made negligible difference to the virality distribution. The result is not that troll participation was absent, but that direct leadership in viral spread was limited.
6. Countermeasures, adversarial adaptation, and the unrelated acronym “TROLL”
Moderation research increasingly studies how to respond rather than only how to detect. ELF22 formalizes seven counter-response strategies—Engage, Ignore, Expose, Challenge, Critique, Mock, and Reciprocate—and uses them for controlled generation (Lee et al., 2022). In the reported experiments, BERT and RoBERTa achieved weighted F1 around 0.63–0.64 for overt-versus-covert troll classification and up to 0.47 for seven-way response-strategy classification, while BART fine-tuned on ELF22 reached ROUGE-L 0.10, BLEU-1 0.15, METEOR 0.09, and BERTScore 0.40 in response generation. Human evaluation showed significantly improved strategy-controlled sentence generation, with BART-ELF22 outperforming GPT-2 in both relevance and compatibility.
Adversarial pressure is now a first-class concern. “TrollHunter” targeted trolling tweets during the COVID-19 pandemic using trolling hashtags, trolling tropes, sentiment via VADER, tweet type, and TF-IDF vectorization over text and hashtags, producing more than 43,000 features (Jachim et al., 2020). On 1.3 million tweets, with 25,000 hand-labeled examples and , a linear-kernel SVM with SMOTE achieved 98.5% accuracy, 75.4% precision, 69.8% recall, and 68.7% F1. The companion TrollHunter-Evader, using Test Time Evasion and a Markov chain-based rewriting mechanism, reduced accuracy to 96.9%, precision to 45.4%, recall to 41.9%, and F1 to 39.8%, which the paper summarizes as an approximately 40% decrease in the detector’s ability to correctly identify trolling tweets (Jachim et al., 2020).
In a different subfield, “TROLL” denotes a reinforcement-learning optimization method rather than an online actor. “TROLL: Trust Regions improve Reinforcement Learning for LLMs” replaces the PPO-like clip objective with a discrete differentiable trust region projection that enforces token-level KL constraints on a sparse subset of important logits (Becker et al., 4 Oct 2025). The core projection is
with a closed-form solution expressed as a geometric interpolation between old and new policies. The method is presented as a direct replacement for PPO-like clipping during training, does not alter inference behavior, and across datasets, model families, and advantage-estimation methods consistently outperforms PPO-like clipping in training speed, stability, and final success rates (Becker et al., 4 Oct 2025). The acronym is therefore homonymous with, but conceptually unrelated to, the study of online trolling.