Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Hate Cannot Drive out Hate: Forecasting Conversation Incivility following Replies to Hate Speech (2312.04804v1)

Published 8 Dec 2023 in cs.CY and cs.CL

Abstract: User-generated replies to hate speech are promising means to combat hatred, but questions about whether they can stop incivility in follow-up conversations linger. We argue that effective replies stop incivility from emerging in follow-up conversations - replies that elicit more incivility are counterproductive. This study introduces the task of predicting the incivility of conversations following replies to hate speech. We first propose a metric to measure conversation incivility based on the number of civil and uncivil comments as well as the unique authors involved in the discourse. Our metric approximates human judgments more accurately than previous metrics. We then use the metric to evaluate the outcomes of replies to hate speech. A linguistic analysis uncovers the differences in the language of replies that elicit follow-up conversations with high and low incivility. Experimental results show that forecasting incivility is challenging. We close with a qualitative analysis shedding light into the most common errors made by the best model.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. Not All Counterhate Tweets Elicit the Same Replies: A Fine-Grained Analysis. In Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023), 71–88. Toronto, Canada: Association for Computational Linguistics.
  2. Civility vs. incivility in online social interactions: An evolutionary approach. PloS one, 11(11): e0164286.
  3. Inter-Coder Agreement for Computational Linguistics. Comput. Linguist., 34(4): 555–596.
  4. Predicting Responses to Microblog Posts. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 602–606. Montréal, Canada: Association for Computational Linguistics.
  5. Characterizing and Curating Conversation Threads: Expansion, Focus, Volume, Re-Entry. In Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, WSDM ’13, 13–22. New York, NY, USA: Association for Computing Machinery. ISBN 9781450318693.
  6. Conversations Gone Alright: Quantifying and Predicting Prosocial Outcomes in Online Conversations. In Proceedings of the Web Conference 2021, WWW ’21, 1134–1145. New York, NY, USA: Association for Computing Machinery. ISBN 9781450383127.
  7. Artificial intelligence against hate: Intervention reducing verbal aggression in the social network environment. Aggressive behavior, 47(3): 260–266.
  8. AMPERSAND: Argument Mining for PERSuAsive oNline Discussions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2933–2943. Hong Kong, China: Association for Computational Linguistics.
  9. Don’t Let Me Be Misunderstood:Comparing Intentions and Perceptions in Online Discussions. In Proceedings of The Web Conference 2020, WWW ’20, 2066–2077. New York, NY, USA: Association for Computing Machinery. ISBN 9781450370233.
  10. Trouble on the Horizon: Forecasting the Derailment of Online Conversations as they Develop. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 4743–4754. Hong Kong, China: Association for Computational Linguistics.
  11. Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW ’17, 1217–1230. New York, NY, USA: Association for Computing Machinery. ISBN 9781450343350.
  12. CONAN - COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2819–2829. Florence, Italy: Association for Computational Linguistics.
  13. Sentiment Analysis and Social Cognition Engine (SEANCE): An automatic tool for sentiment, social cognition, and social-order analysis. Behavior research methods, 49(3): 803–821.
  14. Would Your Tweet Invoke Hate on the Fly? Forecasting Hate Intensity of Reply Threads on Twitter. In Zhu, F.; Ooi, B. C.; and Miao, C., eds., KDD ’21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14-18, 2021, 2732–2742. ACM.
  15. Developing a New Classifier for Automated Identification of Incivility in Social Media. In Proceedings of the Fourth Workshop on Online Abuse and Harms, 95–101. Online: Association for Computational Linguistics.
  16. Automated Hate Speech Detection and the Problem of Offensive Language. In Proceedings of the Eleventh International Conference on Web and Social Media, ICWSM 2017, Montréal, Québec, Canada, May 15-18, 2017, 512–515. AAAI Press.
  17. Latent Hatred: A Benchmark for Understanding Implicit Hate Speech. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 345–363. Online and Punta Cana, Dominican Republic: Association for Computational Linguistics.
  18. Neural Networks For Negation Scope Detection. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 495–504. Berlin, Germany: Association for Computational Linguistics.
  19. Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speech. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 3226–3240. Online: Association for Computational Linguistics.
  20. A Survey on Automatic Detection of Hate Speech in Text. ACM Comput. Surv., 51(4).
  21. Impact and dynamics of hate and counter speech online. EPJ Data Science, 11(1): 3.
  22. A Report on the 2020 Sarcasm Detection Shared Task. In Proceedings of the Second Workshop on Figurative Language Processing, 1–11. Online: Association for Computational Linguistics.
  23. An Expert Annotated Dataset for the Detection of Online Misogyny. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 1336–1350. Online: Association for Computational Linguistics.
  24. Empathy-based counterspeech can reduce racist hate speech in a social media field experiment. Proceedings of the National Academy of Sciences, 118(50): e2116310118.
  25. Detecting Attackable Sentences in Arguments. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1–23. Online: Association for Computational Linguistics.
  26. Conversational Resilience: Quantifying and Predicting Conversational Outcomes Following Adverse Events. In Proceedings of the International AAAI Conference on Web and Social Media, volume 16, 548–559.
  27. Understanding conflicts in online conversations. In Proceedings of the ACM Web Conference 2022, 2592–2602.
  28. Forecasting the Presence and Intensity of Hostility on Instagram Using Linguistic and Social Features. In Proceedings of the Twelfth International Conference on Web and Social Media, ICWSM 2018, Stanford, California, USA, June 25-28, 2018, 181–190. AAAI Press.
  29. RoBERTa: A Robustly Optimized BERT Pretraining Approach. CoRR, abs/1907.11692.
  30. Thou Shalt Not Hate: Countering Online Hate Speech. In Pfeffer, J.; Budak, C.; Lin, Y.; and Morstatter, F., eds., Proceedings of the Thirteenth International Conference on Web and Social Media, ICWSM 2019, Munich, Germany, June 11-14, 2019, 369–380. AAAI Press.
  31. McNemar, Q. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2): 153–157.
  32. Perverse Downstream Consequences of Debunking: Being Corrected by Another User for Posting False Political News Increases Subsequent Sharing of Low Quality, Partisan, and Toxic Content in a Twitter Field Experiment. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI ’21. New York, NY, USA: Association for Computing Machinery. ISBN 9781450380966.
  33. Munger, K. 2017. Tweetment effects on the tweeted: Experimentally reducing racist harassment. Political Behavior, 39(3): 629–649.
  34. Linguistic Harbingers of Betrayal: A Case Study on an Online Strategy Game. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 1650–1659. Beijing, China: Association for Computational Linguistics.
  35. Abusive Language Detection in Online User Content. In Bourdeau, J.; Hendler, J.; Nkambou, R.; Horrocks, I.; and Zhao, B. Y., eds., Proceedings of the 25th International Conference on World Wide Web, WWW 2016, Montreal, Canada, April 11 - 15, 2016, 145–153. ACM.
  36. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32, 8024–8035. Curran Associates, Inc.
  37. Towards Debate Automation: a Recurrent Model for Predicting Debate Winners. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2465–2475. Copenhagen, Denmark: Association for Computational Linguistics.
  38. DEBAGREEMENT: A comment-reply dataset for (dis)agreement detection in online debates. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
  39. A Study of Cyber Hate on Twitter with Implications for Social Media Governance Strategies. In Liakata, M.; and Vlachos, A., eds., Proceedings of the 2019 Truth and Trust Online Conference (TTO 2019), London, UK, October 4-5, 2019.
  40. jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 109–117. Online: Association for Computational Linguistics.
  41. A Benchmark Dataset for Learning to Intervene in Online Hate Speech. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 4755–4764. Hong Kong, China: Association for Computational Linguistics.
  42. Counterspeech 2000: A new look at the old remedy for bad speech. BYU L. Rev., 553.
  43. SemEval-2017 Task 4: Sentiment Analysis in Twitter. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), 502–518. Vancouver, Canada: Association for Computational Linguistics.
  44. Incivility Detection in Online Comments. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), 283–291. Minneapolis, Minnesota: Association for Computational Linguistics.
  45. Governing hate speech by means of counterspeech on Facebook. In 66th ica annual conference, at fukuoka, japan, 1–23.
  46. A Survey on Hate Speech Detection using Natural Language Processing. In Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media, 1–10. Valencia, Spain: Association for Computational Linguistics.
  47. Will it Blend? Blending Weak and Strong Labeled Data in a Neural Network for Argumentation Mining. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 599–605. Melbourne, Australia: Association for Computational Linguistics.
  48. Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-Faith Online Discussions. In Proceedings of the 25th International Conference on World Wide Web, WWW ’16, 613–624. Republic and Canton of Geneva, CHE: International World Wide Web Conferences Steering Committee. ISBN 9781450341431.
  49. Generating Counter Narratives against Online Hate Speech: Data and Strategies. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 1177–1190. Online: Association for Computational Linguistics.
  50. Predicting the volume of comments on online news stories. In Proceedings of the 18th ACM conference on Information and knowledge management, 1765–1768.
  51. Challenges and frontiers in abusive content detection. In Proceedings of the Third Workshop on Abusive Language Online, 80–93. Florence, Italy: Association for Computational Linguistics.
  52. Introducing CAD: the Contextual Abuse Dataset. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2289–2303. Online: Association for Computational Linguistics.
  53. Effects of the Prevention Program “HateLess. Together against Hatred” on Adolescents’ Empathy, Self-efficacy, and Countering Hate Speech. Journal of youth and adolescence, 52(6): 1115–1128.
  54. Making Online Communities ’Better’: A Taxonomy of Community Values on Reddit.
  55. What Makes Online Communities ‘Better’? Measuring Values, Consensus, and Conflict across Thousands of Subreddits. Proceedings of the International AAAI Conference on Web and Social Media, 16(1): 1121–1132.
  56. Detection of Abusive Language: the Problem of Biased Datasets. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 602–608. Minneapolis, Minnesota: Association for Computational Linguistics.
  57. Transformers: State-of-the-Art Natural Language Processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 38–45. Online: Association for Computational Linguistics.
  58. What’s Worthy of Comment? Content and Comment Volume in Political Blogs. Proceedings of the International AAAI Conference on Web and Social Media, 4(1): 359–362.
  59. Hate Speech and Counter Speech Detection: Conversational Context Does Matter. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 5918–5930. Seattle, United States: Association for Computational Linguistics.
  60. Conversation Modeling to Predict Derailment. Proceedings of the International AAAI Conference on Web and Social Media, 17(1): 926–935.
  61. Conversations Gone Awry: Detecting Early Signs of Conversational Failure. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1350–1361. Melbourne, Australia: Association for Computational Linguistics.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now