CounterQuill: Investigating the Potential of Human-AI Collaboration in Online Counterspeech Writing

Abstract: Online hate speech has become increasingly prevalent on social media, causing harm to individuals and society. While automated content moderation has received considerable attention, user-driven counterspeech remains a less explored yet promising approach. However, many people face difficulties in crafting effective responses. We introduce CounterQuill, a human-AI collaborative system that helps everyday users with writing empathetic counterspeech, not by generating automatic replies, but by educating them through reflection and response. CounterQuill follows a three-stage workflow grounded in computational thinking: (1) a learning session to build understanding of hate speech and counterspeech, (2) a brainstorming session to identify harmful patterns and ideate counterspeech ideas, and (3) a co-writing session that helps users refine their counter responses while preserving personal voice. Through a user study \r{ho}(N=20), we found that CounterQuill helped participants develop the skills to brainstorm and draft counterspeech with increased confidence and control throughout the process. Our findings highlight how AI systems can scaffold complex communication tasks through structured, human-centered workflows that educate users on how to recognize, reflect on, and respond to online hate speech.

• The paper introduces CounterQuill, a human-AI collaborative system designed with learning, brainstorming, and co-writing sessions to help users create effective, empathetic counterspeech against online hate.
• A user study comparing CounterQuill to ChatGPT found the AI-mediated system resulted in significantly higher performance scores and lower user frustration during counterspeech drafting, despite requiring greater effort.
• Using CounterQuill enhanced participants' sense of ownership over their writing, improved their learning of hate speech dynamics, and increased their willingness to post personalized counterspeech online.

An Examination of Human-AI Collaboration in Counterspeech Writing: Introducing CounterQuill

The research paper titled "CounterQuill: Investigating the Potential of Human-AI Collaboration in Online Counterspeech Writing" focuses on addressing the challenge of online hate speech through an innovative system termed CounterQuill. As social media continues to be a breeding ground for hate speech, this study is positioned within a growing body of research exploring user-driven solutions that transcend traditional content moderation methods.

Problem Context and Proposed Solution

The study identifies existing barriers to effective user-driven counterspeech, such as apprehensions about retaliation and challenges related to writing skills. It highlights the inadequacies in existing AI-generated counterspeech, which often lacks personalization and emotional resonance. In this context, the researchers introduce CounterQuill, an AI-mediated system designed to support users in drafting nuanced, empathetic counterspeech. CounterQuill is structured into three distinct sessions—learning, brainstorming, and co-writing—to equip users with both the knowledge and the tools necessary for effective counterspeech creation.

1. Learning Session: This initial phase educates users about the nuances of hate speech and effective counterspeech strategies, particularly focusing on empathy-based approaches. This educational foundation aims to heighten awareness and understanding of both explicit and implicit hate speech dynamics.
2. Brainstorming Session: By guiding users through a structured reflection process, this session aids in identifying the critical components of hate speech, such as targeted identities and dehumanizing language. With AI-driven feedback, users are encouraged to consider potential empathetic counter-responses.
3. Co-Writing Session (CounterQuill): In this session, AI assistants provide real-time feedback and refinement suggestions, empowering users to tailor their counterspeech. Features such as selective rewriting and tone customization help align the output with personal user expression.

Methodology and Evaluation

The effectiveness of CounterQuill was empirically evaluated through a within-subjects user study involving 20 participants. The design compared users' experiences and outputs when utilizing CounterQuill versus ChatGPT, a baseline generative AI tool. A multifaceted approach was employed to assess participants' perceptions, including NASA-TLX workload assessments and custom measures addressing users' confidence, satisfaction, and willingness to post counterspeech online.

The statistical analysis of the NASA-TLX ratings revealed that CounterQuill was associated with significantly higher performance scores compared to ChatGPT, albeit requiring greater physical and temporal demand—indicative of the enhanced effort integral to the AI-human collaborative process. Notably, users reported lower frustration levels with CounterQuill, highlighting its supportive framework in mitigating the challenges of counterspeech writing.

Key Findings and Implications

Findings from the study suggest several dimensions in which CounterQuill enhances the counterspeech writing experience:

• Enhanced Ownership and Engagement: Participants expressed a stronger sense of ownership over the counterspeech co-authored with CounterQuill, correlating with their level of cognitive effort and customization during the writing process. This sense of investment influenced a greater willingness to share their responses publicly on social media.
• Adapted Learning and Empathy: Through guided learning and feedback mechanisms, participants demonstrated improved capabilities in identifying and articulating the impacts of hate speech. The integration of empathy-based approaches was effective in crafting responses that were not only assertive but also constructive.
• User-Centric Customization: CounterQuill’s capacity to assist users in selectively refining and personalizing their counterspeech was recognized as a significant improvement over conventional AI writing assistants, which often fail to accommodate individual user perspectives and emotional tones.

Conclusion and Future Directions

This research underscores the potential of human-AI collaboration in enhancing user agency and effectiveness in counterspeech creation. The study paves the way for further exploration into embedding such AI-mediated tools within social media platforms, allowing real-time assistance in the face of online hate speech. Future work may involve long-term field studies to validate CounterQuill's impact on user behavior and its role in fostering healthier online discourse.

CounterQuill exemplifies an incremental step towards equipping users with the necessary skills and confidence to engage constructively with online hate, setting a foundation for future developments in AI-mediated counterspeech strategies.