Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
135 tokens/sec
GPT-4o
10 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
3 tokens/sec
DeepSeek R1 via Azure Pro
55 tokens/sec
2000 character limit reached

A Piece of Theatre: Investigating How Teachers Design LLM Chatbots to Assist Adolescent Cyberbullying Education (2402.17456v1)

Published 27 Feb 2024 in cs.HC, cs.AI, and cs.CL

Abstract: Cyberbullying harms teenagers' mental health, and teaching them upstanding intervention is crucial. Wizard-of-Oz studies show chatbots can scale up personalized and interactive cyberbullying education, but implementing such chatbots is a challenging and delicate task. We created a no-code chatbot design tool for K-12 teachers. Using LLMs and prompt chaining, our tool allows teachers to prototype bespoke dialogue flows and chatbot utterances. In offering this tool, we explore teachers' distinctive needs when designing chatbots to assist their teaching, and how chatbot design tools might better support them. Our findings reveal that teachers welcome the tool enthusiastically. Moreover, they see themselves as playwrights guiding both the students' and the chatbot's behaviors, while allowing for some improvisation. Their goal is to enable students to rehearse both desirable and undesirable reactions to cyberbullying in a safe environment. We discuss the design opportunities LLM-Chains offer for empowering teachers and the research opportunities this work opens up.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (78)
  1. Understanding the Bystander Effect on Toxic Twitter Conversations. https://doi.org/10.48550/ARXIV.2211.10764
  2. Kimberley R Allison and Kay Bussey. 2016. Cyber-bystanding in context: A review of the literature on witnesses’ responses to cyberbullying. Children and Youth Services Review 65 (2016), 183–194.
  3. PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts. https://doi.org/10.48550/ARXIV.2202.01279
  4. Cyberbullying among adolescent bystanders: Role of the communication medium, form of violence, and empathy. Journal of Community & Applied Social Psychology 23, 1 (2013), 37–51.
  5. ‘Can I afford to help?’How affordances of communication modalities guide bystanders’ helping intentions towards harassment on social network sites. Behaviour & Information Technology 34, 4 (2015), 425–435.
  6. Menucha Birenbaum. 2023. The Chatbots’ Challenge to Education: Disruption or Destruction? Education Sciences 13, 7 (2023), 711.
  7. How HCI Interprets the Probes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’07). Association for Computing Machinery, New York, NY, USA, 1077–1086. https://doi.org/10.1145/1240624.1240789
  8. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
  9. Angela Busacca and Melchiorre Alberto Monaca. 2023. Deepfake: Creation, Purpose, Risks. In Innovations and Economic and Social Changes due to Artificial Intelligence: The State of the Art. Springer, 55–68.
  10. The situational-cognitive model of adolescent bystander behavior: Modeling bystander decision-making in the context of bullying and teen dating violence. Psychology of violence 7, 1 (2017), 33.
  11. An education-based approach to aid in the prevention of cyberbullying. Acm Sigcas Computers and Society 47, 4 (2018), 17–28.
  12. We’re just playing: The influence of a modified tabletop role-playing game on ELA students’ in-class reading. Simulation & Gaming 48, 2 (2017), 199–218.
  13. John M Darley and Bibb Latané. 1968. Bystander intervention in emergencies: diffusion of responsibility. Journal of personality and social psychology 8, 4p1 (1968), 377.
  14. Thomas S Dee and Dan Goldhaber. 2017. Understanding and addressing teacher shortages in the United States. The Hamilton Project 5 (2017), 1–28.
  15. Mobilizing bystanders of cyberbullying: an exploratory study into behavioural determinants of defending the victim. Annual review of cybertherapy and telemedicine 10 (2012), 58–63.
  16. Determinants of self-reported bystander behavior in cyberbullying incidents amongst adolescents. Cyberpsychology, Behavior, and Social Networking 17, 4 (2014), 207–215.
  17. Upstanding by design: Bystander intervention in cyberbullying. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1–12.
  18. A systematic literature review of factors that moderate bystanders’ actions in cyberbullying. Cyberpsychology: Journal of Psychosocial Research on Cyberspace 12, 4 (2018).
  19. Eva Durall and Evangelos Kapros. 2020. Co-design for a competency self-assessment chatbot and survey in science education. In Learning and Collaboration Technologies. Human and Technology Ecosystems: 7th International Conference. Springer, 13–24.
  20. Ownership of learning in monitoring technology: Design case of self-monitoring tech in independent study. Interaction Des. Architecture (s) J 45 (2020), 133–154.
  21. Towards Measuring the Representation of Subjective Global Opinions in Language Models. CoRR abs/2306.16388 (2023). https://doi.org/10.48550/arXiv.2306.16388 arXiv:2306.16388
  22. Laura Faulkner. 2003. Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, & Computers 35 (2003), 379–383.
  23. WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023. Association for Computational Linguistics, 9126–9140. https://doi.org/10.18653/v1/2023.acl-long.507
  24. A chatbot-based coaching intervention for adolescents to promote life skills: pilot study. JMIR Human Factors 7, 1 (2020), e16762.
  25. Exploring the impact of AI on teacher leadership: regressing or expanding? Education and Information Technologies (2023), 1–19.
  26. Evaluating Large Language Models in Generating Synthetic HCI Research Data: A Case Study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 433, 19 pages. https://doi.org/10.1145/3544548.3580688
  27. Gunnar Harboe and Elaine M. Huang. 2015. Real-World Affinity Diagramming Practices: Bridging the Paper-Digital Gap. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 95–104. https://doi.org/10.1145/2702123.2702561
  28. Report on Indicators of School Crime and Safety: 2020. (2021). https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2021092
  29. Chasing New Worlds: Stories of Roleplaying in Classroom Spaces. Journal of language and literacy education 17, 1 (2021), n1.
  30. CTRL: A Conditional Transformer Language Model for Controllable Generation. CoRR abs/1909.05858 (2019). arXiv:1909.05858 http://arxiv.org/abs/1909.05858
  31. DV Kiriukhina. 2019. Cyberbullying among young users of social networks. Journal of Modern Foreign Psychology 8, 3 (2019), 53–59.
  32. Vasiliy Kolchenko. 2018. Can modern AI replace teachers? Not so fast! Artificial intelligence and adaptive learning: Personalized education in the AI age. HAPS educator 22, 3 (2018), 249–252.
  33. Robin M Kowalski and Cristin Fedina. 2011. Cyber bullying in ADHD and Asperger Syndrome populations. Research in Autism Spectrum Disorders 5, 3 (2011), 1201–1208.
  34. Joel Kupperstein. 2023. AI Can’t Replace High-quality Teaching: Using the Technology as a Tool. (2023).
  35. Bibb Latané and John M Darley. 1970. The unresponsive bystander: Why doesn’t he help? Prentice Hall.
  36. The changing face of bullying: An empirical comparison between traditional and internet bullying and victimization. Computers in Human Behavior 28, 1 (2012), 226–232. https://doi.org/10.1016/j.chb.2011.09.004
  37. Qing Li. 2007. Bullying in the new playground: Research into cyberbullying and cyber victimisation. Australasian Journal of Educational Technology 23, 4 (2007).
  38. Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. arXiv:2107.13586 [cs.CL]
  39. Chung Kwan Lo. 2023. What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences 13, 4 (2023), 410.
  40. Andrés Lucero. 2015. Using affinity diagrams to evaluate interactive prototypes. In Human-Computer Interaction–INTERACT 2015: 15th IFIP TC 13 International Conference, Bamberg, Germany, September 14-18, 2015, Proceedings, Part II 15. Springer, 231–248.
  41. Peer victimisation and depressive symptoms: Can specific coping strategies buffer the negative impact of cybervictimisation? Emotional and Behavioural Difficulties 17, 3-4 (2012), 403–420.
  42. Rethinking bullying interventions for high school students: A qualitative study. Journal of Child and Adolescent Counseling 4, 2 (2018), 146–163.
  43. Effectiveness of Artificial Intelligence–Based Cyberbullying Interventions From Youth Perspective. Social Media+ Society 9, 1 (2023), 20563051221147325.
  44. More human than human: Measuring ChatGPT political bias. Public Choice (2023), 1–21.
  45. Bystanders’ behavior in cyberbullying episodes: Active and passive patterns in the context of personal–socio-emotional factors. Journal of interpersonal violence 32, 1 (2017), 23–48.
  46. Julia Othlinghaus-Wulhorst and H Ulrich Hoppe. 2020. A technical and conceptual framework for serious role-playing games in the area of social skill training. Frontiers in Computer Science 2 (2020), 28.
  47. Generative Agents: Interactive Simulacra of Human Behavior. CoRR abs/2304.03442 (2023). https://doi.org/10.48550/arXiv.2304.03442 arXiv:2304.03442
  48. Social Simulacra: Creating Populated Prototypes for Social Computing Systems. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (Bend, OR, USA) (UIST ’22). Association for Computing Machinery, New York, NY, USA, Article 74, 18 pages. https://doi.org/10.1145/3526113.3545616
  49. Justin W Patchin and Sameer Hinduja. 2012. Cyberbullying prevention and response: Expert perspectives. Routledge.
  50. Chatbots to support children in coping with online threats: Socio-technical requirements. In Designing Interactive Systems Conference 2021. 1504–1517.
  51. Megan Price and John Dalgleish. 2010. Cyberbullying: Experiences, impacts and coping strategies as described by Australian young people. Youth studies australia 29, 2 (2010), 51–59.
  52. Rhiarne E Pronk and Melanie J Zimmer-Gembeck. 2010. It’s “mean,” but what does it mean to adolescents? Relational aggression described by victims, aggressors, and their peers. Journal of Adolescent research 25, 2 (2010), 175–204.
  53. A Recipe for Arbitrary Text Style Transfer with Large Language Models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, 837–848. https://doi.org/10.18653/v1/2022.acl-short.94
  54. Sara E Rimm-Kaufman. 2020. SEL from the Start: Building Skills in K-5 (Social and Emotional Learning Solutions). WW Norton & Company.
  55. Bullying as a group process: Participant roles and their relations to social status within the group. Aggressive Behavior: Official Journal of the International Society for Research on Aggression 22, 1 (1996), 1–15.
  56. Multitask Prompted Training Enables Zero-Shot Task Generalization. https://doi.org/10.48550/ARXIV.2110.08207
  57. Jeff Sauro and James R Lewis. 2016. Quantifying the user experience: Practical statistics for user research. Morgan Kaufmann.
  58. Cyberbullying, school bullying, and psychological distress: A regional census of high school students. American journal of public health 102, 1 (2012), 171–177.
  59. Jessica Shieh. 2023. Best practices for prompt engineering with openai API. https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api
  60. RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting. CoRR abs/2305.15685 (2023). https://doi.org/10.48550/arXiv.2305.15685 arXiv:2305.15685
  61. Creating a WhatsApp Dataset to Study Pre-teen Cyberbullying. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2). Association for Computational Linguistics, Brussels, Belgium, 51–59. https://doi.org/10.18653/v1/W18-5107
  62. Controlled Language Generation for Language Learning Items. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: EMNLP 2022 - Industry Track, Abu Dhabi, UAE, December 7 - 11, 2022. Association for Computational Linguistics, 294–305. https://doi.org/10.18653/v1/2022.emnlp-industry.30
  63. Seda Gökçe Turan. 2021. Deepfake and digital citizenship: A long-term protection method for children and youth. In Deep fakes, fake news, and misinformation in online teaching and learning technologies. IGI Global, 124–142.
  64. Cyberbullying Mitigation by a Proxy Persuasion of a Chat Member Hijacked by a Chatbot. In Proceedings of the 9th International Conference on Human-Agent Interaction. 202–208.
  65. U.S. Department of Education. 2023. Teacher Shortage Areas. https://tsa.ed.gov
  66. “Thinking before posting?” Reducing cyber harassment on social networking sites through a reflective message. Computers in human behavior 66 (2017), 345–352.
  67. Nationality Bias in Text Generation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2023, Dubrovnik, Croatia, May 2-6, 2023. Association for Computational Linguistics, 116–122. https://aclanthology.org/2023.eacl-main.9
  68. Sofia Villatoro Moral and Barbara de Benito. 2021. An Approach to Co-Design and Self-Regulated Learning in Technological Environments. Systematic Review. Journal of New Approaches in Educational Research 10, 2 (2021), 234–250.
  69. Emily Vogels. 2022. Teens and Cyberbullying 2022. Pew Research Center (2022).
  70. Taxonomy of Risks Posed by Language Models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea) (FAccT ’22). Association for Computing Machinery, New York, NY, USA, 214–229. https://doi.org/10.1145/3531146.3533088
  71. Alone: A dataset for toxic behavior among adolescents on twitter. In International Conference on Social Informatics. Springer, 427–439.
  72. PromptChainer: Chaining Large Language Model Prompts through Visual Programming. https://doi.org/10.48550/ARXIV.2203.06566
  73. AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts. In Proceedings of the 2022 CHI conference on human factors in computing systems.
  74. Sensemaking, Support, Safety, Retribution, Transformation: A Restorative Justice Approach to Understanding Adolescents’ Needs for Addressing Online Harm. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 146, 15 pages. https://doi.org/10.1145/3491102.3517614
  75. Sketching nlp: A case study of exploring the right things to design with language intelligence. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.
  76. Herding AI Cats: Lessons from Designing a Chatbot by Prompting GPT-3. In Proceedings of the 2023 ACM Designing Interactive Systems Conference (Pittsburgh, PA, USA) (DIS ’23). Association for Computing Machinery, New York, NY, USA, 2206–2220. https://doi.org/10.1145/3563657.3596138
  77. Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21.
  78. Are children involved in cyberbullying low on empathy? A systematic review and meta-analysis of research on empathy versus different cyberbullying roles. Aggression and violent behavior 45 (2019), 83–97.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets