Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Writer-Defined AI Personas for On-Demand Feedback Generation (2309.10433v2)

Published 19 Sep 2023 in cs.HC and cs.CL

Abstract: Compelling writing is tailored to its audience. This is challenging, as writers may struggle to empathize with readers, get feedback in time, or gain access to the target group. We propose a concept that generates on-demand feedback, based on writer-defined AI personas of any target audience. We explore this concept with a prototype (using GPT-3.5) in two user studies (N=5 and N=11): Writers appreciated the concept and strategically used personas for getting different perspectives. The feedback was seen as helpful and inspired revisions of text and personas, although it was often verbose and unspecific. We discuss the impact of on-demand feedback, the limited representativity of contemporary AI systems, and further ideas for defining AI personas. This work contributes to the vision of supporting writers with AI by expanding the socio-technical perspective in AI tool design: To empower creators, we also need to keep in mind their relationship to an audience.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. 2020. How to Set Goals in the Grammarly Editor, and Why You Should | Grammarly Spotlight. https://www.grammarly.com/blog/set-goals-grammarly-editor/
  2. 2023. MeMo:KI - Opinion Monitor Artificial Intelligence. https://www.cais-research.de/en/research/memoki/
  3. The Team at AMI. 2020. Precursors to a Digital Muse. https://medium.com/artists-and-machine-intelligence/precursors-to-a-digital-muse-2653a3025700
  4. Predictive Text Encourages Predictable Writing. In Proceedings of the 25th International Conference on Intelligent User Interfaces (Cagliari, Italy) (IUI ’20). Association for Computing Machinery, New York, NY, USA, 128–138. https://doi.org/10.1145/3377325.3377523
  5. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (Virtual Event, Canada) (FAccT ’21). Association for Computing Machinery, New York, NY, USA, 610–623. https://doi.org/10.1145/3442188.3445922
  6. Soylent: A Word Processor with a Crowd Inside. In Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology (New York, New York, USA) (UIST ’10). Association for Computing Machinery, New York, NY, USA, 313–322. https://doi.org/10.1145/1866029.1866078
  7. Interacting with Next-Phrase Suggestions: How Suggestion Systems Aid and Influence the Cognitive Processes of Writing. In Proceedings of the 28th International Conference on Intelligent User Interfaces (Sydney, NSW, Australia) (IUI ’23). Association for Computing Machinery, New York, NY, USA, 436–452. https://doi.org/10.1145/3581641.3584060
  8. Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (jan 2006), 77–101. https://doi.org/10.1191/1478088706qp063oa
  9. Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health 11, 4 (2019), 589–597. https://doi.org/10.1080/2159676X.2019.1628806 arXiv:https://doi.org/10.1080/2159676X.2019.1628806
  10. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 1877–1901. https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
  11. How to Support Users in Understanding Intelligent Systems? An Analysis and Conceptual Framework of User Questions Considering User Mindsets, Involvement, and Knowledge Outcomes. ACM Trans. Interact. Intell. Syst. 12, 4, Article 29 (nov 2022), 27 pages. https://doi.org/10.1145/3519264
  12. The Second Workshop on Intelligent and Interactive Writing Assistants. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI EA ’23). Association for Computing Machinery, New York, NY, USA, Article 369, 5 pages. https://doi.org/10.1145/3544549.3573826
  13. About Face: The Essentials of Interaction Design (4 ed.). Wiley, Indianapolis, IN.
  14. Juliet Corbin and Anselm Strauss. 2014. Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage publications.
  15. Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (Bend, OR, USA) (UIST ’22). Association for Computing Machinery, New York, NY, USA, Article 98, 13 pages. https://doi.org/10.1145/3526113.3545672
  16. Linda Flower and John R Hayes. 1981. A cognitive process theory of writing. College composition and communication 32, 4 (1981), 365–387.
  17. A Design Space for Writing Support Tools Using a Cognitive Process Model of Writing. In Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2022). Association for Computational Linguistics, Dublin, Ireland, 11–24. https://doi.org/10.18653/v1/2022.in2writing-1.2
  18. Social Dynamics of AI Support in Creative Writing. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 245, 15 pages. https://doi.org/10.1145/3544548.3580782
  19. Susanne Göpferich. 2009. Comprehensibility assessment using the Karlsruhe comprehensibility concept. The Journal of Specialised Translation 11, 2009 (2009), 31–52.
  20. Emanuela Haller and Traian Rebedea. 2013. Designing a Chat-bot that Simulates an Historical Figure. In 2013 19th International Conference on Control Systems and Computer Science. 582–589. https://doi.org/10.1109/CSCS.2013.85
  21. Evaluating Large Language Models in Generating Synthetic HCI Research Data: A Case Study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 433, 19 pages. https://doi.org/10.1145/3544548.3580688
  22. AI-Mediated Communication: Definition, Research Agenda, and Ethical Considerations. Journal of Computer-Mediated Communication 25, 1 (01 2020), 89–100. https://doi.org/10.1093/jcmc/zmz022 arXiv:https://academic.oup.com/jcmc/article-pdf/25/1/89/32961176/zmz022.pdf
  23. John Hayes and Linda Flower. 1980. Identifying the organization of writing processes.
  24. John R. Hayes. 2012. Modeling and Remodeling Writing. Written Communication 29, 3 (July 2012), 369–388. https://doi.org/10.1177/0741088312451260 Publisher: SAGE Publications Inc.
  25. Hendrik Heuer and Elena Leah Glassman. 2023a. Accessible Text Tools for People with Cognitive Impairments and Non-Native Readers: Challenges and Opportunities. In Proceedings of Mensch Und Computer 2023 (Rapperswil, Switzerland) (MuC ’23). Association for Computing Machinery, New York, NY, USA, 250–266. https://doi.org/10.1145/3603555.3603569
  26. Hendrik Heuer and Elena Leah Glassman. 2023b. Accessible Text Tools: Where They Are Needed & What They Should Look Like. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI EA ’23). Association for Computing Machinery, New York, NY, USA, Article 20, 7 pages. https://doi.org/10.1145/3544549.3585749
  27. Jess Hohenstein and Malte Jung. 2020. AI as a moral crumple zone: The effects of AI-mediated communication on attribution and trust. Computers in Human Behavior 106 (2020), 106190. https://doi.org/10.1016/j.chb.2019.106190
  28. Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2022). Association for Computational Linguistics, Dublin, Ireland. https://aclanthology.org/2022.in2writing-1.0
  29. Co-Writing with Opinionated Language Models Affects Users’ Views. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 111, 15 pages. https://doi.org/10.1145/3544548.3581196
  30. AI-Mediated Communication: How the Perception That Profile Text Was Written by AI Affects Trustworthiness. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300469
  31. The Ghost in the Machine has an American accent: value conflict in GPT-3. arXiv preprint arXiv:2203.07785 (2022).
  32. Who Answers It Better? An In-Depth Analysis of ChatGPT and Stack Overflow Answers to Software Engineering Questions. arXiv preprint arXiv:2308.02312 (2023).
  33. Judy Kay and Bob Kummerfeld. 2013. Creating Personalized Systems That People Can Scrutinize and Control: Drivers, Principles and Experience. ACM Trans. Interact. Intell. Syst. 2, 4, Article 24 (jan 2013), 42 pages. https://doi.org/10.1145/2395123.2395129
  34. Cells, Generators, and Lenses: Design Framework for Object-Oriented Interaction with Large Language Models. In The 36th Annual ACM Symposium on User Interface Software and Technology (UIST ’23), October 29-November 1, 2023, San Francisco, CA, USA (San Francisco, CA, USA) (UIST ’23). Association for Computing Machinery, New York, NY, USA, 18 pages. https://doi.org/979-8-4007-0132-0/23/10
  35. LMCanvas: Object-Oriented Interaction to Personalize Large Language Model-Powered Writing Environments. http://arxiv.org/abs/2303.15125 arXiv:2303.15125 [cs].
  36. Matthew G. Kirschenbaum. 2016. Track Changes: A Literary History of Word Processing. Harvard University Press.
  37. Vivek Kulkarni and Vipul Raheja. 2023. Writing Assistants Should Model Social Factors of Language. https://doi.org/10.48550/arXiv.2303.16275 arXiv:2303.16275 [cs].
  38. Sich verständlich ausdrücken. Ernst Reinhardt Verlag.
  39. CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI’22). Association for Computing Machinery, New Orleans, LA, USA. https://doi.org/10.1145/3491102.3502030
  40. The writer’s workbench: Computer aids for text analysis. IEEE Transactions on Communications 30, 1 (1982), 105–110.
  41. Philipp Mayring. 2021. Qualitative content analysis: A step-by-step guide. Qualitative Content Analysis (2021), 1–100.
  42. AI-Mediated Communication: Language Use and Interpersonal Effects in a Referential Communication Task. Proc. ACM Hum.-Comput. Interact. 5, CSCW1, Article 17 (apr 2021), 14 pages. https://doi.org/10.1145/3449091
  43. Dead Man Tweeting. In Workshop on Collecting and Generating Resources for Chatbots and Conversational Agents-Development and Evaluation, Portorož (Slovenia), May 28, 2016.
  44. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442 (2023).
  45. Social Simulacra: Creating Populated Prototypes for Social Computing Systems. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (Bend, OR, USA) (UIST ’22). Association for Computing Machinery, New York, NY, USA, Article 74, 18 pages. https://doi.org/10.1145/3526113.3545616
  46. Peter Rechenberg. 2006. Technisches Schreiben. In Technisches Schreiben. Carl Hanser Verlag GmbH & Co. KG, 1–10. https://doi.org/10.3139/9783446409576.fm
  47. Kevin Roose. 2022. The brilliance and weirdness of ChatGPT. https://www.nytimes.com/2022/12/05/technology/chatgpt-ai-twitter.html. The New York Times (2022).
  48. Oliver Schmitt and Daniel Buschek. 2021. CharacterChat: Supporting the Creation of Fictional Characters through Conversation and Progressive Manifestation with a Chatbot. In Creativity and Cognition (Virtual Event, Italy) (C&C ’21). Association for Computing Machinery, New York, NY, USA, Article 10, 10 pages. https://doi.org/10.1145/3450741.3465253
  49. BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data. https://doi.org/10.48550/arXiv.2106.06169 arXiv:2106.06169 [cs].
  50. Exploiting Persona Information for Diverse Generation of Conversational Responses. https://doi.org/10.48550/arXiv.1905.12188 arXiv:1905.12188 [cs].
  51. Jared Spataro. 2023. Introducing Microsoft 365 Copilot – your copilot for work. https://blogs.microsoft.com/blog/2023/03/16/introducing-microsoft-365-copilot-your-copilot-for-work/
  52. Nothing About Us Without Us: Investigating the Role of Critical Disability Studies in HCI. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3375150
  53. Digital support for academic writing: A review of technologies and pedagogies. Computers & Education 131 (2019), 33–48. https://doi.org/10.1016/j.compedu.2018.12.005
  54. Schreibkompetenz, Schreibmotivation, Schreibförderung: Grundlagen und Modelle zum Schreiben als soziale Praxis. Klett Kallmeyer.
  55. Inspire creativity with ORIBA: Transform Artists’ Original Characters into Chatbots through Large Language Model. arXiv:2306.09776 [cs.MM]
  56. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824–24837.
  57. Sketching NLP: A Case Study of Exploring the Right Things To Design with Language Intelligence. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300415
  58. Re-Examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376301
  59. Wordcraft: Story Writing With Large Language Models. In 27th International Conference on Intelligent User Interfaces (IUI ’22). Association for Computing Machinery, New York, NY, USA, 841–852. https://doi.org/10.1145/3490099.3511105
  60. Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 437, 21 pages. https://doi.org/10.1145/3544548.3581388
  61. Personalizing dialogue agents: I have a dog, do you have pets too? arXiv preprint arXiv:1801.07243 (2018).
  62. A Pre-Training Based Personalized Dialogue Generation Model with Persona-Sparse Data. Proceedings of the AAAI Conference on Artificial Intelligence 34, 05 (April 2020), 9693–9700. https://doi.org/10.1609/aaai.v34i05.6518 Number: 05.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Karim Benharrak (6 papers)
  2. Tim Zindulka (4 papers)
  3. Florian Lehmann (14 papers)
  4. Hendrik Heuer (21 papers)
  5. Daniel Buschek (41 papers)
Citations (10)
View on Semantic Scholar