Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

The Value, Benefits, and Concerns of Generative AI-Powered Assistance in Writing (2403.12004v1)

Published 18 Mar 2024 in cs.HC

Abstract: Recent advances in generative AI technologies like LLMs raise both excitement and concerns about the future of human-AI co-creation in writing. To unpack people's attitude towards and experience with generative AI-powered writing assistants, in this paper, we conduct an experiment to understand whether and how much value people attach to AI assistance, and how the incorporation of AI assistance in writing workflows changes people's writing perceptions and performance. Our results suggest that people are willing to forgo financial payments to receive writing assistance from AI, especially if AI can provide direct content generation assistance and the writing task is highly creative. Generative AI-powered assistance is found to offer benefits in increasing people's productivity and confidence in writing. However, direct content generation assistance offered by AI also comes with risks, including decreasing people's sense of accountability and diversity in writing. We conclude by discussing the implications of our findings.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. Daron Acemoglu and Pascual Restrepo. 2018. The race between man and machine: Implications of technology for growth, factor shares, and employment. American economic review 108, 6 (2018), 1488–1542.
  2. On suggesting phrases vs. predicting words for mobile text composition. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. 603–608.
  3. The limits of expert text entry speed on mobile keyboards with autocorrect. In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services. 1–12.
  4. Both complete and correct? multi-objective optimization of touchscreen keyboard. In Proceedings of the SIGCHI conference on human factors in computing systems. 2297–2306.
  5. From tool to companion: Storywriters want AI writers to respect their personal values and writing strategies. In Designing Interactive Systems Conference. 1209–1227.
  6. Latent dirichlet allocation. Journal of machine Learning research 3, Jan (2003), 993–1022.
  7. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
  8. ResearchIME: A mobile keyboard application for studying free typing behaviour in the wild. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–14.
  9. The impact of multiple parallel phrase suggestions on email input and composition behaviour of native and non-native english writers. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13.
  10. Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writing. arXiv preprint arXiv:2210.13669 (2022).
  11. Gmail smart compose: Real-time assisted writing. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2287–2295.
  12. TaleBrush: Sketching stories with generative pretrained language models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–19.
  13. Creative writing with a machine in the loop: Case studies on slogans and stories. In 23rd International Conference on Intelligent User Interfaces. 329–340.
  14. Does prediction really help in Marathi text input? Empirical analysis of a longitudinal study. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services. 35–46.
  15. Choice over control: How users write with large language models using diegetic and non-diegetic prompting. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17.
  16. Zijian Ding and Joel Chan. 2023. Mapping the Design Space of Interactions in Human-AI Text Co-creation Tasks. arXiv e-prints (2023), arXiv–2303.
  17. Mark Dunlop and John Levine. 2012. Multidimensional pareto optimization of touchscreen keyboards for speed, familiarity and improved spell checking. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2669–2678.
  18. Effects of language modeling and its personalization on touchscreen typing performance. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. 649–658.
  19. Comparing Sentence-Level Suggestions to Message-Level Suggestions in AI-Mediated Communication. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13.
  20. The economic and cognitive costs of annoying display advertisements. Journal of Marketing Research 51, 6 (2014), 742–752.
  21. Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183.
  22. Morten Størling Hedegaard and Jean-Robert Tyran. 2018. The price of prejudice. American Economic Journal: Applied Economics 10, 1 (2018), 40–63.
  23. Jess Hohenstein and Malte Jung. 2020. AI as a moral crumple zone: The effects of AI-mediated communication on attribution and trust. Computers in Human Behavior 106 (2020), 106190.
  24. How much do platform workers value reviews? An experimental method. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–11.
  25. https://huggingface.co/sentence-transformers/paraphrase MiniLM-L6-v2. 2023. Sentence transformers: paraphrase-MiniLM-L6-v2.
  26. Co-writing with opinionated language models affects users’ views. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–15.
  27. AI-mediated communication: How the perception that profile text was written by AI affects trustworthiness. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–13.
  28. Smart reply: Automated response suggestion for email. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 955–964.
  29. Coauthor: Designing a human-ai collaborative writing dataset for exploring language model capabilities. In Proceedings of the 2022 CHI conference on human factors in computing systems. 1–19.
  30. The hidden costs and benefits of monitoring in the gig economy. Information Systems Research 34, 1 (2023), 297–318.
  31. Prompt Middleware: Mapping Prompts for Large Language Models to UI Affordances. arXiv preprint arXiv:2307.01142 (2023).
  32. Alexandre Mas and Amanda Pallais. 2017. Valuing alternative work arrangements. American Economic Review 107, 12 (2017), 3722–3759.
  33. Co-Writing Screenplays and Theatre Scripts with Language Models: Evaluation by Industry Professionals. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–34.
  34. Shakked Noy and Whitney Zhang. 2023. Experimental evidence on the productivity effects of generative artificial intelligence. Science 381, 6654 (2023), 187–192. https://doi.org/10.1126/science.adh2586 arXiv:https://www.science.org/doi/pdf/10.1126/science.adh2586
  35. OpenAI. 2023. GPT-4 Technical Report. (2023).
  36. How do people type on mobile devices? Observations from a study with 37,000 volunteers. In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services. 1–12.
  37. Philip Quinn and Shumin Zhai. 2016. A cost-benefit study of text entry suggestion interaction. In Proceedings of the 2016 CHI conference on human factors in computing systems. 83–88.
  38. Directed diversity: Leveraging language embedding distances for collective creativity in crowd ideation. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–35.
  39. Exploring the space of topic coherence measures. In Proceedings of the eighth ACM international conference on Web search and data mining. 399–408.
  40. Sherwin Rosen. 1986. The theory of equalizing differences. Handbook of labor economics 1 (1986), 641–692.
  41. Where to hide a stolen elephant: Leaps in creative writing with multimodal machine intelligence. ACM Transactions on Computer-Human Interaction (2022).
  42. VelociTap: Investigating fast mobile text entry using sentence-based decoding of touchscreen keyboard input. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 659–668.
  43. AI as an Active Writer: Interaction strategies with generated text in human-AI collaborative fiction writing. In Joint Proceedings of the ACM IUI Workshops, Vol. 10. CEUR-WS Team.
  44. Running out of time: The impact and value of flexibility in on-demand crowdwork. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1–11.
  45. Wordcraft: story writing with large language models. In 27th International Conference on Intelligent User Interfaces. 841–852.
  46. Synthetic lies: Understanding ai-generated misinformation and evaluating algorithmic and human solutions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Zhuoyan Li (7 papers)
  2. Chen Liang (140 papers)
  3. Jing Peng (32 papers)
  4. Ming Yin (70 papers)
Citations (22)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets