Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Inspo: Writing with Crowds Alongside AI (2311.16521v3)

Published 28 Nov 2023 in cs.HC

Abstract: The use of AI to support creative writing has bloomed in recent years. However, it is less well understood how AI compares to on-demand human support. We explored how writers interact with both AI and crowd worker writing assistants in creative writing. We replicated the interface of the prior crowd-writing system, Heteroglossia, and developed Inspo, a text editor allowing users to request suggestions from AI models and crowd workers. In a one-week deployment study involving eight creative writers, we examined how often participants selected crowd workers when fluent AI text generators were also available. Findings showed a consistent decline in crowd worker usage, with participants favoring AI due to its faster responses and more consistent quality. We conclude with suggestions for future systems, recommending designs that account for the unique strengths and weaknesses of human versus AI assistants, strategies to address automation bias, and sociocultural views of writing.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. Mark S Ackerman. 2000. The intellectual challenge of CSCW: the gap between social requirements and technical feasibility. Human–Computer Interaction 15, 2-3 (2000), 179–203.
  2. How Novelists Use Generative Language Models: An Exploratory User Study.
  3. Generative Models can Help Writers without Writing for Them.. In IUI Workshops.
  4. Alton L Becker. 2000. Beyond translation: Essays toward a modern philology. University of Michigan Press.
  5. Patricia Bizzell. 2003. Cognition, convention, and certainty. Cross-talk in comp theory: A reader (2003), 387–411.
  6. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 1877–1901. https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
  7. Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writing. arXiv preprint arXiv:2210.13669 (2022).
  8. The Shmoop Corpus: A Dataset of Stories with Loosely Aligned Summaries. arXiv:1912.13082 (2019).
  9. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 (2022).
  10. Creative Writing with a Machine in the Loop: Case Studies on Slogans and Stories. In 23rd International Conference on Intelligent User Interfaces (Tokyo, Japan) (IUI ’18). ACM, New York, NY, USA, 329–340. https://doi.org/10.1145/3172944.3172983
  11. Choice over control: How users write with large language models using diegetic and non-diegetic prompting. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17.
  12. Jessica L Feuston and Jed R Brubaker. 2021. Putting Tools in Their Place: The Role of Time and Perspective in Human-AI Collaboration for Qualitative Analysis. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–25.
  13. InkWell: A Creative Writer’s Creative Assistant. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition. ACM, 93–102.
  14. Katy Ilonka Gero. 2022. Ai reveals the most human parts of writing. https://www.wired.com/story/artificial-intelligence-writing-art/
  15. Katy Ilonka Gero and Lydia B. Chilton. 2019. Metaphoria: An Algorithmic Companion for Metaphor Creation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300526
  16. Sparks: Inspiration for Science Writing Using Language Models. In Designing Interactive Systems Conference (Virtual Event, Australia) (DIS ’22). Association for Computing Machinery, New York, NY, USA, 1002–1019. https://doi.org/10.1145/3532106.3533533
  17. Social dynamics of AI support in creative writing. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–15.
  18. An introduction to technological support for writing. Technical Report. Technical Report. Microsoft Research Tech Report MSR-TR-2016-001.
  19. A Data-Driven Analysis of Workers’ Earnings on Amazon Mechanical Turk. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3173574.3174023
  20. Heteroglossia: In-Situ Story Ideation with the Crowd. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376715
  21. Chieh-Yang Huang and Ting-Hao Huang. 2021. Semantic Frame Forecast. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Online, 2702–2713. https://doi.org/10.18653/v1/2021.naacl-main.215
  22. Conveying the Predicted Future to Users: A Case Study of Story Plot Prediction. In Proc. AAAI 2023 Workshop of Creative AI Across Modalities.
  23. Creative Writing with an AI-Powered Writing Assistant: Perspectives from Professional Writers. (2022).
  24. An introduction to statistical learning. Vol. 112. Springer.
  25. Lightgbm: A highly efficient gradient boosting decision tree. In Advances in neural information processing systems. 3146–3154.
  26. M. Jimmie Killingsworth. 1992. Discourse Communities. Local and Global. Rhetoric Review 11, 1 (1992), 110–122. http://www.jstor.org/stable/465883
  27. When is a Tool a Tool? User Perceptions of System Agency in Human–AI Co-Creative Drawing. In Proceedings of the 2023 ACM Designing Interactive Systems Conference. 1978–1996.
  28. CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities. In CHI Conference on Human Factors in Computing Systems. ACM.
  29. Collaborating with mature English language learners to combine peer and automated feedback: A user-centered approach to designing writing support. International Journal of Artificial Intelligence in Education 31 (2021), 638–679.
  30. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017).
  31. Brian Lubars and Chenhao Tan. 2019. Ask not what AI can do, but what AI should do: Towards a framework of task delegability. Advances in Neural Information Processing Systems 32 (2019).
  32. Paul Kei Matsuda. 2001. Voice in Japanese written discourse: Implications for second language writing. Journal of Second language writing 10, 1-2 (2001), 35–53.
  33. Nora McCook. 2016. Literacy contact zones: A framework for research. Literacy in Composition Studies 4, 1 (2016), 50–72.
  34. Heidi A McKee and James E Porter. 2020. Ethics for AI writing: The importance of rhetorical context. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 110–116.
  35. Co-Writing Screenplays and Theatre Scripts with Language Models: Evaluation by Industry Professionals. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 355, 34 pages. https://doi.org/10.1145/3544548.3581225
  36. Animesh Nighojkar and John Licato. 2021. Improving Paraphrase Detection with the Adversarial Paraphrasing Task. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Online, 7106–7116. https://doi.org/10.18653/v1/2021.acl-long.552
  37. James E. Porter. 1986. Intertextuality and the Discourse Community. Rhetoric Review 5, 1 (1986), 34–47. http://www.jstor.org/stable/466015
  38. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. https://arxiv.org/abs/1908.10084
  39. Jeba Rezwana and Mary Lou Maher. 2023. User Perspectives on Ethical Challenges in Human-AI Co-Creativity: A Design Fiction Study. In Proceedings of the 15th Conference on Creativity and Cognition. 62–74.
  40. William S Robinson. 1951. The logical structure of analytic induction. American Sociological Review 16, 6 (1951), 812–818.
  41. Melissa Roemmele. 2021. Inspiration through Observation: Demonstrating the Influence of Automatically Generated Text on Creative Writing. arXiv preprint arXiv:2107.04007 (2021).
  42. Melissa Roemmele and Andrew S Gordon. 2015. Creative help: a story writing assistant. In International Conference on Interactive Digital Storytelling. Springer, 81–92.
  43. A participative inquiry in a TESOL program: Development of three NNES graduate students’ legitimate peripheral participation to fuller participation. The Modern Language Journal 95, 4 (2011), 558–574.
  44. Ben Shneiderman and Pattie Maes. 1997. Direct manipulation vs. interface agents. interactions 4, 6 (1997), 42–61.
  45. Reading on LCD vs e-Ink displays: effects on fatigue and visual strain. Ophthalmic and Physiological Optics 32, 5 (2012), 367–374.
  46. Using deepspeed and megatron to train megatron-turing nlg 530b, a large-scale generative language model. arXiv preprint arXiv:2201.11990 (2022).
  47. John M Swales. 2017. The concept of discourse community: Some recent personal history. In Composition Forum, Vol. 37.
  48. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239 (2022).
  49. ” It Felt Like Having a Second Mind”: Investigating Human-AI Co-creativity in Prewriting with Large Language Models. arXiv preprint arXiv:2307.10811 (2023).
  50. Etienne Wenger. 2000. Communities of Practice and Social Learning Systems. Organization 7, 2 (2000), 225–246. https://doi.org/10.1177/135050840072002 arXiv:https://doi.org/10.1177/135050840072002
  51. Etienne Wenger. 2010. Communities of practice and social learning systems: the career of a concept. In Social learning systems and communities of practice. Springer, 179–198.
  52. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019).
  53. Sketching nlp: A case study of exploring the right things to design with language intelligence. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.
  54. Wordcraft: Story Writing With Large Language Models. In 27th International Conference on Intelligent User Interfaces (Helsinki, Finland) (IUI ’22). Association for Computing Machinery, New York, NY, USA, 841–852. https://doi.org/10.1145/3490099.3511105
  55. Li Yujian and Liu Bo. 2007. A Normalized Levenshtein Distance Metric. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 6 (2007), 1091–1095. https://doi.org/10.1109/TPAMI.2007.1078
  56. Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, 2979–2989. https://doi.org/10.18653/v1/D17-1323
  57. Hao Zheng and Mirella Lapata. 2019. Sentence Centrality Revisited for Unsupervised Summarization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Florence, Italy, 6236–6247. https://doi.org/10.18653/v1/P19-1628
  58. Q. Zhu and J. Luo. 2022. Generative Pre-Trained Transformer for Design Concept Generation: An Exploration. Proceedings of the Design Society 2 (2022), 1825–1834. https://doi.org/10.1017/pds.2022.185
  59. Qihao Zhu and Jianxi Luo. 2023. Generative Transformers for Design Concept Generation. Journal of Computing and Information Science in Engineering 23, 4 (01 2023). https://doi.org/10.1115/1.4056220 arXiv:https://asmedigitalcollection.asme.org/computingengineering/article-pdf/23/4/041003/6971757/jcise_23_4_041003.pdf 041003.
  60. Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books. In The IEEE International Conference on Computer Vision (ICCV).
  61. Florian Znaniecki. 1934. The method of sociology. Farrar & Rinehart.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Chieh-Yang Huang (24 papers)
  2. Sanjana Gautam (11 papers)
  3. Shannon McClellan Brooks (1 paper)
  4. Ya-Fang Lin (5 papers)
  5. Ting-Hao 'Kenneth' Huang (42 papers)
  6. Tiffany Knearem (7 papers)
Citations (5)
View on Semantic Scholar