Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
60 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
8 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

How AI Ideas Affect the Creativity, Diversity, and Evolution of Human Ideas: Evidence From a Large, Dynamic Experiment (2401.13481v2)

Published 24 Jan 2024 in cs.CY, cs.AI, cs.CL, and cs.HC

Abstract: Exposure to LLM output is rapidly increasing. How will seeing AI-generated ideas affect human ideas? We conducted an experiment (800+ participants, 40+ countries) where participants viewed creative ideas that were from ChatGPT or prior experimental participants and then brainstormed their own idea. We varied the number of AI-generated examples (none, low, or high exposure) and if the examples were labeled as 'AI' (disclosure). Our dynamic experiment design -- ideas from prior participants in an experimental condition are used as stimuli for future participants in the same experimental condition -- speaks to the interdependent process of cultural creation: creative ideas are built upon prior ideas. Hence, we capture the compounding effects of having LLMs 'in the culture loop'. We find that high AI exposure (but not low AI exposure) did not affect the creativity of individual ideas but did increase the average amount and rate of change of collective idea diversity. AI made ideas different, not better. There were no main effects of disclosure. We also found that self-reported creative people were less influenced by knowing an idea was from AI and that participants may knowingly adopt AI ideas when the task is difficult. Our findings suggest that introducing AI ideas may increase collective diversity but not individual creativity.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. Anna Abraham. 2016. Gender and creativity: an overview of psychological and neuroscientific literature. Brain Imaging and Behavior 10, 2 (June 2016), 609–618. https://doi.org/10.1007/s11682-015-9410-8
  2. Cues to gender and racial identity reduce creativity in diverse social networks. Scientific Reports 11, 1 (May 2021), 10261. https://doi.org/10.1038/s41598-021-89498-5
  3. Roger E. Beaty and Dan R. Johnson. 2021. Automating creativity assessment with SemDis: An open platform for computing semantic distance. Behavior Research Methods 53, 2 (April 2021), 757–780. https://doi.org/10.3758/s13428-020-01453-w
  4. Semantic Distance and the Alternate Uses Task: Recommendations for Reliable Automated Assessment of Originality. Creativity Research Journal 34, 3 (July 2022), 245–260. https://doi.org/10.1080/10400419.2022.2025720
  5. Forward flow and creative thought: Assessing associative cognition and its role in divergent thinking. Thinking Skills and Creativity 41 (Sept. 2021), 100859. https://doi.org/10.1016/j.tsc.2021.100859
  6. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21). Association for Computing Machinery, New York, NY, USA, 610–623. https://doi.org/10.1145/3442188.3445922
  7. Charles F. Bond and Linda J. Titus. 1983. Social facilitation: A meta-analysis of 241 studies. Psychological Bulletin 94, 2 (1983), 265–292. https://doi.org/10.1037/0033-2909.94.2.265
  8. Robert Boyd and Peter J. Richerson. 1988. Culture and the Evolutionary Process. University of Chicago Press.
  9. The cultural niche: Why social learning is essential for human adaptation. Proceedings of the National Academy of Sciences 108, supplement_2 (June 2011), 10918–10925. https://doi.org/10.1073/pnas.1100290108
  10. Collaborative Storytelling with Human Actors and AI Narrators. http://arxiv.org/abs/2109.14728
  11. Vincent R. Brown and Paul B. Paulus. 2002. Making Group Brainstorming More Effective: Recommendations From an Associative Memory Perspective. Current Directions in Psychological Science 11, 6 (Dec. 2002), 208–212. https://doi.org/10.1111/1467-8721.00202
  12. How Novelists Use Generative Language Models: An Exploratory User Study.. In HAI-GEN+ user2agent IUI.
  13. Ted Chiang. 2023. ChatGPT Is a Blurry JPEG of the Web. The New Yorker (Feb. 2023). https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web
  14. The Idea Machine: LLM-based Expansion, Rewriting, Combination, and Suggestion of Ideas. In Proceedings of the 14th Conference on Creativity and Cognition (C&C ’22). Association for Computing Machinery, New York, NY, USA, 623–627. https://doi.org/10.1145/3527927.3535197
  15. Four Text-Mining Methods for Measuring Elaboration. The Journal of Creative Behavior 55, 2 (2021), 517–531. https://doi.org/10.1002/jocb.471
  16. How Generative AI Can Augment Human Creativity. Harvard Business Review (July 2023). https://hbr.org/2023/07/how-generative-ai-can-augment-human-creativity
  17. Katy Ilonka Gero. 2023. AI and the Writer: How Language Models Support Creative Writers. Ph.D. Columbia University, United States – New York. https://www.proquest.com/docview/2753687892/abstract/ACF7F21F1E274995PQ/1
  18. Katy Ilonka Gero and Lydia B. Chilton. 2019. Metaphoria: An Algorithmic Companion for Metaphor Creation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow Scotland Uk, 1–12. https://doi.org/10.1145/3290605.3300526
  19. Sparks: Inspiration for Science Writing using Language Models. In Proceedings of the 2022 ACM Designing Interactive Systems Conference (DIS ’22). Association for Computing Machinery, New York, NY, USA, 1002–1019. https://doi.org/10.1145/3532106.3533533
  20. Automation bias: Empirical results assessing influencing factors. International Journal of Medical Informatics 83, 5 (May 2014), 368–375. https://doi.org/10.1016/j.ijmedinf.2014.01.001
  21. J.P. Guilford. 1967. The nature of human intelligence. McGraw-Hill, New York, NY, US.
  22. Joy Paul Guilford. 1978. Alternate uses. Sheridan supply Company.
  23. AI-Mediated Communication: Definition, Research Agenda, and Ethical Considerations. Journal of Computer-Mediated Communication 25, 1 (March 2020), 89–100. https://doi.org/10.1093/jcmc/zmz022
  24. Does human–AI collaboration lead to more creative art? Aesthetic evaluation of human-made and AI-generated haiku poetry. Computers in Human Behavior (Oct. 2022), 107502. https://doi.org/10.1016/j.chb.2022.107502
  25. Krystal Hu. 2023. ChatGPT sets record for fastest-growing user base - analyst note. Reuters (Feb. 2023). https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/
  26. Heteroglossia: In-Situ Story Ideation with the Crowd. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376715
  27. Angel Hsing-Chi Hwang and Andrea Stevenson Won. 2021. IdeaBot: Investigating Social Facilitation in Human-Machine Team Creativity. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama Japan, 1–16. https://doi.org/10.1145/3411764.3445270
  28. Human Heuristics for AI-Generated Language Are Flawed. https://doi.org/10.48550/arXiv.2206.07271
  29. Jared Henderson. 2022. ChatGPT Will Make You Less Creative. https://www.youtube.com/watch?v=1K8PiMNoR7A
  30. Modulation of aesthetic value by semantic context: An fMRI study. NeuroImage 44, 3 (Feb. 2009), 1125–1132. https://doi.org/10.1016/j.neuroimage.2008.10.009
  31. Nils Köbis and Luca D. Mossink. 2021. Artificial intelligence versus Maya Angelou: Experimental evidence that people cannot differentiate AI-generated from human-written poetry. Computers in Human Behavior 114 (Jan. 2021), 106553. https://doi.org/10.1016/j.chb.2020.106553
  32. Krish Naik. 2023. Will Chatgpt Kill Your Creativity? https://www.youtube.com/watch?v=0m2r9elReBY
  33. CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI ’22). Association for Computing Machinery, New York, NY, USA, 1–19. https://doi.org/10.1145/3491102.3502030
  34. Jasmine Mangalaseril. 2023. The Incredible Blandness of ChatGPT. https://cardamomaddict.substack.com/p/the-incredible-blandness-of-chatgpt
  35. Arthur I. Miller. 2019. The Artist in the Machine: The World of AI-Powered Creativity. The MIT Press. https://doi.org/10.7551/mitpress/11585.001.0001
  36. Co-Writing Screenplays and Theatre Scripts with Language Models: Evaluation by Industry Professionals. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23). Association for Computing Machinery, New York, NY, USA, 1–34. https://doi.org/10.1145/3544548.3581225
  37. Coming to Terms: Automatic Formation of Neologisms in Hebrew. In Findings of the Association for Computational Linguistics: EMNLP 2020. Association for Computational Linguistics, Online, 4918–4929. https://doi.org/10.18653/v1/2020.findings-emnlp.442
  38. Automation Bias, Accountability, and Verification Behaviors. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 40, 4 (Oct. 1996), 204–208. https://doi.org/10.1177/154193129604000413
  39. Michael D. Mumford and Sven Hemlin. 2017. Handbook of Research on Leadership and Creativity. Edward Elgar Publishing.
  40. Reem Nadeem. 2022. How Americans think about artificial intelligence. https://www.pewresearch.org/internet/2022/03/17/how-americans-think-about-artificial-intelligence/
  41. Reem Nadeem. 2023. Public Awareness of Artificial Intelligence in Everyday Activities. https://www.pewresearch.org/science/2023/02/15/public-awareness-of-artificial-intelligence-in-everyday-activities/
  42. Nation World News. 2023. Why Does ChatGPT Increase Creativity? https://nationworldnews.com/why-does-chatgpt-increase-creativity/
  43. J. Nickerson and Yasuaki Sakamoto. 2010. Crowdsourcing Creativity: Combining Ideas in Networks. https://www.semanticscholar.org/paper/Crowdsourcing-Creativity%3A-Combining-Ideas-in-Nickerson-Sakamoto/340a7645d1402287e151e83981f8a4085227e317
  44. Bernard A. Nijstad and Wolfgang Stroebe. 2006. How the Group Affects the Mind: A Cognitive Model of Idea Generation in Groups. Personality and Social Psychology Review 10, 3 (Aug. 2006), 186–213. https://doi.org/10.1207/s15327957pspr1003_1
  45. University of Minnestota. [n. d.]. UM Research: AI Tests Into Top 1% for Original Creative Thinking. https://www.umt.edu/news/2023/07/070523test.php
  46. American Psychological Association Dictionary of Psychology. [n. d.]. divergent thinking. https://dictionary.apa.org/divergent-thinking
  47. Markus Ojala and Gemma C. Garriga. 2009. Permutation Tests for Studying Classifier Performance. In 2009 Ninth IEEE International Conference on Data Mining. IEEE, Miami Beach, FL, USA, 908–913. https://doi.org/10.1109/ICDM.2009.108
  48. Beyond Semantic Distance: Automated Scoring of Divergent Thinking Greatly Improves with Large Language Models. https://doi.org/10.13140/RG.2.2.32393.31840
  49. Beyond semantic distance: Automated scoring of divergent thinking greatly improves with large language models. Thinking Skills and Creativity 49 (Sept. 2023), 101356. https://doi.org/10.1016/j.tsc.2023.101356
  50. BunCho: AI Supported Story Co-Creation via Unsupervised Multitask Learning to Increase Writers’ Creativity in Japanese. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama Japan, 1–10. https://doi.org/10.1145/3411763.3450391
  51. Paul B. Paulus and Vincent R. Brown. 2007. Toward More Creative and Innovative Group Idea Generation: A Cognitive-Social-Motivational Perspective of Brainstorming. Social and Personality Psychology Compass 1, 1 (2007), 248–265. https://doi.org/10.1111/j.1751-9004.2007.00006.x
  52. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. http://arxiv.org/abs/1908.10084
  53. Katharina Reinecke and Krzysztof Z. Gajos. 2015. LabintheWild: Conducting Large-Scale Online Experiments With Uncompensated Samples. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW ’15). Association for Computing Machinery, New York, NY, USA, 1364–1378. https://doi.org/10.1145/2675133.2675246
  54. European Business Review. 2023. ChatGPT: Ushering in the Age of Creativity. https://www.europeanbusinessreview.com/chatgpt-ushering-in-the-age-of-creativity/
  55. Peter J. Richerson and Robert Boyd. 2008. Not By Genes Alone: How Culture Transformed Human Evolution. University of Chicago Press.
  56. Melissa Roemmele. 2021. Inspiration through Observation: Demonstrating the Influence of Automatically Generated Text on Creative Writing. https://doi.org/10.48550/arXiv.2107.04007
  57. Interacting with Large Language Models: A Case Study on AI-Aided Brainstorming for Guesstimation Problems. In HHAI 2023: Augmenting Human Intellect. IOS Press, 153–167. https://doi.org/10.3233/FAIA230081
  58. On the Influence of Explainable AI on Automation Bias. https://doi.org/10.48550/arXiv.2204.08859
  59. The Curse of Recursion: Training on Generated Data Makes Models Forget. http://arxiv.org/abs/2305.17493
  60. Toward Collaborative Ideation at Scale: Leveraging Ideas from Others to Generate More Creative and Diverse Ideas, In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, 937–945. https://doi.org/10.1145/2675133.2675239
  61. How to do better with gender on surveys: a guide for HCI researchers. Interactions 26, 4 (June 2019), 62–65. https://doi.org/10.1145/3338283
  62. Putting GPT-3’s Creativity to the (Alternative Uses) Test. https://doi.org/10.48550/arXiv.2206.08932
  63. Tubefilter. 2023. 86% of creators believe AI has a positive effect on creativity. ChatGPT offered its own opinions. https://www.tubefilter.com/2023/06/02/lightricks-creator-artificial-intelligence-ai-survey-chat-gpt-wired/
  64. Chetan Walia. 2019. A Dynamic Definition of Creativity. Creativity Research Journal 31, 3 (July 2019), 237–247. https://doi.org/10.1080/10400419.2019.1641787
  65. Electronic Brainstorming With a Chatbot Partner: A Good Idea Due to Increased Productivity and Idea Diversity. Frontiers in Artificial Intelligence 5 (Sept. 2022), 880673. https://doi.org/10.3389/frai.2022.880673
  66. Wilcot. 2023. Using Chat-GPT for Innovators: Enhancing Creativity and Innovation. https://www.boardofinnovation.com/blog/using-chat-gpt-for-innovators-enhancing-creativity-and-innovation/
  67. Jamie Williams. 2018. Should AI Always Identify Itself? It’s More Complicated Than You Might Think. https://www.eff.org/deeplinks/2018/05/should-ai-always-identify-itself-its-more-complicated-you-might-think
  68. AI as an Active Writer: Interaction Strategies with Generated Text in Human-AI Collaborative Fiction Writing 56-65. https://www.semanticscholar.org/paper/AI-as-an-Active-Writer%3A-Interaction-Strategies-with-Yang-Zhou/15ddeb7765e2a3ea692a27d9b30e8f9446d74742
  69. Automatic Assessment of Divergent Thinking in Chinese Language with TransDis: A Transformer-Based Language Model Approach. https://doi.org/10.48550/arXiv.2306.14790
  70. Lixiu Yu and Jeffrey V. Nickerson. 2011. Cooks or cobblers? crowd creativity through combination. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’11). Association for Computing Machinery, New York, NY, USA, 1393–1402. https://doi.org/10.1145/1978942.1979147
  71. Lixiu Yu and Jeffrey V. Nickerson. 2013. An internet-scale idea generation system. ACM Transactions on Interactive Intelligent Systems 3, 1 (April 2013), 2:1–2:24. https://doi.org/10.1145/2448116.2448118
  72. A MAD method to assess idea novelty: Improving validity of automatic scoring using maximum associative distance (MAD). Psychology of Aesthetics, Creativity, and the Arts (2023), No Pagination Specified–No Pagination Specified. https://doi.org/10.1037/aca0000573
  73. Wordcraft: Story Writing With Large Language Models. In 27th International Conference on Intelligent User Interfaces (IUI ’22). Association for Computing Machinery, New York, NY, USA, 841–852. https://doi.org/10.1145/3490099.3511105
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Joshua Ashkinaze (6 papers)
  2. Julia Mendelsohn (13 papers)
  3. Li Qiwei (6 papers)
  4. Ceren Budak (16 papers)
  5. Eric Gilbert (20 papers)
Citations (6)
View on Semantic Scholar