Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On the Creativity of Large Language Models (2304.00008v4)

Published 27 Mar 2023 in cs.AI, cs.CL, and cs.CY

Abstract: LLMs are revolutionizing several areas of Artificial Intelligence. One of the most remarkable applications is creative writing, e.g., poetry or storytelling: the generated outputs are often of astonishing quality. However, a natural question arises: can LLMs be really considered creative? In this article, we first analyze the development of LLMs under the lens of creativity theories, investigating the key open questions and challenges. In particular, we focus our discussion on the dimensions of value, novelty, and surprise as proposed by Margaret Boden in her work. Then, we consider different classic perspectives, namely product, process, press, and person. We discuss a set of easy'' andhard'' problems in machine creativity, presenting them in relation to LLMs. Finally, we examine the societal impact of these technologies with a particular focus on the creative industries, analyzing the opportunities offered, the challenges arising from them, and the potential associated risks, from both legal and ethical points of view.

An Expert Overview of "On the Creativity of LLMs"

The paper "On the Creativity of LLMs" by Giorgio Franceschelli and Mirco Musolesi explores the concept of creativity attributed to LLMs, providing an analytical exposition underpinned by philosophical and cognitive frameworks. The authors focus on situating LLMs within creativity theories, especially those proposed by Margaret Boden, and interrogate whether LLMs can truly be regarded as creative entities.

Firstly, the authors contextualize the emergence and development of LLMs by reflecting on historical discourse, including Ada Lovelace's and Alan Turing's speculations on machine creativity. The evolution of automatic text generation traces its lineage from early computational attempts, through neural network advents marked by recurrent neural networks and GANs, to the transformative impact brought by transformers and attention mechanisms. The authors effectively summarize this journey to underpin their exploration of LLM creativity.

Central to the paper are Boden's three creativity criteria: value, novelty, and surprise. The authors contend that while LLMs undeniably produce valuable and coherent artifacts due to their well-documented impacts across various industries, achieving originality and unpredictability is more challenging. They dissect novelty into P-creativity and H-creativity and critique LLMs' generative probabilistic processes that predominantly encourage combinatorial rather than transformational creativity. The criteria of surprise in creativity, categorized into combinatorial, exploratory, and transformational levels, are primarily unreachable by LLMs in their current form.

Beyond the creative products, the authors extend their discussion to the considerations of process, press, and person. They elucidate the process of creativity in LLMs, emphasizing their lack of intent, self-assessment, and motivational force, which are pivotal in human creativity. The immutability of trained LLMs, lacking continual real-world adaptation, is highlighted as a barrier when viewed from the press perspective, wherein a creative system dynamically interacts with and evolves alongside its domain. The person component remains abstract and speculative, noting its entwinement with consciousness—a frontier largely unexplored by artificial systems.

The paper also explores the practical and theoretical implications of LLMs within creative industries. There is an acknowledgment of the multifaceted legal challenges, chiefly surrounding copyright, as creations by LLMs do not align with conventional originality attributes tied to human creation. Furthermore, the authors suggest potential societal risks, such as the obsolescence of traditional writing occupations and intellectual property disputes. Nevertheless, they posit that LLMs have the potential to facilitate novel opportunities in human-AI co-creativity, augmenting human creative processes and conceiving unique artistic collaborations.

In closing, Franceschelli and Musolesi provide a critical examination of LLMs' contributions to creativity while outlining a speculative yet cautious research agenda. They advocate for the advancement of adaptive and continual learning paradigms to allow LLMs closer integration with dynamic societal frameworks. The ongoing dialogue in understanding and implementing creativity within AI demands holistic advancements understanding both theoretical underpinnings and pragmatic implementations, ensuring alignment with evolving ethical, societal, and technological landscapes.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (110)
  1. T. M. Amabile. The social psychology of creativity: A componential conceptualization. Journal of Personality and Social Psychology, 45(2):357–376, 1983.
  2. T. M. Amabile. Creativity In Context. Routledge, 1996.
  3. Is transformative use eating the world? Boston College Law Review, 61(3):905–970, 2020.
  4. C. Babbage. Of the analytical engine. In Passages from the Life of a Philosopher, volume 3, pages 112–141. Longman, Green, Longman, Roberts, & Green, 1864.
  5. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Representations (ICLR’15), 2015.
  6. F. Barron. The disposition toward originality. Journal of Abnormal Psychology, 51(3):478–485, 1955.
  7. E. M. Bender and A. Koller. Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL’20), pages 5185–5198, 2020.
  8. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT’21), page 610–623, 2021.
  9. D. E. Berlyne. Conflict, Arousal, and Curiosity. McGraw-Hill Book Company, 1960.
  10. D. E. Berlyne. Aesthetics and Psychobiology. Appleton-Century-Crofts, New York, NY, 1971.
  11. M. A. Boden. The Creative Mind: Myths and Mechanisms. Routledge, 2003.
  12. M. A. Boden. Computer models of creativity. AI Magazine, 30(3):23–34, 2009.
  13. On the opportunities and risks of foundation models, 2021. arXiv:2108.07258 [cs.LG].
  14. E. Bonadio and L. McDonagh. Artificial intelligence as producer and consumer of copyright works: Evaluating the consequences of algorithmic creativity. Intellectual Property Quarterly 2020, 2:112–137, 2020.
  15. T. Bond and S. Blair. Artificial intelligence & copyright: Section 9(3) or authorship without an author. Journal of Intellectual Property Law & Practice, 14(6):423–423, 2019.
  16. Generating sentences from a continuous space. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNNL’16), pages 10–21, 2016.
  17. Language models are few-shot learners. In Advances in Neural Information Processing Systems (NIPS’20), pages 1877–1901, 2020.
  18. J. S. Bruner. The conditions of creativity. In Contemporary approaches to creative thinking: A symposium held at the University of Colorado, pages 1–30. Atherton Press, 1962.
  19. Sparks of Artificial General Intelligence: Early experiments with GPT-4, 2023. arXiv:2303.12712 [cs.CL].
  20. Learning to surprise: A composer-audience architecture. In Proceedings of the 10th International Conference on Computational Creativity (ICCC’19), pages 41–48, 2019.
  21. How novelists use generative language models: An exploratory user study. In Proceedings of the IUI’20 Workshop on Human-AI Co-Creation with Generative Models, 2020.
  22. Help me write a poem: Instruction tuning as a vehicle for collaborative poetry writing. In Proceedings of the AAAI-23 Workshop on Creative AI Across Modalities, 2023.
  23. D. J. Chalmers. The Conscious Mind: In Search of a Fundamental Theory. Oxford University Press, 1996.
  24. On the properties of neural machine translation: Encoder-decoder approaches. In Proceedings of SSST-8, 8th Workshop on Syntax, Semantics and Structure in Statistical Translation, pages 103–111, 2014.
  25. PaLM: Scaling language modeling with pathways, 2022. arXiv:2204.02311 [cs.CL].
  26. S. Colton and G. A. Wiggins. Computational creativity: The final frontier? In Proceedings of the 20th European Conference on Artificial Intelligence (ECAI’12), volume 12, pages 21–26, 2012.
  27. Machine generated text: A comprehensive survey of threat models and detection methods, 2022. arXiv:2210.07321 [cs.CL].
  28. M. Csikszentmihalyi. Society, culture, and person: A systems view of creativity. In The nature of creativity: Contemporary psychological perspectives, pages 325–339. Cambridge University Press, 1988.
  29. Plug and play language models: A simple approach to controlled text generation. In Proceedings of the 8th International Conference on Learning Representations (ICLR’20), 2020.
  30. Intrinsic Motivation and Self-Determination in Human Behavior. Springer, 1985.
  31. N. Dehouche. Plagiarism in the age of massive generative pre-trained transformers (GPT-3). Ethics in Science and Environmental Politics, 21:17–23, 2021.
  32. J.-M. Deltorn. Deep creations: Intellectual property and the automata. Frontiers in Digital Humanities, 4(3):1–13, 2017.
  33. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (NAACL’19), pages 4171–4186, 2019.
  34. GLaM: Efficient scaling of language models with mixture-of-experts. In Proceedings of the 39th International Conference on Machine Learning (ICML’22), pages 5547–5569, 2022.
  35. E. Eisenstein. The Printing Press as an Agent of Change: Communications and Cultural Transformations in Early-Modern Europe. Cambridge University Press, Cambridge, United Kingdom, 1979.
  36. L. Floridi and M. Chiriatti. GPT-3: Its nature, scope, limits, and consequences. Minds and Machines, 30(4):681–694, 2020.
  37. G. Franceschelli and M. Musolesi. Creativity and machine learning: A survey, 2021. arXiv:2104.02726 [cs.LG].
  38. G. Franceschelli and M. Musolesi. Copyright in generative deep learning. Data & Policy, 4:e17, 2022.
  39. B. Gaut. Creativity and imagination. In The Creation of Art: New Essays in Philosophical Aesthetics, pages 148–173. Cambridge University Press, 2003.
  40. B. Gaut. The philosophy of creativity. Philosophy Compass, 5(12):1034–1046, 2010.
  41. Sparks: Inspiration for science writing using language models. In Proceedings of the 2022 Designing Interactive Systems Conference (DIS’22), page 1002–1019, 2022.
  42. D. J. Gervais. Feist goes global: A comparative analysis of the notion of originality in copyright law. Journal of the Copyright Society of the U.S.A., 49:949–981, 2002.
  43. P. Gervás. Computational modelling of poetry generation. In Symposium on Artificial Intelligence and Poetry (AISB’13), pages 11–16, 2013.
  44. Can GPT-3 write an academic paper on itself, with minimal human input?, 2022. https://hal.archives-ouvertes.fr/hal-03701250v1.
  45. A. Grinbaum and L. Adomaitis. The ethical need for watermarks in machine-generated language, 2022. arXiv:2209.03118 [cs.CL].
  46. A. Guadamuz. Do androids dream of electric copyright? comparative analysis of originality in artificial intelligence generated works. Intellectual Property Quarterly, 2017.
  47. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735–178, 1997.
  48. E. Hoel. The banality of ChatGPT, 2022. https://erikhoel.substack.com/p/the-banality-of-chatgpt [Retrieved on January 26, 2023].
  49. Training compute-optimal large language models. In Advances in Neural Information Processing Systems (NIPS’22), 2022.
  50. Creative writing with an ai-powered writing assistant: Perspectives from professional writers, 2022. arXiv:2211.05030 [cs.HC].
  51. A. K. Jordanous. Evaluating machine creativity. In Proceedings of the Seventh ACM Conference on Creativity and Cognition (C&C’09), page 331–332, 2009.
  52. D. Jurafsky and J. H. Martin. Speech and Language Processing. Third (draft) edition, 2023.
  53. A. Karpathy. The unreasonable effectiveness of recurrent neural networks, 2015. http://karpathy.github.io/2015/05/21/rnn-effectiveness/ [Retrieved on August 5, 2020].
  54. A watermark for large language models, 2023. arXiv:2301.10226 [cs.LG].
  55. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13):3521–3526, 2017.
  56. M. Lebowitz. Creating a story-telling universe. In Proceedings of the 8th International Joint Conference on Artificial Intelligence (IJCAI’83), page 63–65, 1983.
  57. RecipeGPT: Generative pre-training based cooking recipe generation and evaluation system. In Companion Proceedings of the Web Conference 2020 (WWW’20), page 181–184, 2020.
  58. CoAuthor: Designing a human-ai collaborative writing dataset for exploring language model capabilities. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI’22), pages 1–19, 2022.
  59. Holistic evaluation of language models, 2022. arXiv:2211.09110 [cs.CL].
  60. Modeling forms of surprise in artificial agents: empirical and theoretical study of surprise functions. In Proceedings of the Annual Meeting of the Cognitive Science Society (CogSci’04), pages 873–878, 2004.
  61. M. L. Maher. Evaluating creativity in humans, computers, and collectively intelligent systems. In Proceedings of the 1st DESIRE Network Conference on Creativity and Innovation in Design (DESIRE’10), page 22–28, 2010.
  62. Using genetic algorithms to create meaningful poetic text. Journal of Experimental & Theoretical Artificial Intelligence, 24(1):43–64, 2012.
  63. Event representations for automated story generation with deep neural nets. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence (AAAI’18/IAAI’18/EAAI’18), 2018.
  64. How much do language models copy from their training data? evaluating linguistic novelty in text generation using RAVEN, 2021. arXiv:2111.09509 [cs.CL].
  65. J. R. Meehan. TALE-SPIN, an interactive program that writes stories. In Proceedings of the 5th International Joint Conference on Artificial Intelligence (IJCAI’77), pages 91–98, 1977.
  66. L. F. Menabrea and A. Lovelace. Sketch of the analytical engine invented by charles babbage. In Scientific Memoirs, volume 3, pages 666–731. Richard and John E. Taylor, 1843.
  67. A. I. Miller. The Artist in the Machine. The MIT Press, 2019.
  68. Co-writing screenplays and theatre scripts alongside language models using Dramatron. In Proceedings of the NIPS’22 Workshop on ML for Creativity & Design, 2022.
  69. The processes of creative thinking. In Contemporary approaches to creative thinking: A symposium held at the University of Colorado, pages 63–119. Atherton Press, 1962.
  70. A. Newton and K. Dhole. Is AI art another industrial revolution in the making? In Proceedings of the AAAI-23 Workshop on Creative AI Across Modalities, 2023.
  71. OpenAI. GPT-4 Technical Report, 2023. https://cdn.openai.com/papers/gpt-4.pdf.
  72. Leveraging human preferences to master poetry. In Proceedings of the AAAI-23 Workshop on Creative AI Across Modalities, 2023.
  73. GhostWriter: Using an LSTM for automatic rap lyric generation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP’15), pages 1919–1924, 2015.
  74. Racter. The Policeman’s Beard Is Half Constructed. Warner Books, Inc., 1984.
  75. Language models are unsupervised multitask learners, 2019.
  76. Scaling language models: Methods, analysis & insights from training gopher, 2021. arXiv:2112.11446 [cs.CL].
  77. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67, 2020.
  78. PlotMachines: Outline-conditioned generation with dynamic plot state tracking. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP’20), pages 4274–4295, 2020.
  79. L. Reynolds and K. McDonell. Multiversal views on language models, 2021. arXiv:2102.06391 [cs.HC].
  80. M. Rhodes. An analysis of creativity. The Phi Delta Kappan, 42(7):305–310, 1961.
  81. M. O. Riedl. Computational creativity as meta search, 2018. https://mark-riedl.medium.com/computational-creativity-as-meta-search-6cad95da923b [Retrieved on January 27, 2023].
  82. Narrative planning: Balancing plot and character. Journal of Artificial Intelligence Research, 39(1):217–268, 2010.
  83. G. Ritchie. Some empirical criteria for attributing creativity to a computer program. Minds and Machines, 17:67–99, 2007.
  84. C. Rosset. Turing-NLG: A 17-billion-parameter language model by microsoft, 2020. https://www.microsoft.com/en-us/research/blog/turing-nlg-a-17-billion-parameter-language-model-by-microsoft/ [Retrieved on May 3, 2022].
  85. Training GPT-2 to represent two romantic-era authors: challenges, evaluations and pitfalls. In Proceedings of the 3th International Conference on Computational Creativity (ICCC’22), 2022.
  86. A hybrid convolutional variational autoencoder for text generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP’17), pages 627–637, 2017.
  87. M. Shanahan. Talking about large language models, 2022. arXiv:2212.03551 [cs.CL].
  88. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems (NIPS’17), 2017.
  89. Megatron-LM: Training multi-billion parameter language models using model parallelism, 2019. arXiv:1909.08053 [cs.CL].
  90. E. Silva. How photography pioneered a new understanding of art, 2022. https://www.thecollector.com/how-photography-transformed-art/ [Retrieved on February 10, 2023].
  91. Using DeepSpeed and megatron to train megatron-turing NLG 530b, a large-scale generative language model, 2022. arXiv:2201.11990 [cs.CL].
  92. M. I. Stein. Stimulating Creativity. Volume 1. Academic Press, 1974.
  93. Putting GPT-3’s creativity to the (alternative uses) test. In Proceedings of the 13th International Conference on Computational Creativity (ICCC’22), 2022.
  94. Learning to summarize with human feedback. In Advances in Neural Information Processing Systems (NIPS’20), pages 3008–3021, 2020.
  95. LAMOL: LAnguage MOdeling for Lifelong Language Learning. In Proceedings of the 2020 International Conference on Learning Representations (ICLR’20), 2020.
  96. Story centaur: Large language model few shot learning as a creative writing tool. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations (EACL’21), pages 244–256, 2021.
  97. Understanding the capabilities, limitations, and societal impact of large language models, 2021. arXiv:2102.02503 [cs.CL].
  98. LaMDA: Language models for dialog applications, 2022. arXiv:2201.08239 [cs.CL].
  99. A. M. Turing. Computing machinery and intelligence. Mind, LIX(236):433–460, 1950.
  100. S. R. Turner. The Creative Process: A Computer Model of Creativity and Storytelling. Lawrence Erlbaum Associates, Inc, Hillsdale, NJ, 1994.
  101. Attention is all you need. In Advances in Neural Information Processing Systems (NIPS’17), 2017.
  102. T. Waite. Ai-generated artworks are disappointing at auction, 2019. https://www.dazeddigital.com/art-photography/article/46839/1/ai-generated-artworks-disappointing-at-auction-obvious-artificial-intelligence [Retrieved on January 26, 2023].
  103. Taxonomy of risks posed by language models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT’22), page 214–229, 2022.
  104. L. Wertz and J. Kuhn. Adapting transformer language models for application in computational creativity: Generating german theater plays with varied topics. In Proceedings of the 13th International Conference on Computational Creativity (ICCC’22), 2022.
  105. G. A. Wiggins. A preliminary framework for description, analysis and comparison of creative systems. Knowledge-Based Systems, 19(7):449–458, 2006.
  106. Pretrained language model in continual learning: A comparative study. In Proceedings of the 2022 International Conference on Learning Representations (ICLR’22), 2022.
  107. SeqGAN: Sequence generative adversarial nets with policy gradient. In Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI’17), page 2852–2858, 2017.
  108. Adversarial feature matching for text generation. In Proceedings of the 34th International Conference on Machine Learning (ICML’17), page 4006–4015, 2017.
  109. Fine-tuning language models from human preferences, 2019. arXiv:1909.08593 [cs.CL].
  110. Neural Poetry: Learning to Generate Poems Using Syllables. In Proceedings of the 2019 International Conference on Artificial Neural Networks (ICANN’19), pages 313–325, 2019.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Giorgio Franceschelli (11 papers)
  2. Mirco Musolesi (81 papers)
Citations (36)