Does ChatGPT Have a Poetic Style? (2410.15299v2)

Abstract: Generating poetry has become a popular application of LLMs, perhaps especially of OpenAI's widely-used chatbot ChatGPT. What kind of poet is ChatGPT? Does ChatGPT have its own poetic style? Can it successfully produce poems in different styles? To answer these questions, we prompt the GPT-3.5 and GPT-4 models to generate English-language poems in 24 different poetic forms and styles, about 40 different subjects, and in response to 3 different writing prompt templates. We then analyze the resulting 5.7k poems, comparing them to a sample of 3.7k poems from the Poetry Foundation and the Academy of American Poets. We find that the GPT models, especially GPT-4, can successfully produce poems in a range of both common and uncommon English-language forms in superficial yet noteworthy ways, such as by producing poems of appropriate lengths for sonnets (14 lines), villanelles (19 lines), and sestinas (39 lines). But the GPT models also exhibit their own distinct stylistic tendencies, both within and outside of these specific forms. Our results show that GPT poetry is much more constrained and uniform than human poetry, showing a strong penchant for rhyme, quatrains (4-line stanzas), iambic meter, first-person plural perspectives (we, us, our), and specific vocabulary like "heart," "embrace," "echo," and "whisper."

• The paper evaluates ChatGPT's poetic style, comparing AI poems to human ones and finding models adhere to common forms but exhibit distinct, predictable stylistic patterns.
• Distinct patterns include preference for quatrains, iambic meter, rhyming couplets, and words like "heart" and "embrace," with GPT-4 showing better form adherence.
• Findings contribute to computational creativity by showing AI can mimic form but has stylistic limits, guiding prompt engineering and future AI development.

Analysis of "Does ChatGPT Have a Poetic Style?"

The paper "Does ChatGPT Have a Poetic Style?" by Melanie Walsh, Anna Preus, and Elizabeth Gronski explores the stylistic tendencies of ChatGPT, specifically focusing on its poetic capabilities. By evaluating over 5,700 poems generated by GPT-3.5 and GPT-4, the researchers provide an extensive comparison with a sample of 3,700 poems from the Poetry Foundation and the Academy of American Poets. The paper investigates the ability of these models to emulate various forms of English-language poetry, seeking to understand not only the technical prowess of the models but also their inherent stylistic biases.

Key Findings

1. Adherence to Poetic Forms: The GPT models exhibit proficiency in adhering to structural constraints of common poetic forms, such as the correct line lengths for sonnets, villanelles, and sestinas. This suggests that the models can replicate superficial formal elements with consistency, especially in GPT-4.
2. Distinct Stylistic Tendencies: Despite the adherence to form, the poetry generated by GPT models reveals distinct stylistic patterns. These include a pronounced preference for quatrains, iambic meter, and rhyming couplets. The models often default to structured, predictable patterns, indicating a more constrained output compared to human equivalents.
3. Prominent Semantic Elements: Words such as "heart," "embrace," "echo," and "whisper" are prominent in GPT-generated poems. These lexical preferences contribute to the models' characteristic style, alongside a frequent usage of first-person plural perspectives.
4. Differences Between Models: While both GPT-3.5 and GPT-4 exhibit these stylistic tendencies, GPT-4 shows improved consistency in adhering to poetic form compared to GPT-3.5. It presents less variability in the number of lines per poem but maintains the rigid stylistic patterns evident across both versions.

Theoretical and Practical Implications

This research contributes significantly to the field of computational creativity, particularly in understanding how LLMs like ChatGPT navigate the challenges of artistic expression. The findings emphasize the models' potential to mimic human-like creativity within set parameters, but also highlight their stylistic boundaries. These insights are pivotal for further exploration of creativity in AI, providing a foundation for developing more diverse and adaptive generation mechanisms.

Practically, the results have implications for the deployment of LLMs in creative writing applications. Understanding the models' default tendencies can guide users in crafting prompts that elicit more varied outputs, enhancing the poetic creativity of AI systems.

Future Developments

Future research could explore the impact of different prompts, including author-specific styles, on model outputs. Additionally, further analysis could investigate how alternative training data might affect the models' creative diversity. By addressing these areas, researchers can advance the capabilities of AI in generating more nuanced and diverse artistic content, potentially leading to applications that can emulate or even enhance human creativity.

The paper "Does ChatGPT Have a Poetic Style?" solidifies the argument that while LLMs can replicate poetic forms, their inherent stylistic limitations reflect the current challenges AI faces in achieving true creative expression. Through ongoing advancements and explorations, the pursuit of more sophisticated AI-driven art offers a promising horizon for computational literature.