Homogenization Effects of Large Language Models on Human Creative Ideation

Abstract: LLMs are now being used in a wide variety of contexts, including as creativity support tools (CSTs) intended to help their users come up with new ideas. But do LLMs actually support user creativity? We hypothesized that the use of an LLM as a CST might make the LLM's users feel more creative, and even broaden the range of ideas suggested by each individual user, but also homogenize the ideas suggested by different users. We conducted a 36-participant comparative user study and found, in accordance with the homogenization hypothesis, that different users tended to produce less semantically distinct ideas with ChatGPT than with an alternative CST. Additionally, ChatGPT users generated a greater number of more detailed ideas, but felt less responsible for the ideas they generated. We discuss potential implications of these findings for users, designers, and developers of LLM-based CSTs.

• The paper finds that while large language models (LLMs) like ChatGPT increase fluency and idea count in creative tasks, they reduce semantic diversity at the group level, leading to homogenization.
• A comparative study with 36 participants showed ChatGPT users generated more ideas and elaborated more, but their ideas were less diverse across individuals compared to using non-AI tools.
• Implications suggest designers must mitigate LLM homogenization through techniques like diversifying outputs and scaffolding user creativity to maintain engagement and authorship.

Analyzing the Homogenization Effects of LLMs on Human Creative Ideation

The investigation into the influence of LLMs on creative ideation processes represents an essential frontier in understanding how expanding reliance on artificial intelligence shapes human creativity. The paper "Homogenization Effects of LLMs on Human Creative Ideation" elucidates the impact of these models on creativity, focusing on whether they promote or stifle the diversity of ideas when used as Creativity Support Tools (CSTs).

Study Design and Findings

The study employs a comparative user analysis involving 36 participants, assessing the outputs generated using ChatGPT against those generated using the non-AI CST, the Oblique Strategies (OS) deck. The paper presents several compelling findings:

1. Group-Level Homogenization: Participants using ChatGPT showed a notable decrease in semantic diversity in the ideas produced at the group level, consistent with the homogenization hypothesis. This suggests that LLMs generate more homogenous ideas across different users compared to the OS deck.
2. Individual-Level Effects: Interestingly, individual-level semantic diversity—diversity within the ideas produced by a single user—did not significantly differ between the two CSTs. This implies that while LLMs impact the diversity of ideas among different individuals, they do not necessarily induce creative fixation, where ideas from a single individual become overly similar to one another.
3. User Creativity and Engagement: Quantitatively, ChatGPT enabled users to produce a higher number of ideas with greater fluency, flexibility, and elaboration than the OS deck. However, it also led participants to report a diminished sense of personal creative responsibility, indicating a potential detachment from the ideation process when engaging with AI-generated outputs.
4. Content Reiterations: There was no significant difference in the number of unique ideas between users of ChatGPT and those who used OS. This finding challenges concerns about LLMs entirely stifling uniqueness in ideation but does highlight the tendency towards generating prevalent concepts.

Implications and Future Directions

These results bear significant implications for the deployment and design of CSTs based on LLMs:

• Awareness of Homogenization: A central implication of this study is the potential for LLM-based CSTs to induce homogenization through algorithmic outputs that echo stereotypical ideas. This underscores the necessity for system designers to consider strategies that encourage variable and diverse outputs. Methods such as integrating analogical reasoning prompts or diversifying context inputs could serve to mitigate homogenization.
• Scaffolded Creativity: The study suggests designing LLM interactions that encourage users to progressively elaborate their creative intent rather than offering direct, completed ideas. This could enhance user engagement and maintain a sense of authorship, addressing the observed reduction in perceived personal responsibility for ideas generated with AI assistance.
• Algorithmic Diversity: Future LLMs might incorporate quality-diversity optimization methods and diverse decoding strategies to enhance output variability across different user inputs, thus aiming to counteract the homogenizing force of highly similar generated responses.
• Critical Evaluation in Diverse Contexts: Extending this model evaluation across varying domains and differing LLM implementations might offer insights on how LLMs can be adapted to reduce negative impacts in creativity-centric fields while capitalizing on their ability to support rapid ideation.

Conclusion

The findings articulate a nuanced understanding of how LLMs like ChatGPT influence creative outputs, highlighting a complexity in how they can empower creativity through volume and detail yet concurrently homogenize the creative landscape at a group level. This research is a critical dialogue in advancing both theoretical perspectives and practical methodologies for AI-mediated creativity, paving the way for more refined and beneficial human-AI collaborative creative practices. As technology evolves, continuous assessment and adaptation of these models will be essential in fostering authentic human creativity in conjunction with AI support systems.