Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals (2209.14958v1)

Abstract: LLMs are increasingly attracting interest from writers. However, such models lack long-range semantic coherence, limiting their usefulness for longform creative writing. We address this limitation by applying LLMs hierarchically, in a system we call Dramatron. By building structural context via prompt chaining, Dramatron can generate coherent scripts and screenplays complete with title, characters, story beats, location descriptions, and dialogue. We illustrate Dramatron's usefulness as an interactive co-creative system with a user study of 15 theatre and film industry professionals. Participants co-wrote theatre scripts and screenplays with Dramatron and engaged in open-ended interviews. We report critical reflections both from our interviewees and from independent reviewers who watched stagings of the works to illustrate how both Dramatron and hierarchical text generation could be useful for human-machine co-creativity. Finally, we discuss the suitability of Dramatron for co-creativity, ethical considerations -- including plagiarism and bias -- and participatory models for the design and deployment of such tools.

Exploring Co-Writing Screenplays and Theatre Scripts with LLMs: A Structured Assessment with Industry Experts

The paper "Co-Writing Screenplays and Theatre Scripts with LLMs: An Evaluation by Industry Professionals" investigates the use of LLMs in co-creative writing, specifically for generating longform creative texts such as screenplays and theatre scripts. This paper aims to address the limitations of LLMs in maintaining long-range semantic coherence, which has been a hindrance in their broader application to creative tasks.

The authors present Dramatron, a hierarchical co-creative system that combines prompt chaining with structured generation to produce coherent screenplays. This involves breaking down the narratives into hierarchical layers, starting from a user-provided log line, and progressing through titles, character descriptions, plot outlines, location descriptions, and dialogue generation. The method attempts to surpass earlier models that struggle with long sequences due to limited context windows.

A distinctive aspect of the paper is the engagement with theatre and film professionals for evaluation, marking a departure from the common practice of relying on crowd-sourced evaluations in NLP tasks. The participating experts, who have experience both in the use of AI writing tools and the creative industry, provided qualitative feedback through interviews and a survey, focusing on Dramatron's usability, capabilities, and potential applications.

Numerical analysis of the feedback revealed several key findings:

• Collaboration: Participants generally felt a collaborative interaction with the AI system, indicating Dramatron's effectiveness for co-writing screenplays.
• Surprise and Uniqueness: The AI-generated scripts were often found to be surprising and unique, aligning with creative exploration and ideation.
• Assistance and Creativity: Dramatron was recognized as a helpful tool for expressing creative goals and overcoming writer's block, providing inspiration and alternative creative decisions.

However, experts also noted areas for improvement. The AI's outputs occasionally contained repetitive dialogues, lacked character motivation, exhibited biases, and demonstrated a literal interpretation style. These issues highlight the broader challenges of LLM-generated content in reflecting nuanced, human-like storytelling.

The paper discusses the practical and theoretical implications of using Dramatron in creative industries:

• Practical Implications: Dramatron could be positioned as a supplemental tool in writers' rooms for generating and refining story ideas, particularly in formulaic genres like television series.
• Theoretical Implications: The approach propels the understanding of hierarchical generation as a means to tackle long-range coherence. Although rooted in Western narrative structures, it opens up investigation into diverse narrative compositions and styles for AI models.

The authors acknowledge ethical considerations such as bias and potential plagiarism, advocating for transparency about AI's role in the creative process. Future developments could address these ethical concerns, enhance character and genre modeling, and improve narrative coherence by generating complex character arcs and integrating narrative subtexts within scenes.

In conclusion, this paper not only evaluates the potential of AI-assisted writing in film and theatre but also offers insights into human-AI collaboration for creative pursuits. It sheds light on the iterative and participatory nature of developing AI tools suited for artistic fields, suggesting an evolving synergy between human creativity and machine assistance.