Whose story is it? Personalizing story generation by inferring author styles (2502.13028v2)

Abstract: Personalization is critical for improving user experience in interactive writing and educational applications, yet remains understudied in story generation. We study the task of personalizing story generation, where our goal is to mimic an author's writing style, given other stories written by them. We collect Mythos, a dataset of 3.6k stories from 112 authors, with an average of 16 stories per author, across five distinct sources reflecting diverse story-writing settings. We propose a two-stage pipeline for personalized story generation: first, we infer authors' implicit writing characteristics and organize them into an Author Writing Sheet, which is validated by humans to be of high quality; second, we simulate the author's persona using tailored persona descriptions and personalized story rules. We find that stories personalized using the Author Writing Sheet outperform a non-personalized baseline, achieving a 78% win-rate in capturing authors' past style and 59% in similarity to ground-truth author stories. Human evaluation supports these findings and further highlights trends, such as Reddit stories being easier to personalize, and the Creativity and Language Use aspects of stories being easier to personalize than the Plot.

Personalizing Story Generation by Inferring Author Styles: A Technical Overview

The paper "Whose story is it? Personalizing story generation by inferring author styles" presents an innovative approach to personalized story generation by leveraging the individual styles of authors. The research introduces a two-stage pipeline designed to first infer and then emulate the unique story-writing characteristics of authors, thus advancing the role of LLMs in personalized content generation.

Objective and Methodology

The primary objective is to enhance the personalization of story generation by tailoring narrative outputs to mimic individual author styles, which could significantly benefit interactive writing environments. The proposed methodology involves a two-step process:

1. Author Writing Sheet Construction: This initial stage involves deriving a structured representation—termed the Author Writing Sheet—of an author's style by analyzing their past writings. This is accomplished by comparing author-written stories to average stories generated by LLMs in response to identical prompts. The comparison yields Claim-Evidence pairs that encapsulate distinct narrative dimensions such as plot structure, creativity, character development, and language use.
2. Personalized Story Generation: In the second stage, LLMs utilize the derived Author Writing Sheet to simulate the author's narrative style. This is achieved by integrating persona descriptions and specific storytelling rules into the story generation process, ensuring that the output conforms to the personalized style parameters set forth in the first stage.

Dataset and Evaluation

The paper introduces a new dataset consisting of 590 stories from 64 authors across platforms such as Reddit, AO3, Storium, Narrative Magazine, and the New Yorker. This dataset facilitates the evaluation of personalized story generation methods by offering diverse writing settings and thematic varieties.

The effectiveness of the personalization process is assessed through both automatic and human evaluations. Automatic evaluations employ an LLM-as-a-judge framework to compare generated stories against the original author stories, focusing on dimensions like faithfulness to the author's historical writing style and overall similarity to the author's typical narrative output. Human evaluation further substantiates these findings, revealing that personalized story generation, particularly using the Author Writing Sheet, offers significant advantages over non-personalized baselines. Notably, the research finds that narrative categories such as creativity and language use benefit more from personalization than plot and character development.

Implications and Future Directions

The paper's contributions are multi-faceted. First, it extends the capabilities of LLMs by embedding personalization into story generation, thus enhancing the utility of AI in creative disciplines and educational tools. Second, it proposes a novel framework—Author Writing Sheet—that can potentially be adapted for other personalized text generation applications where individual writing styles play a critical role.

Despite the promising results, the research acknowledges limitations, mainly the relatively small size of the dataset due to data crawling policies and the model-dependence of persona descriptions. Future research could focus on expanding dataset size, enhancing model cross-compatibility for persona descriptions, and developing advanced multi-agent systems for more granular control over narrative generation.

Conclusion

Overall, this paper represents a significant stride in personalizing AI-driven narrative generation to match specific author styles. By systematically incorporating unique story-writing characteristics, it opens new avenues for AI applications in creative writing, providing a foundation for future developments in personalized content generation. The integration of narrative theory into computational models marks an evolution in how machines can effectively engage with nuanced human creativity.