2000 character limit reached
SWAG: Storytelling With Action Guidance (2402.03483v2)
Published 5 Feb 2024 in cs.CL and cs.AI
Abstract: Automated long-form story generation typically employs long-context LLMs for one-shot creation, which can produce cohesive but not necessarily engaging content. We introduce Storytelling With Action Guidance (SWAG), a novel approach to storytelling with LLMs. Our approach frames story writing as a search problem through a two-model feedback loop: one LLM generates story content, and another auxiliary LLM is used to choose the next best "action" to steer the story's future direction. Our results show that SWAG can substantially outperform previous end-to-end story generation techniques when evaluated by GPT-4 and through human evaluation. Our SWAG pipeline using only small open-source models surpasses GPT-3.5-Turbo.
- Automatic story generation: Challenges and attempts, 2021a.
- Goal-directed story generation: Augmenting generative language models with reinforcement learning, 2021b.
- Training a helpful and harmless assistant with reinforcement learning from human feedback, 2022.
- Language models are few-shot learners. CoRR, abs/2005.14165, 2020. URL https://arxiv.org/abs/2005.14165.
- Robust preference learning for storytelling via contrastive reinforcement learning, 2022.
- Learning to generate better than your llm, 2023.
- Charniak, E. Toward a model of children’s story comprehension. 10 2004.
- Longlora: Efficient fine-tuning of long-context large language models, 2023.
- Deep reinforcement learning from human preferences, 2023.
- Talebrush: Sketching stories with generative pretrained language models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pp. 1–19, 2022.
- Wordcraft: a human-ai collaborative editor for story writing, 2021.
- Dao, T. Flashattention-2: Faster attention with better parallelism and work partitioning, 2023.
- Alpacafarm: A simulation framework for methods that learn from human feedback. arXiv preprint arXiv:2305.14387, 2023.
- Hierarchical neural story generation, 2018.
- Plan, write, and revise: an interactive system for open-domain story generation. In Ammar, W., Louis, A., and Mostafazadeh, N. (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pp. 89–97, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-4016. URL https://aclanthology.org/N19-4016.
- Lora: Low-rank adaptation of large language models, 2021.
- Mistral 7b, 2023.
- Mixtral of experts, 2024.
- The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691, 2021.
- Plug-and-blend: A framework for plug-and-play controllable story generation with sketches. In Artificial Intelligence and Interactive Digital Entertainment Conference, 2021. URL https://api.semanticscholar.org/CorpusID:236470168.
- Decoupled weight decay regularization, 2019.
- Improvisational storytelling agents. In Workshop on Machine Learning for Creativity and Design (NeurIPS 2017), volume 8, 2017.
- Co-writing screenplays and theatre scripts with language models: An evaluation by industry professionals, 2022.
- Oatley, K. Book reviews: The creative process: A computer model of storytelling and creativity. Computational Linguistics, 21(4), 1995. URL https://aclanthology.org/J95-4007.
- OpenAI. Gpt-4 technical report. ArXiv, abs/2303.08774, 2023. URL https://arxiv.org/abs/2303.08774.
- Guiding neural story generation with reader models, 2022.
- Learning how to ask: Querying lms with mixtures of soft prompts. arXiv preprint arXiv:2104.06599, 2021.
- Direct preference optimization: Your language model is secretly a reward model, 2023.
- Plotmachines: Outline-conditioned generation with dynamic plot state tracking, 2020.
- Proximal policy optimization algorithms, 2017.
- Autoprompt: Eliciting knowledge from language models with automatically generated prompts, 2020.
- Controllable neural story plot generation via reward shaping. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-2019. International Joint Conferences on Artificial Intelligence Organization, August 2019. doi: 10.24963/ijcai.2019/829. URL http://dx.doi.org/10.24963/ijcai.2019/829.
- Llama 2: Open foundation and fine-tuned chat models, 2023.
- Zephyr: Direct distillation of lm alignment, 2023.
- Trl: Transformer reinforcement learning. https://github.com/huggingface/trl, 2020.
- Playing story creation games with large language models: Experiments with gpt-3.5. In International Conference on Interactive Digital Storytelling, pp. 297–305. Springer, 2023.
- Chain-of-thought prompting elicits reasoning in large language models. 2022. doi: 10.48550/ARXIV.2201.11903. URL https://arxiv.org/abs/2201.11903.
- A temporal variational model for story generation, 2021.
- A skeleton-based model for promoting coherence among sentences in narrative story generation, 2018.
- Judging llm-as-a-judge with mt-bench and chatbot arena, 2023.
- End-to-end story plot generator, 2023.
- Controllable generation from pre-trained language models via inverse prompting. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD ’21, pp. 2450–2460, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450383325. doi: 10.1145/3447548.3467418. URL https://doi.org/10.1145/3447548.3467418.
- Zeeshan Patel (5 papers)
- Karim El-Refai (5 papers)
- Jonathan Pei (3 papers)
- Tianle Li (25 papers)