Idea2Story: Structured Narrative Generation

• Idea2Story is a research framework that uses explicit intermediate representations (e.g., dialogue history, event graphs, JSON schemas) to guide narrative generation.
• It converts diverse inputs like conversational turns, images, and code into structured outputs including prose, storyboards, and scientific narratives through iterative planning.
• Empirical studies show improved iteration, coherence, and control in narrative generation across fiction, educational, and scientific applications.

Searching arXiv for the cited Idea2Story-related papers to ground the article and confirm metadata. Idea2Story denotes a family of research systems that transform underspecified inputs into stories through explicit intermediate structure rather than direct long-form generation. Across the literature, the input may be conversational turns, trope seeds, event graphs, images, code, structural articles, or research intents, and the output may be prose, storyboards, web stories, comics, illustrated books, or scientific narratives. A recurring pattern is to externalize narrative state—through dialogue history, event graphs, JSON schemas, trope canvases, executable descriptions, or methodological knowledge graphs—and then use that state to guide generation, iteration, and revision (Burtenshaw, 2020, Le et al., 15 Jun 2026, Liang et al., 2024, Xu et al., 28 Jan 2026).

1. Conceptual scope and recurring design pattern

In its broadest research usage, Idea2Story refers to systems that bridge an initial idea and a narrative artifact by inserting one or more intermediate representations. This is explicit in systems that describe themselves as “idea-to-story” or are analyzed as such: AI Stories is a conversational “idea-to-story” machine in which each child utterance becomes a new idea-injection into an evolving story world; GraphStory is described as targeting the stage of turning early, messy ideas into structured, coherent stories; StoryDiffusion is characterized as an “Idea2Story2Storyboard” pipeline; and Idea2Story in autonomous scientific discovery converts underspecified research intents into complete scientific narratives by aligning them with reusable research patterns (Burtenshaw, 2020, Le et al., 15 Jun 2026, Liang et al., 2024, Xu et al., 28 Jan 2026).

A recurring position in this literature is that Idea2Story is not adequately modeled as “prompt → wall of text.” AI Stories treats story creation as a continuous, chat-like process; GraphStory focuses on structuring and connecting ideas rather than only generating prose; TaleFrame decomposes story structure into entities, events, relationships, and story outline; and StoryDiffusion emphasizes back-and-forth movement between narrative iteration and image generation (Burtenshaw, 2020, Le et al., 15 Jun 2026, Wang et al., 2 Dec 2025, Liang et al., 2024).

System	Intermediate representation	Output orientation
AI Stories	Dialogue history and active topics	Conversational narrative world
GraphStory	Chunks, ordered events, directed edges	Branching prose generation
TaleFrame	Entities, events, relationships, story outline	JSON2Story text generation
StoryDiffusion	Scene sequence plus style and prompt parameters	UX storyboard frames
Idea2Story	Method units, research patterns, methodological knowledge graph	Scientific narratives

This pattern suggests that Idea2Story is best understood as a control architecture for narrative generation: the system first organizes the idea into manipulable structure, then realizes that structure as narrative text or multimodal story output.

2. Dialogic and socially distributed Idea2Story

AI Stories implements Idea2Story as dialogue. A child types or speaks something, the system routes the utterance through several subsystems, a selector evaluates candidate responses, and the chosen response becomes the next turn of the story-dialogue. Over many turns, the dialogue history forms a loose, evolving story world. The main components are a topic-based question answering subsystem grounded in external knowledge like Wikipedia, a context-based seq2seq neural dialogue subsystem, a template-based poetry and humor subsystem, and a selector that uses reinforcement learning with hand-coded lexical rules. The selector is framed as a simplified POMDP $(S, A, R)$, where $S$ is previous dialogue turns, $A$ is the set of candidate responses, and $R(s,a)$ is defined as the similarity between the chosen response and the actual next sentence in training dialogues; the selector learns a Q-function $Q(s,a)$ and chooses the candidate with highest estimated $Q(s,a)$, subject to lexical constraints (Burtenshaw, 2020).

This dialogic formulation replaces explicit plot planning with local responsiveness, context memory, topic continuity, and selector optimization for “longer consistency.” Story state is represented implicitly as current dialogue history plus active topics, and the system explicitly embraces “chatty and nonsensical” language as part of children’s language play. The resulting story is performative, temporal, and social rather than static and linear (Burtenshaw, 2020).

Heteroglossia distributes Idea2Story across a crowd rather than across model subsystems. Embedded inside Google Docs, it allows a writer to select a snippet of a working draft, choose a team of characters, and solicit follow-up story ideas from MTurk workers. Its distinctive strategy is role play: each worker is assigned a fictional character and asked to brainstorm from that character’s perspective. In the main experiment using 14 stories from the Creative Help corpus, role-play ideas were rated as less relevant than no-role ideas—No-role mean relevance was $3.998$ and Role mean relevance was $3.869$—and automatic semantic distance measures also placed role-play ideas farther from the prompt, supporting the claim that role play induces more semantically distant follow-up ideas at moments of impasse (Huang et al., 2020).

Collective Story Writing through Linking Images externalizes the same process in an image graph. An Image Chain is “a sequence of images which depicts a flow of narrative,” beginning from a Base Image and extended by crowd workers who upload or select further images. Users can write multiple stories for the same chain and vote on alternatives; duplicate creation of an already existing chain is treated as an increase in votes rather than a new chain. In experiments with 25 contributors, users showed interest in growing shorter Image Chains but voting longer Image Chains: the difference in extension behavior across length groups had $p = 0.0086$, and longer Image Chains obtained significantly higher numbers of votes with $p = 0.0366$ (Mandal et al., 2018).

3. Externalized structure as the main control surface

A major branch of Idea2Story research replaces free-form prompting with explicit narrative data structures. GraphStory treats the story as a graph whose nodes are macro-level chunks and whose micro-level contents are ordered event lists. For each chunk,

$S$0

and directed edges connect chunks into alternative flows. The Event-Graph Constructor accepts abstract ideas, structured outlines, or complete stories and converts them into a multi-resolution event graph; the Story Generator then performs two-stage event refinement—first intra-chunk, then inter-chunk—before feeding the selected path plus style, tone, and theme to GPT‑4o. The system’s stories, flows, and versions mechanism makes the graph, rather than the latest prose output, the primary artifact (Le et al., 15 Jun 2026).

TaleFrame adopts a stricter schema. It decomposes story structure into four basic units: entities, events, relationships, and story outline. Entities have fields such as entity_name, entity_identity, entity_motivation, and personality_traits; events include event_time, event_location, event_details, event_importance, earlier_event, and later_event; relationships specify included entities, emotional type, action type, action direction, relationship strength, and relationship evolution; and the outline stores title, story description, and beginning–middle–climax–ending structure. The system fine-tunes a local Llama model on 9,851 JSON-formatted preference entries and uses JSON2Story to transform structured data into coherent stories (Wang et al., 2 Dec 2025).

TaleStream proposes tropes as an intermediate representation for story ideation. It models a story concept as a canvas of trope cards, text cards, movie cards, and image cards, and computes trope suggestions from trope–index relations and trope–movie co-occurrences. Its semantic similarity for multiple trope inputs is defined as

$S$1

while co-occurrence similarity incorporates description tropes and movie overlap. In evaluation, trope suggestions generated by its methods provided better story ideation materials than random tropes 97% of the time for the index-based method (Chou et al., 2023).

STORYTELLER formalizes plot planning through linguistically grounded triplets. Its node definitions are

$S$2

and

$S$3

These nodes are organized as Chapter Begin Nodes, Chapter Plot Nodes, and Chapter End Nodes, while a Neo4j-based narrative entity knowledge graph continuously tracks characters, locations, objects, and their relations. The framework generates a Pseudo CPN, retrieves relevant context from the graph, reviews the candidate for logic, theme, emotion, mystery, or redundancy issues, and then accepts or revises it before text generation (Li et al., 3 Jun 2025).

4. Multimodal Idea2Story: storyboarding, illustration, and interactive visual narrative

StoryDiffusion turns UX ideas into storyboards by combining GPT‑4 and Stable Diffusion in a single pipeline. A designer may begin with a few words or a full narrative, after which GPT‑4 performs Story-to-Style extraction, narrative segmentation into scenes, and Story-to-Prompt parameterization with fields such as General description, Object, Person, Action, Emotion, Background, Style, and Shot. Stable Diffusion then renders each scene as a 512×512 image, about 1 second on an RTX 4070. The system supports both AI-directed and user-directed workflows and found that concept ideation and concept illustration lead to different strategies and preferences (Liang et al., 2024).

ImageTeller approaches Idea2Story from images rather than text. It uses a multi-agent architecture consisting of a Visual Analyzer AI Agent, Storywriter AI Agent, Illustrator AI Agent, and Plot Manager. Genre is represented as

$S$4

and prompt composition is made explicit: $S$5 or, in other modes, by replacing $S$6 and $S$7 with $S$8 as required. GPT‑4o Vision describes each image, GPT‑4o generates a markdown story with a title and chapters, and Juggernaut XL produces chapter illustrations from optimized prompts (Lima et al., 2024).

S2ED addresses story-to-image consistency by compiling a full story into per-frame executable descriptions. For each frame $S$9, it defines

$A$0

where $A$1 is the character registry, $A$2 appearance attributes, $A$3 layout, and $A$4 affect. Three agents segment the story, ground character identity, and enrich spatial and emotional cues; prompt-carried state propagation lets each frame description $A$5 inherit commitments from $A$6. This makes state interpretable and editable and allows local repair of drift without retraining the image generator (Yin et al., 21 May 2026).

TaleCrafter implements a four-stage visualization pipeline—story-to-prompt generation, text-to-layout generation, controllable text-to-image generation, and image-to-video animation—for multiple novel characters. Its controllable image model conditions on prompt embeddings, layout embeddings, optional sketch embeddings, and actor-specific LoRA identifiers, while sequential inpainting composes multiple personalized characters in one image. The system is explicitly interactive: prompts, layouts, sketches, and individual characters can all be revised (Gong et al., 2023).

FairyTailor and TaleForge extend multimodal Idea2Story toward human-in-the-loop fairytale co-creation and identity-centered personalization. FairyTailor combines fine-tuned GPT‑2, CLIP-based image retrieval from 2M Unsplash images, fast neural style transfer, and a web canvas in which users accept, edit, or delete generated text and images; TaleForge uses Llama 3 for story and metadata generation, StoryMaker and InstantID for personalized character images, DALL·E 3 and DreamBooth for background generation, and bounding boxes plus SAM and SwapAnything for composition, so that the user appears as protagonist in both narrative and illustrations (Bensaid et al., 2021, Nguyen et al., 27 Jun 2025).

5. Domain-specific Idea2Story systems

In autonomous scientific discovery, Idea2Story is the explicit name of a pre-computation-driven framework that shifts literature understanding from online reasoning to offline knowledge construction. It continuously collects accepted NeurIPS and ICLR papers from the last three years together with review artifacts, anonymizes and safety-filters them, extracts core methodological units, embeds papers from those units, reduces dimensionality with UMAP, clusters with DBSCAN into research patterns, and organizes canonicalized method units and meta-methods into a methodological knowledge graph $A$7. At runtime, a user idea $A$8 is aligned to patterns by multi-view retrieval: $A$9 with idea-level, domain-level, and paper-level scores, after which a review-guided refinement loop produces a structured research pattern rather than open-ended speculative prose (Xu et al., 28 Jan 2026).

CodeToon treats source code as the input idea and educational comic strips as the narrative output. Python code is parsed into an AST, transpiled into a story AST that becomes a line-aligned story template, and then mapped into a comic AST based on Visual Narrative Grammar categories such as Establisher, Initial, Prolongation, Peak, and Release. Variables are mapped to nouns or noun phrases, assignment to verbs or verb phrases, and values to contextualized story values; object imagery is drawn from 345 Quick, Draw! categories. The resulting pipeline is code → story template → human-authored story → auto-generated comic (Suh et al., 2022).

Wiki2Story begins from a structural article rather than a fiction idea. It parses a Wikipedia article into a section tree, summarizes sections with PEGASUS, and decides whether to create a single compact story or a multi-path set of web stories. If the number of content sections $R(s,a)$0 satisfies

$R(s,a)$1

with $R(s,a)$2 pages per story, it creates a compact story; otherwise it creates a Main Story plus Section Stories. MERGE and SPLIT heuristics allocate subsection summaries or section-level summary segments to pages, and text–image pairing is done by embedding similarity over WIT images and section summaries (Nkemelu et al., 2023).

These systems show that Idea2Story is not confined to fictional prose. It can be specialized for scientific narratives, code-driven educational stories, and mobile web-story conversion while retaining the same basic principle: explicit intermediate structure mediates between the initial idea and the final narrative artifact.

6. Empirical findings

GraphStory reports one of the clearest quantitative demonstrations that an event-graph interface changes the ideation experience. In a study with 16 students, GraphStory was significantly better than ChatGPT on Ease of Iteration ($R(s,a)$3), Efficiency in Providing Useful Plot Points ($R(s,a)$4), Efficiency in Task Completion ($R(s,a)$5), and Comfort and Naturalness ($R(s,a)$6); GraphStory-only functionality ratings included $R(s,a)$7 for “Graph view gives a comprehensive overview” and $R(s,a)$8 for “Iterating across versions and flows is easy” on a 7-point scale (Le et al., 15 Jun 2026).

TaleFrame reports that its Full Units model achieved the highest scores among ablations in Functionality (3.91), Technical (3.95), Readability (3.97), Thoughtfulness (3.54), Emotional (3.77), and Clarity (3.88), and user study means reached 4.74 for one framework-construction item and 4.58 for one narrative-visualization item on 5-point scales. Storyteller reports an 84.33% average win rate through human preference evaluation and, in automatic comparative scoring against GPT‑4o as reference, records 74.7 for Creativity, 72.8 for Coherence, 66.5 for Engagement, 77.6 for Relevance, and 89.4 for Overall (Wang et al., 2 Dec 2025, Li et al., 3 Jun 2025).

Multimodal systems report both usability gains and substantial consistency improvements. StoryDiffusion’s satisfaction ratings were 6.67 for Concept Ideation and 6.88 for Concept Illustration, while TaleForge’s user study found Story–Concept Alignment around 4.25 for Tasks 1, 2, and 4, Story Engagement up to 4.42, but Character Integrity falling to 2.92 and Visual Naturalness to 2.83 in the multi-character setting. S2ED’s human evaluation showed mean opinion scores of 4.4 for character consistency, 4.5 for story relevance, and 4.3 for visual quality, and pairwise preferences of 82%, 78%, and 74% respectively (Liang et al., 2024, Nguyen et al., 27 Jun 2025, Yin et al., 21 May 2026).

CodeToon reports a System Usability Scale of 81.0 for the full system, compared with 78.5 for the baseline, and significantly shorter mean comic-authoring time than the baseline (11:45 vs 18:35). In the second study, CodeToon comics were rated significantly better than baseline comics on overall mapping accuracy, code execution accuracy, code semantic accuracy, code illustration quality, concept illustration quality, usefulness for learning, and usefulness for teaching (Suh et al., 2022).

Not all evidence is quantitative. AI Stories reports no formal user studies or quantitative results and states that early prototypes “produce interesting forms of dialogue that reflect the multi-tonal tendencies of human conversation” but are “not enough to be called dialogue nor narrative” yet. Heteroglossia’s deployment with two experienced writers found the tool easy to use and capable of generating interesting ideas, with median latency of 14.42 minutes to the first idea per task and 53.32 minutes to at least one idea per character (Burtenshaw, 2020, Huang et al., 2020).

7. Limitations, misconceptions, and open directions

A common misconception is that Idea2Story research is primarily about long-form text generation. The literature repeatedly points elsewhere. AI Stories lacks a single plot planner and instead relies on context memory, topic grounding, and selector optimization; GraphStory targets “structuring and connecting ideas” rather than only generating prose; TaleFrame insists on structured information and JSON2Story; StoryDiffusion emphasizes the interchange between narrative iteration and image generation; and S2ED makes prompt-carried state explicit and editable (Burtenshaw, 2020, Le et al., 15 Jun 2026, Wang et al., 2 Dec 2025, Liang et al., 2024, Yin et al., 21 May 2026).

The main technical limitation is still long-horizon coherence. AI Stories acknowledges outputs that are “not enough to be called dialogue nor narrative,” GraphStory notes non-deterministic generation and imperfect graph–narrative alignment, TaleFrame currently supports only linear temporal evolution and lacks explicit causal relationships, and STORYTELLER still falls short of full novel-level depth despite strong gains in coherence and relevance (Burtenshaw, 2020, Le et al., 15 Jun 2026, Wang et al., 2 Dec 2025, Li et al., 3 Jun 2025).

Multimodal systems face persistent identity, layout, and safety problems. StoryDiffusion reports continuity issues and difficulty visualizing nuanced actions such as “shaking the phone”; ImageTeller explicitly identifies future work on character trait consistency across illustrations; TaleForge reports multi-character inconsistency, visual artifacts, and the absence of explicit privacy mechanisms for user faces; TaleCrafter notes dependence on Stable Diffusion v1.4 and the burden of manual sketches; S2ED documents multi-entity interference and attribute leakage (Liang et al., 2024, Lima et al., 2024, Nguyen et al., 27 Jun 2025, Gong et al., 2023, Yin et al., 21 May 2026).

Collaborative and crowd-mediated systems raise different issues. Heteroglossia shows that role play increases semantic distance but can reduce legitimacy and willingness-to-read scores relative to no-role ideation; Image Chain collaboration needs content filtering, better recommendation than vote-only ranking, and stronger incentives than a leaderboard (Huang et al., 2020, Mandal et al., 2018).

The scientific-discovery version of Idea2Story adds a separate concern: bias toward existing paradigms. Because its knowledge graph is built from accepted NeurIPS and ICLR papers plus reviews, the framework inherits the limitations of accepted literature and may force very novel ideas into existing patterns. The paper therefore presents the framework as a paradigm demonstration rather than a full solution for experimental automation or result generation (Xu et al., 28 Jan 2026).

Taken together, these systems indicate that Idea2Story is less a single algorithm than a research program centered on narrative state representation, controllable transformation, and iterative realization. The open problems—causal planning, scalable visualization, explicit safety layers, finer narrative editing, robust multi-character consistency, and stronger evaluation beyond surface fluency—remain central to its development across fiction, education, multimodal design, and scientific reasoning.