Aether Weaver: Multimodal Narrative System
- Aether Weaver is a multimodal narrative framework that co-generates text, visuals, scene graphs, and affective soundscapes for consistent storytelling.
- It employs a dynamic scene graph and narrative arc controllers to maintain persistent world-state tracking and emotional alignment across modalities.
- The system demonstrates enhanced spatio-temporal and relational consistency compared to traditional sequential narrative generation pipelines.
Aether Weaver is a multimodal affective narrative co-generation framework for story generation in which narrative text, dynamic scene graphs, visual scenes, and affective soundscape cues are generated together rather than through a sequential text-to-visual cascade (Ghorbani, 29 Jul 2025). Its defining premise is that a narrative should be maintained as a dynamic multimodal world-state: the system uses a continuously updated scene graph as a structured representation of entities, relations, and attributes, while explicit narrative-arc and affective controllers condition subsequent generation so as to improve spatio-temporal consistency, relational continuity, and emotional coherence across modalities (Ghorbani, 29 Jul 2025). The framework was introduced as an integrated alternative to brittle pipelines in which text is produced first and visual or audio assets are added afterward, a design choice the authors associate with losses in world consistency and affective alignment (Ghorbani, 29 Jul 2025).
1. Conceptual definition and problem setting
Aether Weaver is presented as an integrated system for multimodal narrative co-generation whose outputs comprise four coordinated products: narrative text, dynamic scene graphs, visual scenes, and affective soundscape cues (Ghorbani, 29 Jul 2025). The central problem it addresses is a failure mode of conventional story-generation pipelines: when text is generated first and image or audio modules operate only downstream, the resulting multimodal artifact often lacks persistent world-state tracking, yielding inconsistencies in character placement, object state, inter-entity relations, and emotional expression (Ghorbani, 29 Jul 2025).
The framework therefore treats the story world as an evolving structured state rather than as an ephemeral by-product of the latest textual segment. This design is operationalized through a dynamic scene graph that functions as a central representation of the fictional environment and is updated after each generation step (Ghorbani, 29 Jul 2025). A plausible implication is that the framework is closer to stateful world modeling than to prompt-chaining alone, because subsequent text and rendering are conditioned on explicit, incrementally maintained structure rather than on unstructured context windows.
The paper characterizes the system as suitable for immersive storytelling, interactive fiction, and rapid creative prototyping (Ghorbani, 29 Jul 2025). Those use cases follow directly from the framework’s combination of co-generation, persistent state tracking, and user-intervenable narrative control.
2. System architecture and generation loop
The architecture is organized around four major components: the Narrator, the Narrative Arc Controller (NAC), the Affective Tone Mapper (ATM), and the Director (Ghorbani, 29 Jul 2025). These modules are coupled with a visual scene generator and a multimodal character grounding process (Ghorbani, 29 Jul 2025). The overall flow begins with user-provided story premise, arc choice, affective tones, character and setting profiles, and style preferences; NAC and ATM then provide high-level guidance; the Narrator generates the next narrative segment and soundscape cues; the Director parses that segment into a dynamic scene graph and maintains consistency; and the updated graph, together with the generated text and affective directives, informs visual generation and subsequent narrative steps (Ghorbani, 29 Jul 2025).
The Narrator is a pre-trained LLM that serves both as text generator and multimodal prompt composer (Ghorbani, 29 Jul 2025). For each story segment, it generates the next piece of narrative text, soundscape cues, and contextual prompts for later visual generation (Ghorbani, 29 Jul 2025). Its prompting is explicitly multi-turn and iterative: each segment prompt contains user inputs, a story-so-far summary, NAC outputs, ATM outputs, and the current scene graph state (Ghorbani, 29 Jul 2025). This means the LLM is not free-running; it is conditioned on a structured and changing representation of the story world.
The Narrative Arc Controller functions as a state machine over narrative beats such as Exposition, Rising Action, Climax, Falling Action, and Resolution (Ghorbani, 29 Jul 2025). The supported arcs listed in the paper include Classic Arc, Tragedy, Hero’s Journey, Rags to Riches, Quest, and Character Arc (Ghorbani, 29 Jul 2025). For each stage, NAC outputs the current stage name, a goal directive, and associated affective cues (Ghorbani, 29 Jul 2025). Its progression mechanism is ratio-based: each stage is assigned a minimum proportion of the total story length, and stage transitions occur once those boundaries are crossed (Ghorbani, 29 Jul 2025). For the Classic Arc, the approximate thresholds given are Exposition at minimum 5%, Rising Action at minimum 25%, Climax at minimum 65%, Falling Action at minimum 80%, and Resolution at minimum 90% (Ghorbani, 29 Jul 2025).
The Affective Tone Mapper converts emotional intent into multimodal directives (Ghorbani, 29 Jul 2025). It takes user-selected emotions, intensity levels, and optional narrative-arc affective cues, then produces a directive for the Narrator together with soundscape and visual cues (Ghorbani, 29 Jul 2025). Intensities are specified as Low, Medium, or High (Ghorbani, 29 Jul 2025). If enabled, ATM incorporates the current NAC stage and can down-modulate conflicting emotional tones; the paper gives as an example a tragic arc stage reducing a user-requested positive tone such as “Joy - High” to a weaker intensity (Ghorbani, 29 Jul 2025). For the genre label “Mystery,” the supplementary material specifies LLM modifiers focused on secrets, puzzles, and hidden truths, sound cues including low-frequency drones, whispers, and unresolved musical phrases, and visual cues such as low-key lighting, fog, obscured figures, and Dutch angles (Ghorbani, 29 Jul 2025).
The Director is the core coherence module and is described as an LLM-based Dynamic Scene Graph Manager (Ghorbani, 29 Jul 2025). Its role is to parse each new narrative segment, update the structured world model, and enforce cross-scene consistency (Ghorbani, 29 Jul 2025). In architectural terms, the Director is the mechanism that binds the textual, visual, and affective subsystems to a shared evolving state.
3. Dynamic scene graph formalism and world-state maintenance
The formal object used to represent story state is the scene graph
where denotes nodes representing entities such as characters, objects, and locations, denotes directed edges representing relationships such as “holds,” “inside,” and “near,” and denotes attributes attached to nodes or edges, such as “weary” or “flickering” (Ghorbani, 29 Jul 2025).
After the Narrator produces a segment, the Director applies a dynamic update pipeline consisting of Named Entity Recognition, Relation Extraction, Attribute Updates, Spatial Reasoning, and Consistency Enforcement (Ghorbani, 29 Jul 2025). Named Entity Recognition identifies new or existing characters, objects, and locations; Relation Extraction identifies actions, spatial relations, and causal links, often via dependency parsing and semantic role labeling; Attribute Updates revise state changes such as “tarnished” or “weary”; Spatial Reasoning tracks relations such as near, inside, and on; and Consistency Enforcement removes invalid relationships, updates outdated locations, and flags inconsistencies or sends corrective prompts (Ghorbani, 29 Jul 2025).
The graph is dynamically modified by adding nodes, removing nodes, adding or removing edges, updating attributes, and enforcing consistency after state changes (Ghorbani, 29 Jul 2025). The supplementary example contains locations including Whispering Woods and Forgotten Shrine, a character Elara, an object Elderwood, and an event Spirit Revelation, together with edges such as Elara Elderwood, Elderwood Whispering Woods, and Forgotten Shrine Whispering Woods (Ghorbani, 29 Jul 2025). This suggests that the graph is intended not merely as a static entity list but as an event-sensitive, relation-rich memory substrate for the unfolding narrative.
The paper emphasizes four specific benefits of this representation: spatio-temporal consistency, relational consistency, state continuity, and logical plausibility (Ghorbani, 29 Jul 2025). In the system description, these correspond to preventing unexplained teleportation or disappearance of characters, preserving object relations across scenes unless the story changes them, retaining updated conditions of characters and artifacts, and flagging conflicting state updates (Ghorbani, 29 Jul 2025). The distinction from sequential pipelines is explicit: instead of generating text, extracting some entities, and then rendering from a local textual slice, Aether Weaver feeds each new generation step with the current world model rather than only the immediately preceding sentence (Ghorbani, 29 Jul 2025).
4. Multimodal grounding: visuals, soundscapes, and affective control
The visual scene generator consumes the current narrative text, character appearance profiles, the scene graph, style preferences, and affective mood from NAC and ATM (Ghorbani, 29 Jul 2025). An LLM assembles these into a detailed prompt for an image generation model (Ghorbani, 29 Jul 2025). This prompt-construction stage is therefore conditioned jointly by semantic content, world-state structure, and affective direction.
A notable feature is multimodal character grounding (Ghorbani, 29 Jul 2025). If a portrait is provided by the user, it becomes the character’s anchor; otherwise, the system generates a full-body image from text, a vision model analyzes the image to extract detailed appearance traits, and those features are written back into the character profile (Ghorbani, 29 Jul 2025). When the character reappears, the stored attributes are injected into the image prompt (Ghorbani, 29 Jul 2025). The stated effect is improved visual consistency across scenes and reduced drift in appearance, clothing, or key features (Ghorbani, 29 Jul 2025). In practical terms, this establishes a stable visual-textual grounding loop rather than relying on repeated re-description.
Soundscape generation is not described through an explicit synthesis objective, but through cue generation tightly coordinated with the narrative and affective controllers (Ghorbani, 29 Jul 2025). The Narrator outputs soundscape cues for each segment, while ATM contributes emotion-conditioned audio directives (Ghorbani, 29 Jul 2025). The qualitative example of “growing dread” is mapped to low, rising dissonance, heartbeat-like audio, and darker visual framing (Ghorbani, 29 Jul 2025). This indicates a deliberate cross-modal alignment strategy: affect is not merely stated in text but translated into synchronized textual, auditory, and visual descriptors.
The system also supports direct user intervention through narrative-text or scene-graph editing and through plot-twist injection (Ghorbani, 29 Jul 2025). When edited text is saved, the Director re-analyzes it and updates the graph; newly introduced characters may be discovered and profiled (Ghorbani, 29 Jul 2025). Plot twists can be selected from generated suggestions or added directly by the user, after which they become new directives for the Narrator (Ghorbani, 29 Jul 2025). This interactive design positions the framework as a co-generation environment rather than a one-shot generation pipeline.
5. Evaluation protocol and reported results
The evaluation is qualitative and human-centered, using expert raters rather than an automated benchmark (Ghorbani, 29 Jul 2025). The prompt set comprises 10 narrative prompts spanning Sci-Fi, Fantasy, different character archetypes, different initial settings, and complex plot hooks (Ghorbani, 29 Jul 2025). For each prompt, the study manually assigned 1–3 affective tones with intensities, such as Mystery: High, Hope: Medium, and Loneliness: Low, together with a narrative arc such as Classic Arc, Tragedy, or Quest (Ghorbani, 29 Jul 2025).
The baseline is a sequential cascade: an LLM generates the full narrative, a separate module extracts entities with basic NER, those entities feed an image prompt generator, sound is generated from crude textual cues such as “ominous sounds,” and the system includes neither a dynamic scene graph nor an affective co-generation loop (Ghorbani, 29 Jul 2025). The evaluators consisted of 10 expert human judges identified as professional writers, game designers, and AI researchers with narrative-system experience (Ghorbani, 29 Jul 2025). Rating used a 5-point Likert scale with 1 = Poor and 5 = Excellent (Ghorbani, 29 Jul 2025).
| Metric | Aether Weaver | Baseline |
|---|---|---|
| Narrative Fluency | 4.62 | 4.21 |
| MM Coherence (Visual) | 4.23 | 2.85 |
| MM Coherence (Auditory) | 4.76 | 4.10 |
| Affective Alignment | 4.67 | 3.28 |
| Narrative Arc Adherence | 4.30 | 3.45 |
The reported result is that Aether Weaver outperforms the baseline on all metrics (Ghorbani, 29 Jul 2025). The largest gain appears in multimodal coherence on the visual side, which the paper attributes to Director-mediated tracking of entities, states, and relations (Ghorbani, 29 Jul 2025). Narrative fluency improves more modestly, with the interpretation that NAC and Director feedback make the narrative more logically consistent (Ghorbani, 29 Jul 2025). Auditory coherence improves because ATM generates emotionally and narratively aligned cues rather than generic descriptors, affective alignment improves because emotional tone is controlled throughout the story rather than only initialized at the start, and narrative arc adherence improves because NAC enforces beat progression (Ghorbani, 29 Jul 2025).
The qualitative findings reinforce the metric table. Evaluators reported better narrative depth, better visual fidelity, stronger emotional resonance, and better multimodal cohesion (Ghorbani, 29 Jul 2025). Examples of baseline failures reduced by Aether Weaver include forgotten objects, inconsistent object states, incorrect character locations, and pose or scene mismatches (Ghorbani, 29 Jul 2025). This suggests that the framework’s primary benefit lies less in raw text-generation quality than in cross-modal coordination under persistent world-state constraints.
6. Interpretation, scope, and technical significance
The technical significance of Aether Weaver lies in its treatment of narrative generation as a coordinated multimodal control problem rather than as a text-first generation task followed by downstream decoration (Ghorbani, 29 Jul 2025). Its main mechanisms are a shared dynamic state through the scene graph, explicit structural guidance through NAC, emotional direction through ATM, and iterative feedback of updated context into future generation (Ghorbani, 29 Jul 2025). In this respect, the framework combines world modeling, narrative planning, and affective conditioning into a single generation loop.
The paper is primarily architectural. It explicitly states that it does not present optimization losses or training objectives beyond the human-evaluation Likert-scoring setup, and the main formal notation provided is the scene graph definition (Ghorbani, 29 Jul 2025). A common misconception would therefore be to read the system as a new end-to-end trained multimodal foundation model. The description instead indicates an integrated orchestration framework composed of an LLM-based Narrator, an LLM-based Director, explicit controllers for story arc and affect, and a visual scene generator coupled through iterative prompting and graph maintenance (Ghorbani, 29 Jul 2025).
Another potential misconception is that the framework’s value resides only in stylistic enhancement. The reported gains are broader: the system is designed to improve spatio-temporal consistency, relational continuity, and emotional coherence, and the measured improvements include not only affective alignment but also narrative arc adherence and visual coherence (Ghorbani, 29 Jul 2025). At the same time, the evaluation remains qualitative and small-scale, involving 10 prompts and 10 expert evaluators (Ghorbani, 29 Jul 2025). A plausible implication is that the paper’s contribution is best understood as a systems demonstration of architectural integration rather than as a large-scale benchmark study.
Within multimodal storytelling, Aether Weaver is therefore notable for making the scene graph the central state variable of generation, for pairing that state with explicit narrative-arc scheduling and affective modulation, and for allowing user edits and plot-twist interventions inside the same control loop (Ghorbani, 29 Jul 2025). The framework’s stated application domains—interactive fiction, game prototyping, VR/AR storytelling, creative authoring tools, and emotionally guided narrative generation—follow from that integration of persistent structure, multimodal conditioning, and controllable co-generation (Ghorbani, 29 Jul 2025).
