StoryWeaver: A Unified World Model for Knowledge-Enhanced Story Character Customization

Abstract: Story visualization has gained increasing attention in artificial intelligence. However, existing methods still struggle with maintaining a balance between character identity preservation and text-semantics alignment, largely due to a lack of detailed semantic modeling of the story scene. To tackle this challenge, we propose a novel knowledge graph, namely Character Graph (\textbf{CG}), which comprehensively represents various story-related knowledge, including the characters, the attributes related to characters, and the relationship between characters. We then introduce StoryWeaver, an image generator that achieve Customization via Character Graph (\textbf{C-CG}), capable of consistent story visualization with rich text semantics. To further improve the multi-character generation performance, we incorporate knowledge-enhanced spatial guidance (\textbf{KE-SG}) into StoryWeaver to precisely inject character semantics into generation. To validate the effectiveness of our proposed method, extensive experiments are conducted using a new benchmark called TBC-Bench. The experiments confirm that our StoryWeaver excels not only in creating vivid visual story plots but also in accurately conveying character identities across various scenarios with considerable storage efficiency, \emph{e.g.}, achieving an average increase of +9.03\% DINO-I and +13.44\% CLIP-T. Furthermore, ablation experiments are conducted to verify the superiority of the proposed module. Codes and datasets are released at https://github.com/Aria-Zhangjl/StoryWeaver.

• The paper presents StoryWeaver, a novel AI model employing a Character Graph (CG) to significantly improve consistent characterization and text-to-image alignment in multi-character story visualization.
• The model's Character Graph and Knowledge-Enhanced Spatial Guidance mechanism improve identity preservation and semantic text alignment, achieving superior results on TBC-Bench like +9.03% DINO-I and +13.44% CLIP-T.
• This research expands AI's potential in creative industries like animation by automating high-accuracy story visualization and provides a framework for future character-driven visual storytelling.

An Analysis of StoryWeaver: A Unified Model for Knowledge-Enhanced Story Character Customization

StoryWeaver presents a significant advancement in the field of artificial intelligence, particularly within the domain of story visualization. The primary challenge addressed by this research is the consistent characterization and precise text-to-image alignment in story visualization tasks, where a balance between these elements has largely proved difficult to obtain with conventional methods. This paper introduces a novel methodology incorporating a Character Graph (CG) within the StoryWeaver model to enhance knowledge representation and application in multi-character visual storytelling.

The proposed Character Graph serves as a comprehensive repository of story-related knowledge, including character identities, associated attributes, and inter-character relationships. This rich, semantic structure allows for more detailed and accurate character representations, which surpasses the efficacy of simple token-based and contextual models employed in prior frameworks like IP-Adapter and Dreambooth.

The StoryWeaver model adopts a Customization via Character Graph (C-CG) approach, where image generation captures consistent story visuals grounded in rich text semantics. The Character Graph is constructed by embedding detailed semantic components—objects (characters), their attributes, and interactions—into the story world. In contrast to previous models, the incorporation of CG into StoryWeaver significantly improves identity preservation and semantic text alignment, ably demonstrated by superior performance metrics such as a +9.03% increase in DINO-I and a +13.44% improvement in CLIP-T.

Furthermore, the paper integrates a Knowledge-Enhanced Spatial Guidance (KE-SG) mechanism to optimize cross-attention distributions during image synthesis. This innovation addresses the issue of character identity blending by modifying attention maps within the diffusion model, ensuring that character-specific knowledge is accurately applied to corresponding visual regions. Such precision in feature representation is crucial for generating coherent and semantically aligned multi-character interactions—a notable improvement over conventional methods that often struggle with identity preservation and semantic fidelity in complex scenes.

Significant quantitative results on the newly proposed TBC-Bench further underscore the model's abilities. When benchmarked against leading approaches such as StoryGEN, Mix-of-Show, and LoRA-Composer, StoryWeaver consistently delivers enhanced character identity preservation and semantic representation across diverse story contexts.

The implications of this research are manifold. Practically, it expands the potential applications for AI in creative industries, such as animation and story-based content creation, by automating the visualization of narratives with high accuracy and detail. Theoretically, it offers a robust framework for future explorations in character-driven visual storytelling, suggesting potential extensions into dynamic and interactive storytelling domains.

Future research directions could explore the integration of temporal dynamics to further enhance story visualization, extending the model's capacities to handle evolving narratives in real-time. Additionally, fine-tuning the interplay between semantic constraints and character interactions within complex scenes could offer refined control over narrative coherence and visual storytelling fidelity. Overall, StoryWeaver represents a substantial progression in the field, laying a foundation for increasingly sophisticated AI-driven storytelling solutions.