AniMaker Framework: Automated Story Animations

• AniMaker Framework is a multi-agent system designed for generating cohesive animated stories from textual input using modular agents and MCTS-driven clip selection.
• It combines specialized agents for storyboarding, clip generation, evaluation, and post-production to ensure narrative fidelity and production efficiency.
• Empirical benchmarks show AniMaker's superior performance in scene coherence, image quality, and text-video alignment compared to previous generation systems.

AniMaker is a multi-agent animation generation framework targeting automated creation of story-coherent, multi-scene animated videos from textual input. Developed to overcome the limitations of earlier framewise or purely keyframe-based video generators, AniMaker introduces a modular pipeline combining generative modeling, multi-candidate clip selection, and story-level evaluation. By structuring the process into specialized "agents" and employing a Monte Carlo Tree Search-inspired strategy for candidate curation, AniMaker achieves globally consistent, narratively faithful animated stories with computational efficiency suitable for production pipelines (Shi et al., 12 Jun 2025).

1. System Architecture

AniMaker implements a four-agent modular architecture that mirrors a traditional animation studio workflow:

• Director Agent: Parses the input story text $T$ using a LLM (LLM, e.g. Gemini 2.0 Flash) to generate an ordered list of shots $\{\mathrm{shot}_k\}_{k=1}^{N_\mathrm{clips}}$, with each shot described by $(d_k, C_k, B_k)$—the description, character set, and background. Transition indices $K_\mathrm{cut}$ signal when a new keyframe is needed. This agent synthesizes keyframes $F_k^{\mathrm{key}}$ through visualization prompts to generative models (e.g., GPT-4o), assembling character banks (from Hunyuan3D) and background banks (from FLUX).
• Photography Agent: Transforms storyboards into candidate video clips. Employing Wan 2.1 as the base generator, it wraps clip synthesis inside an efficient MCTS-Gen strategy, pruning and expanding the candidate tree to balance computational cost and output diversity.
• Reviewer Agent: Implements AniEval, a context- and story-aware multi-shot evaluation protocol. It evaluates generated clip candidates on 14 granular sub-scores grouped into Overall Video Quality, Text-Video Alignment, Video Consistency, and Motion Quality, leveraging immediate context to assess transitions and global coherence.
• Post-Production Agent: Finalizes the animation by generating and synchronizing the script-based voiceover (Gemini 2.0, CosyVoice2), adds subtitles, and assembles the final video using MoviePy.

2. Monte Carlo Tree Search-Driven Clip Generation

The MCTS-Gen module in the Photography Agent addresses the instability and inconsistency prevalent in generative video models by efficiently exploring a tree of possible continuation clips:

• Node Definition: Each node represents a prefix sequence $(v_1, ..., v_k)$, maintaining $N_\mathrm{visits}$, $Score_\mathrm{sum}$, and its children.
• Exploration-Exploitation: Selection is governed by a UCT-inspired formula:

$$\mathrm{UCT}(j) = \frac{2.0}{\mathrm{rank}(j)+1} + \alpha \sqrt{\frac{2.0}{\mathrm{child\_count}(j)+1}}$$

• Algorithmic Flow: Beginning at root, $w_1$ initial candidates are spawned and evaluated via AniEval. Over $w_2$ further iterations, the tree is selectively expanded at promising nodes, with scores backpropagated.
• Efficiency: In empirical studies, $w_1 = 3$, $w_2 = 3$ yields an average of 4.37 generations per shot, compared to 9 for exhaustive k-ary exploration, reducing compute by approximately 51% (Shi et al., 12 Jun 2025).

3. Story-Aware Evaluation: AniEval

AniEval systematically scores candidate clips and sequences with respect to both local and global animation qualities:

• Overall Video Quality: Aesthetic (VQA_Aesthetic), technical (VQA_Technical), and frame-level (MusIQ) measures.
• Text-Video Alignment: Consistency between description and generated video, via cosine similarity of CLIP embeddings and BLEU score of BLIP-captioned frames.
• Video Consistency: Perceptual similarity across frames (DreamSim), face tracking invariance (FaceCons), warping error, and semantic continuity.
• Motion Quality: Action correctness (Action Recognition), strength (Action Strength), and qualitative matching of motion vectors.
• Contextual Scoring: By redefining clip score as $$S_\mathrm{AniEval}(v_k) = \frac{1}{3}(S(v_{k-1}) + S(v_k) + S(v_{k+1}))$$, the system penalizes abrupt discontinuities and incomplete transitions.

4. End-to-End Pipeline

AniMaker's workflow is characterized by sequential agent-driven operations:

1. Text Parsing: Story text $T$ is segmented into shot list and visual keyframes.
2. Clip Tree Generation: Each shot is expanded via MCTS-Gen, exploring a bounded number of video candidate paths.
3. Evaluation and Selection: Clips are scored with context by AniEval; the highest-ranked prefix is extended iteratively.
4. Post-Production: Audio and subtitles are synthesized and synchronized, and the final artifact is assembled.

A summary of modular responsibilities is presented in the following table.

Agent	Primary Function	Technical Backbone
Director	Script & storyboard construction	LLMs + visual models
Photography	Multi-path video clip generation	Wan 2.1, MCTS-Gen
Reviewer	Multi-modal evaluation (AniEval)	Feature extractors
Post-Production	Audio, subtitles, and final assembly	TTS, MoviePy

5. Quantitative Results and Benchmarks

AniMaker's performance is benchmarked on both standard and framework-specific metrics:

• Scene-Level Coherence (CLIP similarity): 0.81 (best among evaluated systems).
• Image–Image Similarity: 0.83.
• Text–Image Similarity: 0.31 (19.2% improvement over the best baseline).
• VBench Video Metrics: Highest average rank (2.50), with superior scores in Image Quality (76.96), Semantic Consistency (84.27), Background Consistency (89.06), and Motion Smoothness (98.50).
• AniEval Aggregate Score: Total 76.72 (14.6% above next-best system), including gains in video consistency (15.5% over baseline).
• Human Evaluation: Across criteria (character consistency, narrative coherence, script faithfulness, visual appeal), AniMaker recorded 3.22/5 versus 2.07 for the next best system (Shi et al., 12 Jun 2025).

6. Limitations and Extensions

AniMaker's dependency on state-of-the-art video generation models imposes key limitations, such as persistent failure modes in low-quality clip generation and potential weaknesses in long-range temporal coherence if the search width ($w_1$, $w_2$) is reduced. The current evaluation loop is limited to agent-based, not user-in-the-loop, correction. Extended support for cross-modal prompts (e.g., integrating storyboards or sketches) and deeper user guidance could be incorporated as future directions—several of which are explored in related systems such as Manimator for STEM visualization (P et al., 18 Jul 2025) and Sketch2Anim for storyboard-to-3D motion translation (Zhong et al., 27 Apr 2025).

A plausible implication is that further modularizing the refinement process and integrating interactive or iterative feedback could extend AniMaker's applicability to a broader set of creative and educational content domains.

7. Cross-System Comparisons and Research Trajectory

AniMaker distinguishes itself from contemporaneous frameworks by combining global search-based candidate curation (MCTS-Gen) with explicit context-aware evaluation (AniEval), optimizing for story-level narrative integrity and action completeness under practical compute budgets. This contrasts with animation systems specializing in either LLM-to-code (Manimator) (P et al., 18 Jul 2025), storyboard-to-3D motion (Sketch2Anim) (Zhong et al., 27 Apr 2025), or user-guided video–3D retargeting (VidAnimator) (Ye et al., 3 Aug 2025).

The architecture and methodology underlying AniMaker set a research direction towards automated, semantically driven animation pipelines capable of global narrative planning and per-clip quality assurance, scalable to heterogeneous visual and narrative domains.