A²Bench: Animated Anthropomorphic Benchmark

• The paper introduces A²Bench, a novel benchmark for evaluating image-to-video animation on 500 non-human, anthropomorphic characters with diverse stylizations.
• A²Bench is constructed using a synthetic pipeline with GPT-4o and KLing AI, categorizing data into three difficulty levels to probe generalization gaps in human-only datasets.
• Animate-X++ demonstrates robust performance on A²Bench with high PSNR*, SSIM, and user study preferences (e.g., 98.5% for identity preservation), underscoring the benchmark’s diagnostic value.

Searching arXiv for the benchmark and closely related papers. The Animated Anthropomorphic Benchmark, written as A²Bench, is introduced in "Animate-X++: Universal Character Image Animation with Dynamic Backgrounds" (Tan et al., 13 Aug 2025) as an anthropomorphic character benchmark for image-to-video evaluation. It is defined as an Animated Anthropomorphic character Benchmark ($A^2$Bench), which includes 500 anthropomorphic characters along with corresponding dance videos, to evaluate the performance of Animate-X++ on other types of characters. Within that work, A²Bench addresses a gap left by human-centric animation datasets by centering non-human anthropomorphic characters, stylized visual regimes, and cases in which pose extraction is unreliable or impossible (Tan et al., 13 Aug 2025).

1. Definition and motivation

A²Bench is motivated by the claim that the main public animation datasets used by prior work, including TikTok dance data and UBC Fashion, are almost exclusively human figures with human anatomy and mostly static backgrounds, and are therefore insufficient for evaluating methods on non-human anthropomorphic characters, stylized or highly non-realistic characters, and cases where pose extraction is unreliable or impossible (Tan et al., 13 Aug 2025). The benchmark is consequently positioned as a standardized evaluation set for anthropomorphic character animation and as a stress test for universal animation frameworks that claim to animate “any character X” beyond humans.

The benchmark’s target domain is explicitly anthropomorphized characters rather than ordinary human actors. The motivating examples include cartoons, game mascots, personified objects, and animals, often with non-human body structures, strong stylization, and imaginative semantics such as a “personified refrigerator” or a “human-like bee” (Tan et al., 13 Aug 2025). The paper states that models which appear strong on human benchmarks often fail badly on such material, with identity distortion, unnatural limbs, and the imposition of human skeletons onto non-human shapes. A plausible implication is that A²Bench is intended not merely as another test set, but as a diagnostic instrument for failure modes that human-only leaderboards systematically underexpose.

In the Animate-X++ study, A²Bench serves three roles: a primary evaluation benchmark for non-human anthropomorphic animation, a demonstration of generalization because Animate-X++ is trained only on human video datasets but evaluated on A²Bench, and a design driver for architectural choices such as the Pose Indicator and misalignment training strategies (Tan et al., 13 Aug 2025).

2. Construction, composition, and formal structure

A²Bench contains 500 anthropomorphic image-video pairs. For each character, the benchmark provides one high-quality image and one corresponding dance video generated through an image-to-video pipeline. The data-building process is explicitly staged: GPT-4o generates 500 prompts, KLing AI’s text-to-image module produces 500 anthropomorphic character images, and KLing AI’s image-to-video module produces the corresponding dance videos (Tan et al., 13 Aug 2025).

The benchmark also stores the text prompt associated with each entry, and pose sequences extracted by DWPose where those poses are reliable enough to be used. The paper formalizes each entry as

$$\text{Entry}_i = \{ I^a_i,\, V^a_{i,1:F},\, P^a_{i,1:F},\, \text{prompt}_i \},$$

where $I^a_i$ is the anthropomorphic reference image, $V^a_{i,1:F}$ is the ground-truth dance video, $P^a_{i,1:F}$ is the pose image or keypoint sequence for the screened subset, and $\text{prompt}_i$ is the prompt used to generate the image and video (Tan et al., 13 Aug 2025).

Component	Specification
Total entries	500 anthropomorphic characters
Per-entry media	1 image and 1 corresponding dance video
Quantitative subset	100 manually screened videos with accurate poses
Remaining entries	400 videos used for qualitative comparisons

The construction pipeline is deliberately synthetic. GPT-4o is instructed to generate prompts for “anthropomorphized”, cute, high-quality characters with arms and legs, standing or dancing. Style diversification is then introduced through trigger words such as “Watercolor Painting”, “Cyberpunk Style”, “Van Gogh”, “Ukiyo-E”, and “Pixel Art” (Tan et al., 13 Aug 2025). DWPose is run on all 500 videos, but because it is trained on humans, the authors manually screen 100 videos with accurate poses for quantitative evaluation and retain the remaining 400 primarily as qualitative material and as a prospective resource for anthropomorphic-specific pose extraction research.

3. Character space, stylization, and difficulty levels

A²Bench explicitly targets anthropomorphized plants, animals, food, furniture, everyday objects, and mixed or complex entities. The examples highlighted in the paper include flowers with faces, plant-like characters with arms and legs, ducks, dragons, rabbits, anthropomorphic bananas, cups of juice, fruits with faces and limbs, refrigerators, chairs, and combinations of objects and creatures (Tan et al., 13 Aug 2025). These categories are not treated as decorative variation; they are the source of the benchmark’s structural difficulty, because articulation, body proportions, and even the existence of human-like limbs vary widely.

The benchmark also maximizes diversity in stylization and scene structure. It includes cartoon and 2D animation, game-like characters and mascots, and stylized artwork spanning painterly and retro aesthetics. Backgrounds are often nontrivial, and some of the hardest examples involve interaction with background objects, props, or lighting effects. The paper states that this diversity forces animation methods to cope with non-standard skeletons, unusual occlusion patterns, stylized textures, and environments where foreground-background separation matters (Tan et al., 13 Aug 2025).

The expanded version of A²Bench adds three difficulty levels. Level 1 contains characters whose body shape and appearance are similar to humans. Level 2 contains characters that balance anthropomorphism and original form. Level 3 contains characters that preserve non-human anatomical structures, including ducks with wings, dragons, complex creature-object hybrids, and characters interacting with environment or props (Tan et al., 13 Aug 2025). The paper characterizes Level 3 as the hardest regime because of the conflict between identity preservation and motion transfer.

This stratification is empirically reflected in Animate-X++ results: Level 1 yields $\text{PSNR}^* = 13.96$, $\text{SSIM} = 0.461$, $\text{FID} = 24.24$, and $\text{FVD} = 681.53$; Level 2 yields $$\text{Entry}_i = \{ I^a_i,\, V^a_{i,1:F},\, P^a_{i,1:F},\, \text{prompt}_i \},$$0, $$\text{Entry}_i = \{ I^a_i,\, V^a_{i,1:F},\, P^a_{i,1:F},\, \text{prompt}_i \},$$1, $$\text{Entry}_i = \{ I^a_i,\, V^a_{i,1:F},\, P^a_{i,1:F},\, \text{prompt}_i \},$$2, and $$\text{Entry}_i = \{ I^a_i,\, V^a_{i,1:F},\, P^a_{i,1:F},\, \text{prompt}_i \},$$3; Level 3 yields $$\text{Entry}_i = \{ I^a_i,\, V^a_{i,1:F},\, P^a_{i,1:F},\, \text{prompt}_i \},$$4, $$\text{Entry}_i = \{ I^a_i,\, V^a_{i,1:F},\, P^a_{i,1:F},\, \text{prompt}_i \},$$5, $$\text{Entry}_i = \{ I^a_i,\, V^a_{i,1:F},\, P^a_{i,1:F},\, \text{prompt}_i \},$$6, and $$\text{Entry}_i = \{ I^a_i,\, V^a_{i,1:F},\, P^a_{i,1:F},\, \text{prompt}_i \},$$7 (Tan et al., 13 Aug 2025). The monotonic degradation confirms that the benchmark is not only diverse but explicitly graded by structural difficulty.

4. Evaluation protocol, metrics, and reported results

A²Bench is used for image-to-video character animation. In formal terms, the evaluation task is written as

$$\text{Entry}_i = \{ I^a_i,\, V^a_{i,1:F},\, P^a_{i,1:F},\, \text{prompt}_i \},$$8

where $$\text{Entry}_i = \{ I^a_i,\, V^a_{i,1:F},\, P^a_{i,1:F},\, \text{prompt}_i \},$$9 is the reference anthropomorphic image, $I^a_i$0 is the driving pose sequence, and $I^a_i$1 is the generated video (Tan et al., 13 Aug 2025). The paper uses two settings. In the rescaled-pose setting, a human pose sequence is sampled from a human dataset and used as an anchor to realign and rescale the anthropomorphic pose, yielding a misalignment test between anthropomorphic identity and human-like pose shapes. In the self-driven setting, the model receives pose derived from the same anthropomorphic video and reconstructs that video.

Quantitative evaluation is performed on the 100-video test subset with accurate poses. The metrics are PSNR or the modified $I^a_i$2, SSIM, L1, LPIPS, FID, FID-VID, and FVD (Tan et al., 13 Aug 2025). In the rescaled-pose setting, Animate-X++ with a DiT backbone reports $I^a_i$3, $I^a_i$4, $I^a_i$5, $I^a_i$6, $I^a_i$7, $I^a_i$8, and $I^a_i$9. The paper contrasts this with Unianimate-DiT, which reports $V^a_{i,1:F}$0, $V^a_{i,1:F}$1, $V^a_{i,1:F}$2, and $V^a_{i,1:F}$3 (Tan et al., 13 Aug 2025). In the self-driven setting, Animate-X++ reports $V^a_{i,1:F}$4, $V^a_{i,1:F}$5, $V^a_{i,1:F}$6, $V^a_{i,1:F}$7, and $V^a_{i,1:F}$8.

The benchmark is also validated through a blind user study. The protocol uses 10 participants, 10 characters from A²Bench, and 10 driving videos from the web. For six methods—Moore-AnimateAnyone, MimicMotion, ControlNeXt, MusePose, Unianimate, and Animate-X++—participants view paired comparisons and choose the better output (Tan et al., 13 Aug 2025). Animate-X++ receives 98.5% preference for identity preservation, 93.4% for temporal consistency, and 95.8% for visual quality. These values are presented in the paper as evidence that A²Bench exposes shortcomings in competing methods and supports human-discriminable ranking.

The paper further compares A²Bench itself with TikTok and Fashion using VBench-style metrics. A²Bench reports a CLIP Score of 33.24, Background Consistency of 96.66%, Motion Smoothness of 99.39%, Aesthetic Quality of 69.86%, and Image Quality of 69.32% (Tan et al., 13 Aug 2025). TikTok reports 26.92, 94.10%, 99.05%, 55.14%, and 62.54%; Fashion reports 20.18, 98.25%, 99.45%, 49.62%, and 49.96%. The paper interprets this as evidence that A²Bench is comparable to human datasets in spatial and temporal consistency while offering stronger semantic alignment and higher aesthetic quality.

5. Methodological position within anthropomorphic benchmarking

Although A²Bench is explicitly introduced for anthropomorphic character animation, adjacent work shows that it belongs to a broader family of anthropomorphic evaluation problems. "See What I Mean? Expressiveness and Clarity in Robot Display Design" (Ebisu et al., 19 Jun 2025) studies animated anthropomorphic eyes and animated icons in a collaborative maze-navigation task with asymmetric partial information, real-time decisions, and eight display cues. That work does not define A²Bench as a released benchmark, but it provides a template in which animated anthropomorphic displays are evaluated through success ratio, interpretation accuracy, trust, and satisfaction. This suggests one lineage of A²Bench-like evaluation centered on human-robot communication rather than video generation.

"Can Large Models Fool the Eye? A New Turing Test for Biological Animation" (Chen et al., 8 Aug 2025) introduces BioMotion Arena, a pairwise preference benchmark for point-light biological motion generated by LLMs and MLLMs. It uses 15-marker humanoid skeletons, 90 biological motion variants, more than 45,000 votes, and Elo aggregation. BioMotion Arena is framed as a visual Turing test for biological animation and is described as a strong foundation for an Animated Anthropomorphic Benchmark. Its emphasis, however, is sparse humanoid motion plausibility rather than image-to-video character animation.

"HeartBench: Probing Core Dimensions of Anthropomorphic Intelligence in LLMs" (Liu et al., 26 Dec 2025) shifts the benchmark focus from motion and appearance to socio-emotional and ethical behavior. It defines anthropomorphic intelligence as the capacity to navigate complex social, emotional, and ethical nuances, structures evaluation around Personality, Emotion, Sociality, Morality, and Motivation, and uses case-specific rubrics with a reasoning-before-scoring protocol. HeartBench is not an animation benchmark, but it establishes a standardized metric for anthropomorphic AI evaluation in counseling-like dialogue and provides a theory-driven taxonomy that can be extended to multimodal or embodied settings.

"AnimationBench: Are Video Models Good at Character-Centric Animation?" (Wu et al., 16 Apr 2026) later develops a benchmark for animation image-to-video generation that operationalizes the Twelve Basic Principles of Animation, IP Preservation, and broader quality dimensions such as semantic consistency, motion rationality, and camera motion consistency. It is not the same benchmark as A²Bench, but it addresses many of the same concerns: stylized appearance, exaggerated motion, character-centric consistency, expressive acting, and the insufficiency of realism-oriented benchmarks. A plausible implication is that A²Bench and AnimationBench represent convergent responses to the same evaluation gap, with A²Bench specialized for anthropomorphic character generalization and AnimationBench formalizing a wider animation-theoretic metric suite.

6. Limitations, intended use, and future directions

The A²Bench paper identifies several limitations. Pose annotation quality is the most immediate one: only 100 of the 500 videos have reasonably accurate poses from DWPose, because DWPose is trained on humans and performs poorly on stylized anthropomorphic characters (Tan et al., 13 Aug 2025). Dataset size is another limitation: 500 videos is described as relatively modest compared with web-scale human datasets. The data are synthetic, created through GPT-4o and KLing AI, which may constrain variety relative to real-world footage. Domain bias is also explicit: the benchmark focuses on cute anthropomorphized characters with arms and legs, standing, dancing, and therefore does not cover non-cute anthropomorphic designs, non-dance motions, or broader interactional repertoires. The benchmark’s videos also have mostly static backgrounds; the dynamic-background capability in Animate-X++ comes from separate TI2V training rather than from A²Bench itself.

Within Animate-X++, A²Bench is used only for testing. The paper states that, to ensure a fair comparison, the data in the $V^a_{i,1:F}$9Bench are not included in the training set to train our model (Tan et al., 13 Aug 2025). This preserves the benchmark’s role as an out-of-domain generalization test for models trained on human videos.

The future directions named in the paper are concrete. They include extending A²Bench beyond dancing, expanding character variety and dataset size, providing better pose annotations or training pose estimators specifically for anthropomorphic characters, and studying interactions between characters and environment or background as a more complex task (Tan et al., 13 Aug 2025). The authors also state that the dataset could serve as an important resource for future pose extraction algorithms tailored to anthropomorphic datasets. Taken together, these directions indicate that A²Bench is both a benchmark in its own right and a scaffold for a larger research program on non-human anthropomorphic motion, identity preservation, and cross-domain animation generalization.