BioMotion Arena: Visual Turing Test

• BioMotion Arena is a framework that uses point-light animations to evaluate LLMs by mimicking biological motion and capturing subtle kinematic cues.
• It converts code-generated outputs into visual displays, using pairwise human comparisons to provide perceptually clear assessments beyond traditional scalar scores.
• The method leverages psychophysical principles, with over 45k crowd-sourced votes, to reveal discrepancies in animation realism and model performance.

Searching arXiv for the specified BioMotion Arena paper and closely related work on biological motion and point-light evaluation. {"query":"arXiv (Chen et al., 8 Aug 2025) BioMotion Arena Can Large Models Fool the Eye? A New Turing Test for Biological Animation", "max_results": 5} {"query":"BioMotion Arena biological animation point-light large models", "max_results": 10} BioMotion Arena is a framework for evaluating LLMs and multimodal LLMs (MLLMs) through visual biological animation rather than static answer scoring or open-ended chatbot preference judgments. It is designed as a ground-truth-free, human-preference benchmark in which models generate Python or Pygame scripts that render point-light animations of living motion, and human raters compare the resulting outputs side by side. The framework is motivated by the psychophysical observation that human vision is highly sensitive to the kinematic and geometric regularities of biological motion, so even small departures from plausible motion become visually salient. In the reported study, BioMotion Arena evaluates 53 mainstream LLMs and MLLMs on 90 biological motion variants using more than 45k crowd-sourced votes (Chen et al., 8 Aug 2025).

1. Evaluation rationale and conceptual basis

BioMotion Arena was introduced in response to two limitations attributed to prevailing large-model benchmarks. First, closed-ended benchmarks with ground-truth annotations often reduce performance to scalar scores that are difficult to interpret perceptually. Second, open human-preference evaluations in chatbot form can yield textual judgments that are comparatively indistinct at a glance. BioMotion Arena therefore converts model behavior into short visual animations whose quality can be assessed immediately through pairwise human comparison (Chen et al., 8 Aug 2025).

The framework draws on classical point-light displays, in which a small number of moving dots is sufficient to induce a vivid percept of a living body in motion. The motivating claim is that point-light walkers can communicate gait, gender, emotion, and weight, while biologically implausible motion is readily detected. BioMotion Arena adopts this “Turing test of the eye” as an evaluation principle: instead of asking whether a model can produce text that appears correct, it asks whether a model can generate code that yields motion percepts consistent with human expectations of living organisms (Chen et al., 8 Aug 2025).

This setup also defines the scope of the benchmark. It targets code-generating LLMs