Code4D Benchmark for Text-to-4D Generation
- Code4D Benchmark is a comprehensive evaluation framework designed to assess executable code that generates 4D simulations with attention to physical laws and dynamic evolution.
- It measures model performance across object, scene, and dynamic generation using metrics such as O-CLIP, S-CLIP, HRS, and failure rates.
- The benchmark exposes the semantic–physical execution gap by testing models on their ability to translate dense text instructions into physically and temporally coherent 4D worlds.
Searching arXiv for the benchmark and closely related work to ground the article in current papers. Code4D Benchmark is a task and evaluation benchmark for text-to-4D world generation via executable code. It is defined as a benchmark for generating physically grounded 4D environments and is explicitly positioned against text-to-3D datasets that focus solely on static 3D structures. Its central purpose is to evaluate whether a model can translate semantically dense natural-language instructions into 3D scenes and 4D rollouts that preserve spatial structure, temporal evolution, physical interactions, and atmospheric change. In the formulation used by "Code2Worlds: Empowering Coding LLMs for 4D World Generation" (Zhang et al., 12 Feb 2026), the benchmark assesses performance across three dimensions—object, scene, and dynamic generation—and compares code-centric systems with leading text-to-video models.
1. Definition, scope, and benchmark objective
The benchmark is introduced as a unified testbed for code-based 4D simulation. Its stated focus is not merely visual plausibility, but the generation of worlds that obey physical laws and match dense textual specifications. The benchmark is therefore designed to expose what the paper terms the "semantic–physical execution gap": cases where a model writes code that is syntactically valid yet produces physically incorrect behavior, such as objects ignoring gravity, interpenetrating, or exhibiting unrealistic motion magnitude (Zhang et al., 12 Feb 2026).
The three benchmark dimensions are defined as follows. Object generation concerns fine-grained asset structure and appearance. Scene generation concerns global layout, environmental richness, semantics, atmosphere, and lighting. Dynamic generation concerns 4D motion, physical plausibility, and temporal consistency. This organization makes Code4D broader than a static geometry benchmark and narrower than a fully supervised simulation dataset, because it evaluates rendered outputs and executable behavior rather than ground-truth state trajectories.
A common misunderstanding is to treat Code4D as a static 3D prompt set with animation added after the fact. The benchmark is explicitly curated to challenge models on temporal evolution, physical interactions, and atmospheric changes, and its evaluation protocol includes video stability, motion quality, visual-physical plausibility, and manual inspection of physics failures. Another misunderstanding is to treat it as a code-only benchmark. In practice, it is model-agnostic at evaluation time: code-based methods output Blender and Infinigen-compatible Python scripts, whereas pure video models can be evaluated directly from generated videos under the same prompt set (Zhang et al., 12 Feb 2026).
2. Inputs, outputs, and benchmark composition
The inputs are natural-language instructions that are deliberately semantically dense and require long-context reasoning and precise attribute binding. Prompts specify scene type, atmosphere, time of day, object categories, object attributes, and detailed dynamics. The benchmark covers both nature and indoor environments. Nature prompts include autumn forests, time-lapse forest lighting transitions, dense rainstorms, underwater scenes with jellyfish deformation, burning chopped trees with flames and ash, and desert scenes with flowing sand. Indoor prompts include water pouring from a ceramic cup onto a living room table, a brown glass bottle rolling across a floor, and steam rising from a coffee cup in a bedroom (Zhang et al., 12 Feb 2026).
The required outputs differ by model family. For code-based systems, outputs are executable simulation code using Blender and Infinigen APIs, together with the rendered artifacts produced by executing that code: static 3D scenes, object images, and 4D videos. For video diffusion baselines such as Stable Video Diffusion, AnimateDiff, CogVideoX, and HunyuanVideo, the output is video only. In the reported implementation, Blender 4.3 with bpy is the execution substrate, Cycles is the rendering engine, and nature scenes are rendered at , 240 frames, 128 samples per frame, while indoor scenes use , 120 frames, 196 samples per frame, with OpenImageDenoise for denoising (Zhang et al., 12 Feb 2026).
The scene list is physics-typed. Each prompt row includes Scene Type ∈ {Nature, Indoor} and a Primary Dynamics label such as "Soft Body / Wind", "Lighting / Atmosphere", "Rigid Body / Gravity", "Fluid Dynamics", "Particle (Steam)", or "Particle (Fire/Smoke)". This typing is not merely descriptive; it structures the intended failure modes and clarifies what counts as a faithful 4D rollout.
The paper does not provide the exact number of benchmark scenes beyond the ten examples reproduced in the prompt table, and it does not define train/validation/test splits. It presents Code4D as a pure evaluation benchmark on which models are evaluated zero-shot. The authors state that they will release the full benchmark, including prompt texts and evaluation scripts, upon acceptance (Zhang et al., 12 Feb 2026).
3. Evaluation protocol and metric suite
Code4D uses a multi-channel evaluation protocol that combines automatic similarity measures, video-quality metrics, prompted GPT-4o evaluation, and manual inspection. The metric design is axis-specific.
| Axis | Outputs scored | Metrics |
|---|---|---|
| Object generation | Rendered object images | O-CLIP, SGS, Style-CLIP |
| Scene generation | Static scene images | S-CLIP, Richness, HRS, Failure Rate |
| Dynamic generation | Rendered videos | Motion Smoothness, Subject Consistency, Background Consistency, Temporal Flickering, HRS, Failure Rate |
For object and scene semantics, the benchmark uses CLIP-based measures. O-CLIP is the CLIP similarity between a generated object image and its prompt text. S-CLIP is the CLIP similarity between a static scene image and the scene prompt text. Style-CLIP measures contextual compatibility between the generated object and a target scene image. The paper describes standard CLIP scoring as
For video-level quality, the benchmark uses VBench metrics: Motion Smoothness, Subject Consistency, Background Consistency, and Temporal Flickering. These are automatic measures of temporal coherence and video stability rather than direct physical simulation error (Zhang et al., 12 Feb 2026).
The GPT-4o-based metrics are more specialized. SGS is used for fine-grained object attributes and is produced by prompting GPT-4o as an "expert 3D object generation quality evaluator" on a 0–100 scale. HRS is defined as a visual-physical plausibility score and is produced by prompting GPT-4o as an "expert VFX Supervisor and Physics Simulation Evaluator" to jointly consider physics plausibility, visual aesthetics, and temporal stability. Richness is a scene-complexity metric that explicitly ignores text alignment and scores object variety, object count, detail level, and scene complexity from the visible scene image alone (Zhang et al., 12 Feb 2026).
The paper does not provide closed-form analytical definitions for SGS, HRS, or Richness, but it does present them as averages of per-sample GPT-4o scores. Conceptually,
The Physics Failure Rate is derived from manual inspection and counts severe simulation violations, including rigid-body interpenetration, unnatural gravitational detachment, and collision mishandling. In conceptual form,
A crucial property of this protocol is the absence of ground-truth geometry or motion trajectories. Evaluation is therefore perceptual and semantic rather than supervised against reference states. This makes the benchmark suitable for open-ended world generation, but it also means that GPT-4o judgments and human failure annotations remain integral components of the scoring stack (Zhang et al., 12 Feb 2026).
4. Physics-typed dynamic scenarios and the semantic–physical gap
Code4D is explicitly designed around dynamic phenomena that are easily recognizable when simulated incorrectly. The prompt set includes fluid dynamics, rigid-body motion, soft-body or cloth-like motion, particle systems, and atmospheric or lighting evolution. Examples include water flooding a tabletop and dripping downward, leaves detaching and falling straight down while spinning, jellyfish bell pulsation and tentacle drift, burning wood with embers and ash, heavy diagonal rain with swaying branches, and time-lapse lighting changes across a full day (Zhang et al., 12 Feb 2026).
This design separates visual plausibility from physical faithfulness. Visual plausibility is captured by CLIP alignment, Richness, and the visual-aesthetic component of HRS. Physical faithfulness is captured by the physics component of HRS, VBench motion metrics, Temporal Flickering, and the manual Failure Rate. The distinction is central to the benchmark’s purpose: a model may generate visually compelling frames while failing on gravity, collision, or motion semantics.
The benchmark therefore operationalizes the semantic–physical execution gap. Prompts specify motion in semantically precise language—for example, an object may be required to fall, spin, drift, or come to rest in a particular way—while evaluation penalizes videos that remain visually plausible but violate these physical constraints. The ablation results in the Code2Worlds paper make this explicit: removing the VLM-Motion self-reflection component increases Failure Rate from to and reduces HRS from $55.4$ to $47.0$, showing that Code4D’s dynamic scenarios are sensitive to failures in closed-loop physics correction (Zhang et al., 12 Feb 2026).
A plausible implication is that Code4D is not simply measuring whether a model can animate a scene, but whether it can preserve a binding between linguistic event structure and physically credible simulation outcomes.
5. Reported benchmark results and ablation sensitivity
Code4D is used to compare Code2Worlds against both code-centric baselines and text-to-video generators. Among code-centric methods, the benchmark includes MeshCoder, Infinigen, 3D-GPT, SceneCraft, ImmerseGen, and Code2Worlds. On the object-generation slice, Code2Worlds reports O-CLIP 0.2655, SGS 61.4, and Style-CLIP 0.6734. The best prior SGS in the same table is 43.5 from ImmerseGen, which corresponds to the paper’s abstract claim of a 41% SGS gain. On the scene-generation slice, Code2Worlds reports S-CLIP 0.2432, Failure 10%, HRS 55.4, and Richness 62.3. The best prior Richness in the table is 41.7 from 3D-GPT, corresponding to the paper’s claim of 49% higher Richness (Zhang et al., 12 Feb 2026).
Against video diffusion models, Code2Worlds reports Motion Smoothness 0.9952, Subject Consistency 0.9415, Background Consistency 0.9710, Temporal Flickering 0.9949, and Failure Rate 10%. The comparison table reports substantially higher failure rates for several diffusion baselines: 70% for AnimateDiff, 50% for Stable Video Diffusion and CogVideoX, and 30% for HunyuanVideo. The benchmark is therefore used to argue that explicit simulation can improve physical reliability even when video-only systems remain competitive on appearance (Zhang et al., 12 Feb 2026).
The ablations further clarify what the benchmark rewards. In object generation, removing retrieval reduces SGS from 61.4 to 23.5, and removing the parameter library reduces SGS to 48.8. Removing the VLM-Critic reduces SGS to 58.6. In the scene stream, removing both Planner and Solver reduces Richness from 62.3 to 50.9, while removing the Scene Stream reduces it to 26.4. In dynamics, removing VLM-Motion increases Failure Rate to 60% and lowers HRS to 47.0 (Zhang et al., 12 Feb 2026).
These results show that the benchmark is not only comparative but also diagnostic. It distinguishes between failures of fine-grained object grounding, failures of environmental orchestration, and failures of physics-aware temporal control.
6. Relation to adjacent benchmark paradigms and current limitations
Code4D occupies an intersection between two adjacent benchmark traditions. On one side, "4DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Models" defines a unified four-axis evaluation of Perceptual Quality, Condition-4D Alignment, Physical Realism, and 4D Consistency, and it maps different conditioning modalities into a unified textual space while combining LLM-as-judge, MLLM-as-judge, and traditional network-based metrics (Lu et al., 25 Nov 2025). On the other side, "3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code" formalizes the pipeline condition code 0 engine execution 1 geometry, adds executability and penalized metrics, and validates automated scores with human pairwise preferences in 3DCodeArena (Gao et al., 31 May 2026).
This suggests that Code4D can be understood as a benchmark that inherits the code-grounded execution emphasis of 3DCodeBench and the dynamic, physics-aware evaluation emphasis of 4DWorldBench. The benchmark as reported in Code2Worlds is narrower than 4DWorldBench in metric breadth, because it does not present the same explicit four-axis decomposition into perceptual quality, condition alignment, physical realism, and 4D consistency. It is also narrower than 3DCodeBench in executability analysis, because its reported protocol emphasizes rendered outputs, GPT-4o scoring, and manual failure inspection rather than conditional versus penalized aggregation over execution success. Yet it extends both paradigms into executable 4D world generation specified by dense text (Lu et al., 25 Nov 2025, Gao et al., 31 May 2026).
The present limitations are explicit. The benchmark is evaluation-only, the exact number of scenes beyond the ten example prompts is not given, and no train/validation/test partition is defined. The current physics coverage is limited to what Blender and Infinigen can simulate in the reported setup: fluids, rigid bodies, soft bodies, particles, and atmospheric effects. The metric suite also inherits subjective elements from GPT-4o prompting and manual inspection (Zhang et al., 12 Feb 2026).
Even with those limits, Code4D defines a concrete benchmark notion for executable 4D world generation: semantically dense prompts, physics-typed scenarios, code-compatible simulation backends, and a scoring protocol that jointly probes object fidelity, scene richness, temporal stability, and physical plausibility.