WorldLens-Agent Evaluation Model

• WorldLens-Agent is an evaluation model that objectively scores generative driving worlds across visual, geometric, physical, and behavioral dimensions.
• It integrates a frozen multimodal Qwen3-VL-8B transformer with LoRA-based adaptation to ensure high human alignment and effective zero-shot performance.
• Leveraging the WorldLens-26K dataset, the model consistently mirrors human judgments and generalizes well to diverse and complex driving scenarios.

WorldLens-Agent is an evaluation model designed to deliver scalable, explainable, and objective scoring of generative driving world models, functioning as a distilled proxy for human evaluators across visual, geometric, physical, and behavioral dimensions. Grounded in the WorldLens-26K dataset of over 26,000 human-annotated video evaluations, it standardizes the assessment of world fidelity by producing fine-grained scores and natural-language rationales, thereby aligning agent-based judgments with human perception and expert rubrics (Liang et al., 11 Dec 2025).

1. Architecture and Core Components

WorldLens-Agent is implemented atop the Qwen3-VL-8B multimodal transformer, incorporating both a vision encoder—structured as a ViT-style image encoder—and a language modeling component (encoder/decoder), all of which are held frozen during training. Adaptation is introduced exclusively via Low-Rank Adaptation (LoRA) modules inserted into every self- and cross-attention layer of the Qwen3-VL decoder. LoRA modules use a rank $r=16$ and dropout $p=0.05$, constraining optimization to only these added parameters while preserving the representational integrity of the pretrained backbone.

Video inputs are processed into per-frame embeddings by the frozen vision encoder, and textual prompts (combining evaluation dimension names and scoring rubrics) are tokenized by the frozen Qwen3 tokenizer. Vision token embeddings are projected via a linear layer into the LLM embedding space. The resulting sequence—comprising both vision and text tokens—is then fed to the Qwen3-VL decoder for joint processing.

2. Data Foundation and Training Paradigm

The WorldLens-Agent derives its evaluative knowledge directly from the WorldLens-26K dataset, which consists of $N=26{,}808$ annotated records, each comprising:

• A generated driving video clip $y$,
• A target evaluation dimension $$d\in\{\text{overall\_realism}, \text{vehicle\_realism}, \text{pedestrian\_realism}, \text{3D\_consistency}, \text{physical\_plausibility}, \text{behavioral\_safety}\}$$,
• A discrete integer score $s\in\{1,\ldots,10\}$,
• A free-form textual rationale $r$.

Supervised fine-tuning is performed to teach the agent to emit structured JSON outputs containing both the score and rationale, optimizing a conditional language modeling objective: $$p_\theta(\tau\mid y,u,d) = \prod_{t=1}^T p_\theta(\tau_t \mid \tau_{<t}, \phi_V(y), \mathrm{Tok}(u, d))$$ where $\theta$ denotes the set of LoRA parameters, $\phi_V(y)\in\mathbb{R}^{F\times D}$ are vision encoder features, and $\mathrm{Tok}(u, d)$ encodes the system prompt. The training loss is standard cross-entropy: $$\mathcal{L}(\theta)= -\frac{1}{N}\sum_{i=1}^{N} \sum_{t=1}^{T_i} \log p_\theta\bigl(\tau^{(i)}_t \mid \tau^{(i)}_{<t},\,y^{(i)},\,u^{(i)},\,d^{(i)}\bigr)$$

Optimization is performed with AdamW, learning rate $1 \times 10^{-4}$, cosine decay schedule, and 10% warmup over three epochs with a batch size distributed across eight A100 GPUs (bfloat16 precision).

3. Input/Output Specification and Inference Protocol

Inputs to the agent consist of a batch of video clips $y$ (multi-frame, temporally synchronized), paired with system prompts $u$ detailing the evaluation dimension $d$ and an abridged rubric. Inputs are encoded as $\phi_V(y)$ and $\mathrm{Tok}(u, d)$. The agent produces, for each clip, a single JSON object with two fields:

{ "score": 2.5, "reason": "Frequent texture flicker on vehicles and unstable shadows reduce realism, but geometry and traffic behaviors remain mostly plausible." }

Scoring is rubric-guided. Each evaluation dimension $d$ is anchored by five rubric levels (1, 3, 5, 7, 9) with precise criteria; the agent interpolates outputs in 0.5 increments within $[1, 10]$.

Inference proceeds as follows:

1. Encode $y$ with $\phi_V$.
2. Prepend $u$ (specifying the evaluation dimension and rubric).
3. Decode the output sequence autoregressively until well-formed JSON is produced.

The learned mapping is thus formalized as

$$f_{\theta}(y,d) = (s, r), \qquad s \in [1,10],\; r \in \text{Text}$$

4. Alignment, Metrics, and Human Consistency

Human alignment is ensured by direct distillation: $$(y, \text{prompt}) \to (\text{score}, \text{rationale})$$ mappings are supervised from real annotator judgments, including both quantitative and causal or perceptual reasoning. This produces an evaluator capable of rationalizing its assessments using evidence of scene artifacts, behavioral anomalies, and visual or physical inconsistencies.

Quantitative consistency with human annotators is measured on held-out splits using:

• Spearman’s rank correlation,
• Mean absolute error (MAE).

The results exhibit Spearman’s $\rho\geq0.9$ and $\mathrm{MAE}\approx0.4$ across all evaluation axes in zero-shot generalization, indicating robust fidelity to human scoring distributions.

The correlation metric is defined as: $$\rho = 1 - \frac{6\sum_{i=1}^M ( {\rm rank}(s^H_i)-{\rm rank}(s^A_i))^2}{M(M^2-1)}$$ where $s^H$ and $s^A$ are human and agent scores, respectively, and $M$ the split size.

5. Generalization and Empirical Behavior

WorldLens-Agent demonstrates reliable zero-shot generalization to out-of-distribution video data, including scenes from Gen3C and CARLA domains. The model exhibits high sensitivity to violations of physics (teleportation, object interpenetration), behavioral abnormalities (illegal maneuvers, vehicle-pedestrian collisions), and reductions in visual realism (texture flicker, game-engine artifacts). Output JSONs typically cite concrete evidence with temporal or spatial localization, e.g., “object interpenetration at 3.2 s” or “motion jitter across frames 5–7”.

A summary of the agent evaluation workflow:

Step	Process (verbatim from data)	Key Output
Video encoding	Use $\phi_V(y)$ to represent frames	Per-frame features
Prompt assembly	Tokenize $u$ with rubric and dimension $d$	System prompt tokens
Fusion and decoding	Concatenate vision/text tokens, feed to Qwen3-VL decoder	Autoregressive JSON output $(s$, $r)$

6. Significance and Ecosystem Role

WorldLens-Agent, in conjunction with the WorldLens-26K dataset and full-spectrum benchmarks, establishes a unified and explainable ecosystem for evaluating generative driving environments. By standardizing the bridge between low-level quantitative metrics and high-level perceptual human judgment, WorldLens-Agent mitigates subjective variance, increases reproducibility of evaluations, and allows model designers to simultaneously optimize for visual fidelity, physical correctness, geometric consistency, and behavioral reliability (Liang et al., 11 Dec 2025).

A plausible implication is that this methodology could extend beyond driving into general embodied simulation, wherever high-throughput, human-aligned model evaluation is needed. The agent closes the loop between scored realism and functional behavior, enabling robust, explainable benchmarking for real-world deployment validation.