Vision-Language Physics Verification Gate

• The Vision-Language Physics-Verification Gate validates simulation outputs by analyzing rendered frames to ensure physical plausibility, directly addressing the code-physics gap.
• It integrates with a multi-agent pipeline that uses vision-language models to compute accuracy scores and generate structured, criterion-based corrective feedback.
• Domain-specific checks spanning mechanics, fluid dynamics, electromagnetics, and more enable precise detection of numerical instabilities and implementation errors.

Searching arXiv for the primary paper and closely related verification-gate work to ground the article in current literature. The Vision-Language Physics-Verification Gate (VLPVG) is the perceptual checkpoint in a multi-agent physics-simulation pipeline that decides whether a generated simulation is physically correct by inspecting rendered frames rather than code, thereby targeting the “oracle gap” in which code can compile, run, and display while still violating domain laws (Shende et al., 12 Feb 2026). In the formulation introduced in “Perceptual Self-Reflection in Agentic Physics Simulation Code Generation” (Shende et al., 12 Feb 2026), the gate sits between physics code generation and iterative refinement, consumes rendered frames together with validation metadata, and produces a decision, a physics accuracy score, and structured corrective feedback. Closely related work places similar emphasis on explicit physical context extraction, executable falsification, trajectory verification, and typed state-transition auditing, but the VLPVG is distinguished by using vision-language analysis of animation outputs as the primary physics oracle (Balazadeh et al., 2024, Liang et al., 9 Feb 2026, Huang et al., 21 Nov 2025, Bourigault, 28 May 2026).

1. Definition, formalization, and the oracle gap

The VLPVG is defined as the component that decides whether a generated simulation is physically correct by inspecting its rendered frames, not its code (Shende et al., 12 Feb 2026). Its inputs are $K$ uniformly sampled PNG frames from the video, validation metadata including domain tag, scaled parameters, boundary conditions, and per-scenario criteria, and execution context such as sampling times, frame size, and FPS. Its outputs are a decision, structured feedback with per-criterion rationales and “how-to-fix” suggestions, and a routing hint that distinguishes requirements_error from implementation_error (Shende et al., 12 Feb 2026).

The central motivation is the distinction between execution-level success and physics-level correctness. The paper defines a code-level oracle $O_{\text{code}}(\text{code}) \in \{0,1\}$ that checks compile, run, and display, and a physics oracle $O_{\text{phys}}(A) \in \{0,1\}$ that checks physical plausibility from visual evidence in the rendered animation $A = \psi(\text{code})$ (Shende et al., 12 Feb 2026). The oracle gap is then

$$\Delta = P[O_{\text{code}} = 1 \wedge O_{\text{phys}} = 0],$$

the probability that code passes execution checks yet violates physics. In continuous form, with $S_{\text{phys}} \in [0,1]$ and $S_{\text{code}} \in \{0,1\}$, the expected discrepancy is

$\Gamma = E[S_{\text{code}}] - E[S_{\text{phys}}].$

Large $\Gamma$ indicates code that runs but is physically wrong, which is precisely the failure mode the gate is designed to detect (Shende et al., 12 Feb 2026).

This formulation explicitly distinguishes emergent physical behavior from software correctness. Unit tests and execution checks operate on explicit I/O relations, exceptions, syntax, runtime, or timeouts, whereas physics correctness is visually encoded in spatiotemporal patterns in $A(t,x)$ and may not admit a straightforward reference solution for novel simulations (Shende et al., 12 Feb 2026). A plausible implication is that VLPVG belongs to a broader class of verification mechanisms that shift from answer-only or code-only validation toward explicit state, transition, or perceptual commitments, as also seen in execution-based world reconstruction (Liang et al., 9 Feb 2026) and typed trace auditing (Bourigault, 28 May 2026).

2. Placement in the multi-agent architecture

The architecture described in (Shende et al., 12 Feb 2026) contains four agents. Agent 1, the Natural Language Interpreter, converts user intent into a physics-grounded scene description. Agent 1A, the Technical Requirements Generator, scales parameters for 24 FPS visibility and emits scenario-specific validation criteria. Agent 2, the Physics Code Generator with self-correction, produces executable Python/matplotlib animation code and fixes syntax, runtime, timeout, and silent-failure issues with at most 5 attempts and a 30 s timeout. Agent 3 is the VLPVG itself: the Physics Validator with perceptual self-reflection (Shende et al., 12 Feb 2026).

The end-to-end workflow is iterative. A user request is first converted into a physics-formalized description; parameters are then scaled and validation criteria emitted; code is generated and execution issues are repaired; finally, the VLPVG executes the animation, captures 8 evenly spaced frames, evaluates them with a vision-capable model, computes the aggregate physics score $O_{\text{code}}(\text{code}) \in \{0,1\}$0, compares it to the threshold $O_{\text{code}}(\text{code}) \in \{0,1\}$1, and either accepts or emits structured feedback for another iteration (Shende et al., 12 Feb 2026). If $O_{\text{code}}(\text{code}) \in \{0,1\}$2, the pipeline stops. Otherwise, the gate classifies the issue as a requirements_error or implementation_error, routes feedback upstream, and iterates with a budget of at most 10 steps and plateau early stopping (Shende et al., 12 Feb 2026).

The gate’s decision rule is criterion-based. Given validation criteria $O_{\text{code}}(\text{code}) \in \{0,1\}$3 with weights $O_{\text{code}}(\text{code}) \in \{0,1\}$4 satisfying $O_{\text{code}}(\text{code}) \in \{0,1\}$5, the vision-LLM returns per-criterion confidences $O_{\text{code}}(\text{code}) \in \{0,1\}$6, labels $O_{\text{code}}(\text{code}) \in \{0,1\}$7, and rationales $O_{\text{code}}(\text{code}) \in \{0,1\}$8; the aggregate physics accuracy is

$O_{\text{code}}(\text{code}) \in \{0,1\}$9

The acceptance rule is PASS if $O_{\text{phys}}(A) \in \{0,1\}$0, else FAIL, with the target threshold in the paper set to $O_{\text{phys}}(A) \in \{0,1\}$1 (Shende et al., 12 Feb 2026).

This architecture is notable for locating verification after rendering rather than before execution. Related systems place verification gates at other points in the pipeline: Physics Context Builders use physical context extraction as a gate over question answering (Balazadeh et al., 2024), SketchVerify ranks candidate motion plans before full synthesis (Huang et al., 21 Nov 2025), and execution-based frameworks gate visual-to-code reconstructions via re-simulation and comparison (Liang et al., 9 Feb 2026). The VLPVG instead validates the rendered consequences of generated simulator code.

3. Perceptual scoring and visual analysis methodology

The perceptual validator in (Shende et al., 12 Feb 2026) uses Claude Sonnet 4.5 for physics analysis and Claude Haiku 4.5 for error-source classification and cost-efficient routing. The validator samples 8 frames uniformly across the animation duration, tags each with normalized time $O_{\text{phys}}(A) \in \{0,1\}$2, and provides all frames together, ordered with timestamps, so that the model reasons over temporal evolution rather than isolated images (Shende et al., 12 Feb 2026). The prompt explicitly instructs the model to judge each criterion from frames only and return JSON entries containing criterion, label, confidence, rationale, and fix (Shende et al., 12 Feb 2026).

The paper also specifies quantitative probes that can augment VLM judgment. For object trajectories, salient objects are segmented and centroid-tracked to estimate position $O_{\text{phys}}(A) \in \{0,1\}$3, velocity

$O_{\text{phys}}(A) \in \{0,1\}$4

and acceleration

$O_{\text{phys}}(A) \in \{0,1\}$5

For scalar or vector fields, pixel intensities may be treated as $O_{\text{phys}}(A) \in \{0,1\}$6 and optical flow as $O_{\text{phys}}(A) \in \{0,1\}$7; spatial derivatives are computed via finite differences and temporal derivatives via frame differencing (Shende et al., 12 Feb 2026).

Representative estimators include parabola fitting for projectile motion, wave-speed estimation from crest spacing and oscillation frequency, and diffusion-profile fitting via variance growth (Shende et al., 12 Feb 2026). The paper lists error metrics such as MSE, MAE, relative error

$O_{\text{phys}}(A) \in \{0,1\}$8

divergence penalty

$O_{\text{phys}}(A) \in \{0,1\}$9

and spectrum-shape measures such as radial power concentration around $A = \psi(\text{code})$0 (Shende et al., 12 Feb 2026). Thresholds $A = \psi(\text{code})$1 are domain- and scale-dependent and are supplied by Agent 1A to match the visualization scaling (Shende et al., 12 Feb 2026).

This methodology makes the gate neither purely symbolic nor purely heuristic. It primarily relies on the VLM’s confidence scores but is designed to be augmentable with explicit probes when available (Shende et al., 12 Feb 2026). A plausible implication is that the VLPVG occupies an intermediate position between fully learned perceptual verification and rule-based physical auditing. That broader pattern also appears in systems that combine extracted facts with rule checking (Balazadeh et al., 2024), hybrid executable verification with RAFT and holistic judgment (Liang et al., 9 Feb 2026), and typed-trace verification with schema, state, transition, and faithfulness checks (Bourigault, 28 May 2026).

4. Domain-specific physics criteria

The VLPVG operates across seven tested domains: classical mechanics, fluid dynamics, thermodynamics, electromagnetics, wave physics, reaction-diffusion systems, and non-physics visualization (Shende et al., 12 Feb 2026). Agent 1A instantiates a weighted subset of criteria for each scenario (Shende et al., 12 Feb 2026).

In classical mechanics, the paper includes checks derived from Newton’s second law, projectile motion, and simple harmonic motion. For free fall, it uses

$A = \psi(\text{code})$2

and estimates acceleration from tracked positions to test whether $A = \psi(\text{code})$3. For projectile motion, it checks consistency with

$A = \psi(\text{code})$4

fitting a parabola and testing curvature and residuals. For the simple harmonic oscillator, it uses

$A = \psi(\text{code})$5

and checks whether the inferred frequency matches expectation (Shende et al., 12 Feb 2026).

In incompressible fluid dynamics, the gate refers to the Navier–Stokes equations

$A = \psi(\text{code})$6

and evaluates divergence-free behavior, visually coherent vortex shedding, and numerical stability, including absence of checkerboard or aliasing artifacts (Shende et al., 12 Feb 2026). In thermodynamics and heat conduction, it checks the heat equation

$A = \psi(\text{code})$7

monotone smoothing through decreasing $A = \psi(\text{code})$8, and linear growth of $A = \psi(\text{code})$9 under diffusion (Shende et al., 12 Feb 2026).

In electromagnetics, the gate uses Maxwell’s equations in source-free regions,

$$\Delta = P[O_{\text{code}} = 1 \wedge O_{\text{phys}} = 0],$$0

and checks wave propagation speed, field coupling, numerical stability, and boundary behavior such as PML or hard-wall reflections (Shende et al., 12 Feb 2026). In wave physics, it verifies

$$\Delta = P[O_{\text{code}} = 1 \wedge O_{\text{phys}} = 0],$$1

dispersion-free propagation, consistency of $$\Delta = P[O_{\text{code}} = 1 \wedge O_{\text{phys}} = 0],$$2, and qualitative superposition (Shende et al., 12 Feb 2026). In reaction–diffusion, it uses

$$\Delta = P[O_{\text{code}} = 1 \wedge O_{\text{phys}} = 0],$$3

checks Turing-pattern formation and temporal saturation, and explicitly disables inappropriate conservation checks because the system is dissipative (Shende et al., 12 Feb 2026).

For non-physics visualization, the gate avoids false positives by switching to task-specific visual correctness criteria rather than physics laws, and it does not enforce physics constraints when the domain tag is “non-physics” (Shende et al., 12 Feb 2026). This distinction is important because it prevents the validator from overextending physical priors to tasks outside the intended domain.

A recurrent issue in related work is exactly the need to separate perception, state extraction, and downstream reasoning. Physics Context Builders explicitly separate physical context generation from reasoning (Balazadeh et al., 2024), while SeePhys shows that many failures arise from miscoupling diagram interpretation and symbolic modeling (Xiang et al., 25 May 2025). The VLPVG addresses that coupling problem by embedding domain laws directly into criterion-level visual validation.

5. Empirical performance, corrections, and robustness

The evaluation in (Shende et al., 12 Feb 2026) spans seven domains, including classical mechanics, fluid dynamics with Rayleigh–Taylor, double-diffusive convection, and shallow water, electromagnetics with FDTD, wave physics with KdV soliton, reaction–diffusion with Gray–Scott, statistical mechanics with hard-sphere gas, and a non-physics visualization task. The primary baseline is single-shot generation without a validation loop (Shende et al., 12 Feb 2026).

Quantitatively, the perceptual validator’s average physics accuracy across scenarios is 91%, and the share of scenarios achieving the target threshold $$\Delta = P[O_{\text{code}} = 1 \wedge O_{\text{phys}} = 0],$$4 is 86% (Shende et al., 12 Feb 2026). Most scenarios converged in 1–2 iterations, the upper bound was 10 with plateau early stopping, Agent 2 resolved syntax, runtime, timeout, and silent failures in testing, and the cost was approximately $0.20 per animation across all agents (Shende et al., 12 Feb 2026).

The paper reports several concrete corrections relative to single-shot generation. In CFD, missing vortex shedding in the baseline was replaced by coherent shedding after perceptual refinement. In FDTD, checkerboard numerical instability was removed after feedback. In heat diffusion, a wrong problem variant—steady-state instead of transient—was corrected. In reaction–diffusion, the initial use of an energy conservation check was found to be inappropriate for a dissipative system and was corrected by disabling that criterion (Shende et al., 12 Feb 2026). These examples show that the gate catches both implementation faults and criterion-design mismatches.

The reported findings support the paper’s stated hypothesis that feeding visual simulation outputs back to a vision-LLM for iterative refinement significantly outperforms single-shot code generation for physics simulation tasks (Shende et al., 12 Feb 2026). A related observation appears elsewhere in the literature: trajectory-verification before final generation improves physical realism and long-term consistency in video synthesis (Huang et al., 21 Nov 2025), while verifier-guided reranking improves trace validity without sacrificing answer accuracy in typed physical-state auditing (Bourigault, 28 May 2026). In each case, the verifier adds value by screening latent failure modes that are not exposed by surface-level success criteria.

6. Failure modes, deployment guidance, and broader significance

The VLPVG catches non-physical trajectories, numerical instabilities such as checkerboard artifacts and spurious oscillations, boundary mishandling, silent failures with no visible evolution, and wrong problem variants such as steady versus transient formulations (Shende et al., 12 Feb 2026). Its limitations are also explicit. Generic conservation checks are invalid for dissipative systems and must be disabled or replaced. Slow dynamics can yield low confidence because inter-frame differences are small. Low resolution, poor contrast, or visual clutter can degrade judgments. Domain shift to exotic physics or unfamiliar visualization styles can reduce reliability. Sampling only $$\Delta = P[O_{\text{code}} = 1 \wedge O_{\text{phys}} = 0],$$5 frames may be insufficient for high-frequency or multi-scale phenomena (Shende et al., 12 Feb 2026).

The paper offers concrete deployment recommendations. It suggests calibrating $$\Delta = P[O_{\text{code}} = 1 \wedge O_{\text{phys}} = 0],$$6 by domain, for example $$\Delta = P[O_{\text{code}} = 1 \wedge O_{\text{phys}} = 0],$$7 for rigid-body mechanics and $$\Delta = P[O_{\text{code}} = 1 \wedge O_{\text{phys}} = 0],$$8 for turbulent flows; weighting criteria by importance; requesting one more iteration when $$\Delta = P[O_{\text{code}} = 1 \wedge O_{\text{phys}} = 0],$$9; increasing $S_{\text{phys}} \in [0,1]$0 or extending duration when UNCERTAIN dominates; using human-in-the-loop review when repeated FAILs persist; logging the best-so-far run; and including both early and late frames, with optional jittered sample times to avoid aliasing (Shende et al., 12 Feb 2026). It also proposes extensions such as hybrid validators

$S_{\text{phys}} \in [0,1]$1

differentiable-physics cross-checks, automatic dataset generation from archived decisions, modular criteria for multi-physics, multi-view inputs, and calibrated uncertainty estimation via temperature scaling or prompt ensembling (Shende et al., 12 Feb 2026).

Within the wider literature, the VLPVG can be situated among several verification paradigms. Physics Context Builders provide modular physical context that can gate reasoning without fine-tuning the large foundation model (Balazadeh et al., 2024). VisPhyWorld replaces answer-only evaluation with executable world hypotheses and re-simulation (Liang et al., 9 Feb 2026). NICE FACT introduces model-agnostic post-hoc verification for kinematic reasoning and confidence calibration (Lan et al., 8 May 2026). EVPV decouples perceptual uncertainty from process-level reward by explicitly verifying visual premises before scoring reasoning steps (Wang et al., 17 Mar 2026). World Models in Words audits whether a model’s stated initial state, transition, resulting state, and answer can all be true at once (Bourigault, 28 May 2026). The VLPVG is most directly aligned with these approaches in its insistence that successful outputs must be checked against physically meaningful evidence rather than surface plausibility alone.

A common misconception is that passing compilation, unit tests, or even visually attractive rendering is already sufficient for physics-grounded acceptance. The formulation in (Shende et al., 12 Feb 2026) directly rejects that view: the relevant target is not executable code but visually observed dynamics that satisfy scenario-specific physical criteria. Another misconception is that a single general-purpose criterion such as energy conservation can be applied uniformly across domains; the reaction–diffusion correction in the paper shows that inappropriate criteria can themselves become a source of error (Shende et al., 12 Feb 2026).

The broader significance of the VLPVG lies in making physics verification operational in agentic generation pipelines. It transforms rendered animations into auditable evidence, turns physical plausibility into a criterion-weighted decision process, and routes failure information to the part of the system most likely to fix it. This suggests a general design pattern for vision-language systems in physics: verification should be perceptual when the target property is perceptually manifest, structured when explicit state can be extracted, and iterative when error localization can improve subsequent generations (Shende et al., 12 Feb 2026, Balazadeh et al., 2024, Bourigault, 28 May 2026).