PhyCritic: Multimodal Physical AI Critic
- PhyCritic is a multimodal critic for physical AI that uses a self-referential two-stage RLVR pipeline to judge responses based on physical, causal, and temporal validity.
- It evaluates candidate answers in robotics, autonomous driving, and embodied interaction by first generating its own prediction and reasoning before comparative judgment.
- The model produces structured outputs combining internal predictions, comparative critiques, and final verdicts, achieving state-of-the-art accuracy in physical evaluation benchmarks.
PhyCritic is a multimodal critic model for physical AI: a vision-language model trained not just to answer questions about images or videos, but to evaluate other models’ responses in physically grounded settings—robotics, embodied interaction, and autonomous driving—where correctness depends on perception, causal reasoning, affordances, temporal order, and action/planning validity. It is introduced as a multimodal critic model optimized for physical AI through a two-stage RLVR pipeline consisting of a physical skill warmup stage and a self-referential critic finetuning stage, with the explicit thesis that a strong physical critic should behave more like a human expert: before judging others, it should try to solve the problem itself [2602.11124].
1. Conceptual scope and target of evaluation
PhyCritic is motivated by the claim that physical AI needs a different kind of judge than standard multimodal evaluators. In ordinary visual tasks, a critic may only need to decide whether a caption or answer is plausible or image-grounded. In physical AI, by contrast, a judge must determine whether a response is physically correct: whether it respects what is visible, the causal state of the world, what actions are feasible next, and whether a plan follows a sensible temporal and spatial sequence. The paper identifies three main gaps in prior multimodal critics: they are largely trained on general visual domains such as captioning, STEM reasoning, or VQA; they often lack physics awareness; and they usually judge candidate responses without first solving the task themselves [2602.11124].
The model is a multimodal VLM-based judge handling images and videos paired with language. It is trained to judge responses involving physical perception, visual grounding, spatial reasoning, affordance reasoning, causal reasoning, action prediction, next-step selection, plan evaluation, embodied task understanding, and autonomous-driving decision questions. Its outputs are generative rather than scalar-only: during training and often inference, it produces structured text containing an internal prediction/reasoning, an explicit comparative critique, and a final pairwise verdict such as “Response 1 is better” or “Response 2 is better.”
This design places PhyCritic within the broader class of judge and critic models used for pairwise preferences, numerical scores, and explanatory justifications, but it narrows the notion of correctness to physical truth rather than superficial fluency. That emphasis is especially consequential in safety-critical domains, where a response may be linguistically coherent yet physically impossible or causally wrong.
2. Formal problem setting and self-referential judgment
The training and evaluation unit is a tuple
[
(Q, L_A, L_B, A_Q, P),
]
where (Q) is a multimodal prompt containing visual input plus a user question, (L_A) and (L_B) are two candidate responses to compare, (A_Q) is the ground-truth answer to the question, and (P \in {A,B}) is the preferred response label. The critic therefore receives an image or video, a question, and two model responses, and must choose which response is better while assessing both the reasoning process quality and the final answer correctness in a physical context [2602.11124].
The defining technical novelty is the self-referential mechanism. Instead of directly comparing two candidate responses, the model is first asked to generate its own reasoning for the original task and its own answer to the question, and only then compare the two responses using that self-generated answer as an internal anchor. The prompt explicitly asks it to put its own reasoning in <pred_think>...</pred_think>, its own answer in <pred>...</pred>, then perform comparison reasoning in <think>...</think>, and finally output the decision in \boxed{}.
This differs from prior multimodal critics that simply inspect response A and response B. The intended effect is judgment stability and physical correctness: if the critic first forms its own physically grounded belief, it can better discriminate whether one candidate response respects that physical state and another does not. The paper’s qualitative examples present this as a shift away from preference for stylistic surface structure and toward grounded evaluation of the physical situation.
3. Two-stage RLVR pipeline and reward design
PhyCritic is trained from Qwen2.5-VL-7B-Instruct using a two-stage RLVR pipeline implemented in veRL. Stage 1 is a physical skill warmup stage trained with vanilla GRPO for 80 steps on ((Q, A_Q)) pairs from Cosmos-Reason1-RL. Its reward is a simple accuracy reward,
[
r = \mathbb{I}\big(\hat{A}_{pred}(Q) = A_Q\big).
]
This stage is intended to improve what the authors call the model’s “intrinsic ability” in the physical domain before it is trained as a judge [2602.11124].
Stage 2 is self-referential critic finetuning, run for 300 steps on tuples ((Q, L_A, L_B, A_Q, P)). In one rollout, the model performs both self-prediction and preference judgment. The prompt instructs it to evaluate each candidate response on Truthfulness, Visual Groundedness, Logical Validity, Efficiency and Clarity, and Final Answer Accuracy, while still requiring self-generated reasoning and answer before critique.
The total Stage 2 reward is
[
r_{total} = r_{acc} + r_{form} * \alpha_{form},
]
with
[
r_{acc} = \alpha_{sp} r_{sp} + \alpha_{crit} r_{crit},
]
where
[
r_{sp} = \mathbb{I}(\hat{A}{pred} = A_Q), \qquad
r{crit} = \mathbb{I}(\hat{P}{crit}(Q, L_A, L_B) = P).
]
The reported hyperparameters are (\alpha{sp}=0.2), (\alpha_{crit}=0.7), and (\alpha_{form}=0.1).
The format reward enforces the self-referential template:
[
r_{form} =
\begin{cases}
1.0 & \text{if }
0.5 & \text{if only }
0 & \text{otherwise.}
\end{cases}
]
Optimization uses Group Relative Policy Optimization (GRPO), with objective
[
\begin{aligned}
\mathcal{L}{\text{GRPO}}
&= \mathbb{E}{o\sim\pi_\theta}\Big[ \min(\rho_{o}A_{o}, \text{clip}(\rho_o,1-\epsilon,1+\epsilon)A_o) \Big] \
&\quad - \beta\, \mathcal{D}{\mathrm{KL}(\pi\theta \Vert \pi_{\mathrm{ref}})}
\end{aligned}
]
where
[
\rho_o = \frac{\pi_\theta(o)}{\pi_{ref}(o)}, \qquad
A_o = \frac{r_o - \bar{r}}{\text{std}(r)}.
]
The remaining reported training settings are batch size 128, learning rate (1\times10{-6}), KL coefficient 0.01, and total training with only 4,058 samples and 80 + 300 RL steps.
4. Dataset construction and PhyCritic-Bench
The critic training set is constructed from four embodied or robotics video sources: RoboVQA, BridgeData V2, HoloAssist, and AgiBot World. These cover egocentric and third-person viewpoints and tasks such as grasping, stacking, folding, and assembly. Questions are built using Cosmos-Reason1 RL data, which provides 800 high-quality physical question–answer pairs involving physical perception, planning, and reasoning [2602.11124].
Candidate responses are drawn from seven multimodal models: GPT-4o, Gemini-2.5-Flash, Qwen2.5-VL-72B, InternVL3-38B, Cosmos-Reason1-7B, Video-R1, and MiMo-VL-7B. All are prompted to generate chain-of-thought responses. GPT-4o then verifies each response against ground truth and assigns a binary preference score. Chosen and rejected responses are paired, and after balancing, the final critic training set contains 3,258 samples.
For evaluation, the paper introduces PhyCritic-Bench, a physical-domain multimodal judge benchmark containing 225 evaluation samples. It spans robotics tasks from RoboVQA, BridgeData V2, HoloAssist, AgiBot, and RoboFail, with questions adapted from CosmosReason1-Bench, together with autonomous driving from LingoQA. Each benchmark sample is a pairwise preference tuple ((q, l_a, l_b, p)). For each prompt, one model generates (N=8) chain-of-thought responses via temperature sampling; GPT-4o verifies them against ground truth, and one correct and one incorrect response are paired.
Performance is measured as pairwise preference accuracy:
[
Acc = \mathbb{I}\big(\text{VLM}(q,l_a,l_b) = p\big).
]
In addition to PhyCritic-Bench, the model is evaluated on VL-RewardBench and Multimodal RewardBench for general multimodal judge performance, and on CosmosReason1-Bench, CV-Bench, and EgoPlanBench2 to test whether critic training also improves policy-like reasoning.
5. Quantitative results, transfer, and ablations
On PhyCritic-Bench, PhyCritic-7B achieves 68.0 overall accuracy, best among open-source 7B/8B models. The reported baselines are Eagle-2.5-8B: 56.0, Qwen2.5-VL-7B: 51.6, RoboBrain2.0-7B: 54.7, and Cosmos-R1-7B: 51.1. Subsuite scores are AgiBot: 78.8, HoloAssist: 65.5, RoboVQA: 86.7, Bridge: 65.6, RoboFail: 57.4, and LingoQA: 60.0. Proprietary reference points are Gemini-2.5-Pro: 78.2, Gemini-2.5-Flash: 67.1, and GPT-4o: 64.7, so the model exceeds GPT-4o on this benchmark but not Gemini-2.5-Pro [2602.11124].
The model also improves general multimodal judging despite being trained only on physical critic data. On VL-RewardBench, it scores 57.3 versus 53.2 for Qwen2.5-VL-7B; on Multimodal RewardBench, 65.9 versus 64.0. The paper contrasts this with UnifiedReward-Think-Qwen-7B, which is stronger on the general reward benchmarks but scores only 52.4 on PhyCritic-Bench and 51.8 on CosmosReason1-Bench.
A further result is that PhyCritic also acts as a stronger policy model. On CosmosReason1-Bench, PhyCritic-7B scores 63.9, compared with 63.0 for Cosmos-R1-7B and 54.3 for Qwen2.5-VL-7B. On CV-Bench, it reaches 79.7 overall, including best open-source 3D score 83.9. On EgoPlanBench2, it reaches 42.3 overall.
The ablation studies are central to the method’s interpretation. Full two-stage RL gives 68.0 on PhyCritic-Bench, 63.9 on CosmosR1-Bench, and 57.3 on VL-Reward. Removing self-reference yields 64.4 / 62.6 / 56.6; removing the self-prediction reward yields 65.8 / 63.5 / 56.5; removing the hand-crafted evaluation criteria yields 63.9 / 62.0 / 55.1. The appendix also varies (\alpha_{sp}), reporting that (\alpha_{sp}=0.2) gives the best balance among the tested settings.
The paper’s judgment-stability analysis reports a strong positive dependence between the critic’s own answer correctness and its judging correctness: after Stage 1 warmup,
[
\chi2 = 51.07,\quad p = 8.93 \times 10{-13},
]
and in the final self-referential model,
[
\chi2 = 161.76,\quad p = 4.66 \times 10{-37}.
]
This is presented as direct evidence for the claim that better self-predictions lead to better judgments.
6. Downstream use, limitations, and significance
PhyCritic is used not only as a judge but also as a reward model for improving policies. In best-of-(N) test-time selection, using Qwen2.5-VL-7B-Instruct as a policy and PhyCritic as a pairwise knockout judge on CosmosReason1-Bench, the paper reports an improvement from 54.3 to 60.8 at (N=32). In a downstream DPO experiment, starting from Qwen2.5-VL-7B-Instruct, the reported scores on CosmosReason1-Bench are 54.3 for the base model, 57.5 for answer-verify DPO, and 60.0 for PhyCritic-7B DPO [2602.11124].
The paper assigns three forms of significance to the system. First, it reframes critic modeling for physical AI around physically grounded and causally valid evaluation rather than generic plausibility. Second, it proposes self-referential judging as a concrete training principle: solve first, judge second. Third, it argues that critic training and policy improvement are intertwined, since the critic itself becomes a stronger physically grounded reasoner.
The main acknowledged limitation is that self-referential critic finetuning relies on access to ground-truth answers (A_Q), because the self-prediction reward
[
r_{sp} = \mathbb{I}(\hat{A}_{pred} = A_Q)
]
requires verifiable supervision. A second limitation is that the current setup uses a relatively simple binary preference construction based on correctness labels verified by GPT-4o, which may not capture more subtle human preferences beyond correctness, such as risk sensitivity, style of explanation, or graded planning quality. The suggested future directions are self-verification or meta-judging to replace explicit self-prediction supervision, extension toward multi-round critic self-refinement, and use of critic signals to iteratively improve self-generation.
Within the broader literature on critique-centered AI, PhyCritic occupies a distinct position. Whereas SCALAR studies an Actor--Critic--Judge pipeline for quantum field theory and string theory and reports that multi-turn dialogue improves over single-shot attempts throughout, PhyCritic addresses multimodal judging in physically grounded perception, causal reasoning, and planning [2605.06772]. A plausible implication is that critique-centric architectures are becoming domain-specific: in theoretical physics the central question is iterative derivational refinement, while in physical AI it is physically correct pairwise evaluation under multimodal grounding.