---
title: 'PhyCritic: Multimodal Physical AI Critic'
url: https://www.emergentmind.com/topics/phycritic
type: topic
---

# PhyCritic: Multimodal Physical AI Critic

PhyCritic is a multimodal critic model for physical AI: a vision-language model trained not just to answer questions about images or videos, but to evaluate other models’ responses in physically grounded settings—robotics, embodied interaction, and autonomous driving—where correctness depends on perception, causal reasoning, affordances, temporal order, and action/planning validity. It is introduced as a multimodal critic model optimized for physical AI through a two-stage RLVR pipeline consisting of a physical skill warmup stage and a self-referential critic finetuning stage, with the explicit thesis that a strong physical critic should behave more like a human expert: before judging others, it should try to solve the problem itself [2602.11124].

## 1. Conceptual scope and target of evaluation

PhyCritic is motivated by the claim that physical AI needs a different kind of judge than standard multimodal evaluators. In ordinary visual tasks, a critic may only need to decide whether a caption or answer is plausible or image-grounded. In physical AI, by contrast, a judge must determine whether a response is physically correct: whether it respects what is visible, the causal state of the world, what actions are feasible next, and whether a plan follows a sensible temporal and spatial sequence. The paper identifies three main gaps in prior multimodal critics: they are largely trained on general visual domains such as captioning, STEM reasoning, or VQA; they often lack physics awareness; and they usually judge candidate responses without first solving the task themselves [2602.11124].

The model is a multimodal VLM-based judge handling images and videos paired with language. It is trained to judge responses involving physical perception, visual grounding, spatial reasoning, affordance reasoning, causal reasoning, action prediction, next-step selection, plan evaluation, embodied task understanding, and autonomous-driving decision questions. Its outputs are generative rather than scalar-only: during training and often inference, it produces structured text containing an internal prediction/reasoning, an explicit comparative critique, and a final pairwise verdict such as “Response 1 is better” or “Response 2 is better.”

This design places PhyCritic within the broader class of judge and critic models used for pairwise preferences, numerical scores, and explanatory justifications, but it narrows the notion of correctness to physical truth rather than superficial fluency. That emphasis is especially consequential in safety-critical domains, where a response may be linguistically coherent yet physically impossible or causally wrong.

## 2. Formal problem setting and self-referential judgment

The training and evaluation unit is a tuple
\[
(Q, L_A, L_B, A_Q, P),
\]
where \(Q\) is a multimodal prompt containing visual input plus a user question, \(L_A\) and \(L_B\) are two candidate responses to compare, \(A_Q\) is the ground-truth answer to the question, and \(P \in \{A,B\}\) is the preferred response label. The critic therefore receives an image or video, a question, and two model responses, and must choose which response is better while assessing both the reasoning process quality and the final answer correctness in a physical context [2602.11124].

The defining technical novelty is the self-referential mechanism. Instead of directly comparing two candidate responses, the model is first asked to generate its own reasoning for the original task and its own answer to the question, and only then compare the two responses using that self-generated answer as an internal anchor. The prompt explicitly asks it to put its own reasoning in `<pred_think>...</pred_think>`, its own answer in `<pred>...</pred>`, then perform comparison reasoning in `<think>...</think>`, and finally output the decision in `\boxed{}`.

This differs from prior multimodal critics that simply inspect response A and response B. The intended effect is judgment stability and physical correctness: if the critic first forms its own physically grounded belief, it can better discriminate whether one candidate response respects that physical state and another does not. The paper’s qualitative examples present this as a shift away from preference for stylistic surface structure and toward grounded evaluation of the physical situation.

## 3. Two-stage RLVR pipeline and reward design

PhyCritic is trained from **Qwen2.5-VL-7B-Instruct** using a two-stage RLVR pipeline implemented in **veRL**. Stage 1 is a physical skill warmup stage trained with vanilla **GRPO** for **80 steps** on \((Q, A_Q)\) pairs from **Cosmos-Reason1-RL**. Its reward is a simple accuracy reward,
\[
r = \mathbb{I}\big(\hat{A}_{pred}(Q) = A_Q\big).
\]
This stage is intended to improve what the authors call the model’s “intrinsic ability” in the physical domain before it is trained as a judge [2602.11124].

Stage 2 is self-referential critic finetuning, run for **300 steps** on tuples \((Q, L_A, L_B, A_Q, P)\). In one rollout, the model performs both self-prediction and preference judgment. The prompt instructs it to evaluate each candidate response on **Truthfulness**, **Visual Groundedness**, **Logical Validity**, **Efficiency and Clarity**, and **Final Answer Accuracy**, while still requiring self-generated reasoning and answer before critique.

The total Stage 2 reward is
\[
r_{total} = r_{acc} + r_{form} * \alpha_{form},
\]
with
\[
r_{acc} = \alpha_{sp} r_{sp} + \alpha_{crit} r_{crit},
\]
where
\[
r_{sp} = \mathbb{I}(\hat{A}_{pred} = A_Q), \qquad
r_{crit} = \mathbb{I}(\hat{P}_{crit}(Q, L_A, L_B) = P).
\]
The reported hyperparameters are \(\alpha_{sp}=0.2\), \(\alpha_{crit}=0.7\), and \(\alpha_{form}=0.1\).

The format reward enforces the self-referential template:
\[
r_{form} =
\begin{cases}
1.0 & \text{if } <pred\_think>, <pred>, <think> \text{ and } \boxed{\{\}} \text{ exist},\\[3pt]
0.5 & \text{if only } <think> \text{ and } \boxed{\{\}} \text{ exist},\\[3pt]
0 & \text{otherwise.}
\end{cases}
\]

Optimization uses **Group Relative Policy Optimization (GRPO)**, with objective
\[
\begin{aligned}
\mathcal{L}_{\text{GRPO}}
&= \mathbb{E}_{o\sim\pi_\theta}\Big[ \min(\rho_{o}A_{o}, \text{clip}(\rho_o,1-\epsilon,1+\epsilon)A_o) \Big] \\
&\quad - \beta\, \mathcal{D}_{\mathrm{KL}(\pi_\theta \Vert \pi_{\mathrm{ref}})}
\end{aligned}
\]
where
\[
\rho_o = \frac{\pi_\theta(o)}{\pi_{ref}(o)}, \qquad
A_o = \frac{r_o - \bar{r}}{\text{std}(r)}.
\]

The remaining reported training settings are batch size **128**, learning rate **\(1\times10^{-6}\)**, KL coefficient **0.01**, and total training with only **4,058 samples** and **80 + 300 RL steps**.

## 4. Dataset construction and PhyCritic-Bench

The critic training set is constructed from four embodied or robotics video sources: **RoboVQA**, **BridgeData V2**, **HoloAssist**, and **AgiBot World**. These cover egocentric and third-person viewpoints and tasks such as grasping, stacking, folding, and assembly. Questions are built using **Cosmos-Reason1 RL** data, which provides **800 high-quality physical question–answer pairs** involving physical perception, planning, and reasoning [2602.11124].

Candidate responses are drawn from seven multimodal models: **GPT-4o**, **Gemini-2.5-Flash**, **Qwen2.5-VL-72B**, **InternVL3-38B**, **Cosmos-Reason1-7B**, **Video-R1**, and **MiMo-VL-7B**. All are prompted to generate chain-of-thought responses. **GPT-4o** then verifies each response against ground truth and assigns a binary preference score. Chosen and rejected responses are paired, and after balancing, the final critic training set contains **3,258 samples**.

For evaluation, the paper introduces **PhyCritic-Bench**, a physical-domain multimodal judge benchmark containing **225 evaluation samples**. It spans robotics tasks from RoboVQA, BridgeData V2, HoloAssist, AgiBot, and RoboFail, with questions adapted from CosmosReason1-Bench, together with autonomous driving from **LingoQA**. Each benchmark sample is a pairwise preference tuple \((q, l_a, l_b, p)\). For each prompt, one model generates **\(N=8\)** chain-of-thought responses via temperature sampling; **GPT-4o** verifies them against ground truth, and one correct and one incorrect response are paired.

Performance is measured as pairwise preference accuracy:
\[
Acc = \mathbb{I}\big(\text{VLM}(q,l_a,l_b) = p\big).
\]

In addition to PhyCritic-Bench, the model is evaluated on **VL-RewardBench** and **Multimodal RewardBench** for general multimodal judge performance, and on **CosmosReason1-Bench**, **CV-Bench**, and **EgoPlanBench2** to test whether critic training also improves policy-like reasoning.

## 5. Quantitative results, transfer, and ablations

On **PhyCritic-Bench**, **PhyCritic-7B** achieves **68.0** overall accuracy, best among open-source 7B/8B models. The reported baselines are **Eagle-2.5-8B: 56.0**, **Qwen2.5-VL-7B: 51.6**, **RoboBrain2.0-7B: 54.7**, and **Cosmos-R1-7B: 51.1**. Subsuite scores are **AgiBot: 78.8**, **HoloAssist: 65.5**, **RoboVQA: 86.7**, **Bridge: 65.6**, **RoboFail: 57.4**, and **LingoQA: 60.0**. Proprietary reference points are **Gemini-2.5-Pro: 78.2**, **Gemini-2.5-Flash: 67.1**, and **GPT-4o: 64.7**, so the model exceeds GPT-4o on this benchmark but not Gemini-2.5-Pro [2602.11124].

The model also improves general multimodal judging despite being trained only on physical critic data. On **VL-RewardBench**, it scores **57.3** versus **53.2** for **Qwen2.5-VL-7B**; on **Multimodal RewardBench**, **65.9** versus **64.0**. The paper contrasts this with **UnifiedReward-Think-Qwen-7B**, which is stronger on the general reward benchmarks but scores only **52.4** on PhyCritic-Bench and **51.8** on CosmosReason1-Bench.

A further result is that PhyCritic also acts as a stronger policy model. On **CosmosReason1-Bench**, **PhyCritic-7B** scores **63.9**, compared with **63.0** for **Cosmos-R1-7B** and **54.3** for **Qwen2.5-VL-7B**. On **CV-Bench**, it reaches **79.7** overall, including best open-source 3D score **83.9**. On **EgoPlanBench2**, it reaches **42.3** overall.

The ablation studies are central to the method’s interpretation. Full two-stage RL gives **68.0** on PhyCritic-Bench, **63.9** on CosmosR1-Bench, and **57.3** on VL-Reward. Removing self-reference yields **64.4 / 62.6 / 56.6**; removing the self-prediction reward yields **65.8 / 63.5 / 56.5**; removing the hand-crafted evaluation criteria yields **63.9 / 62.0 / 55.1**. The appendix also varies \(\alpha_{sp}\), reporting that \(\alpha_{sp}=0.2\) gives the best balance among the tested settings.

The paper’s judgment-stability analysis reports a strong positive dependence between the critic’s own answer correctness and its judging correctness: after Stage 1 warmup,
\[
\chi^2 = 51.07,\quad p = 8.93 \times 10^{-13},
\]
and in the final self-referential model,
\[
\chi^2 = 161.76,\quad p = 4.66 \times 10^{-37}.
\]
This is presented as direct evidence for the claim that better self-predictions lead to better judgments.

## 6. Downstream use, limitations, and significance

PhyCritic is used not only as a judge but also as a reward model for improving policies. In best-of-\(N\) test-time selection, using **Qwen2.5-VL-7B-Instruct** as a policy and PhyCritic as a pairwise knockout judge on CosmosReason1-Bench, the paper reports an improvement from **54.3** to **60.8** at \(N=32\). In a downstream **DPO** experiment, starting from **Qwen2.5-VL-7B-Instruct**, the reported scores on CosmosReason1-Bench are **54.3** for the base model, **57.5** for answer-verify DPO, and **60.0** for **PhyCritic-7B DPO** [2602.11124].

The paper assigns three forms of significance to the system. First, it reframes critic modeling for physical AI around physically grounded and causally valid evaluation rather than generic plausibility. Second, it proposes self-referential judging as a concrete training principle: solve first, judge second. Third, it argues that critic training and policy improvement are intertwined, since the critic itself becomes a stronger physically grounded reasoner.

The main acknowledged limitation is that self-referential critic finetuning relies on access to ground-truth answers \(A_Q\), because the self-prediction reward
\[
r_{sp} = \mathbb{I}(\hat{A}_{pred} = A_Q)
\]
requires verifiable supervision. A second limitation is that the current setup uses a relatively simple binary preference construction based on correctness labels verified by GPT-4o, which may not capture more subtle human preferences beyond correctness, such as risk sensitivity, style of explanation, or graded planning quality. The suggested future directions are **self-verification** or **meta-judging** to replace explicit self-prediction supervision, extension toward **multi-round critic self-refinement**, and use of critic signals to iteratively improve self-generation.

Within the broader literature on critique-centered AI, PhyCritic occupies a distinct position. Whereas SCALAR studies an Actor--Critic--Judge pipeline for quantum field theory and string theory and reports that multi-turn dialogue improves over single-shot attempts throughout, PhyCritic addresses multimodal judging in physically grounded perception, causal reasoning, and planning [2605.06772]. A plausible implication is that critique-centric architectures are becoming domain-specific: in theoretical physics the central question is iterative derivational refinement, while in physical AI it is physically correct pairwise evaluation under multimodal grounding.

Source: https://www.emergentmind.com/topics/phycritic