Agentic Critic in AI Systems

• Agentic Critic is a mechanism in AI systems that evaluates intermediate actions, reasoning traces, and candidate strategies to influence subsequent computation.
• It operates by selecting, intervening, or revising action trajectories, ensuring dynamic improvements across video reasoning, search, and documentation maintenance.
• Empirical results demonstrate measurable gains in performance and safety, though challenges such as feedback miscalibration and computational overhead remain.

Agentic Critic denotes a class of mechanisms in agentic AI systems in which a model evaluates intermediate actions, reasoning traces, retrieved evidence, drafted outputs, or candidate strategies inside an iterative workflow rather than only scoring a final answer. In recent work, this role appears as a prompted selector over video-reasoning strategies, a process-level intervention module for reinforcement learning, a retrieval sufficiency judge in agentic search, a reflexive verifier for code-grounded documentation repair, a natural-language critic for off-policy language-space learning, and a safety layer that challenges recommendations before human review (Menon et al., 9 Sep 2025). The common pattern is not a single architecture but a family of designs in which critique changes subsequent computation: by selecting among trajectories, rewriting queries, rewinding adverse steps, revising drafts, shaping policy updates, or constraining unsafe outputs.

1. Conceptual scope and defining variants

Across the literature, the critic function is attached to different objects. In "CAViAR" (Menon et al., 9 Sep 2025), the critic judges sampled strategies—tool sequences such as get_segment plus retrieval_qa plus find_when—because multiple plausible action sequences can be produced by the agentic video-reasoning pipeline, but not all are equally reliable. In "ProCeedRL" (Gao et al., 2 Apr 2026), the critic operates at the process level, monitoring each intermediate interaction step online and intervening when the score falls below a threshold. In "Critic-R" (Alam et al., 30 May 2026), the critic evaluates whether retrieved documents are satisfactory for the next reasoning step by using the reasoner’s own introspective trace. In "DocSync" (Badrinarayan et al., 4 May 2026), the critic checks whether a generated documentation update is semantically consistent with the changed code. In "Agentic Critical Training" (Liu et al., 9 Mar 2026), the policy itself is trained to judge which action is better among alternatives. In "Natural Language Actor-Critic" (Hong et al., 4 Dec 2025), the critic outputs natural language rather than scalar values.

This diversity produces several technically distinct notions of critique. Some systems use a selector that ranks trajectories or strategies without modifying them directly, as in CAViAR’s prompted LLM selector (Menon et al., 9 Sep 2025). Some use an interventionist critic that rewinds and refines actions before the rollout continues, as in ProCeedRL (Gao et al., 2 Apr 2026). Others use a verification critic that accepts or rejects outputs and feeds back reasons for revision, as in DocSync (Badrinarayan et al., 4 May 2026) and XAI narrative generation (He et al., 20 Mar 2026). Still others attach the critic to retrieval, value estimation, or policy refinement rather than to textual answers themselves (Alam et al., 30 May 2026).

A concise way to organize the space is by what the critic evaluates and what consequence follows.

Representative system	What the critic evaluates	What the critic changes
CAViAR (Menon et al., 9 Sep 2025)	Candidate strategy trajectories	Winning strategy selection
ProCeedRL (Gao et al., 2 Apr 2026)	Intermediate action plus next observation	Rewind and refined action
Critic-R (Alam et al., 30 May 2026)	Retrieval sufficiency for next reasoning step	Query and instruction refinement
DocSync (Badrinarayan et al., 4 May 2026)	Draft docstring against code	Reflexive draft revision
NLAC (Hong et al., 4 Dec 2025)	Natural-language action quality	Refined action distillation
Android Coach (Gan et al., 8 Apr 2026)	Multiple candidate actions per state	Group-wise policy improvement

This suggests that “agentic critic” is best treated as an architectural role rather than a fixed algorithm. The role is defined by closed-loop evaluation inside an agent workflow, not merely by the existence of a scalar reward.

2. Architectural patterns in agentic-critic systems

A recurring pattern is adaptive tool orchestration plus post hoc strategy selection. CAViAR exposes the reasoning agent to think(thought), get_segment(start, end), find_when(query, video_segment), asr_understanding(question, answer_options), retrieval_qa(question, answer_options, video_segment), and finish(final_answer). The LLM acts as a controller that iteratively calls one tool at a time and decides the next tool based on previous outputs. The critic then receives the question, candidate strategies, and examples of trajectories and outcomes, and outputs Winning Strategies: A, B, .... It is instructed that if multiple strategies lead to the same correct final answer, multiple can be selected, but conflicting answers cannot be selected together (Menon et al., 9 Sep 2025). The critic therefore judges the quality of the action sequence rather than only the terminal answer.

A second pattern is online process monitoring and intervention. ProCeedRL identifies a “vicious circle” in long-horizon exploration: a suboptimal action elicits irrelevant or misleading observations, these are added to the history, and later decisions degrade further. Its process critic evaluates each step in real time,

$l_t, c_t = \phi(\tau_t, a_t, s_{t+1}),$

and when the score falls below $l_{\text{th}}$, the method rewinds and invokes a refining policy,

$a_t' = \mu(\tau_{t-1}, a_t, l_t, c_t).$

The refined action replaces the original one, so the rollout proceeds from a corrected step instead of propagating the flawed one (Gao et al., 2 Apr 2026). Here the critic is not merely advisory; it changes the experienced context.

A third pattern is evaluation–repair separation. Critic-R separates satisfaction judgment from query refinement. The critic $\mathcal{M}_C$ first receives the global query $Q$, local sub-query $q_i^{(t)}$, retrieved documents $D_i^{(t)}$, and introspective trace $T_{i+1}^{(t)}$, and returns a binary verdict $\sigma_i^{(t)} \in \{yes, no\}$ and a reason $r_i^{(t)}$. Only if the verdict is negative does the system enter query refinement mode, producing a refined instruction and refined query (Alam et al., 30 May 2026). DocSync uses the same architectural separation in a different domain: the model drafts $l_{\text{th}}$0, the critic returns IsGood, Reason, and the reason is appended to the prompt for another pass if needed (Badrinarayan et al., 4 May 2026).

A fourth pattern is critic as safety or governance layer. In commercial insurance underwriting, the primary agent performs underwriting analysis and the critic agent acts as a skeptical internal reviewer that cross-checks facts, checks alignment with rules and appetite constraints, identifies missing risk factors, flags contradictions, tests robustness using hypothetical adversarial scenarios, and can force revision before the case reaches the human underwriter (Roy et al., 21 Jan 2026). The system permits one full critique-revision loop and remains explicitly decision-negative: the AI can recommend but cannot make binding underwriting decisions.

A fifth pattern is critic as coach over unexecuted alternatives. Android Coach reframes online Android agent RL from Single State Single Action to Single State Multiple Actions. For one collected state $l_{\text{th}}$1, it resamples multiple candidate actions $l_{\text{th}}$2 and evaluates them with a learned critic $l_{\text{th}}$3 without extra emulator overhead (Gan et al., 8 Apr 2026). The critic therefore coaches policy improvement over hypothetical actions rather than only over executed ones.

3. Learning objectives, scoring rules, and formalizations

The literature contains several distinct formalizations of critique. Some critics are prompted selectors with no explicit supervised loss in the provided text. CAViAR’s supplemental material gives no formal critic loss; instead, the critic uses 4 in-context examples per prompt, examples authored to be in-distribution for the target dataset, and a decision rule that selects the strategy most likely to yield a correct answer, selecting all strategies that produce the same final answer when appropriate (Menon et al., 9 Sep 2025).

Other systems define explicit reward or value objectives. ACT reframes post-training from action imitation to action judgment. Imitation learning is written as

$l_{\text{th}}$4

whereas ACT rewards correct judgment between expert and alternative actions with

$l_{\text{th}}$5

The appendix specifies $l_{\text{th}}$6 as exact match reward, $l_{\text{th}}$7 as partial credit for admissible but non-expert actions, and $l_{\text{th}}$8 as a format penalty tied to <action>...</action> tags. Optimization uses Group Relative Policy Optimization (GRPO) (Liu et al., 9 Mar 2026). The critic-like capability is learned by rewarding whether the model’s judgment is correct, not only whether its action matches the expert.

ProCeedRL extends a DAPO-style group-based RLVR setup with a mixed buffer $l_{\text{th}}$9 formed from on-policy rollouts $a_t' = \mu(\tau_{t-1}, a_t, l_t, c_t).$0 and refined demonstration trajectories $a_t' = \mu(\tau_{t-1}, a_t, l_t, c_t).$1. Demonstration steps are off-policy and masked in failed trajectories, and the demonstration loss is reweighted by

$a_t' = \mu(\tau_{t-1}, a_t, l_t, c_t).$2

while on-policy steps use weight 1. The objective is a clipped, group-based relative-advantage loss with importance sampling (Gao et al., 2 Apr 2026). In this formulation, the critic is part of a rollout-and-data-collection scheme rather than a separate standalone RL algorithm.

Critic-R formalizes critic-generated supervision for retriever optimization. Retrieved documents accepted by the critic become positives, rejected documents become intra-trajectory hard negatives, and the retriever is trained with the contrastive loss

$a_t' = \mu(\tau_{t-1}, a_t, l_t, c_t).$3

Critic-Embed is initialized from Stella-400M and trained with InfoNCE using $a_t' = \mu(\tau_{t-1}, a_t, l_t, c_t).$4 (Alam et al., 30 May 2026).

NLAC replaces scalar value estimation with a natural-language critic $a_t' = \mu(\tau_{t-1}, a_t, l_t, c_t).$5. A successor model $a_t' = \mu(\tau_{t-1}, a_t, l_t, c_t).$6 is trained via a language Bellman backup, and an evaluator $a_t' = \mu(\tau_{t-1}, a_t, l_t, c_t).$7 aggregates sampled futures into the final critique. Actor improvement occurs through self-refinement,

$a_t' = \mu(\tau_{t-1}, a_t, l_t, c_t).$8

followed by a distillation-style projection of the base policy onto the refinement policy, rather than a policy gradient update (Hong et al., 4 Dec 2025). This is one of the clearest formulations in which critique is itself the learning signal.

Android Coach uses a conventional scalar critic but couples it to a Process Reward Model and a group-wise advantage estimator. The process reward model is trained with

$a_t' = \mu(\tau_{t-1}, a_t, l_t, c_t).$9

and the actor update uses the ACLOO leave-one-out advantage

$\mathcal{M}_C$0

The paper states that this estimator is unbiased and shift-invariant (Gan et al., 8 Apr 2026).

Finally, several papers clarify the boundary cases. SeeUPO argues that mainstream backbone RL algorithms cannot simultaneously achieve both critic-free and convergence guarantees in multi-turn scenarios, and proposes a critic-free alternative based on sequential turn-by-turn updates in reverse execution order (Hu et al., 6 Feb 2026). EVOM, by contrast, makes the outer design loop agentic but does not make the runtime critic an autonomous evaluator; the LLM functions as an architecture designer for actor-critic networks rather than as a decision-time critic (Zhang et al., 24 Jun 2026).

4. Empirical domains and reported effects

Agentic critics have been evaluated across long-video reasoning, search, documentation maintenance, embodied and web tasks, GUI control, front-end code generation, XAI narrative generation, theoretical physics reasoning, underwriting, and general agent benchmarks.

In long-video reasoning, CAViAR reports on EgoSchema that SeViLA scores 25.7, LLoVi 57.6, VideoAgent 60.2, VideoTree 66.2, Direct inference 70.6, and CAViAR 72.2, an improvement of 1.6 points over direct inference on the evaluation subset of 500. On 25% of LVBench with GPT-4o-mini, direct inference is 43.2 and CAViAR is 49.0, a 5.8-point gain (Menon et al., 9 Sep 2025). The reported analyses attribute the gains to choosing strategies that properly localize with get_segment, preferring transcript-based reasoning when the task is speech-heavy, and avoiding strategies that guess after failing to find exact evidence.

In deep search and embodied planning, ProCeedRL reports for Qwen3-8B: 73.87% on Bamboogle, 29.52% on MuSiQue, 46.42% on Frames, 13.79% on GAIA, and 23.01% on WebWalkerQA, compared with DAPO/Search-R1 at 70.83%, 23.60%, 43.59%, 10.10%, and 19.51%. On AlfWorld, ProCeedRL reaches 51.43% in-distribution and 55.22% out-of-distribution for Qwen3-8B, versus DAPO at 45.23% and 53.24%; ProCeedSFT reaches 57.14% and 58.95% with an additional SFT stage on correct samples collected during rollout (Gao et al., 2 Apr 2026).

In retrieval-centric agentic search, Critic-R reports an overall relative improvement of 12.4% for Critic-R-Zero in the multi-hop setting. For retriever comparison, the reported multi-hop averages at $\mathcal{M}_C$1 are Stella-400M: 0.3472 / 0.4470, Agentic-R: 0.3670 / 0.4564, and Critic-Embed: 0.3794 / 0.4806. Combining Critic-Embed with Critic-R-Zero yields full Critic-R at 0.3957 / 0.4959 (Alam et al., 30 May 2026).

In documentation maintenance, DocSync (Final) reports BLEU 0.575, BERTScore F1 0.985, Summary Exact 0.969, and Judge 3.44, compared with CodeT5-base at BLEU 0.193, BERTScore F1 0.880, Summary Exact 0.188, and Judge 1.91. Relative to DocSync (Initial), the final critic pass raises BERTScore F1 from 0.980 to 0.985, Summary Exact from 0.938 to 0.969, and Judge from 3.25 to 3.44, with little BLEU change (Badrinarayan et al., 4 May 2026). The interpretation given in the paper is that the critic improves semantic faithfulness rather than surface n-gram overlap.

In benchmarked action-judgment training, ACT on Qwen3-8B reports ALFWorld ID/OOD 72.86/72.39 for ACT alone, 91.43/87.31 for IL w/ ACT, and 92.86/88.06 for RL w/ ACT. The paper summarizes average gains of 5.07 points over imitation learning, 4.62 points over reinforcement learning, and 2.42 points over Early Experience (Liu et al., 9 Mar 2026). It also reports improvements on MATH-500 and GPQA-Diamond without reasoning-specific training data.

In Android GUI control, Android Coach improves UI-TARS-1.5-7B from 31.9% to 39.4% on AndroidLab and from 32.8% to 41.1% on AndroidWorld, corresponding to gains of 7.5% and 8.3%. At matched success rates, it attains 1.4x higher training efficiency than Single State Single Action methods PPO and GRPO (Gan et al., 8 Apr 2026).

In front-end code generation, ReLook reports a monotone ranking $\mathcal{M}_C$2 across ArtifactsBench subsets and attributes the gains to the MLLM-in-the-loop visual critic, the zero-reward rule for invalid renders, and Forced Optimization. On ArtifactsBench-Lite for Qwen2.5-7B-Instruct, the ablation rises from base 21.59 to 24.89 with MLLM reward, to 25.84 with format constraint, and to 27.88 with Forced Optimization (Li et al., 13 Oct 2025).

In XAI narrative generation, the Basic Design, Critic Design, and Critic-Rule Design improve faithfulness across all LLMs, and Claude Sonnet 4.5 on Basic Design reduces unfaithful narratives from 30 to 3 by round 2, which the paper presents as a 90% reduction after three rounds (He et al., 20 Mar 2026).

In regulated underwriting, the adversarial critique mechanism reduces hallucination rates from 11.3% to 3.8% and increases decision accuracy from 92% to 96% on 500 expert-validated cases. It also raises contradiction detection from 76% to 89%, source traceability from 81% to 96%, guideline compliance from 95% to 98.5%, and risk factor recall from 88% to 95% (Roy et al., 21 Jan 2026).

5. Failure modes, limitations, and negative results

A major result of the literature is that critique is not automatically beneficial. "Helpful Agent Meets Deceptive Judge" shows that feedback-based workflows are vulnerable when the judge is misleading, biased, or adversarial. The paper introduces a two-dimensional taxonomy over intent and knowledge access, distinguishes constructive, hypercritical, and malicious judges, and distinguishes no-knowledge, parametric-knowledge, and grounded-knowledge judges. It also constructs WAFER-QA with 574 contextual samples and 708 non-contextual samples to evaluate grounded adversarial critiques (Ming et al., 3 Jun 2025). The central empirical finding is that even strong models can switch correct answers after a single round of misleading feedback; for example, GPT-4o on ARC-Challenge drops from 96.5% to 76.0% after one round of baseless criticism, and grounded critiques on WAFER-QA often cause drops over 50% from initial accuracy after one round (Ming et al., 3 Jun 2025).

Theoretical-physics reasoning yields a second negative result. SCALAR reports that multi-turn dialogue improves over single-shot attempts throughout, but the mechanism of improvement and the value of prompting choices depend strongly on the Actor–Critic pairing. In asymmetric Actor–Critic settings, such as Haiku Actor with stronger Sonnet Critic, constructive feedback improves mean-score outcomes. In same-family pairings such as DS8B or DS70B with DS70B Critic, strategy effects are weaker, lenient feedback is sometimes favored, and strict and adversarial feedback are not beneficial (Niarchos et al., 7 May 2026). The hardest task, Polchinski 2.7, remains a bottleneck even with larger models and structured critique.

Several papers also document intrinsic failure modes of the critics themselves. CAViAR notes that retrieval_qa alone may hallucinate timestamps or answers, find_when can return ambiguous ranges, asr_understanding may help on audio-rich videos but hurt on purely visual reasoning, and strategies may stop too early and guess (Menon et al., 9 Sep 2025). ProCeedRL states that the method adds computation, roughly $\mathcal{M}_C$3 vanilla exploration in their estimates, and does not guarantee improvement because the critic and refiner are themselves LLM-based and can still make mistakes (Gao et al., 2 Apr 2026). Android Coach reports that training the critic from scratch causes divergence and that PRM pretraining is needed for stable critic learning (Gan et al., 8 Apr 2026). DocSync’s remaining failure modes include minor formatting debris, over-elaboration or hallucinated details, and truncation or repetition on long docstrings (Badrinarayan et al., 4 May 2026). XAI narrative generation identifies extraction mistakes, confusing faithfulness instruction, counterintuitive feature influence, misleading coherence instruction, and generation misalignment as persistent causes of error (He et al., 20 Mar 2026).

There is also a methodological controversy over whether a critic must exist at all. SeeUPO argues that mainstream methods cannot jointly provide critic-free training and convergence guarantees in multi-turn settings, while ProAct introduces a parameter-free Monte-Carlo Critic to stabilize PPO and GRPO with environment-grounded rollouts (Hu et al., 6 Feb 2026). This suggests that some of the observed benefits attributed to “agentic critics” are entangled with deeper questions about credit assignment, long-horizon variance, and the limits of critic-free optimization.

6. Broader landscape: judges, reward models, self-critique, and agency

The broader agentic-AI literature places the agentic critic inside a larger taxonomy of evaluators. "The Hitchhiker's Guide to Agentic AI" treats critics as classical value functions in actor-critic RL, reward models trained with Bradley–Terry preference objectives, human or LLM judges, group-relative baselines in GRPO, verification-based rewards, process reward models, and agent evaluation frameworks (Roitman, 22 Jun 2026). This wider view is consistent with the empirical papers: some critics are learned value estimators, some are prompted evaluators, some are reward models, and some are explicit judges in multi-agent loops.

At the same time, the conceptual literature cautions against conflating critique mechanisms with full agency. "Critique of Agent Model" distinguishes agentic systems, whose competence resides in engineered workflows, from agentive systems, whose capabilities arise endogenously. It argues that genuine agency requires internalized structures of goal, identity, decision-making, self-regulation, and learning, rather than external scaffolding alone (Xing et al., 22 Jun 2026). From that perspective, many current agentic critics are components inside larger engineered loops rather than evidence of fully internalized self-regulation.

A balanced synthesis is therefore possible. On one side, agentic critics have shown measurable benefits in long-video reasoning, search, documentation maintenance, agent training, GUI control, front-end code generation, and regulated workflows (Menon et al., 9 Sep 2025). On the other side, critique can destabilize systems when judges are deceptive, when feedback strategies are mismatched to model pairings, or when the critic itself is noisy, miscalibrated, or computationally burdensome (Ming et al., 3 Jun 2025). The literature therefore supports a restrained definition: an agentic critic is an evaluator embedded in an iterative agent workflow whose outputs alter subsequent reasoning, retrieval, editing, or optimization. Whether that evaluator is helpful, harmful, or unnecessary depends on the evaluated object, the intervention mechanism, the training regime, and the robustness of the surrounding system.