Behavior-Grounded LLM Judge

• Behavior-grounded LLM judges are evaluation systems that tie model judgments to explicit behavioral targets and observable signals like rubric criteria and hidden passages.
• They integrate diverse grounding signals—including document evidence, rubric-based scores, and persona preferences—to provide structured partial credit over multiple evaluative dimensions.
• Their formal architectures support reinforcement learning, prompt adaptation, and reliability diagnostics, which drive improved performance and robust evaluation outcomes.

A behavior-grounded LLM judge is an evaluation system in which an LLM’s judgments are anchored to observable behaviors rather than to a single holistic impression. In the most explicit formulation, it evaluates what a policy does against structured criteria grounded in external evidence, and its judgments can be used directly as a reward signal or as an evaluation score (Bhattarai et al., 8 May 2026). Across recent work, the grounding signal may be a hidden document passage, a panel of rubric-conditioned subjudges, human- or persona-like preference behavior, internal representations of a smaller model, pairwise consistency structure, or trajectory-level agent behavior (Bhattarai et al., 8 May 2026). The concept therefore spans both judge construction and judge diagnosis: it concerns how the judge is constrained, what behavioral dimensions it scores, how those scores are aggregated, and whether the resulting evaluator is reliable on the kinds of behaviors that matter in deployment (Gupta et al., 16 Apr 2026).

1. Definition and conceptual scope

The defining feature of a behavior-grounded LLM judge is that evaluation is tied to explicit behavioral targets rather than left as an opaque scalar preference. In the rubric-grounded formulation, a frozen LLM judge reads a question, hidden grounding, the policy’s answer, and a rubric of weighted criteria, then returns per-criterion scores that are aggregated into reward (Bhattarai et al., 8 May 2026). This makes the optimization target the judged behavior itself: correctness, coverage, terminology, explanation, caveats, or other rubric dimensions. The same work characterizes this as behavior-grounded in three senses: the rubric decomposes quality into explicit behavioral dimensions, the judge’s evaluations are grounded in a hidden document passage, and the policy is optimized to produce behaviors that the judge can verify even though the policy never sees the passage (Bhattarai et al., 8 May 2026).

A broader formulation appears in multi-judge aggregation work, where the goal is not to trust a single rubric or judge but to learn how multiple rubric-conditioned judges map onto observed preference behavior. There, the relevant object is an aggregator

$f_\theta : \mathbb{R}^K \to \mathbb{R},$

trained to predict persona-based preference labels from a vector of judge scores (Sprejer et al., 29 Oct 2025). In that setting, judges encode normative dimensions such as truthfulness or helpfulness, while personas encode heterogeneous preference behavior. A plausible implication is that behavior grounding need not mean document grounding alone; it can also mean grounding judge outputs in empirically modeled preference behavior.

Other papers extend the same idea to prompt optimization, representation probing, and reliability analysis. A dynamic multi-agent system can iteratively redesign a judge prompt so that its behavior better matches human semantic judgments across tasks and answer styles (Cao et al., 1 Apr 2025). A probing-based evaluator can infer aspect-level scores from internal representations of a small LM, motivated by the claim that evaluation requires less semantic capacity than generation (Li et al., 30 Jan 2026). Reliability diagnostics treat judges as stochastic measurement processes with observable properties such as transitivity violations and conformal uncertainty (Gupta et al., 16 Apr 2026). Taken together, these strands define behavior-grounding as a family of methods for constraining, interpreting, and calibrating LLM judges through behavioral evidence rather than through unconstrained free-form scoring.

2. Grounding signals and behavioral dimensions

Behavior-grounded judges differ chiefly in what grounds the judgment. One common pattern is document grounding. In rubric-grounded RL, task instances are triples $(x,g,R)$, where $x$ is the prompt shown to the policy, $g$ is auxiliary grounding shown only to the judge, and $R$ is a rubric of criteria (Bhattarai et al., 8 May 2026). The judge is therefore privileged relative to the policy: it can check whether claims are supported by the source passage, penalize hallucinations, and verify whether required concepts and caveats are present. This asymmetry is central to the method’s notion of grounding.

A second pattern is rubric grounding without direct exposure of the rubric’s latent weights to the policy. In the same framework, the rubric is

$$R=\{c_1,\ldots,c_M\}, \qquad c_j=(w_j,\eta_j,\mathcal{E}_j,\kappa_j,\nu_j),$$

where each criterion has a non-negative weight, a natural-language description, required elements, expected keywords, and a verification method (Bhattarai et al., 8 May 2026). Because the judge scores each criterion separately, the reward represents partial credit over a structured behavior vector rather than a binary correctness bit.

A third pattern is grounding in human-like preference behavior. Multi-judge aggregation constructs persona-based labels by prompting LLM personas such as “Professor,” “CEO,” “Parent,” “Data Scientist,” or “Lawyer” to score candidate answers on a 0–10 scale, then learns an aggregator over rubric-conditioned judges to approximate those labels (Sprejer et al., 29 Oct 2025). The behavioral target is not a universal rubric but the way different personas trade off normative dimensions. This suggests a descriptive form of grounding: a judge can be grounded in modeled preference heterogeneity rather than in a single canonical standard.

A fourth pattern is trajectory and state grounding. In production multi-turn transactional agents, the core missing dimensions are not surface quality but state-tracking, guardrails, recovery, and safety (Zhang et al., 9 Jun 2026). That case study shows that a judge with only intent, brand-voice, and personalization axes misses most systematic cross-turn defects because the rubric has no category for the behavioral dimensions where the failures cluster. A behavior-grounded judge in that setting would require access to the conversation arc, tool traces, and state snapshots rather than only the latest response.

The same contrast appears in contextual safety evaluation. Safety is treated as a contextual property depending on policy, language, and domain, but current LLM judges often revert to rigid internal safety priors when the explicit definition conflicts with those priors (Alloula et al., 5 Jun 2026). Here the intended grounding signal is the prompt-level policy specification and the attached contextual explanation, but the observed behavior shows incomplete grounding when the model’s internal boundary dominates.

Grounding mode	Primary signal	Representative paper
Document-grounded rubric judge	Hidden passage $g$ plus rubric $R$	(Bhattarai et al., 8 May 2026)
Persona-grounded aggregation	Persona-scored preference labels	(Sprejer et al., 29 Oct 2025)
Prompt-optimized semantic judge	Task examples plus STS-style rubric	(Cao et al., 1 Apr 2025)
Representation-grounded evaluator	Hidden states and attention features	(Li et al., 30 Jan 2026)
Trajectory-grounded agent judge	Full transcript, state, and guardrails	(Zhang et al., 9 Jun 2026)

3. Formal architectures and scoring mechanisms

The most explicit formalization is the rubric-grounded reward model. Given a response $y \sim \pi_\theta(\cdot \mid x)$, the judge returns criterion scores

$$\{s_j\}_{j=1}^M=\mathcal{J}(x,g,y,R), \qquad s_j \in [0,w_j],$$

and the normalized reward is

$(x,g,R)$0

Equivalently, with normalized criterion scores $(x,g,R)$1 and normalized weights $(x,g,R)$2,

$(x,g,R)$3

This makes the scalar reward a projection of a multidimensional behavior vector onto the rubric weight vector (Bhattarai et al., 8 May 2026). The formulation is notable because it preserves interpretability at the criterion level while remaining compatible with scalar RL updates.

A different architecture aggregates multiple rubric-conditioned judge outputs. If judges $(x,g,R)$4 assign scalar scores $(x,g,R)$5 to a prompt–answer pair, the aggregator learns

$(x,g,R)$6

with mean squared error against persona-based targets (Sprejer et al., 29 Oct 2025). Two concrete implementations are examined: a Generalized Additive Model,

$(x,g,R)$7

and a small MLP. The GAM exposes judge-wise response curves and feature importance, while the MLP captures interactions at the cost of interpretability (Sprejer et al., 29 Oct 2025).

Representation-as-a-judge replaces generated evaluation text with probing over hidden states. For aspect $(x,g,R)$8, a small LM processes an aspect-specific evaluation prompt $(x,g,R)$9, hidden states $x$0 are featurized into $x$1, and a lightweight classifier predicts a score

$x$2

This is a decoding-free evaluation path: no natural-language judgment need be generated at inference time (Li et al., 30 Jan 2026). The central claim is that evaluative signals are already encoded in mid-to-upper layers of small models, even when those models are weak generative judges.

Another line treats the judge explicitly as a measurement device. In SummEval, pairwise judges induce a tournament graph over summaries for each document and criterion, while direct scorers emit Likert scores $x$3 (Gupta et al., 16 Apr 2026). Transitivity violations are measured by directed 3-cycles, and conformal prediction sets are built over score residuals. This does not define a new judge architecture so much as a new formal layer for characterizing judge behavior per instance.

4. Optimization roles: evaluation, reward, and adaptation

Behavior-grounded judges are used in at least three optimization roles. The first is as a direct reward mechanism in reinforcement learning. In rubric-grounded RL, a frozen judge provides the reward for GRPO training of Llama-3.1-8B-Instruct on roughly 100,000 OSTI-derived question–passage–rubric triples (Bhattarai et al., 8 May 2026). The objective includes the rubric-judged reward and a KL penalty to a reference model. For each prompt, GRPO samples a group of responses, computes judged rewards under the same hidden passage and rubric, and forms group-relative advantages. On held-out rubric evaluation, the GRPO-tuned policy reaches $x$4 normalized reward, versus $x$5 for the base model and $x$6 for the SFT baseline (Bhattarai et al., 8 May 2026). The same paper reports improvements over the base model on GSM8K, MATH, GPQA Main, and GPQA Diamond, which it interprets as evidence that structured, document-grounded rewards induce transferable reasoning behaviors beyond the training corpus (Bhattarai et al., 8 May 2026).

The second role is as a learned reward or routing layer. In the multi-judge aggregation framework, the learned function $x$7 can replace a scalar reward model in RLHF, DPO, or routing systems by scoring candidate answers from multiple judges and approximating persona-based preference behavior (Sprejer et al., 29 Oct 2025). This introduces a modular design in which rubric-conditioned judges remain frozen while the aggregator is trained to match the desired behavioral target.

The third role is prompt-level adaptation. A multi-agent system with a Sample Selection Agent, Evaluation Agent, and ReWrite Agent iteratively refines judge prompts against representative examples until the induced judge behavior matches human semantic criteria more closely (Cao et al., 1 Apr 2025). The final prompt optimized by this loop improves AUC on Instruct-QA from $x$8 for the initial prompt and $x$9 for a few-shot prompt to $g$0, and improves Pearson correlation on STS-B from $g$1 for the initial prompt to $g$2 (Cao et al., 1 Apr 2025). This is a behavior-grounded optimization process because the object being improved is the judge’s behavior on concrete samples rather than its weights.

5. Reliability, bias, and failure modes

A central theme in the recent literature is that behavior-grounding requires explicit reliability analysis, not just aggregate agreement. One major failure mode is asymmetry in judge error. In open-ended code-feedback validation, LLM judges exhibit strong “agreeableness bias”: they have high true positive rates on valid outputs but very low true negative rates on invalid ones (Jain et al., 13 Oct 2025). For 14 validators, the paper reports $g$3 for most validators and $g$4 typically, with overall invalid feedback comprising roughly $g$5 of items (Jain et al., 13 Oct 2025). This produces inflated precision estimates for generators unless the bias is corrected. The paper introduces an optimal minority-veto ensemble and a regression-based framework that models validator TPR and TNR using a small human-annotated calibration set. On 366 high-school Python programs, the regression approach reduces maximum absolute error to $g$6, a $g$7 improvement over the best ensemble of 14 LLMs (Jain et al., 13 Oct 2025).

Another failure mode is prompt, rubric, and calibration instability. The multi-judge aggregation paper motivates its framework by rubric sensitivity, bias, and instability in single judges, and shows that learned aggregators are more robust than naive means under monotonic distortions of judge scales (Sprejer et al., 29 Oct 2025). On mixed-persona ground truth, the MLP and GAM aggregators reach $g$8 and $g$9, versus $R$0 for the 10-judge mean and $R$1 for the best single judge (Sprejer et al., 29 Oct 2025). A plausible implication is that behavior-grounding benefits from modeling judge calibration explicitly, not only from decomposing evaluation into dimensions.

Per-instance inconsistency appears in transitivity analysis. On SummEval, aggregate directed 3-cycle rates are low, $R$2, but $R$3 of documents exhibit at least one cycle depending on judge and criterion (Gupta et al., 16 Apr 2026). Split conformal prediction sets over 1–5 Likert scores achieve the promised coverage and yield set width as a practical per-instance reliability indicator, with pooled Spearman $R$4 between width and absolute error, and cross-judge width agreement $R$5 for several criteria (Gupta et al., 16 Apr 2026). The same study finds that criterion matters more than judge: relevance is judged most reliably, coherence moderately so, while fluency and consistency remain unreliable with average set sizes near 4.9 on a 1–5 scale (Gupta et al., 16 Apr 2026).

Prompt-format and perturbation robustness are a separate concern. The Judge Reliability Harness builds validation suites that test label flip, format invariance, paraphrase, verbosity bias, stochastic stability, synthetic ordinal robustness, and agentic perturbations across four benchmarks (Dev et al., 5 Mar 2026). No evaluated judge is uniformly reliable across benchmarks; format changes, paraphrases, verbosity shifts, and label flips all produce meaningful performance variation (Dev et al., 5 Mar 2026). For agentic behavior, the harness distinguishes binary judgment accuracy from ordinal grading and shows that perturbation sensitivity can remain high even when overall correlation with human scores is strong (Dev et al., 5 Mar 2026).

Temperature itself is a behavioral control parameter. Systematic experiments over temperatures $R$6 show that temperature causally reduces consistency and increases formatting errors in LLM-as-a-judge pipelines, especially for CoT prompts (Li et al., 30 Mar 2026). The paper reports that high temperature has the strongest causal effect on consistency degradation and a major effect on error rate, while its effect on agreement is smaller and task-dependent (Li et al., 30 Mar 2026). This matters for behavior-grounding because decoding settings alter the judge’s operational behavior even when the underlying prompt and model are fixed.

A more radical critique emerges in production transactional agents. There, a built-in LLM judge for a food-and-beverage ordering agent captures only 2 of 9 human-confirmed systematic patterns in one batch and flags 0 of 100 rounds in another batch where humans find 23 distinct defects and 7 new cross-cutting patterns (Zhang et al., 9 Jun 2026). The judge’s rubric has only intent, brand-voice, and personalization axes, while many failures are in state-tracking, guardrails, recovery, and safety. The paper argues the failure is structured: 113 of 114 raw judge notes describing confirm-gate or cart-state defects are routed to “brand voice,” and none trigger the operational failure gate (Zhang et al., 9 Jun 2026). This is a strong argument that behavior-grounding requires trajectory-level behavioral axes and correct wiring from judge outputs to operational decisions.

6. Domains, applications, and benchmark design

Behavior-grounded judges now appear across multiple application classes. Scientific and technical reasoning is one of the clearest settings because documents, terminology, and caveats offer naturally verifiable rubric criteria (Bhattarai et al., 8 May 2026). Summarization and semantic similarity are another, where STS-style scales provide a semantically interpretable continuum and prompt optimization can adapt the judge to open-domain, multi-hop, and conversational QA styles (Cao et al., 1 Apr 2025). In legal summarization, role-conditioned outputs reveal selective inclusion of facts and legal reasoning aligned with stakeholder perspectives, motivating evaluation frameworks based on fact inclusion, reasoning inclusion, and favorability bias rather than lexical overlap (Cho et al., 30 Aug 2025). That work warns that role-conditioned judges and assistants can drift toward stakeholder-aligned motivated reasoning even when prompts contain balancing instructions (Cho et al., 30 Aug 2025).

Agentic and multi-turn systems introduce the strongest demand for behavior-grounding. RankJudge is a synthetic benchmark generator for multi-turn, document-grounded conversations that pairs a benign conversation with another containing a single injected flaw at a known turn and of a known type (Tang et al., 20 May 2026). The judge must output not only which conversation is better but also the flawed turn and the failure type, and is counted correct only if all three are right: $R$7 This strict joint correctness criterion is intended to separate “right verdict for the wrong reason” from true behavioral understanding (Tang et al., 20 May 2026). The taxonomy includes self-contradiction, evasion, disorganized behavior, fabricated answers, instruction forgetting, no clarification, and unnecessary refusal (Tang et al., 20 May 2026). A plausible implication is that benchmark design itself must be behavior-grounded: if the benchmark only scores final verdicts, it cannot reveal whether the judge understood the behavioral error.

Safety evaluation provides another domain-specific lesson. Safety judges can incorporate novel contextual information when priors are weak, as in post-cutoff slang, but are broadly unlikely to adjust when explicit safety definitions contradict their internal safety priors (Alloula et al., 5 Jun 2026). Remarkably, the same policy variations become easier for judges to follow when safety is reframed as a generic A/B classification task rather than explicit safe/unsafe labeling (Alloula et al., 5 Jun 2026). This suggests that behavior-grounding in safety may require judge training or interface design that weakens unwanted prior interference.

7. Measurement, interpretation, and future directions

Several papers argue that judges should be reported as measurement instruments rather than as single scalar accuracies. A psychometric “Judge Datasheet” protocol evaluates dark current under vacuum inputs, cross-sensitivity to same-quality surface variation, positional false preference, target sensitivity on a controlled quality ladder, and the criterion induced by tie instructions (Usami et al., 14 Jun 2026). In the case study, Llama-3.1-8B shows high dark current and strong positional false preference, Qwen2.5-14B is vacuum-clean but mixes stable cross-sensitivity and position bias, and Qwen2.5-32B is vacuum-clean with low stable cross-sensitivity and low positional false preference (Usami et al., 14 Jun 2026). The same study concludes that prompt changes mainly move the criterion rather than the resolution, reinforcing the distinction between what the judge can discriminate and how willing it is to output a decisive preference.

Automated concept discovery offers a complementary interpretability route. By extracting latent concept features from embedding differences across paired responses, sparse-autoencoder methods recover interpretable preference drivers such as refusal versus substantive answer, concreteness, empathy, detail, formality, or aversion to active legal guidance (Wedgwood et al., 9 Feb 2026). This line of work shows that judge behavior can be decomposed into latent concept preferences even when those concepts were not specified in advance. A plausible implication is that behavior-grounding can also operate retrospectively: one can analyze what a judge is actually rewarding and compare it with the intended construct.

Across these works, several design principles recur. A behavior-grounded judge should evaluate explicit behavioral dimensions, be grounded in external evidence or behaviorally meaningful signals, expose uncertainty or per-instance difficulty, and be calibrated for the deployment criterion rather than reported only by average agreement (Bhattarai et al., 8 May 2026). When used in optimization, it should provide structured partial credit rather than only binary or holistic scores (Bhattarai et al., 8 May 2026). When used in evaluation, it should be audited for calibration drift, bias, transitivity violations, perturbation sensitivity, and criterion effects (Gupta et al., 16 Apr 2026). And in high-stakes settings, especially multi-turn systems, it should not be treated as a substitute for human review without direct evidence of defect recall on the relevant behavioral dimensions (Zhang et al., 9 Jun 2026).

The overall trajectory of the field suggests that “LLM judge” is no longer a single pattern of prompting a frontier model for a scalar verdict. It is becoming a broader family of structured evaluators, aggregators, probes, and diagnostics whose central concern is the same: to ensure that what is being measured is anchored to the behaviors that matter.