Towards Provably Unbiased LLM Judges via Bias-Bounded Evaluation

Abstract: As AI models progress beyond simple chatbots into more complex workflows, we draw ever closer to the event horizon beyond which AI systems will be utilized in autonomous, self-maintaining feedback loops. Any autonomous AI system will depend on automated, verifiable rewards and feedback; in settings where ground truth is sparse or non-deterministic, one practical source of such rewards is an LLM-as-a-Judge. Although LLM judges continue to improve, the literature has yet to introduce systems capable of enforcing standards with strong guarantees, particularly when bias vectors are unknown or adversarially discovered. To remedy this issue, we propose average bias-boundedness (A-BB), an algorithmic framework which formally guarantees reductions of harm/impact as a result of any measurable bias in an LLM judge. Evaluating on Arena-Hard-Auto with four LLM judges, we achieve (tau=0.5, delta=0.01) bias-bounded guarantees while retaining 61-99% correlation with original rankings across formatting and schematic bias settings, with most judge-bias combinations exceeding 80%. The code to reproduce our findings is available at https://github.com/penfever/bias-bounded-evaluation.

• The paper introduces Bias-Bounded Evaluation, a sensitivity-aware Gaussian mechanism that bounds judgment changes to tolerance τ with failure probability δ for measured perturbations.
• Experiments on 500 Arena-Hard-Auto queries show certificates at τ=0.5 and δ=0.01 while retaining 61–99% of original rank correlation across formatting and schematic biases.
• The framework can expose bias-driven overconfidence and improve robustness through Lipschitz shrinkage, but its guarantees depend on complete bias coverage and reliable finite-sample sensitivity estimates.

Motivation and problem statement

LLM-as-a-Judge systems are increasingly positioned as the feedback mechanism for autonomous agentic workflows, where errors compound and human oversight is sparse. The paper's motivating observation is that existing judge deployments exhibit well-documented failure modes—formatting sensitivity, preference leakage, schematic non-adherence, agreeableness bias—yet no framework provides formal guarantees on the impact of these biases, particularly when the bias sources are unknown or adversarially discovered (2603.05485). The authors' response is bias-bounded evaluation (BBE), a framework that measures a judge's sensitivity to contextual perturbations and injects calibrated Gaussian noise so that the resulting scores carry a probabilistic certificate bounding how much any measurable bias can shift the judgment.

The A-BB definition

The core object is average bias-boundedness (A-BB). For a fixed judgment context $D$, a randomized neighbor generator $T$ producing perturbed contexts $D' \sim_T D$, and a mechanism $M$, the guarantee is:

$\Pr[\|M(D) - M(D')\|_2 > \tau] \leq \delta$

with probability over both the neighbor draw and the mechanism's internal randomness. The design choice that distinguishes A-BB from differential privacy is deliberate: worst-case global sensitivity analysis would tune to adversarially chosen contexts differing on the full judgment scale, which the authors argue is too conservative for LLM judging. Instead, A-BB fixes $D$—analogous to local/smooth sensitivity in DP (2603.05485)—and controls the tail probability under an average-case RMS sensitivity:

$$\Delta_2^*(f,D) = \left(\mathbb{E}_{D' \sim_T D}[\|f(D) - f(D')\|_2^2]\right)^{1/2}$$

The certificate is explicitly local: it holds for the fixed context $D$ and generator $T$ (plus any unknown generators whose biasing effect is no larger than $T$), not as a distributional generalization over unseen contexts.

The Gaussian mechanism and failure-budget splitting

The main theorem gives a Gaussian mechanism $T$0 with $T$1 satisfying $T$2-A-BB. The proof splits the failure event via the triangle inequality into a bias term ($T$3, bounded by Markov's inequality using $T$4) and a noise term ($T$5, where $T$6 is chi-squared, bounded via Laurent–Massart). Setting $T$7 yields an explicit split of the failure budget $T$8, with the symmetric split $T$9 giving the operational rule $D' \sim_T D$0 and a closed-form maximal admissible $D' \sim_T D$1.

Two refinements improve utility. First, the split calibration strictly tightens the noise radius relative to a no-split bound at the same $D' \sim_T D$2, since $D' \sim_T D$3. Second, a deterministic Lipschitz shrinkage post-processing step—an affine contraction $D' \sim_T D$4 toward a center—contracts RMS sensitivity by exactly $D' \sim_T D$5, allowing smaller $D' \sim_T D$6 at the cost of attenuated score variance. The appendix extends this to data-dependent centers (holdout means, frozen EMA) and to conservative calibration against a family of biases via the envelope $D' \sim_T D$7, which also covers convex mixtures of measured generators.

Experimental results

Experiments use Arena-Hard-Auto (500 queries) with four judges: GPT-4o-mini, QwQ-32B, DeepSeek-R1-Distill-32B, and GPT-3.5-Turbo. Two bias families are measured: formatting sensitivity (via LLM-generated semantic variants of responses) and schematic adherence (the fraction of overall-score variance unexplained by per-criteria scores, $D' \sim_T D$8). Notably, measured $D' \sim_T D$9 ranges from 0.10 (DeepSeek-R1-32B) to 0.74 (GPT-4o-mini)—meaning some judges' verdicts are largely unexplainable by their own stated rubrics.

The headline empirical claim is that BBE achieves $M$0 certificates while retaining 61–99% rank correlation with original judgments across formatting and schematic settings, with most combinations above 80%. Specific results include 88% correlation retention for QwQ-32B under formatting bias and near-perfect correlation for GPT-3.5-Turbo under schematic bias despite large measured bias. The authors interpret the compressed post-BBE score distributions as exposing "bias-induced false confidence": extreme judgments with narrow confidence intervals in the raw judge output are revealed as artifacts rather than genuine signal.

Relation to prior work

Compared to Trust or Escalate—the closest prior framework with guarantees—A-BB differs on several axes: it provides guarantees on all evaluations without abstention, requires no human-labeled calibration data, handles general scoring beyond pairwise comparison, and bounds bias impact directly rather than sidestepping uncertainty. Its principal concession relative to Trust or Escalate is the absence of a direct human-agreement guarantee, though the authors note this can be recovered compositionally with conformal prediction methods. A-BB is also complementary to conformal interval estimation, which bounds per-judgment uncertainty rather than batch-level systematic bias impact.

Limitations

The paper is candid about scope. BBE makes no claims about absolute accuracy, cross-judge calibration, or whether judges attend to all input aspects; the guarantee is only as good as the reliability of the bias measurements used to calibrate it—a missed bias source larger than any measured one voids the certificate. Judge preferences themselves constitute a bias pattern, and A-BB retains true signal best when intrinsic jitter and systematic bias are already weak. Most concretely, the theory assumes access to the true $M$1, but the algorithm estimates it from $M$2 sampled neighbors; finite-sample underestimation can push actual failure probability above $M$3. The authors suggest upper-confidence-bound margins or formal concentration-based incorporation into the $M$4 budget as unresolved future work.

Conclusion

This paper contributes a formally grounded mechanism for certifying that measurable biases in LLM judges cannot shift judgments beyond a tolerance $M$5 except with probability at most $M$6, achieved through sensitivity-aware Gaussian noise calibrated by a split failure budget and optional Lipschitz shrinkage. Empirically, the approach preserves most ranking signal while compressing bias-inflated score distributions on Arena-Hard-Auto. Its value rests on the assumption that measured sensitivities dominate unknown ones—a condition the paper identifies but does not itself verify—and the finite-sample gap between estimated and true sensitivity remains the clearest open technical question.