Effectiveness of correlation-aware judge aggregation in RL

Establish whether correlation-aware aggregation of multiple language-model judges improves reinforcement-learning outcomes when judges have correlated errors.

Background

The paper describes conflicting offline evidence about aggregating language-model judges. One study found that confounder-aware aggregation improved over majority voting and neutralized master-key attacks, while another found that aggregation closed only a small fraction of the gap to independent voting. Crucially, the paper states that these aggregation methods had not been evaluated within reinforcement learning, leaving their training-time value unresolved.

References

Correlation-aware judge aggregation helped in one study and closed a little of the gap in another, and neither was tested in RL.

— Checking the Checkers: A Synthesis of Verifier Noise and Robustness in RLVR  (2610.17824 - Gupta et al., 13 Oct 2026) in Section 7.2, “Resolved and Open Disagreements”; Section 6, “Detecting and Suppressing Hacking”