Realistic constraints on manipulation of honest calibration episodes

Ascertain realistic constraints on the degree to which an untrusted language model agent could manipulate “honest” calibration episodes to raise monitor suspiciousness without triggering shutdown, and incorporate these constraints into monitor calibration and control evaluations to prevent exploitable degradation.

Background

Monitors are calibrated to a fixed false positive rate using honest episodes. The authors show in Appendix E that allowing the red team to influence this calibration data can degrade monitor performance by several percentage points, revealing a potential vulnerability.

However, the authors state it is difficult to know what degree of such manipulation is realistic in practice. Determining realistic bounds on this manipulation is necessary to design robust calibration procedures and fair, defensible control evaluations.