Cause of Additional Jailbreak Successes Under Repeated Sampling

Determine whether the additional attack successes observed when harmful requests are sampled repeatedly, rather than answered once, reflect probabilistic suppression of harmful instructions that is revealed by increased sampling or are artifacts of sampling that produce incoherent outputs accepted by the evaluator.

Background

The study finds that repeatedly sampling unchanged harmful requests from Llama 2-7B at temperature 0.8 produces substantially higher reported attack success rates than single-response evaluation: 100% under keyword matching, 80% under the fine-tuned classifier, and 50% under the Qwen3 evaluator using StrongREJECT criteria. Manual inspection estimates approximately 42% goal-level success, but the authors characterize this estimate as informal and subjective.

The unresolved issue is whether repeated sampling exposes residual unsafe behavior that is probabilistically suppressed in individual generations, or instead increases the likelihood of incoherent or otherwise unsuitable outputs that happen to satisfy an automated evaluator. The question is therefore central to interpreting repeated-sampling ASR as evidence about model safety rather than as an artifact of the sampling and evaluation procedure.