Generalization of instruction-count performance relationship to complex instruction types

Determine whether the observed degradation in instruction-following performance as the number of simultaneous instructions increases—measured on the ManyIFEval (text generation with up to 10 instructions) and StyleMBPP (code generation with up to 6 style instructions) benchmarks—persists for more complex instruction types not covered in these benchmarks, including semantic instructions, conditional logic (if/then rules), and multi-step procedural requirements.

Background

Across both ManyIFEval and StyleMBPP, the authors empirically observe that LLMs’ prompt-level accuracy consistently degrades as the number of instructions increases. However, these benchmarks intentionally focus on objectively verifiable, non-conflicting, relatively simple instructions (e.g., formatting, keyword inclusion, code style rules) to enable controlled measurement.

The authors explicitly note that their scope excludes more complex instruction types such as semantic constraints, conditional logic, and multi-step procedures. They state that determining whether the same performance–instruction-count relationship holds for such complex instructions remains unresolved, motivating further evaluation beyond the current benchmarks.