Prompt design effects on LLM compliance

Determine whether particular zero-shot prompt design choices reliably increase or decrease the compliance of large language models with input instructions when performing text annotation tasks.

Background

The paper motivates its large-scale study by noting a lack of systematic evidence on how prompt design affects compliance—whether models follow the requested output format and label constraints. Prior observations showed instances of non-compliance (e.g., generating labels outside the specified set), but general patterns remained unclear.

Establishing reliable prompt design principles for compliance is important for cost-effective and scalable annotation workflows in computational social science, where invalid outputs increase processing costs and can bias results.

References

In the absence of any systemic evidence, however, it remains unclear whether certain prompt designs are more or less likely to generate compliant outputs.

— Prompt Design Matters for Computational Social Science Tasks but in Unpredictable Ways  (2406.11980 - Atreja et al., 2024) in Section 1 (Introduction)

While system prompts are commonly used to steer model behavior, their reliability as a governance mechanism remains an open question.

— Can Foundation Models Moderate Online Content? Evaluating Instruction- vs. Example-Driven Policy Operationalization  (2609.10410 - Majumdar et al., 9 Sep 2026) in Section 2, Related Work, subsection “Safety of AI Models”

We do not know whether other demand characteristics remain.

— The AI Evaluation Ecosystem  (2610.09296 - Dave et al., 7 Oct 2026) in Appendix, Section Limitations, subsection Methodological bounds of GABMs, paragraph The validation surface is multi-dimensional, item Prompt space

It remains unclear which prompt structures, types of descriptions, or semantics yield the most reliable labels across diverse datasets and domains.

— Improving Semantic Coverage in AI Test & Evaluation through Generative Augmentation  (2610.13697 - Hashemi et al., 9 Oct 2026) in Section "Future Work," subsection "Semantic descriptor design and benchmarking"