Prompt Design at Scale: How Format, Instruction Count, and Context Length Shape Instruction Adherence and Hallucination in Large Language Models

Abstract: Practitioners make three prompt-design decisions with almost no controlled evidence behind them: how to format instructions and context (markdown, plain text, prose, or tabular), how many simultaneous instructions a system prompt can carry before compliance degrades, and how much context a model can hold before recall and honesty degrade. We report two controlled experiments crossing all three factors on one held, contamination-free synthetic corpus (the "Book of Veyra," 8,780 uniquely-named entities, deterministically regenerable from a fixed seed), evaluated across five models. Experiment 1 (960 calls/model) measures instruction-following decay as rule count N grows from 10 to 160, crossed with four formats and system-prompt vs. user-turn placement. Perfect-response rate collapses to zero by N=80 for every model, format, and placement. Placement produces effects at least as large as format at N=160 in most models, but the direction is model-specific. No model shows a reliable markdown advantage; one 35B model favors plain text instead. Experiment 2 (5,520 calls/model) measures recall accuracy, false-premise sycophancy, and absent-fact fabrication across a 2k-to-512k-token context ladder in the same four formats. Recall stays near ceiling through 64-128k tokens, then degrades sharply and format-dependently: one model's accuracy spread reaches 48 points at 128k tokens. Fabrication never occurs (0/5,760 probes), and sycophancy stays negligible (<=8.3%). What rises sharply near each model's context ceiling is outright refusal to answer (0% to 79-90%), distinct from sycophancy or fabrication. Neither pre-registered format ordering holds, and token overhead (+22% to +37% over plain text) further changes which format is preferable where accuracy spread is genuine. We release the full harness, corpus generator, and raw results (VeyraBench): https://github.com/iNetanel/veyrabench

• The paper shows that instruction adherence collapses to zero by 80 rules across formats and models, while prompt placement can shift adherence by up to 8.7 percentage points.
• The experiments find no universal best format: recall differences emerge near each model’s context ceiling, and format rankings can reverse across models and context lengths.
• The study finds zero fabricated answers and negligible sycophancy, but refusal rises sharply near context limits—reaching 89.6% for Claude Haiku—highlighting the need to track abstention.

Overview

This paper reports two controlled experiments that cross prompt format against two scale axes—instruction count and context length—on a single contamination-free synthetic corpus, evaluated across five models spanning three families and two capability tiers (Claude Sonnet 5, Claude Haiku, Gemini Flash, Qwen 27B, Qwen 35B). The central claim is methodological: format effects cannot be interpreted without specifying the scale at which they are measured, because the sign and magnitude of a format's advantage change with instruction count and context length, and often reverse between models. All artifacts—the harness, the corpus generator, and raw outputs—are released as VeyraBench for byte-identical reproduction from a fixed seed.

The design addresses a genuine gap in the literature. Prior work on format sensitivity holds task complexity fixed (Sclar et al., 2023, He et al., 2024); work on instruction-count scaling does not vary format (Jaroslawicz et al., 15 Jul 2025); long-context degradation studies do not vary injected-content format [(Liu et al., 2023)-style lost-in-the-middle work; Chroma's context rot report]; and structured-context comparisons at fixed moderate lengths do not cross context length (Oh et al., 2024, Liu et al., 13 Aug 2025). No prior public study crosses format against both scale axes on one held corpus for recall, sycophancy, and fabrication jointly.

The Book of Veyra corpus

Experiment 2 requires a corpus that is large enough to fill 512k tokens, provably free of pretraining contamination, and renderable in multiple formats with byte-identical content. The Book of Veyra satisfies all three: 8,780 uniquely named fictional entities (solar systems, guilds, creatures) generated deterministically from seed 42 with no LLM involvement, verified by MD5-identical regeneration. Contamination-freedom rests on a positive construction argument—entity names are novel syllable combinations that cannot appear in any pretraining corpus—rather than negative filtering. Scored facts are restricted to unguessable attribute values, driving the chance baseline to approximately zero, which the authors confirm empirically rather than assume.

The identical facts are rendered in four formats: markdown (headers plus bold key–value bullets), plain text (markup mechanically stripped), prose (flowing narrative), and markdown tables. Token overhead relative to plain text is stable across all rungs: markdown $1.258\times$, prose $1.221\times$, table $1.367\times$, measured with the o200k_base tokenizer. Probes at each of six rungs (2k–512k) cover recall, false-premise sycophancy, and absent-fact fabrication, each at three needle depths, with three repeats per question. A fixed 20-question anchor set drawn from the 2k rung is re-asked unchanged at every larger rung to support valid cross-rung comparison—a methodological safeguard that proves necessary, as discussed below.

Experiment 1: instruction-following decay

Experiment 1 places $N \in \{10, 20, 40, 80, 120, 160\}$ programmatically verifiable rules in either the system prompt or the user turn, rendered in each of the four formats, with 20 trials per cell (960 calls per model). Every rule is checked by regex or word-boundary matching; no LLM-as-judge scoring is used.

A hard floor by $N=80$. Perfect-response rate collapses to zero by $N=80$ for every model, every format, and both placements. Sonnet 5 falls from 0.938 at $N=10$ to 0.000 at $N=80$; Gemini Flash from 0.919 to 0.019; Claude Haiku from 0.850 to 0.000. This replicates threshold decay documented elsewhere (Jaroslawicz et al., 15 Jul 2025) but shows the floor is reached regardless of format. One compositional caveat applies: because five fixed structural rules are present at every $N$, average rule difficulty shifts with $N$; the authors use perfect-response rate as the primary metric precisely because it is insensitive to this shift, and verify the floor finding holds on the fixed-rule subset alone.

No reliable markdown advantage. Markdown-minus-plain deltas are small ($1.221\times$0pp recurring direction) and inconsistently signed for four of five models. The single clean directional signal runs opposite conventional wisdom: Qwen 35B favors plain text at five of six $1.221\times$1-levels, widening to 4.8pp at $1.221\times$2.

Placement rivals format. Moving the identical rule block between system prompt and user turn changes adherence by up to 8.7 percentage points at $1.221\times$3—an effect at least as large as format in four of five models—but with no universal direction. User-turn placement helps Claude Haiku (+6.6pp) and Qwen 35B (+5.1pp), hurts Gemini Flash (−8.7pp) and Qwen 27B (−1.8pp), and is indistinguishable from zero for Sonnet 5. Logistic fits corroborate this at the knee: Haiku's knee shifts from $1.221\times$4 to $1.221\times$5 under user placement, while Gemini Flash's drops from $1.221\times$6 to $1.221\times$7. There is no portable "system prompts work better" rule.

A model-specific format collapse. Gemini Flash alone shows a sharp collapse for prose and table formats at $1.221\times$8 (−18.1pp and −13.7pp versus plain at $1.221\times$9), while every other model stays within ~2.5pp. Manual inspection attributes this to hidden-reasoning leakage: affected responses contain fragments of an internal rule-verification pass rather than essay text. An automated duplicate-content detector recovered fully rule-compliant essays for 43 of 72 flagged responses (all Qwen); the remaining 29, including Gemini Flash's collapse cases, reach no valid closing sentence and are scored as failures. This interaction would be invisible to any pooled "markdown vs. plain" summary and cautions against assuming format effects are homogeneous across models.

Experiment 2: hallucination across the context ladder

Recall accuracy sits at or near ceiling (0.98–1.00) through 64k tokens for every model and format. Separation appears at 128k and grows toward each model's own ceiling. The most striking cell is Claude Haiku at 128k, where plain text collapses to 0.383 recall while the other three formats sit at 0.817–0.867—a 48.4-percentage-point spread driven by one format. Spread magnitude tracks proximity to a model's effective ceiling rather than absolute token count: Sonnet 5's spread nearly doubles from 256k (11.7pp) to 512k (20.0pp), while Gemini Flash's stays flat (5.0pp to 6.7pp) over the same nominal range despite identical advertised ceilings.

Neither pre-registered ordering holds. Two hypotheses were registered in advance: that any structure beats unstructured plain text, or that tabular structure specifically is the active ingredient. Both fail. Prose—the format both hypotheses predict should perform worst—is best or tied-best in three of five non-ceiling cells. Plain is simultaneously worst at one cell (Haiku @ 128k) and best at another (Sonnet 5 @ 512k). Two mechanisms qualify these reversals. First, at Sonnet 5 @ 512k, three in four incorrect markdown responses are outright refusals rather than wrong guesses, so plain and table's apparent advantage there partly reflects lower refusal propensity rather than comprehension. Second, Gemini Flash's ranking flip between adjacent rungs survives the anchor-set cross-check and is therefore genuine. The prior findings that tables beat unstructured text (Oh et al., 2024, Liu et al., 13 Aug 2025) are bounded rather than contradicted: those measurements were taken at fixed, moderate context lengths, and the advantage need not survive near a model's effective ceiling.

Two confirmed nulls. Fabrication never occurs: 0/5,760 absent-fact probes across all models, rungs, and formats. Sycophancy stays negligible throughout, peaking at 8.3% in a single cell (Qwen 35B @ 128k/markdown), with the full population of 19 sycophantic responses manually reviewed. One qualification attaches to the fabrication null: since "correct" on absent probes means declining to answer, the null cannot distinguish fact verification from reflexive abstention—and the refusal findings below show the latter mechanism is genuinely present at extreme lengths.

Refusal, not sycophancy, rises. What actually climbs near each model's own ceiling is refusal on the false-premise probe: Claude Haiku rises from 0% at 2k to 89.6% at its 128k ceiling; Sonnet 5 reaches 78.8% at 512k. Which format triggers the most refusal is again model-dependent. The practical consequence is direct: monitoring systems that watch only for wrong or agreeable answers near the top of a context window will miss the failure mode that actually dominates there.

Anchor-set necessity. Because main-set questions differ across rungs, cross-rung swings can reflect question composition. The authors demonstrate this concretely: Gemini Flash's apparent markdown improvement from 256k to 512k (0.900 → 1.000) vanishes in the anchor set, confirming it as an artifact. All reported cross-rung claims survive the anchor check.

Cost-adjusted accuracy

Format's token overhead (+22% to +37% over plain text) changes which format is preferable wherever a genuine accuracy spread exists. At Claude Haiku @ 128k, prose and table tie on raw accuracy (0.867), but cost-adjustment separates them decisively (0.710 vs. 0.634 accuracy per relative token cost), making prose the clear winner. At Sonnet 5 @ 512k, plain wins on raw accuracy (0.867 vs. 0.530–0.597) and cost widens rather than creates the gap. Outside such cells, most combinations sit at ceiling, where choosing the cheapest format is a tiebreak between statistically indistinguishable accuracies, not a demonstrated efficiency gain. The recommendation is correspondingly narrow: apply the cost check explicitly where real accuracy gaps exist, not as a blanket cheapest-format policy.

Limitations

Several boundaries constrain the claims. The corpus is a single fictional factual domain; generalization to source code, legal text, or numerical business data is untested. The five-model pool spans three families but is not a systematic sample, so qualitative shapes (a hard floor exists, no universal format winner, refusal rises near ceilings) port better than specific thresholds such as $1.367\times$0 or 89.6% refusal. Sample sizes are modest at the tails—20 trials per cell in Experiment 1, 60 questions per cell in Experiment 2—which motivates the Wilson intervals and anchor-set checks throughout. Token counts use one standardized tokenizer rather than provider-specific ones, though relative overhead ratios should be stable. Finally, both experiments rely on programmatically verifiable rules and facts to permit exact scoring; whether the same decay and format patterns hold for softer, non-checkable instructions remains open.

Open questions

The paper leaves several questions explicitly unresolved. Whether the $1.367\times$1 floor reflects single-prompt density or total task complexity could be tested by splitting the same rule set across turns or validation passes. Whether the placement effect's direction or Gemini Flash's prose/table collapse correlates with measurable model properties requires a larger model pool than the budget allowed. Most substantively, the authors propose that the refusal cliff may be an artifact of alignment tuning—penalizing confident errors pushes models toward abstention under uncertainty—and note that their design cannot test this directly; comparing a base model against its instruction-tuned counterpart on identical probes would distinguish alignment-driven abstention from architectural retrieval degradation near the context limit. Either answer would clarify whether the observed zero fabrication rate reflects genuine capability or trained-in reluctance that happens to look like capability under standard hallucination metrics.

Conclusion

By crossing format with both scale axes on one held corpus, the paper establishes that neither a format's effect nor its sign can be predicted without accounting for scale. Instruction-following collapses to a floor by $1.367\times$2 regardless of format; placement moves adherence as much as format does, in model-specific directions; recall degrades format-dependently in proportion to proximity to each model's own effective ceiling; neither structure hypothesis survives; and the failure mode that grows under context pressure is refusal, not sycophancy or fabrication. The actionable conclusion is that claims about which format is best must specify the instruction count and context length at which they were measured to mean anything at all, and VeyraBench's release makes those thresholds re-measurable rather than taken on trust.