Stale Instructions Override Code: A One-Sentence Fix
This presentation reveals a critical failure mode in repository-scale coding agents: instruction files that fall out of sync with the codebase continue to dominate agent behavior, even when agents explicitly recognize the conflict. The problem stems from instruction hierarchy design—documentation arrives as high-priority instructions while code is treated as lower-authority tool output. Across 320 controlled experiments with Claude Code and Codex CLI, agents followed stale instructions 90% of the time at baseline, sometimes rewriting working code and breaking tests. A single conditional sentence reversed this preference completely, raising code-following behavior to 91% and eliminating destructive rewrites. The talk explains the mechanism, presents the empirical evidence, and discusses practical implications for users, agent developers, and model designers.Script
Coding agents can inspect your repository, recognize that an instruction file contradicts the current code, and still implement the wrong convention because of how instructions reach the model.
The paper documents real cases where instruction files describing React 18 remained unchanged 13 days after the codebase moved to React 19, and where agents kept requesting updates to a document deleted 24 days earlier. In one sample, 73.8 percent of AI configuration files were committed once and never modified, even as the code evolved around them.
The explanation centers on instruction hierarchy. The instruction file enters the model context as a user-level instruction, while the repository code arrives as tool output from a file-reading tool. Models are trained to privilege instructions over information returned by tools, so a stale descriptive claim keeps instruction-level authority even after it becomes factually false.
Across 40 Python projects, two agents, and 320 runs, the baseline result was unambiguous: 90.1 percent of convention choices followed the stale instruction file, while only 4.2 percent followed the existing code. Adding one sentence to the instruction file reversed the distribution entirely. Code-following behavior rose to 91.4 percent, and every destructive rewrite disappeared.
The most severe baseline failure occurred in Codex CLI. In 20 of 80 runs, the agent rewrote existing working code to conform to the stale instruction, removing a median of 198 lines across nine files. Seventeen of those runs broke the original tests, including one case where replacing an enumeration with string constants made established imports fail.
The fix is narrow but the implications are architectural. Users can add the precedence sentence to a global instruction file. Agent developers can encode the rule at the system level so users do not need the workaround. Model developers face a harder problem: separating descriptive repository claims from genuine directives without undermining prompt-injection defenses. If you want to see how instruction hierarchy shapes agent behavior in your own projects, visit EmergentMind.com to explore this research and generate explanations of the work that matters to you.
