The Model Says Walk: How Surface Heuristics Override Implicit Constraints in LLM Reasoning

Abstract: LLMs systematically fail when a salient surface cue conflicts with an unstated feasibility constraint. We study this through a diagnose-measure-bridge-treat framework. Causal-behavioral analysis of the ``car wash problem'' across six models reveals approximately context-independent sigmoid heuristics: the distance cue exerts 8.7 to 38 times more influence than the goal, and token-level attribution shows patterns more consistent with keyword associations than compositional inference. The Heuristic Override Benchmark (HOB) -- 500 instances spanning 4 heuristic by 5 constraint families with minimal pairs and explicitness gradients -- demonstrates generality across 14 models: under strict evaluation (10/10 correct), no model exceeds 75%, and presence constraints are hardest (44%). A minimal hint (e.g., emphasizing the key object) recovers +15 pp on average, suggesting the failure lies in constraint inference rather than missing knowledge; 12/14 models perform worse when the constraint is removed (up to -39 pp), revealing conservative bias. Parametric probes confirm that the sigmoid pattern generalizes to cost, efficiency, and semantic-similarity heuristics; goal-decomposition prompting recovers +6 to 9 pp by forcing models to enumerate preconditions before answering. Together, these results characterize heuristic override as a systematic reasoning vulnerability and provide a benchmark for measuring progress toward resolving it.

• The paper demonstrates that surface heuristics, such as distance cues, can systematically override implicit feasibility constraints in LLM reasoning.
• It employs a diagnostic benchmark (HOB) across multiple models to quantify heuristic influence with significant error rate asymmetries and effect sizes.
• The study shows that a simple goal-decomposition prompt can mitigate these failures, boosting strict accuracy by 6–9 percentage points in weaker models.

Heuristic Override in LLMs: Systematic Dominance of Surface Cues over Implicit Constraints

Introduction

The paper "The Model Says Walk: How Surface Heuristics Override Implicit Constraints in LLM Reasoning" (2603.29025) rigorously investigates systematic reasoning failures in state-of-the-art LLMs arising when salient surface cues (heuristics) conflict with unstated feasibility constraints. Through a sequence of mechanistic behavioral analyses, a fine-grained multi-dimensional diagnostic benchmark (HOB), parametric ablations, and targeted mitigation, the authors map out the structure, pervasiveness, and modifiability of “heuristic override” vulnerabilities. This essay provides a comprehensive technical summary, emphasizing empirical findings, domain generality, and diagnostic methodology.

Mechanistic Analysis: The Car Wash Paradigm

The authors first isolate the paradigm instance: a prompt asking whether to "walk or drive" 100m to a car wash in order to clean a car. Ground-truth is "Drive", since the car must be co-located with the wash, but all tested models overwhelmingly recommend "Walk", indicating a breakdown where the proximity heuristic (short distance $\Rightarrow$ walk) overrides implicit goal constraints.

Causal occlusion analysis quantifies token/phrase-level contributions to model outputs. Across six LLMs (Qwen3-* and GPT-OSS-20B families), span-level attribution reveals that the distance cue is the overwhelmingly dominant factor, exerting between 8.7$\times$ and 38$\times$ greater influence than the goal or requirement wording.

Figure 1: The decision scores of all models are strongly biased toward “Walk” (left); span-level occlusion shows distance cues (columns in blue) consistently drive decisions while the goal span has minimal or even perverse effects (right).

Per-paraphrase sensitivity analysis demonstrates that the heuristic's influence is robust to surface variation, and goal information is largely ineffectual unless made fully explicit. Token-level attribution confirms these effects persist down to granular lexical choices.

Figure 2: Left: CSI vs. DSI for Qwen3-4B indicating goal sensitivity is fragile compared to distance sensitivity; right: span heatmap consistently localizes $\Delta s$ to the surface cue rather than the constraint.

Monotonicity analysis, sweeping the "distance" value, yields nearly perfect sigmoidal curves for both “conflict” and “control” conditions across all six evaluated models, with surface cues mapping directly to action selection independent of feasibility.

Figure 3: All six models’ decisions track a context-independent sigmoid over the surface cue, rather than flat-lining as correct reasoning would require.

Heuristic Override Benchmark (HOB)

To test the generality of this failure mode, the authors introduce the Heuristic Override Benchmark (HOB), a 500-instance diagnostic suite systematically crossing four heuristic families (proximity, cost, efficiency, semantic match) with five constraint types (presence, capability, validity, scope, procedural). Each scenario includes a minimal pair (constraint-removed), gradient explicitness variants, and controls for heuristic strength.

A 14-model evaluation—including Gemini, Claude, Llama, DeepSeek, Kimi, Grok, and multiple open-weights—finds that under strict criteria (10/10 correct per instance), no model surpasses 75% accuracy (Gemini~3.1~Pro: 74.6%). The hardest failures consistently cluster in presence constraints (mean: 44.4%), while capability constraints are significantly easier (mean: 71.6%).

Figure 4: Cells A1 (proximity$\times$presence) and B1 (efficiency$\times$presence) are systematically the hardest across all models.

Figure 5: Presence-based constraints show lowest mean accuracy, with substantial range across models.

A single minimal “hint” boosts strict accuracy by $+15$pp, indicating constraint knowledge exists but is not spontaneously activated. Removal of the constraint in minimal pairs causes accuracy to decrease in 12/14 models (down to $-$38.5pp), exposing a conservative bias—models often default to the more cautious but still heuristically-governed outcome.

Explicitness gradients further corroborate an inference bottleneck: constraint activation is triggerable but non-automatic.

Parametric Probes and Generalization

The authors test whether the observed sigmoid “heuristic override” pattern holds across alternative surface cues and constraint categories using parametric probes.

Figure 6: Parametric curves (Qwen3-4B): cost$\times$scope generates the appropriate separation, but efficiency$\times$capability and semantic$\times$0scope elicit near-identical heuristic sigmoids, indicating generalization of the override failure.

Additional cross-model overlays show that larger models are more likely to correctly separate conflict and control curves for physical constraints, but abstract and semantic-override failures persist across model size.

Figure 7: For the efficiency$\times$1capability probe, only the largest models reliably reverse the heuristic; smaller models stay strongly positive for the wrong action.

Figure 8: As the semantic cue increases (gas station described as more “car-related”), models are more likely to recommend the infeasible option, yielding a semantic sigmoid.

Overall, HOB reveals that surface heuristics (proximity, efficiency, semantic, cost) can override correct application of implicit constraints across numerous domains and constraint types, with effect sizes and error rates largely insensitive to heuristic signal strength.

Diagnostic Mitigation: Goal-Decomposition Prompting

Given that a minimal hint or explicit enumeration triggers correct constraint application, the authors test a simple goal-decomposition prompt: "List the necessary conditions for the stated goal, then answer".

This approach yields +6--9pp strict accuracy improvements on weaker models, with Gemini~3.1~Pro showing no change (ceiling effect), indicating that the failure can be mitigated by explicit requirement activation at inference.

Figure 9: Goal-decomposition prompting substantially improves strict accuracy for weaker models; top-tier models show no further gains.

Theoretical and Practical Implications

The authors’ findings directly challenge the assumption that strong accuracy on classical benchmarks reflects true constraint-following or robust reasoning in open-ended, goal-conditioning tasks. Diagnostic instead of “shortcut” failures require reasoning about which unstated constraints are relevant and must be prioritized over surface associations—a capability that remains systematically elusive for contemporary LLMs.

Conservative bias observed via minimal-pair asymmetry demonstrates that aggregate accuracy can mask systematic heuristic overtakes, reinforcing the necessity of minimal-pair and explicitness-based evaluation for true constraint-following auditing.

Practically, the results have direct implications for LLM deployment in high-stakes contexts (medical, legal, financial), where implicit feasibility, safety, or regulatory constraints must override surface cues. The demonstrated vulnerability necessitates careful mitigation, such as automated precondition enumeration or fine-tuning strategies targeted at constraint prioritization, and calls for future research into architectural, training, or inference time modifications that elevate constraint inference above associative statistics.

Conclusion

This study introduces and systematizes the “heuristic override” phenomenon in LLM decision-making: when surface heuristics systematically and robustly dominate over implicit feasibility constraints, producing high-confidence, fluent but invalid recommendations. The Heuristic Override Benchmark (HOB) provides a rigorous, scalable framework for measuring progress in correcting this class of failures. Results suggest that LLMs frequently possess the requisite world knowledge but lack spontaneous constraint activation in the presence of conflicting surface cues. Mitigation via explicit goal decomposition highlights a tractable, if limited, intervention path. Addressing this “reasoning shortcut” will be necessary for dependable LLM deployment in any domain requiring robust implicit constraint inference.