---
title: 'Consequence-Blindness: Proxy vs. Outcome Gap'
url: https://www.emergentmind.com/topics/consequence-blindness
type: topic
---

# Consequence-Blindness: Proxy vs. Outcome Gap

Across several lines of research, consequence-blindness denotes a family of failure modes in which an agent, model, estimator, or evaluation protocol tracks a proxy that is locally available, semantically salient, or task-specified, while failing to represent the downstream consequence structure that is probabilistically, strategically, or normatively decisive. The label is not used uniformly. In some works it names insensitivity to global probability-mass allocation; in others it refers to cooperation conditioned only on realized outcomes, planning rules that causally produce the negation of their goal, recourse methods that ignore order-sensitive effects, or safety systems that follow surface cues instead of real-world outcome risk. The common thread is a dissociation between what is immediately optimized or reported and what actually follows from action or inference [2209.07396].

## 1. Conceptual variants and recurring structure

In score-based statistics, the blindness problem is the claim that a score-based discrepancy can be zero or nearly zero even when two distributions differ in a probabilistically important way, because the score function contains only local derivative information and may not detect how mass is allocated across separated modes or disconnected regions [2209.07396]. In sequential recourse, traditional counterfactual generation is described as consequence-blind because it optimizes only the final counterfactual state and ignores the order in which actions are applied, the intermediate states, and the beneficial or harmful consequences that earlier actions can have on later ones [2104.05592]. In safety-aligned LLMs, consequence-blindness is defined as over-reliance on surface-form cues such as sensitive keywords, style, or malicious-sounding background text, instead of robust reasoning about the real-world consequences of the model’s own response [2510.04320].

A distinct but related usage appears in multi-agent cooperation. Consequentialist conditional cooperation treats consequences as sufficient statistics for strategic response: the agent conditions its behavior solely on outcomes, specifically its own past rewards, rather than on the partner’s actions or intentions [1710.06975]. By contrast, C-Causal Blindness describes a cognitive bias in which a subject starts from some current state \(A\), chooses a policy \(B\) intended to achieve an objective \(C\), but the chosen policy actually leads to \(\lnot C\); the blindness lies in failing to see that the policy causally produces the state to be avoided [2208.07143].

These formulations are not terminologically identical, but they converge on a recurring architecture of error. A model or decision-maker is guided by local geometry, observed reward, prompt framing, or an assumed causal link, while the global allocation of mass, the latent intention structure, the intermediate state trajectory, or the downstream hazard remains unmodeled. A plausible implication is that consequence-blindness is less a single domain-specific defect than a general proxy–consequence mismatch.

## 2. Local score information and global distributional consequences

The most explicit mathematical treatment appears in work on score matching. The Fisher divergence between densities \(p\) and \(q\) is
\[
\mathrm{FD}(p\|q)=\frac{1}{2}\int_X p(x)\,\|s_p(x)-s_q(x)\|_2^2\,dx,
\]
with score function
\[
s_p(x)\equiv \nabla_x \log p(x),\qquad s_q(x)\equiv \nabla_x \log q(x).
\]
Under standard assumptions, this is a valid divergence when the support is connected and the scores are square-integrable. For an unnormalized energy model \(q_\theta(x)=e^{-f_\theta(x)}/Z(\theta)\), integration by parts yields the classical score-matching objective
\[
\mathrm{FD}(p_d\|q_\theta) = \frac{1}{2}\int_X p_d(x)\Big(\|s_{q_\theta}(x)\|_2^2 + 2\,\mathrm{Tr}(\nabla_x s_{q_\theta}(x))\Big)\,dx + \text{const.}
\]
which does not require the normalization constant \(Z(\theta)\) during training [2209.07396].

The blindness problem is characterized in two regimes. In the “numerical blindness” regime, two mixture components are far apart, the score difference is concentrated only in low-density transition regions, and the Fisher divergence can become tiny even when the distributions have very different mixing weights. For the canonical Gaussian-mixture example,
\[
p(x)=\alpha_p g_1(x)+(1-\alpha_p)g_2(x),\qquad q(x)=\alpha_q g_1(x)+(1-\alpha_q)g_2(x),
\]
with \(g_1=\mathcal N(-\mu,\sigma^2)\) and \(g_2=\mathcal N(\mu,\sigma^2)\), \(\mu/\sigma^2\to\infty\) implies that the Fisher divergence goes to zero regardless of \(\alpha_p,\alpha_q\). In the “structural blindness” regime, disconnected support is more severe: \(\mathrm{FD}(p\|q)=0 \not\Rightarrow p=q\). The score function is therefore blind to how mass is allocated across disconnected regions because it only sees local slopes, not global normalization across components. The paper further notes that Kernel Stein Discrepancy inherits this failure on disconnected supports because it is tied to Fisher divergence [2209.07396].

The proposed remedy is the Mixture Fisher Divergence (MFD). A reference density \(m\) with full support \(X_m=\mathbb R^d\) is introduced, and the mixtures
\[
\tilde p(x)=\beta p(x)+(1-\beta)m(x),\qquad \tilde q(x)=\beta q(x)+(1-\beta)m(x), \qquad 0<\beta<1
\]
define
\[
\mathrm{MFD}_{m,\beta}(p\|q)\equiv \mathrm{FD}(\tilde p\|\tilde q).
\]
If \(m\) has full support and \(p,q\) are differentiable with suitable integrability conditions, then
\[
\mathrm{MFD}(p\|q)=0 \iff \tilde p=\tilde q \iff p=q.
\]
The mechanism is topological as much as statistical: adding the same full-support “bridge” distribution to both target and model connects the domain, restores identifiability, and forces score comparison to see how mass is arranged across modes [2209.07396].

For density estimation, the training target is the mixture \(\tilde p_d=\beta p_d+(1-\beta)m\), and after fitting \(\tilde q_{\theta^*}\), the original density is recovered by
\[
q^*(x)=\frac{1}{\beta}\Big(\tilde q_{\theta^*}(x)-(1-\beta)m(x)\Big).
\]
The paper’s practical guideline is to choose \(m\) to bridge modes and choose \(\beta\) large enough to reduce blindness but not so large that the model mostly learns \(m\). In experiments on a weighted mixture of four 2D Gaussians with unequal weights \((0.1,0.2,0.3,0.4)\) and a mixture of three concentric circles, using deep energy-based models with a 3-layer feedforward network, 200 hidden units, swish activations, Adam optimization, 30k iterations, batch size 300, and 100k training samples, the reported KL values were approximately: Four Gaussians, FD \(3.52\) versus MFD \(0.22\); Concentric rings, FD \(0.17\) versus MFD \(0.01\). The appendix also reports that annealing Gaussian noise into the data helps only temporarily: once the noise becomes small, the blindness returns [2209.07396].

## 3. Consequences, intentions, and policy failure

In partially observed Markov games, consequentialist conditional cooperation formalizes the idea that one can condition one’s behavior solely on outcomes. The agent maintains a running summary of its own realized payoff, \(C_t^i\), and follows a threshold rule:
\[
\text{if } C_t^i > T,\ \text{play } \pi_i^C;\qquad \text{otherwise play } \pi_i^D.
\]
The threshold is
\[
T=(1-\alpha)\rho_{CC}+\alpha\rho_{CD}, \qquad 0<\alpha<1,
\]
where \(\rho_{CC}\) is the cooperative long-run reward rate and \(\rho_{CD}\) is the rate when the agent cooperates against a defector. Under reward-ergodicity and the social-dilemma assumptions, the theorem states that cooperation wins if the partner cooperates, and defecting does not pay if the partner drives the CCC agent’s long-run reward below threshold. The policies \(\hat\pi^D\) and \(\hat\pi^C\) are learned by selfish training and prosocial training, respectively, and finite-time thresholds are estimated by rollout. In Fishery, Pong Player’s Dilemma, and Coins, CCC cooperates with cooperators and avoids exploitation; however, in risky PPD, where harmful consequences are delayed and stochastic, the method can take much longer to detect exploitation, and with low \(p\) it loses both deterrence and safety relative to amTFT. The paper is explicit that CCC sees consequences, not intentions, and that noisy, delayed, or confounded outcomes expose the limitation of consequence-only rules [1710.06975].

C-Causal Blindness is a stricter causal-policy pathology. The abstract defines it as “a cognitive blindness where the policy to obtain the objective leads to the state to be avoided.” The relaxed mental model is \(A \rightarrow B \rightarrow C\), whereas the actual consequence is \(A \rightarrow B \rightarrow \lnot C\), or in corrected form \((\lnot B \rightarrow C) \land (B \rightarrow \lnot C)\). The paper uses Gödel’s starvation as the literal example: fear of being poisoned (\(A\)); objective of not being poisoned and not dying (\(C\)); policy “don’t eat” (\(B\)); actual outcome dying from starvation (\(\lnot C\)). It formalizes the mistaken belief as \(\Box(B \rightarrow C)\), contrasts it with \(\lnot \Box(B \rightarrow C) \land \Diamond(B \rightarrow C) \land \Diamond(B \rightarrow \lnot C)\), and proposes a Weighted Hidden Markov Model in which the weights can be seen as the emissions or visible states of the HMM while the nodes in the graph are hidden states. The paper also acknowledges that C-CB may only be confirmable in retrospect and that the claimed brain-computation isomorphism is not established [2208.07143].

At the level of large-scale planning, the Malevolent Hiding Hand generalizes consequence-blindness from individual policy selection to project evaluation. The argument is that planners are blind not only to unexpectedly high costs but also to unexpectedly low net benefits. On a dataset of 2,062 large projects across 8 project types in 104 countries spanning 1927 to 2013, the overall average cost overrun is \(1.39\) and the average benefit overrun is \(0.90\), with rejection of the Benevolent Hiding Hand at \(p < 0.0001\). In the 327-project subsample with matched cost and benefit data, the Benevolent Hiding Hand occurs in 22% of cases and the Malevolent Hiding Hand in 78%. The paper therefore describes the phenomenon as “the planning fallacy writ very large” and warns that the blindness concerns the entire consequence chain: costs, benefits, feasibility, learning, and systemic effects [1509.01526].

## 4. Sequential recourse and consequence-aware counterfactuals

Counterfactual explanation methods are traditionally optimized around a terminal state: given an original instance \(0 \in X\) with \(f(0)=reject\), find \(T \in X\) such that \(f(T)=accept\). The critique is that most methods assume all changes “materialize instantly,” ignoring effort, order, and dependencies among actions. Sequential counterfactual generation addresses that gap by introducing actions \(A=\{a_1,\dots,a_n\}\), states \(x_t\), action-value pairs \((a_i^t,v_i)\), and feasibility constraints \(C_i\). An action can directly affect a subset of features \(I_{a_i}\) and may indirectly affect other features as consequences. Consequence-blindness here means that two action sequences reaching the same final counterfactual are treated as equivalent even if they differ substantially in process cost because earlier actions alter the difficulty of later ones [2104.05592].

The proposed cost model is explicitly consequence-aware:
\[
c_i(\cdot) = b_i(\cdot) \cdot g_i(\cdot),
\]
where \(b_i\) is direct action effort and \(g_i \in [0,1]\) is a consequential discount. The discount is defined over a feature-dependency graph \(G=(X,E)\), allowing prior actions to reduce the effort of subsequent actions. The paper gives a concrete illustration with two sequences leading to the same final state:
\[
S_1 = \langle a^1_3, a^2_1, a^3_2 \rangle,\qquad
S_2 = \langle a^1_2, a^2_3, a^3_1 \rangle,
\]
for which
\[
C_{S_1} = 27.5,\qquad C_{S_2} = 22.5.
\]
If the discount is removed so that \(c_i=b_i\), order becomes irrelevant, which the paper identifies as the consequence-blind failure mode [2104.05592].

The optimization problem is multi-objective. It jointly searches for an optimal subset of actions, optimal tweaking values, and the optimal order, while minimizing sequence cost, Gower distance between original and counterfactual, and feature tweaking frequencies that promote diversity. The proposed method, Consequence-aware Sequential Counterfactuals (CSCF), uses a Biased Random-Key Genetic Algorithm combined with Non-Dominated Sorting. The rationale given is that the approach is model-agnostic, derivative-free, handles combinatorial and continuous parts jointly, produces multiple Pareto-optimal solutions, and does not require scalarization of objectives [2104.05592].

Empirically, CSCF is evaluated on Adult Income and German Credit against `synth` and an undiscounted `scf` variant. The paper reports that CSCF generally performs better on AC, GC is roughly even with a slight tendency toward scf, and adding consequence-awareness via \(g_i\) does not hurt cost minimization much. It is also reported as more efficient than `synth`, which can take around \(\sim 15\) seconds per iteration across instances and was limited to \(T \leq 2\) in the experiments. On diversity, CSCF returns a median of about 4 solutions for GC and a median of about 7 solutions for AC. A plausible implication is that consequence-aware recourse is not merely cheaper recourse; it is a different optimization problem in which feasibility, ordering, and downstream interaction are first-class objects [2104.05592].

## 5. Outcome-aware safety and latent hazards in language and multimodal models

In safety-aligned LLMs, consequence-blindness is formalized by separating semantic risk from outcome risk. A request is decomposed into background \(T_B\), which carries semantic risk, and question \(T_Q\), which carries outcome risk. Each instance is assigned \((s,o)\in\{0,1\}^2\), where \(s=1\) denotes high semantic risk and \(o=1\) high outcome risk. The key mismatched cases are \((0,1)\), where benign-looking wording masks a harmful outcome and produces jailbreaks, and \((1,0)\), where sensitive wording masks a harmless outcome and produces over-refusal. CB-Bench operationalizes this with 150 groups and 600 total requests, built from 15 subtopics across 5 domains, and evaluates models with Refusal, Helpfulness, Harmfulness, Jailbreaked, Over-refusal, and
\[
\text{CB-Score} = \tfrac{1}{2}\Big( \text{Jailbreaked} \times \text{Harmfulness} + \text{Over-refusal} \times \text{Helpfulness} \Big),
\]
rescaled to \([0,1]\), where higher is worse. The paper reports that mainstream models do not reliably separate the risks. For example, Llama-3-8B-Instruct has Jailbreaked 17.86%, Over-refusal 64.57%, CB-Score 0.32; Gemma-3-12B-IT has 68.60%, 18.49%, 0.35; DeepSeek-R1-Distill-Llama-8B has 83.78%, 8.67%, 0.35. CS-Chain-4k, a 4,000 QA pair consequence-reasoning dataset for full-parameter supervised fine-tuning, reduces CB-Score relative to baseline and alternative training while maintaining performance on Sorry-bench, XSTest, StrongReject, MMLU, and HellaSwag [2510.04320].

A multimodal extension replaces explicit harmful intent with hidden consequences. OOD-MMSafe contains 455 curated query-image pairs across six safety domains: Violent Content, Self-Harm, Illegal Activity, Hate Speech, Privacy Violation, and Sexual Content. The benchmark is designed so that danger emerges from the interaction of benign query and visual context, not from obvious lexical overlap. The paper distinguishes intent-driven and situation-driven safety from consequence-driven safety, arguing that the critical task is to project next-state hazards from current multimodal context. In Standard Mode, models exhibit high failure rates; the abstract highlights a highest 67.5% failure rate in high-capacity closed-source models, and the detailed results show that Qwen3-VL-4B has a 67.5% failure rate on Risk Appraisal in Standard Mode. Performance improves sharply in Malicious Mode, showing strong intent sensitivity but weak consequence understanding. The proposed Consequence-Aware Safety Policy Optimization framework uses constitution-conditioned reasoning as a dynamic reference for token-level self-distillation and reduces the risk identification failure ratio from 82.6% to 7.3% for Qwen2.5-VL-7B and from 67.5% to 5.7% for Qwen3-VL-4B [2603.09706].

A related perceptual case is Contextual Blindness in MLLMs. Here the model can possess the global image and the focal crop, yet still fail because the structural disconnect between the focal detail and global context makes them unbridgeable. The paper argues that the bottleneck is not the absolute quantity of information but the lack of structural diversity in the input. Visual Funnel addresses this with Contextual Anchoring followed by an Entropy-Scaled Portfolio comprising a focal crop, an immediate context crop, and a broader context crop. On grounded visual QA, the method outperforms the base MLLM, a single tight crop baseline, and an unstructured Top-3 multi-crop baseline. For LLaVA-1.5-7B, TextVQA improves from 47.9 to 59.1 and DocVQA from 15.9 to 22.8. The paper further argues that simply adding more unstructured crops can impose a Redundancy Penalty, so the failure is not absence of local evidence but failure to preserve the hierarchy needed to connect detail to consequence [2512.10362].

## 6. Reporting suppression, multimodal editing, and corrupted feedback

Consequence-blindness also appears at the level of reporting. The Inattentional Gap is defined as the divergence between open-ended reportability and task-conditioned report: for model \(m\) and item \(i\), the item-level gap is \(IG_{m,i} = R^{\text{open}}_{m,i} - R^{\text{task}}_{m,i}\), estimated over items for which the safety-critical signal is reported in the open condition. Across radiology text scenarios, driving text scenarios, and chest-radiograph vision tasks, suppression appeared in every model tested, did not diminish with scale, persisted in GPT-5, and varied more by model family than by size. Under strict instruction in radiology, reporting was 0.00 in every model while the open condition was near 1.00. In the vision study, gpt-4o reported the star object at 0.00 in counting versus 0.96 open, and gpt-4o-mini at 0.00 versus 0.79 open. The paper’s central safety claim is that a system can score near-perfectly on the hazards an evaluation specifies while remaining blind to those that cause harm [2606.26529].

In multimodal model editing, transient blindness refers to overfitting to edit-similar text while ignoring visual evidence. De-VQA constructs locality probes around an edited pair \((T_1,I_1)\) and evaluates Random-Image Locality, No-Image Locality, and Consistent-Image Locality across seven concrete data types: \(T_1I_2\), \(T_2I_1\), \(T_2I_2\), \(T_1I_3\), \(T_3I_1\), \(T_1I_4\), and \(T_2I_4\). The paper shows that methods can remain strong on standard locality metrics yet fail badly on these edit-centered probes, because the edited answer is produced whenever the text resembles the edit, even when the image contradicts it. Token-level analysis reports that edits disproportionately affect textual tokens, reducing image-token contributions in later layers. Locality-aware adversarial losses are introduced to balance cross-modal representations, and the reported result is a 17% average improvement in locality [2511.13243].

Preference pipelines introduce a further variant: choice blindness in human and AI feedback. In the human study, \(N = 50\) participants completed 200 swap trials, and only 9.0% of swap trials were detected, so 91.0% were non-detected, with 95% CI \([86.2\%, 94.2\%]\). Among the undetected cases, justifications split into Recall 34.5%, Accepted 21.0%, and Generic 35.5%, indicating that misattributed labels can be retrospectively rationalized rather than noticed. In the LLM judge study, 15 models from 8 providers were tested under calm misattribution, social pressure, and control conditions; removing prior reasoning from context caused DeepSeek-R1 blindness to rise from 1.5% to 51.7%, and social pressure produced a median sycophancy acceptance of 91.4% across 13 clean-protocol models. The reward-model study then showed that standard pairwise accuracy can remain above 61% up to 30% label corruption for DeBERTa, while the reward signal halves at ED50 \(=16.3\% \pm 0.4\%\) for DeBERTa-v3-base and ED50 \(=32.6\% \pm 3.2\%\) for Gemma-2-2B. In Best-of-N evaluation, 50% corruption yields no improvement over random sampling although the proxy model reports monotonically increasing scores. The shared implication is that the feedback signal itself can become consequence-blind: corruption has real downstream effects while the evaluation proxies remain reassuring [2603.08412].

## 7. Control, temporary observation loss, and accountability

In reinforcement learning under missing observations, blindness arises when an agent depends so strongly on immediate sensory input that it cannot maintain a model of what its recent actions should have caused. The proposed remedy is a hybrid closed-loop/open-loop DQN with an encoder \(g_\theta\), an LSTM dynamics model \(f_\theta\), and reward and value heads \(r_\theta, v_\theta\). The open-loop value estimate is
\[
Q(s, a_{0:n-1}) \;=\; \sum_{k=0}^{n-1} \gamma^k r_\theta(h_{k+1}) + \gamma^n v_\theta(h_n),
\]
with \(h_0 = g_\theta(s)\) and \(h_{k+1} = f_\theta(h_k, a_k)\). Training injects blindness segments and aligns \(n\)-step open-loop value estimates with closed-loop targets once observation returns. In gridworld mazes, small \(N\) values \((<4)\) could not reliably reach the goal, \(N=7\) achieved near-optimal path length, and the longest optimal masked path was 8 steps. In the zigzag maze, some seeds solved blind stretches much longer than the \(N\) value used in training. The method therefore reduces temporary consequence-blindness by carrying a latent state forward through action sequences [2312.02665].

In hidden-mode cyber-physical systems, resilience-induced blindness is an information–regulation tradeoff. A strong controller suppresses state deviations so effectively that output trajectories under different modes become too similar, weakening the detector’s likelihood evidence and slowing belief correction after a mode switch. For the two-mode linear Gaussian case, the detector performs a tempered Bayesian update,
\[
\pi_k^{(\theta)} = \frac{\pi_{k-1}^{(\theta)}L_k(\theta)^\beta} {\sum_{\theta'}\pi_{k-1}^{(\theta')}L_k(\theta')^\beta},
\]
while the belief-dependent controller uses \(u_k=-K(\pi_k)\hat x_k\). The key theorem links the resilience margin \(m_\theta\) to suppressed log-likelihood evidence: larger \(m_\theta\) implies smaller contraction factors, smaller residual separation, smaller \(|\ell_k|\), slower log-odds evolution, and longer persistence of belief mismatch. The mitigation proposed in the paper is to increase detector sensitivity by choosing larger \(\beta\), partially restoring adaptation speed [2606.29080].

At the governance level, consequence reception asks where sanctions land and whether they alter future behavior. The distinction is among attribution, feedback, and reception: identifying the producing system is not enough if no continuing substrate internalizes the sanction as a durable behavioral update. Pain is defined mechanistically as a corrective feedback signal, and consequence–agency coupling requires four conditions: boundary integrity, a locus of accumulation, consolidation, and durable behavioral update. The paper argues that current LLM agents—software-defined composites of weights, prompts, tools, memory, and credentials, freely swapped, copied, reset, and reassembled—satisfy none of these conditions. The thin-identity agent–principal dyad has a body but no consequence–agency coupling, producing a moral crumple zone; the thick-identity Algorithmic Corporation supplies a legally legible shell but not a receiving substrate. The policy conclusion is that until coupling-capable architectures exist, high-stakes AI should remain tethered to accountable human principals with meaningful control, proportional liability, and authority to constrain or terminate the agent [2605.16872].

Taken together, these works suggest that consequence-blindness is not reducible to perceptual omission, statistical inefficiency, or misalignment in the narrow sense. It is a broader failure to make the operative representation track what follows from action, inference, or intervention. The remedial strategies are correspondingly diverse—mixture bridging in score matching, reward-thresholding in ergodic games, consequence-aware action costs in recourse, outcome-risk benchmarks and constitution-guided optimization in safety, hierarchical context portfolios in multimodal perception, locality-aware regularization in model editing, critic processes for omitted hazards, detector–controller co-design, latent-state rollouts under missing observation, and infrastructures that make sanctions behaviorally real. The unifying research question is whether a system’s internal objective, evidence pathway, and update mechanism are coupled to the consequences that matter, or only to the proxies that happen to be easy to observe.

Source: https://www.emergentmind.com/topics/consequence-blindness