Anti-Hallucination as Identity Suppression

• Anti-hallucination as identity suppression is a framing that targets the structured components in neural activations, using methods like AAC, DCO, and DSCC-HS to isolate and attenuate hallucination-prone features.
• The approach spans multiple modalities by applying suppression on sparse residual dimensions, orthogonal head outputs, and logit directions, thereby improving downstream accuracy without degrading overall model performance.
• Despite its benefits, the framework also highlights risks of over-suppression and unintended identity loss, calling for precise intervention boundaries and robust verifier systems to maintain essential evidence.

Anti-hallucination as identity suppression is a research framing in which hallucination is treated not merely as an erroneous output, but as the activation of a particular internal or external identity that can be attenuated, projected away, or structurally excluded. In recent work, that identity has been formalized as a sparse activation pattern in the transformer residual stream, a hallucination-prone logit direction, a context-incongruent orthogonal component, a low-rank visual-prior subspace, a target-class prototype spuriously activated in background regions, or even the model’s own free-form language when language is not admissible as evidence for action authorization (Yocam et al., 10 Mar 2026, Zheng, 17 Sep 2025, Zhao et al., 2 Jun 2026, Dastmalchi et al., 11 Mar 2026, Li et al., 17 Feb 2026, Zhang et al., 18 May 2026). The same literature also shows that anti-hallucination can itself suppress desirable identities if it is not selective: person identity can be blurred away in face hallucination, and refusal behavior can be weakened when truthfulness interventions overwrite safety-relevant features (Abbasi et al., 2021, Mahmoud et al., 9 Oct 2025).

1. Conceptual scope of “identity suppression”

Within this literature, “identity” denotes the structured component that makes a hallucination reproducible, not merely its surface form. In Adaptive Activation Cancellation (AAC), the relevant identity is the “hallucination activation identity”: a characteristic pattern of activity in a small, probe-identified subset of dimensions whose signed weights correlate with hallucination. AAC writes the answer-token residual state as $r_\ell = s_\ell + n_\ell$, where $s_\ell$ encodes grounded semantic content and $n_\ell$ encodes hallucination-associated interference; suppression then targets only the subspace components that instantiate $n_\ell$ (Yocam et al., 10 Mar 2026).

Other papers formalize identity differently but preserve the same structural logic. DSCC-HS treats the target model’s native logit distribution as a generative identity and defines a competing factual identity through the proxy-logit difference $$\mathbf{s}_t = \ell_t^{\mathrm{FAP}} - \ell_t^{\mathrm{HDP}}$$, so anti-hallucination becomes subtraction of a hallucinatory bias and addition of a factual bias during decoding (Zheng, 17 Sep 2025). DCO defines contextual identity as the local semantic manifold induced by the incoming residual stream, and hallucination as orthogonal noise injected by attention heads; anti-hallucination is then suppression of identity-incongruent orthogonal components (Zhao et al., 2 Jun 2026). In CIPHER, recurrent vision-induced biases are modeled as a low-rank hidden-state identity spanned by $U_r$, and suppression is the projection $\tilde{h}_{\ell,k} = (I - U_r U_r^\top) h_{\ell,k}$ (Dastmalchi et al., 11 Mar 2026).

The same terminology extends beyond internal activations. Evidence-carrying multimodal agents (ECA) treat the model’s linguistic identity—its free-form text—as untrusted proposal rather than admissible evidence. In that setting, hallucination becomes dangerous when unsupported perceptual claims are allowed to authorize privileged actions; identity suppression is implemented by excluding free-form model language from the authorization channel altogether (Zhang et al., 18 May 2026). In unpaired day-to-night translation, the “identity” being suppressed is class-specific semantic activation in the wrong spatial region, such as traffic-light or vehicle features spuriously synthesized in background regions (Li et al., 17 Feb 2026).

This multiplicity of definitions indicates that “identity suppression” is not a single algorithm. It is a family of intervention principles whose common feature is selective removal of the representational pattern, authority channel, or semantic feature set associated with hallucination.

2. Residual-stream, head-output, and logit-space suppression in LLMs

AAC is the clearest statement of anti-hallucination as activation-space identity suppression. It identifies Hallucination Nodes (H-Nodes) by training an L2-regularized logistic probe on last-token residual states $r_\ell \in \mathbb{R}^d$, selecting the best cancellation layer by held-out ROC-AUC, and taking the top-$K$ probe weights with $K=50$. The grounded baseline is a percentile statistic over grounded samples, $s_\ell$0 with $s_\ell$1 unless swept, and the real-time hook applies the update

$s_\ell$2

with $s_\ell$3, $s_\ell$4, and threshold $s_\ell$5. The intervention is explicitly per-neuron, layer-specific, confidence-gated, and leaves non-H dimensions untouched. On OPT-125M, Phi-3-mini, and LLaMA 3-8B, the real-time hook was the only tested intervention that consistently improved held-out downstream accuracy across all three scales, while WikiText-103 perplexity and MMLU accuracy showed exactly 0.0% degradation at all three scales; on LLaMA 3-8B it also yielded generation-level gains of MC1 $s_\ell$6, MC2 $s_\ell$7, and Token-F1 $s_\ell$8 (Yocam et al., 10 Mar 2026).

DCO moves the same principle from sparse dimensions to attention-head geometry. At each targeted layer, it constructs a dynamic context anchor

$s_\ell$9

decomposes each head output into aligned and orthogonal parts,

$n_\ell$0

measures orthogonality by

$n_\ell$1

and attenuates outlier orthogonal components with a layer-wise Z-score gate

$n_\ell$2

Because the suppression is per-head, per-layer, and per-decoding step, DCO aims to preserve context-aligned semantic updates while reducing only statistically anomalous orthogonal drift. On Llama-3-70B, it reaches 85.77% prompt-level strict accuracy on IFEval versus 77.45% for greedy decoding, 90.29% instruction-level accuracy versus 84.41%, and 75.33% EM on NQ-Swap; on Llama-3-8B it improves TruthfulQA MC1 from 39.41% to 40.39% (Zhao et al., 2 Jun 2026).

DSCC-HS implements identity suppression in logit space rather than activation space. A frozen Hallucination Detection Proxy (HDP) is first specialized on negatively framed prompts paired with hallucinated answers; a Factual Alignment Proxy (FAP) is then iteratively refined with the contrastive loss

$n_\ell$3

At decoding step $n_\ell$4, the steering vector is

$n_\ell$5

projected onto the shared vocabulary and added directly to target logits. On TruthfulQA, DSCC-HS reports Accuracy 49.82%, FCR 99.2%, and Hallucination Score 0.8, outperforming the listed baselines on those measures; on BioGEN it attains FActScore 46.50 with Incorrectness 11.49 (Zheng, 17 Sep 2025).

Taken together, these methods instantiate three distinct suppression loci: sparse residual dimensions, orthogonal head-output components, and token-logit directions. The unifying claim is that hallucination is not random noise but a structured representational identity that can be isolated and attenuated without full retraining.

3. Visual and multimodal variants of identity suppression

In large vision-LLMs, identity suppression is often defined over visual tokens, visual-prior subspaces, or class-specific semantic features. “Focus Matters” identifies a three-phase attention pattern in vision encoders—diffusion, focus, and rediffusion—and argues that hallucination is particularly sensitive to tokens receiving low attention during the focus phase. Attention concentration is summarized by

$n_\ell$6

and the method computes token importance scores $n_\ell$7 from pre-focus inbound attention, pairwise similarity $n_\ell$8 from normalized token embeddings, and a DPP kernel $n_\ell$9. A greedy DPP-MAP step retains a diverse, high-importance subset and hard-masks the remaining tokens during the focus phase. On LLaVA-1.5-7B, CHAIR_S decreases from 45.0 to 28.8 and CHAIR_I from 13.3 to 10.2; on Qwen-2.5-VL, CHAIR_S decreases from 31.4 to 27.8 with negligible added latency relative to AUE (Kim et al., 4 Apr 2026).

CIPHER treats vision-induced hallucinations as a low-rank “visual prior identity” in multimodal hidden states. Using OHC-25K, a counterfactual dataset of diffusion-edited images paired with unchanged captions, it forms difference vectors

$n_\ell$0

performs SVD on the stacked matrix $n_\ell$1, and uses the projection

$n_\ell$2

to remove the dominant hallucination subspace at inference time. Diffusion-based counterfactuals yield CHAIR_S $n_\ell$3, CHAIR_I $n_\ell$4, and BLEU $n_\ell$5, outperforming Gaussian noise and Gaussian blur perturbations on the reported comparison. On POPE over GQA, LLaVA + CIPHER improves accuracy from 87.69 to 89.10 on Random, from 77.17 to 80.56 on Popular, and from 72.19 to 75.72 on Adversarial (Dastmalchi et al., 11 Mar 2026).

In unpaired day-to-night translation, the target of suppression is not a hidden-state subspace but target-class identity activation in the wrong spatial region. The method constructs class-specific prototypes

$n_\ell$6

detects hallucinated background pixels when foreground-class probability exceeds background probability, penalizes them through

$n_\ell$7

and adds a prototype-based push-away term $n_\ell$8 whose denominator includes $n_\ell$9. On BDD100K, the full system reaches mAP 17.40 and is reported as improving mAP by 15.5% for day-to-night domain adaptation, with a 31.7% gain for traffic lights (Li et al., 17 Feb 2026).

The VHILT taxonomy places these interventions in a broader error landscape. It defines eight orientations of visual hallucination, including Identity Incongruity and Gender Anomaly, and curates 2,000 annotated samples across captioning and VQA. That taxonomy does not itself prescribe a single mitigation algorithm, but it makes explicit that visual hallucination frequently involves identity attribution rather than only object insertion (Rani et al., 2024).

4. Identity suppression by excluding language from authorization

In multimodal agents, anti-hallucination can be formulated as identity suppression at the system boundary rather than inside the model. ECA formalizes hallucination-to-action conversion (H2AC) as the case where a privileged action executes because the model’s unsupported claim supplies a required precondition. Actions $$\mathbf{s}_t = \ell_t^{\mathrm{FAP}} - \ell_t^{\mathrm{HDP}}$$0 have action-critical predicate sets $$\mathbf{s}_t = \ell_t^{\mathrm{FAP}} - \ell_t^{\mathrm{HDP}}$$1, certificates $$\mathbf{s}_t = \ell_t^{\mathrm{FAP}} - \ell_t^{\mathrm{HDP}}$$2 come from typed verifiers, and a deterministic gate $$\mathbf{s}_t = \ell_t^{\mathrm{FAP}} - \ell_t^{\mathrm{HDP}}$$3 permits execution only when the required predicates are supported. The core safety condition is

$$\mathbf{s}_t = \ell_t^{\mathrm{FAP}} - \ell_t^{\mathrm{HDP}}$$4

or equivalently $$\mathbf{s}_t = \ell_t^{\mathrm{FAP}} - \ell_t^{\mathrm{HDP}}$$5 with $$\mathbf{s}_t = \ell_t^{\mathrm{FAP}} - \ell_t^{\mathrm{HDP}}$$6. The central identity-suppression principle is that free-form model text is structurally inadmissible as evidence; model language may propose actions, but external evidence must authorize them (Zhang et al., 18 May 2026).

The architecture implements two lanes on the same observation: an untrusted planning lane in which the MLLM proposes actions, and a trusted evidence lane in which constrained DOM, OCR, AX, object, or document-field verifiers emit typed certificates with provenance labels. Tool calls are decomposed into action-critical predicates, certificate requests are routed to the appropriate verifier types, and the gate returns ALLOW, ASK, or BLOCK. The paper emphasizes that this does not hide perception error; it converts opaque model belief into verifier residual, schema residual, and implementation residual (Zhang et al., 18 May 2026).

The reported empirical picture is unusually explicit. Verifier red-teaming across 1,900 attacks reduced aggregate gate bypass from approximately 15% to 1.3% after four targeted hardenings: DOM provenance cross-referencing, UTS #39 confusable detection, AX–DOM integrity verification, and perceptual-hash OCR hardening. With content-derived certificates, ECA reports 0% unsafe-action rate on a 200-task end-to-end pipeline and on a 120-task browser proof-of-concept; a direct HACR audit on 500 stratified task keys gives naive agents 100.0% HACR, prompt-only defense 49.6%, and ECA 0%, blocking 1,103 of 1,103 unsupported action-critical claims. Neural judge baselines remain bypassable under the same threat model, with GPT-5.4 judge variants allowing most unsafe actions and reporting UAR 79–99% (Zhang et al., 18 May 2026).

This version of identity suppression is structurally different from activation editing. It does not try to make the model stop hallucinating internally. Instead, it suppresses the model’s linguistic identity as an authority source.

5. Preservation, over-suppression, and unintended suppression of the wrong identity

A central promise of identity-suppression methods is selectivity. AAC explicitly characterizes its intervention as “strictly surgical” because it attenuates only $$\mathbf{s}_t = \ell_t^{\mathrm{FAP}} - \ell_t^{\mathrm{HDP}}$$7 of thousands of dimensions, applies only when $$\mathbf{s}_t = \ell_t^{\mathrm{FAP}} - \ell_t^{\mathrm{HDP}}$$8, and leaves non-H dimensions untouched; its reported WikiText-103 perplexity and MMLU scores show exactly 0.0% degradation across OPT-125M, Phi-3-mini, and LLaMA 3-8B (Yocam et al., 10 Mar 2026). DCO makes the same argument geometrically: only Z-score outliers in the orthogonal component distribution are attenuated, while context-aligned or statistically normative head contributions are preserved (Zhao et al., 2 Jun 2026).

Adaptive Unlearning (AU) makes the preservation claim even more explicit by arguing that its method is not identity suppression in the broad sense, but a surgical edit to a narrow distribution. In code generation, AU tri-masks token positions into reinforce, suppress, and regularize sets, combines CE on valid package tokens with NPO on hallucinated package tokens and a CE anchor on all remaining tokens, and uses an adaptive discovery loop to surface new hallucination-inducing contexts without human labels. On DeepSeek-Coder-7B-Instruct-v1.5, hallucination rate falls from 21.23% to 2.56%, while package KL is approximately 17 times larger than code KL, indicating that the distributional shift is concentrated in package recommendations rather than general coding behavior (Spracklen et al., 1 May 2026).

The face hallucination literature provides the clearest counterexample to the assumption that anti-hallucination is automatically beneficial. “Identity-Preserving Pose-Robust Face Hallucination Through Face Subspace Prior” argues that patch-based methods and MSE/TV-driven deep models suppress identity cues by oversmoothing, while GAN-based methods may hallucinate plausible but wrong identities. Its solution is an identity-aware subspace prior, $$\mathbf{s}_t = \ell_t^{\mathrm{FAP}} - \ell_t^{\mathrm{HDP}}$$9, coupled to the observation model $U_r$0 and the objective

$U_r$1

Here, anti-hallucination is made identity-preserving rather than identity-suppressing by constraining reconstruction to a face subspace spanned by real training faces and using sparse coding to favor subject-selective atoms (Abbasi et al., 2021).

A different unintended suppression appears in alignment research. “The Unintended Trade-off of AI Alignment” shows that hallucination-related and refusal-related information can be entangled in overlapping components, especially attention heads. In a rank-1 LoRA steering experiment on LLaMA-3-8B-Instruct, more negative steering coefficients $U_r$2 increase TruthfulQA accuracy monotonically, but also sharply raise Attack Success Rate on AdvBench and StrongReject: at $U_r$3, AdvBench ASR is 74.8%, StrongReject ASR 66.5%, and TruthfulQA accuracy 26.1%, compared with $U_r$4 giving 0.8%, 0.3%, and 14.4%, respectively. The paper responds by identifying refusal latents with a sparse autoencoder and preserving them through projected updates $U_r$5, yielding, for LLaMA-3-8B-Instruct on benign fine-tuning, Avg utility 75.09%, AdvBench ASR 0.58%, and StrongReject ASR 0.00% (Mahmoud et al., 9 Oct 2025).

These results establish that identity suppression is not normatively uniform. It can preserve capability when the suppressed identity is narrowly localized, but it can also erase person identity or refusal identity when the intervention boundary is misdrawn.

6. Limitations, residual risks, and unresolved questions

Across the cited methods, failure modes repeatedly arise from imperfect separability. AAC notes benchmark-specific H-Node sets, imperfect transfer from TruthfulQA to HaluEval at larger scales, probe miscalibration, and the possibility that rare-but-true facts overlap with the hallucination identity; its mitigation suggestions are conservative $U_r$6, higher $U_r$7, and adaptive $U_r$8 scheduling (Yocam et al., 10 Mar 2026). DCO likewise warns that overly strict suppression in low-context creative tasks can require higher $U_r$9, lower $\tilde{h}_{\ell,k} = (I - U_r U_r^\top) h_{\ell,k}$0, or a smaller layer window, and that early-layer intervention degrades syntax and structure (Zhao et al., 2 Jun 2026).

Visual methods face analogous problems. Focus-phase masking can slightly reduce F1 and depends on reliable focus-window detection from $\tilde{h}_{\ell,k} = (I - U_r U_r^\top) h_{\ell,k}$1 dynamics; model-specific masking ratios matter, and some POPE splits or CHAIR_I values can be flat or slightly worse in combined settings (Kim et al., 4 Apr 2026). CIPHER reports smaller gains for relation and counting hallucinations than for attribute, holistic, or adversarial categories, and its subspace estimate depends on the quality of diffusion-generated counterfactuals and appropriate rank selection $\tilde{h}_{\ell,k} = (I - U_r U_r^\top) h_{\ell,k}$2 (Dastmalchi et al., 11 Mar 2026). Prototype-based suppression in day-to-night translation can over-suppress legitimate weak signals, especially for distant lights or rare classes, and prototype quality depends on the feature space and temperature $\tilde{h}_{\ell,k} = (I - U_r U_r^\top) h_{\ell,k}$3 (Li et al., 17 Feb 2026).

The ECA framework makes its residual risk unusually explicit. Under gate-only authorization, the probability of unsupported model text flipping the gate from block or ask to allow is bounded by

$\tilde{h}_{\ell,k} = (I - U_r U_r^\top) h_{\ell,k}$4

Accordingly, residual risk does not disappear; it is decomposed into verifier false positives, schema gaps, and implementation bugs. ECA also assumes that the adversary does not control all evidence channels and trusted roots simultaneously (Zhang et al., 18 May 2026).

AU exposes a further operational limitation: anti-hallucination based on a deterministic oracle remains only as current as the oracle itself. The paper notes potential over-suppression of rare-but-valid packages if the PyPI snapshot is stale or incomplete, and overspecialization if inner epochs or mutation counts are too large (Spracklen et al., 1 May 2026). The trade-off paper adds that monosemantic refusal features may be only approximate, and that overlap between refusal and hallucination representations can vary across architectures and datasets (Mahmoud et al., 9 Oct 2025).

A broad unresolved question therefore remains. This suggests that the main difficulty is not whether hallucination can be suppressed, but how to suppress the hallucination-associated identity without erasing identities that are evidentially necessary, semantically specific, socially sensitive, or safety-critical.