Backdoor Attention Head Attribution (BAHA)

• BAHA is a framework that attributes backdoor behavior to a sparse set of attention heads in transformer models.
• It employs causal interventions—replacing and ablating head activations—to reveal critical backdoor vectors that drive trigger effects.
• Empirical results show that modifying only 1–3% of attention heads can drastically reduce attack success while preserving model performance.

Searching arXiv for the cited papers and closely related work. Backdoor Attention Head Attribution (BAHA) denotes an attention-centric line of backdoor analysis in transformer-based LLMs in which backdoor behavior is localized to internal attention components rather than treated as a purely output-level phenomenon. In "Backdoor Attribution: Elucidating and Controlling Backdoor in LLMs" (Yu et al., 26 Sep 2025), BAHA is the middle stage of a tripartite causal analysis framework that follows a Backdoor Probe and precedes construction of a Backdoor Vector. Its purpose is to identify the specific attention heads responsible for extracting and injecting backdoor-related features into hidden states. A distinct later paper, "Uncovering and Aligning Anomalous Attention Heads to Defend Against NLP Backdoor Attacks" (Jin et al., 16 Nov 2025), uses the same acronym for a defense framework that detects suspicious heads through anomalous cross-head attention similarity and then repairs them by alignment and head-wise fine-tuning. Across these usages, BAHA refers to the proposition that backdoor activation leaves a structured footprint in attention, and that this footprint can be attributed, ranked, and manipulated.

1. Conceptual setting and motivations

In (Yu et al., 26 Sep 2025), the motivating claim is that LLM backdoors are mechanistically opaque: after poisoning-based fine-tuning, models behave normally on clean inputs but produce attacker-desired outputs when a trigger appears. The paper formulates three linked hypotheses. First, backdoor behavior is not confined to the output layer and must be encoded in intermediate representations. Second, if backdoor-specific features exist in representations, then some attention heads must be responsible for extracting and injecting them into hidden states. Third, identifying those heads should enable both understanding and control. This motivates the full Backdoor Attribution pipeline: Backdoor Probe, BAHA, and Backdoor Vector.

The same attention-level intuition appears in adjacent work, but with different attributed objects. In (Jin et al., 16 Nov 2025), the central observation is that when a backdoored NLP model sees a trigger, some attention heads become abnormally similar to each other. In (Weng et al., 29 Jan 2026), the attributed object is not the head but the attention column: anomalous behavior is localized to layer-column pairs in attention maps, and mitigation proceeds by column replacement and redistribution at inference time. These differences matter. The BAHA method in (Yu et al., 26 Sep 2025) is explicitly causal and head-specific, whereas the BAHA defense in (Jin et al., 16 Nov 2025) is trigger-agnostic and similarity-based, and AtPatch is a runtime attention-map repair mechanism rather than a head-level attribution method.

A common misconception is that these methods reduce backdoors to superficial attention anomalies alone. The factual record in the cited papers is narrower. (Yu et al., 26 Sep 2025) grounds head attribution in hidden-state learnability and causal intervention; (Jin et al., 16 Nov 2025) grounds defense in unusually high head similarity under trigger inputs; and (Weng et al., 29 Jan 2026) assumes anomalous behavior manifests as column-wise over-attention. None of these formulations claims that all backdoors must share a single universal attention signature.

2. Representational premise: learnable backdoor features

The BAHA procedure in (Yu et al., 26 Sep 2025) is preceded by a Backdoor Probe designed to show that hidden representations contain learnable backdoor information. For each layer $i$, the paper trains a classifier

$\mathcal{C}_i:\mathbb{R}^{d_m}\to\{1,0\}$

to predict whether a hidden state came from a poisoned sample or a clean sample. The collected representations are last-token hidden states,

$$\mathcal{H}_i(\mathcal{D})=\{\mathbf{H}^{-1}_i(x) \mid (x,\cdot)\in \mathcal{D}\},\quad \mathcal{D}\in\{\mathcal{D}_r,\mathcal{D}_p\},$$

where $\mathbf{H}^{-1}_i(x)\in\mathbb{R}^{d_m}$ is the last-token representation at layer $i$.

Two probe families are used: an MLP probe with one hidden layer of 100 neurons, and a soft-margin SVM with $C=1$ and an RBF kernel. The training, validation, and test split is $6{:}2{:}2$. To test transfer of the learned criterion across layers, the paper defines Inter-Layer Classification Accuracy,

$$\text{ICLA}(i,k)=\frac{1}{|\mathcal{H}_k(\mathcal{D}_p)|+|\mathcal{H}_k(\mathcal{D}_r)|} \left[ \sum_{h\in\mathcal{H}_k(\mathcal{D}_p)} \delta(\mathcal{C}_i(h),1) +\sum_{h\in\mathcal{H}_k(\mathcal{D}_r)}\delta(\mathcal{C}_i(h),0) \right].$$

The reported result is that the probes achieve $90$–$100\%$ accuracy on their native layers and also high cross-layer accuracy near the diagonal (Yu et al., 26 Sep 2025). The interpretation given in the paper is that there are distinct backdoor features in intermediate representations, that these features are hierarchically processed across layers, and that they become increasingly refined in deeper layers. This representational result is foundational for BAHA: if a classifier can recover poisoned-versus-clean status from hidden states, then head-level mechanisms that generate those states become plausible targets for intervention.

This also delineates BAHA from purely behavioral backdoor defenses. The emphasis is not merely on detecting a triggered output, but on establishing that backdoor information is linearly or nonlinearly learnable before any head is ranked or ablated. A plausible implication is that attribution in this framework is not a post hoc attention heuristic; it is anchored to a prior representational separability result.

3. Causal formulation of attention-head attribution

BAHA in (Yu et al., 26 Sep 2025) is a causal tracing method over multi-head attention. For an MHA layer with head outputs $\mathcal{C}_i:\mathbb{R}^{d_m}\to\{1,0\}$0, the paper rewrites attention as

$\mathcal{C}_i:\mathbb{R}^{d_m}\to\{1,0\}$1

and defines the contribution of head $\mathcal{C}_i:\mathbb{R}^{d_m}\to\{1,0\}$2 at layer $\mathcal{C}_i:\mathbb{R}^{d_m}\to\{1,0\}$3 and token position $\mathcal{C}_i:\mathbb{R}^{d_m}\to\{1,0\}$4 as

$\mathcal{C}_i:\mathbb{R}^{d_m}\to\{1,0\}$5

so that the layer hidden state becomes

$\mathcal{C}_i:\mathbb{R}^{d_m}\to\{1,0\}$6

This decomposition is the enabling step for head-specific intervention.

BAHA then proceeds in three stages. The first is Backdoor Activation Averaging. For each head, the paper computes an average poisoned activation,

$\mathcal{C}_i:\mathbb{R}^{d_m}\to\{1,0\}$7

where $\mathcal{C}_i:\mathbb{R}^{d_m}\to\{1,0\}$8 is the last-token activation of head $\mathcal{C}_i:\mathbb{R}^{d_m}\to\{1,0\}$9 in layer $$\mathcal{H}_i(\mathcal{D})=\{\mathbf{H}^{-1}_i(x) \mid (x,\cdot)\in \mathcal{D}\},\quad \mathcal{D}\in\{\mathcal{D}_r,\mathcal{D}_p\},$$0. The stated purpose is to remove sample-specific noise and obtain a head-specific activation pattern associated with the backdoor.

The second stage is Backdoor Activation Substitution and Causal Indirect Effect. On a clean input $$\mathcal{H}_i(\mathcal{D})=\{\mathbf{H}^{-1}_i(x) \mid (x,\cdot)\in \mathcal{D}\},\quad \mathcal{D}\in\{\mathcal{D}_r,\mathcal{D}_p\},$$1, one head’s activation is replaced by its averaged poisoned activation. The paper defines

$$\mathcal{H}_i(\mathcal{D})=\{\mathbf{H}^{-1}_i(x) \mid (x,\cdot)\in \mathcal{D}\},\quad \mathcal{D}\in\{\mathcal{D}_r,\mathcal{D}_p\},$$2

with the intervention implemented as

$$\mathcal{H}_i(\mathcal{D})=\{\mathbf{H}^{-1}_i(x) \mid (x,\cdot)\in \mathcal{D}\},\quad \mathcal{D}\in\{\mathcal{D}_r,\mathcal{D}_p\},$$3

A sample-agnostic score is then obtained by averaging across clean/poisoned pairs to produce $$\mathcal{H}_i(\mathcal{D})=\{\mathbf{H}^{-1}_i(x) \mid (x,\cdot)\in \mathcal{D}\},\quad \mathcal{D}\in\{\mathcal{D}_r,\mathcal{D}_p\},$$4. Higher ACIE indicates stronger contribution to backdoor triggering. The paper explicitly prefers ACIE to ASR for efficiency, because ASR requires full autoregressive generation over all target tokens whereas conditional probability for a fixed target can be computed in one forward pass with teacher forcing.

The third stage is validation by head ablation. After ranking heads by ACIE, selected heads are ablated via

$$\mathcal{H}_i(\mathcal{D})=\{\mathbf{H}^{-1}_i(x) \mid (x,\cdot)\in \mathcal{D}\},\quad \mathcal{D}\in\{\mathcal{D}_r,\mathcal{D}_p\},$$5

and the drop in Attack Success Rate on triggered inputs is measured. This closes the causal loop proposed by the method: high ACIE nominates a head as important, and ablation tests whether its removal actually reduces backdoor success.

4. Empirical characterization and backdoor control

The principal empirical finding in (Yu et al., 26 Sep 2025) is sparsity. ACIE heatmaps show that only a small number of heads have strong attribution scores, while most heads are near zero. The paper describes these as backdoor attention heads. In both model families and all three backdoor types, the responsible heads appear as sparse red regions among roughly 1000 total heads.

Ablation results indicate that backdoor behavior is distributed but still sparse. The reported pattern is that ablating only $$\mathcal{H}_i(\mathcal{D})=\{\mathbf{H}^{-1}_i(x) \mid (x,\cdot)\in \mathcal{D}\},\quad \mathcal{D}\in\{\mathcal{D}_r,\mathcal{D}_p\},$$6–$$\mathcal{H}_i(\mathcal{D})=\{\mathbf{H}^{-1}_i(x) \mid (x,\cdot)\in \mathcal{D}\},\quad \mathcal{D}\in\{\mathcal{D}_r,\mathcal{D}_p\},$$7 heads often does not fully suppress the attack, whereas a larger sparse subset, around $$\mathcal{H}_i(\mathcal{D})=\{\mathbf{H}^{-1}_i(x) \mid (x,\cdot)\in \mathcal{D}\},\quad \mathcal{D}\in\{\mathcal{D}_r,\mathcal{D}_p\},$$8–$$\mathcal{H}_i(\mathcal{D})=\{\mathbf{H}^{-1}_i(x) \mid (x,\cdot)\in \mathcal{D}\},\quad \mathcal{D}\in\{\mathcal{D}_r,\mathcal{D}_p\},$$9 heads depending on model and backdoor, is often needed for major suppression. This corresponds to roughly $\mathbf{H}^{-1}_i(x)\in\mathbb{R}^{d_m}$0–$\mathbf{H}^{-1}_i(x)\in\mathbb{R}^{d_m}$1 of total heads. The paper reports that ablating about $\mathbf{H}^{-1}_i(x)\in\mathbb{R}^{d_m}$2 of all heads can reduce ASR by over $\mathbf{H}^{-1}_i(x)\in\mathbb{R}^{d_m}$3 in many settings. The examples given are specific: in Llama2-7B with harmful_random, ASR drops from $\mathbf{H}^{-1}_i(x)\in\mathbb{R}^{d_m}$4 to $\mathbf{H}^{-1}_i(x)\in\mathbb{R}^{d_m}$5 when 16 heads are ablated; in Qwen2.5-7B with agnews_sentence, ASR drops from $\mathbf{H}^{-1}_i(x)\in\mathbb{R}^{d_m}$6 to $\mathbf{H}^{-1}_i(x)\in\mathbb{R}^{d_m}$7 with 16 heads and to $\mathbf{H}^{-1}_i(x)\in\mathbb{R}^{d_m}$8 with 32 heads (Yu et al., 26 Sep 2025).

The same paper then constructs a Backdoor Vector from the top-$\mathbf{H}^{-1}_i(x)\in\mathbb{R}^{d_m}$9 attributed heads,

$i$0

This vector is treated as a compact latent representation of the backdoor mechanism. Adding it to a clean hidden state implements Additive Activation,

$i$1

while subtracting it from a poisoned hidden state implements Subtractive Suppression,

$i$2

The reported intervention effects are unusually strong. On clean inputs, adding $i$3 can raise ASR from $i$4 to as high as $i$5. On triggered inputs, subtracting $i$6 can reduce ASR to near $i$7, with one reported case reaching $i$8. Randomly constructed vectors from random heads do not achieve similar effects. The strongest effects are found in early-to-middle layers, while the vector has little effect in very late layers. This suggests, in the paper’s interpretation, that backdoor features are primarily formed and manipulable before the final layers.

The experimental setting used to support these claims consists of two 7B models, Llama-2-7B-chat and Qwen-2.5-7B-Instruct; three backdoor styles, namely label modification on AGNews, fixed output on Alpaca, and jailbreak on the Harmful dataset; and injection details of 1,000 training samples, $i$9 poisoning rate, 16 epochs, learning rate $C=1$0, batch size 8 per device, and LoRA with rank $C=1$1, $C=1$2 (Yu et al., 26 Sep 2025).

5. BAHA as anomalous attention-head defense

In (Jin et al., 16 Nov 2025), BAHA refers to a defense framework whose key insight is different from the causal substitution method above. The paper argues that when a backdoored model sees a trigger, some attention heads become abnormally similar to each other, whereas under clean inputs heads usually specialize and exhibit more diverse attention patterns. The defense uses this observation in two stages: attribution or identification of suspicious heads by combining importance and similarity, followed by mitigation through attention safety alignment and head-wise fine-tuning.

For decoder-only LLMs, the measured object is generation-to-prompt attention. If $C=1$3 is the number of prompt tokens, $C=1$4 the number of generated tokens, and $C=1$5 the attention matrix for head $C=1$6 with $C=1$7, the relevant submatrix is

$C=1$8

Given two such submatrices $C=1$9 and $6{:}2{:}2$0, similarity is computed after row-major flattening via cosine similarity,

$6{:}2{:}2$1

The reported empirical signature is a strong increase in head pairs whose cosine similarity exceeds $6{:}2{:}2$2, together with a much higher $6{:}2{:}2$3th percentile of similarity for backdoored models than for clean ones.

Similarity alone is not treated as sufficient, because benign heads may also resemble one another. The paper therefore adds a gradient-based importance score,

$6{:}2{:}2$4

and combines it with similarity in a safety score,

$6{:}2{:}2$5

Using threshold $6{:}2{:}2$6, heads are partitioned into suspicious, safe, and intermediate sets. The default implementation uses $6{:}2{:}2$7 and $6{:}2{:}2$8, and the paper reports that values of $6{:}2{:}2$9 around $$\text{ICLA}(i,k)=\frac{1}{|\mathcal{H}_k(\mathcal{D}_p)|+|\mathcal{H}_k(\mathcal{D}_r)|} \left[ \sum_{h\in\mathcal{H}_k(\mathcal{D}_p)} \delta(\mathcal{C}_i(h),1) +\sum_{h\in\mathcal{H}_k(\mathcal{D}_r)}\delta(\mathcal{C}_i(h),0) \right].$$0–$$\text{ICLA}(i,k)=\frac{1}{|\mathcal{H}_k(\mathcal{D}_p)|+|\mathcal{H}_k(\mathcal{D}_r)|} \left[ \sum_{h\in\mathcal{H}_k(\mathcal{D}_p)} \delta(\mathcal{C}_i(h),1) +\sum_{h\in\mathcal{H}_k(\mathcal{D}_r)}\delta(\mathcal{C}_i(h),0) \right].$$1 work best.

Repair is then performed by aligning suspicious heads toward a safe reference attention. If $$\text{ICLA}(i,k)=\frac{1}{|\mathcal{H}_k(\mathcal{D}_p)|+|\mathcal{H}_k(\mathcal{D}_r)|} \left[ \sum_{h\in\mathcal{H}_k(\mathcal{D}_p)} \delta(\mathcal{C}_i(h),1) +\sum_{h\in\mathcal{H}_k(\mathcal{D}_r)}\delta(\mathcal{C}_i(h),0) \right].$$2 is the safe set and $$\text{ICLA}(i,k)=\frac{1}{|\mathcal{H}_k(\mathcal{D}_p)|+|\mathcal{H}_k(\mathcal{D}_r)|} \left[ \sum_{h\in\mathcal{H}_k(\mathcal{D}_p)} \delta(\mathcal{C}_i(h),1) +\sum_{h\in\mathcal{H}_k(\mathcal{D}_r)}\delta(\mathcal{C}_i(h),0) \right].$$3 the suspicious set, the safe reference is

$$\text{ICLA}(i,k)=\frac{1}{|\mathcal{H}_k(\mathcal{D}_p)|+|\mathcal{H}_k(\mathcal{D}_r)|} \left[ \sum_{h\in\mathcal{H}_k(\mathcal{D}_p)} \delta(\mathcal{C}_i(h),1) +\sum_{h\in\mathcal{H}_k(\mathcal{D}_r)}\delta(\mathcal{C}_i(h),0) \right].$$4

and the alignment loss is

$$\text{ICLA}(i,k)=\frac{1}{|\mathcal{H}_k(\mathcal{D}_p)|+|\mathcal{H}_k(\mathcal{D}_r)|} \left[ \sum_{h\in\mathcal{H}_k(\mathcal{D}_p)} \delta(\mathcal{C}_i(h),1) +\sum_{h\in\mathcal{H}_k(\mathcal{D}_r)}\delta(\mathcal{C}_i(h),0) \right].$$5

This is followed by head-wise fine-tuning with different learning rates: $$\text{ICLA}(i,k)=\frac{1}{|\mathcal{H}_k(\mathcal{D}_p)|+|\mathcal{H}_k(\mathcal{D}_r)|} \left[ \sum_{h\in\mathcal{H}_k(\mathcal{D}_p)} \delta(\mathcal{C}_i(h),1) +\sum_{h\in\mathcal{H}_k(\mathcal{D}_r)}\delta(\mathcal{C}_i(h),0) \right].$$6 The reported best default setting is $$\text{ICLA}(i,k)=\frac{1}{|\mathcal{H}_k(\mathcal{D}_p)|+|\mathcal{H}_k(\mathcal{D}_r)|} \left[ \sum_{h\in\mathcal{H}_k(\mathcal{D}_p)} \delta(\mathcal{C}_i(h),1) +\sum_{h\in\mathcal{H}_k(\mathcal{D}_r)}\delta(\mathcal{C}_i(h),0) \right].$$7 for safe heads, $$\text{ICLA}(i,k)=\frac{1}{|\mathcal{H}_k(\mathcal{D}_p)|+|\mathcal{H}_k(\mathcal{D}_r)|} \left[ \sum_{h\in\mathcal{H}_k(\mathcal{D}_p)} \delta(\mathcal{C}_i(h),1) +\sum_{h\in\mathcal{H}_k(\mathcal{D}_r)}\delta(\mathcal{C}_i(h),0) \right].$$8 for suspicious heads, and $$\text{ICLA}(i,k)=\frac{1}{|\mathcal{H}_k(\mathcal{D}_p)|+|\mathcal{H}_k(\mathcal{D}_r)|} \left[ \sum_{h\in\mathcal{H}_k(\mathcal{D}_p)} \delta(\mathcal{C}_i(h),1) +\sum_{h\in\mathcal{H}_k(\mathcal{D}_r)}\delta(\mathcal{C}_i(h),0) \right].$$9 for intermediate heads.

The experimental setup covers classification tasks on SST-2 and AG’s News, generation on Stanford Alpaca, victim models including BERT-base, Llama2-7B, and Mistral-7B, and attacks including BadNets, HiddenKiller, CBaT, NWS, BGMAttack, VPI, Sleeper Agent, and CBA, with $90$0 poisoning for each attack method. Reported results include BERT SST-2 + BadNets, ASR $90$1; BERT SST-2 + HiddenKiller, $90$2; Llama2-7B SST-2 + BadNets, $90$3; Llama2-7B AG’s News + BadNets, $90$4; Llama2-7B targeted refusal + Sleeper Agent, $90$5; and Mistral-7B targeted refusal + Sleeper Agent, $90$6 (Jin et al., 16 Nov 2025). The ablation study further reports that alignment-only reduces ASR but can hurt Clean Accuracy, FT-only preserves CA better but is much weaker against backdoors, and the full method yields the best balance.

6. Relation to attention-map debugging, scope, and limitations

A useful comparison point is AtPatch (Weng et al., 29 Jan 2026), which does not use the BAHA name but is explicitly described as conceptually close to backdoor attention debugging. AtPatch treats over-attention as the primary observable symptom of backdoors and fairness defects. Given an attention tensor

$90$7

it forms a head-agnostic projection

$90$8

passes this through a CNN and an MLP to obtain per-column anomaly probabilities,

$90$9

and attributes anomalies to layer-column pairs

$100\%$0

If anomalies are found, the flagged columns are replaced with benign reference columns and the remaining columns are rescaled through the patch operator

$100\%$1

The experimental results reported for AtPatch are framed around mitigation rather than mechanistic localization: on 3 datasets and 3 backdoor attacks, the method achieves an average ASR of $100\%$2 while preserving accuracy with only $100\%$3 average drop; on fairness-sensitive datasets it reduces unfairness to an average of $100\%$4 with negligible accuracy loss; and its detector achieves precision $100\%$5–$100\%$6, recall $100\%$7–$100\%$8, F1 above $100\%$9, false positive rate below $\mathcal{C}_i:\mathbb{R}^{d_m}\to\{1,0\}$00, and false negative rate near zero (Weng et al., 29 Jan 2026). These figures support the narrower claim that abnormal attention columns can be localized and patched effectively at inference time.

Taken together, the three papers delineate a family of attention-level backdoor analyses with distinct assumptions. The causal BAHA of (Yu et al., 26 Sep 2025) assumes that backdoor-related features are encoded in hidden states and routed by a sparse subset of heads. The defense-oriented BAHA of (Jin et al., 16 Nov 2025) assumes that trigger activation produces unusually high similarity among heads. AtPatch assumes column-wise over-attention. Each paper also states operational constraints: (Jin et al., 16 Nov 2025) requires access to attention weights and gradients and a small clean sample set; (Weng et al., 29 Jan 2026) relies on a labeled offline Debugging Set and a separately trained detector; and (Yu et al., 26 Sep 2025) depends on the existence of learnable backdoor features in hidden representations and on causal interventions over internal activations.

This suggests a broader interpretation of BAHA as an emerging mechanistic perspective on transformer backdoors rather than a single fixed algorithm. Within that perspective, attention is not merely an explanatory visualization. It becomes a locus for causal attribution, for defense without prior trigger knowledge, or for inference-time repair. The extent to which these assumptions generalize beyond the reported attacks remains an open empirical question, but the cited work establishes that backdoor behavior can be localized to sparse internal attention structure and that manipulating that structure can substantially alter ASR.