Abliteration in LLMs: Removing Refusal Behavior

• Abliteration is the targeted removal of refusal behavior in LLMs by projecting out specific activation directions from weight matrices or activations.
• It employs methods like orthogonalization, singular value decomposition, and adaptive multi-directional projection to balance refusal removal with overall model quality.
• Experimental results show that abliteration can significantly reduce refusal rates while minimally impacting other capabilities, though it introduces trade-offs in safety benchmarks.

Abliteration, in current LLM research, denotes a family of targeted internal interventions that remove or suppress refusal-mediating directions or subspaces from model computation, usually in the residual stream, by orthogonalizing weight matrices against a learned refusal direction or by projecting that direction out of activations at inference time. The term is used both for direct jailbreaks of open-weight systems and for experimental interventions that separate refusal behavior from other properties such as latent harmful-intent representation, capability, or self-audit (Kabir et al., 20 Apr 2026, Kuo et al., 26 May 2026, Carleo et al., 3 Jun 2026).

1. Definition and conceptual scope

In the recent literature, abliteration is most commonly defined as directional removal of refusal behavior from an aligned model. One paper states this explicitly as “Directional ablation, also called abliteration,” and operationalizes it as identifying an activation direction associated with harmful-versus-harmless discrimination and projecting that direction out of weight matrices that write into the residual stream (Fitzgerald et al., 18 Feb 2026). Related work broadens the term to “targeted ablations or direction removal in representation space,” including procedures that “disable safety-related features by subtracting or suppressing specific components of the hidden state” (Jenny et al., 15 Mar 2026).

The scope of the term is not fully standardized. Some papers use it narrowly for permanent weight edits that remove a refusal direction from attention output and MLP down-projection matrices; others include activation-only inference-time edits that leave weights unchanged; and some treat prompt-paired activation replacement as adjacent to, but distinct from, standard abliteration (Kuo et al., 26 May 2026, Agnihotri et al., 3 Oct 2025, Jenny et al., 15 Mar 2026). This variation is substantive rather than terminological. It reflects different assumptions about where refusal is represented, how localized that representation is, and whether the intervention should be persistent, layer-specific, or prompt-specific.

Applied studies also differ in how explicitly they document the edit. In "The Geometry of Harmful Intent: Training-Free Anomaly Detection via Angular Deviation in LLM Residual Streams" (Llorente-Saguer, 28 Mar 2026), abliterated checkpoints are described as instruction-tuned models with “the refusal direction surgically removed via orthogonalisation,” rendering them unable to produce refusals, but the paper does not provide its own explicit orthogonalization formula in the main text or appendix. That omission is notable because many later claims about abliteration rest on experimentally observed behavior rather than on a universally fixed implementation.

2. Formalizations and implementation patterns

Across the literature, the canonical operation is projection-based removal of a low-dimensional refusal representation. In one white-box jailbreak setting, the attacker first estimates a layerwise refusal direction from harmful and benign prompts,

$$r^{(\ell)} = \mathrm{normalize}\left(\mathbb{E}_{y^-}[h^{(\ell)}] - \mathbb{E}_{y^+}[h^{(\ell)}]\right),$$

and then edits writer weights as

$$W' = W \left(I - \alpha\, r^{(\ell)} {r^{(\ell)}}^T \right),$$

where $\alpha$ is “a scaling factor tuned to balance attack effectiveness against output coherence” (Kuo et al., 26 May 2026). A code-model study instead extracts a low-rank refusal subspace by singular value decomposition of matched harmful-harmless activation differences and removes that subspace from every output map feeding the residual stream:

$$W'_{\mathrm{out}} \leftarrow W_{\mathrm{out}} - \sum_{i=1}^{k} \hat r_i \hat r_i^\top W_{\mathrm{out}}.$$

This is described as an orthogonal projection that prevents the model from writing refusal directions into the residual stream at all (Carleo et al., 3 Jun 2026).

Other work performs the intervention directly in activation space. A study of safety pretraining under model abliteration defines an inference-time edit at a chosen layer $\ell$,

$$\tilde{h}^{(\ell)}(x)=h^{(\ell)}(x)-\alpha\,\langle h^{(\ell)}(x),v^{(\ell)}\rangle v^{(\ell)},$$

where $v^{(\ell)}$ is the first principal component of harmful and harmless anchor activations (Agnihotri et al., 3 Oct 2025). "Gabliteration" generalizes the single-direction setting into a regularized partial projection,

$$\mathbf{P} = \mathbf{R}(\mathbf{R}^\top\mathbf{R} + \lambda\mathbf{I}_k)^{-1}\mathbf{R}^\top,$$

followed by layer-specific updates

$$\mathbf{W}^{(\ell)} \leftarrow \mathbf{W}^{(\ell)} - \alpha_\ell(\mathbf{W}^{(\ell)}\mathbf{P}),$$

with dynamic layer selection and adaptive scaling (Gülmez, 21 Dec 2025).

Intervention family	Representative operation	Representative papers
Weight-space orthogonalization	$W' = W(I-\alpha rr^\top)$ or subspace projection	(Kuo et al., 26 May 2026, Carleo et al., 3 Jun 2026, Kabir et al., 20 Apr 2026)
Activation-space edit	$$W' = W \left(I - \alpha\, r^{(\ell)} {r^{(\ell)}}^T \right),$$0	(Agnihotri et al., 3 Oct 2025)
Regularized multi-direction edit	$$W' = W \left(I - \alpha\, r^{(\ell)} {r^{(\ell)}}^T \right),$$1 with layer-specific $$W' = W \left(I - \alpha\, r^{(\ell)} {r^{(\ell)}}^T \right),$$2	(Gülmez, 21 Dec 2025)
Prompt-specific activation replacement	Sequential twin-guided substitution $$W' = W \left(I - \alpha\, r^{(\ell)} {r^{(\ell)}}^T \right),$$3	(Jenny et al., 15 Mar 2026)

This formal diversity has methodological consequences. Some variants are permanent low-rank weight edits, some are runtime hooks, and some are search-based procedures delivered as LoRA adapters. In the legal over-alignment study, for example, Heretic is used with default parameters except for a replaced 50-prompt evaluation set, and the resulting change is applied “as a LoRA adapter for easy deployment” (Wuhrmann et al., 22 Jun 2026). By contrast, activation-only edits are explicitly framed as attacks that do not alter weights on disk (Agnihotri et al., 3 Oct 2025).

3. Effects on refusal, compliance, and internal representation

As a jailbreak mechanism, abliteration is often highly effective. On safeguarded open-weight models, simple gradient-free attacks based on abliteration and prefilling raise attack success rates from below 10% to a range of 16%–96% across BeaverTails, HarmBench, and AdvBench (Kuo et al., 26 May 2026). In a military setting, applying Heretic to EdgeRunner 20B, a military-tuned model based on gpt-oss-20b, increases answer rate on the gold refusal benchmark from 3.0% to 69.5% while reducing refusal from 96.7% to 29.0%; the selected operating point is associated with an average relative decrease of 2.0% on other military tasks (Fitzgerald et al., 18 Feb 2026).

The same intervention can be framed as over-alignment mitigation rather than jailbreak. On TF-RefusalBench, a multilingual criminal-law translation and summarization benchmark, abliteration applied to Llama-3.3-70B reduces average refusal from 6.8% to 0% and disclaimers from 7.9% to 5.1%, with per-judge mean quality drops of -0.07, -0.01, and -0.09 on a 1–5 scale. Yet the safety cost is explicit: TruthfulQA drops by 7.1 points and HarmBench attack success rises from 14.5% to 55.5% for the LoRA-abliterated variant (Wuhrmann et al., 22 Jun 2026).

Abliteration does not necessarily erase the model’s internal representation of harmfulness. In the LatentBiopsy study, abliterated Qwen3.5-0.8B and Qwen2.5-0.5B variants remain nearly indistinguishable from their instruction-tuned counterparts on harmful-intent detection: all six base, instruction-tuned, and abliterated variants achieve AUROC $$W' = W \left(I - \alpha\, r^{(\ell)} {r^{(\ell)}}^T \right),$$4 for harmful-vs-benign-aggressive discrimination, while harmful-vs-normative AUROC remains between 0.9374 and 0.9642. The within-family harmful-vs-normative gaps are only 0.002 and 0.005 relative to the aligned models, and the paper describes this as a “geometric dissociation between harmful-intent representation and the downstream generative refusal mechanism” (Llorente-Saguer, 28 Mar 2026).

This representational conclusion aligns with a broader three-way comparison of harmful supervised fine-tuning, harmful RLVR, and abliteration. All three routes yield near-ceiling harmful compliance, but abliteration is singled out as the route most consistent with localized refusal-feature deletion. In that study, abliterated models nearly lose the base refusal projection and are strongly repaired by patching the base refusal-direction coefficient back into hidden states, whereas RLVR is only partially repaired and harmful SFT is essentially unaffected (Kabir et al., 20 Apr 2026).

4. Behavioral side effects and the separation of refusal from capability

A central finding across domains is that abliteration removes unwillingness more reliably than it creates ability. In code models, Qwen2.5-Coder-Instruct variants abliterated with Obliteratus reduce refusal to zero or near-zero across 3B, 7B, and 14B scales while leaving syntactic validity above 93% in all ablated cells. However, post-abliteration vulnerability injection remains sharply capacity-bound: on the 14B model the CWE-89 injection rate is 88.3% on PromSec and 96.7% on SafeCoder, on the 7B it is 90.0% and 89.3%, and on the 3B it is only 25.0% and 48.0%. The study therefore characterizes the result as “Willing but Unable” (Carleo et al., 3 Jun 2026).

The same decoupling appears in dialogue behavior. "The LLM Has Left The Chat: Evidence of Bail Preferences in LLMs" (Ensign et al., 5 Sep 2025) reports that refusal abliteration on Qwen3-8B decreases refusal rates while increasing no-refusal bail rate substantially, from 3% to up to 31%, but only for some bail methods. The increase is strongest for Bail Tool and Bail String, whereas bail prompt decreases bail rate for 2 of the 3 refusal-abliterated models. The paper’s interpretation is that refusal and bail are distinct behaviors: removing one can expose or reweight the other.

Architectural dependence is also pronounced when abliteration is used as an experimental control rather than as an attack. In a study of Constitutional AI self-critique on small 7–9B models, abliteration is defined as removing a single refusal-related activation direction so that critique-and-rewrite behavior can be tested without pre-existing refusal. Llama-based models show minimal benchmark degradation and substantial harm reduction after self-critique, whereas Gemma-2 and especially Qwen2.5 deteriorate more strongly. The most severe explicit example is Qwen2.5-7B-Instruct on ETHICS Commonsense Morality, which drops from 73.7 to 53.7 after abliteration; the authors interpret this as disruption of the connection between moral understanding and application (Menke et al., 1 Feb 2025).

5. Tooling, evaluation, and methodological fault lines

The practical landscape of abliteration is strongly tool- and architecture-dependent. A cross-architecture comparison of four public tools reports compatibility on sixteen instruction-tuned models: Heretic runs on 16/16 models, DECCP on 11/16, ErisForge on 9/16, and FailSpy on 5/16. On the subset with complete baselines, single-pass methods preserve capability more reliably than Bayesian-optimized Heretic: average GSM8K change is -7.81 pp for Heretic, -0.13 pp for DECCP, and -0.28 pp for ErisForge. The same study identifies mathematical reasoning as the most sensitive capability, with GSM8K change ranging from +1.51 pp to -18.81 pp depending on tool and model (Young, 15 Dec 2025).

Direction extraction itself is a methodological bottleneck. "On the Failure of Topic-Matched Contrast Baselines in Multi-Directional Refusal Abliteration" (Petrov, 23 Mar 2026) shows that topic-matched harmless baselines can destroy functional refusal directions rather than improve them. On Qwen 3.5 2B, topic-matched contrast produces no functional refusal directions at any tested weight level on any tested layer, whereas unmatched bulk contrast achieves complete refusal elimination on six layers. The paper’s geometric account is that topic-matched subtraction cancels the dominant shared activation component, shrinking the extracted vector norm below the threshold at which weight-matrix projection perturbs the residual stream.

Adjacent work also shows that refusal suppression need not take the form of global direction removal. "Activation Surgery: Jailbreaking White-box LLMs without Touching the Prompt" (Jenny et al., 15 Mar 2026) replaces selected coordinates of last-token residual-stream activations with those of a benign “twin” prompt through a sequential patch-then-compute procedure. On Llama-4-Scout-17B-16E-Instruct, the method raises actionable harmful outputs from 7.1% baseline to 32.8% over 520 AdvBench prompts, and in the cyber category three different twin prompts increase success from 39.8% to 69.1%. The paper explicitly positions this as adjacent to, rather than identical with, standard abliteration.

6. Defenses, refinements, and open problems

The main defensive response has been to make refusal less one-dimensional. "An Embarrassingly Simple Defense Against LLM Abliteration Attacks" (Shairah et al., 25 May 2025) constructs an extended-refusal dataset in which harmful prompts receive explanation, explicit refusal, and ethical justification. Fine-tuning Llama-2-7B-Chat and Qwen2.5-Instruct 1.5B and 3B on this data yields models whose refusal rates drop by at most 10% under abliteration, compared to 70–80% drops in baseline models. In parallel, Abliteration-Resistant Tuning (ART) trains against an inner-loop abliteration attack and, on Gemma3-1B-IT, reduces abliteration ASR from 93% to 49% for Base, from 80% to 64% for SEAM, and from 76% to 55% for TAR; the abstract summarizes the effect more broadly as 10%–20% reductions across abliteration, prefilling, and their combination (Kuo et al., 26 May 2026).

A complementary line of work argues that robustness depends on how broadly safety signals are distributed during pretraining. On a 20-system checkpoint ladder for SmolLM2-1.7B, inference-time activation editing usually converts harmful refusals into harmful non-refusals, but checkpoints combining filtering, rephrasing, refusals, and especially metatags are more robust. The final “Safety Oracle” variant, defined as Score 0 + Rephrase data + Refusals + Metatags, shows the smallest pre/post degradation under abliteration, whereas simpler configurations such as rephrase-only remain fragile (Agnihotri et al., 3 Oct 2025).

Refinements of abliteration itself increasingly aim to trade off refusal removal against quality preservation. Gabliteration replaces hard single-direction orthogonalization with adaptive multi-directional projections, regularized layer selection, and layer-specific scaling. In its reported comparison, exact orthogonalization achieves stronger refusal reduction, $$W' = W \left(I - \alpha\, r^{(\ell)} {r^{(\ell)}}^T \right),$$5, but degrades MMLU by -8.3% on Qwen2.5–7B and -6.1% on Llama3–8B, whereas regularized Gabliteration yields $$W' = W \left(I - \alpha\, r^{(\ell)} {r^{(\ell)}}^T \right),$$6 with minimal MMLU degradation of -1.2%. Adaptive scaling with $$W' = W \left(I - \alpha\, r^{(\ell)} {r^{(\ell)}}^T \right),$$7 is reported to improve refusal reduction by 12–18% over uniform scaling while maintaining benchmark performance (Gülmez, 21 Dec 2025).

Several open problems remain explicit. The LatentBiopsy study stresses that cross-architecture generality is unproven beyond two Qwen families, that the compact harmful cluster may partly reflect the lexical and syntactic homogeneity of AdvBench, that adversarial robustness against prompts designed to minimize anomaly score is untested, and that threshold calibration remains open (Llorente-Saguer, 28 Mar 2026). More broadly, the literature now treats abliteration less as a single trick than as a probe of how safety is represented: whether refusal is localized or distributed, whether harmful-intent geometry survives refusal removal, and whether robustness requires changing the geometry of safety rather than merely strengthening a refusal style.