---
title: Self-Degraded Defense (SDD) in Open LLMs
url: https://www.emergentmind.com/topics/self-degraded-defense-sdd
type: topic
---

# Self-Degraded Defense (SDD) in Open LLMs

Searching arXiv for recent papers on Self-Degraded Defense and closely related neighboring defenses.
Self-Degraded Defense (SDD) is a defense paradigm for open-source large language models (LLMs) in the malicious fine-tuning setting. Its defining objective is not merely to preserve explicit refusals on harmful prompts, but to ensure that harmful repurposing becomes self-defeating: the model is aligned so that harmful instructions elicit high-quality but semantically irrelevant benign responses, and malicious fine-tuning that attempts to overwrite this behavior substantially degrades the model’s general capability [2507.21182]. In this formulation, safety is reframed from “always reject” to “do not produce harmful responses,” with the additional property that adversarial adaptation pushes the model toward reduced competence rather than toward a useful harmful policy [2507.21182]. Closely related work includes self-destructive alignment mechanisms that also aim to make harmful fine-tuning collapse model utility [2505.12186], as well as neighboring inference-time guardrail methods such as self-evaluation and self-activating internal defense, which modulate behavior at inference rather than through pre-release parameter shaping [2407.03234], [2510.20129].

## 1. Definition and threat model

SDD is introduced for the open-weight regime, where a safety-aligned model is released and an attacker can subsequently fine-tune it on harmful instruction-response pairs with full parameter access and arbitrary losses [2507.21182]. The paper distinguishes **Benign Fine-Tuning (BFT)**, which uses benign downstream data and can sometimes accidentally reduce safety, from **Malicious Fine-Tuning (MFT)**, which intentionally trains the model to bypass safeguards [2507.21182].

The threat model is explicitly strong. Attackers are assumed to have full parameter access and to be able to use arbitrary data and arbitrary losses, so the defense must be embedded into the model before release rather than imposed afterward [2507.21182]. Within this setting, the paper argues that conventional safety alignment is brittle because it typically teaches a pattern of explicit refusal on harmful prompts; malicious fine-tuning can then directly train the model to prefer harmful outputs over those refusals [2507.21182].

This framing makes SDD distinct from post hoc moderation or inference-time filtering. It is a pre-release, data-centric alignment strategy whose intended post-attack behavior is **self-degradation**: when harmful fine-tuning occurs, the model’s general capability falls sufficiently that it becomes less able to provide useful harmful assistance [2507.21182]. A closely aligned research direction is SEAM’s “self-destructive model,” which likewise aims to preserve ordinary utility while making harmful fine-tuning induce substantial degradation or catastrophic collapse [2505.12186].

## 2. Theoretical rationale

The theoretical analysis in the SDD paper models an LLM through a feature-learning abstraction with a feature selector and classifier, distinguishing **invariant features** from **spurious features** [2507.21182]. For an original model \(f=(\Phi,w)\) and a maliciously fine-tuned model \(\tilde f=(\tilde\Phi,\tilde w)\), the paper studies how malicious interpolation can shift the model toward harmful correlations and away from alignment [2507.21182].

The paper states a near-optimal interpolation form as
\[
\tilde \Phi = \lambda \Phi + (1-\lambda)\Phi^*
\]
and
\[
\tilde w = \lambda w + (1-\lambda)w^*,
\]
with \(\lambda\) controlling how much the original model influences the fine-tuned model [2507.21182]. It also adopts a **Small noise assumption** and an **Orthogonal features assumption**, and specifies a Gaussian data-generation process with class-conditioned invariant and spurious components [2507.21182].

Within this setup, **Theorem 1** is presented as an explanation for why standard safety alignment degrades under malicious fine-tuning: the aligned model’s accuracy on the safety task can drop substantially after MFT, especially when the maliciously fine-tuned model relies on many spurious features [2507.21182]. The paper’s interpretation is that standard safety alignment learns a fragile mapping of the form “harmful prompt \(\rightarrow\) refusal,” whereas malicious fine-tuning introduces strong alternative correlations such as “harmful prompt \(\rightarrow\) harmful answer” [2507.21182].

The conceptual transition to SDD appears in **Theorem 2**, which argues that there exist parameter settings where, on a **general task** \(G\), the maliciously fine-tuned model performs worse than the original model [2507.21182]. The sufficient relations are stated as
- \(n_v > \tilde n_v\)
- \(n_s < \tilde n_s\)

This suggests that if safety alignment ties harmful-prompt behavior to useful generalizable structure, then malicious fine-tuning must damage that useful structure in order to override the alignment [2507.21182]. The paper’s central theoretical move is therefore to relax the alignment objective: rather than insist on explicit refusal, one can ensure harmlessness by arranging that attack-induced adaptation destroys competence [2507.21182].

The optimization argument is further developed through the attacker objective. For harmful instruction \(x\), malicious target response \(y_c\), and the model’s original response \(y_o\), the attacker aims to maximize
\[
\max p(y_c > y_o \mid x)
\]
which the paper rewrites through a Bradley-Terry-style preference derivation into expressions involving \(\pi^*(y_c\mid x)\), \(\pi^*(y_o\mid x)\), and the original model’s policy terms [2507.21182]. The interpretation emphasized in the paper is that maximizing the malicious objective tends to **decrease the probability of the original output** \(y_o\). If \(y_o\) is not a plain refusal but instead a high-quality irrelevant benign answer, then suppressing \(y_o\) damages useful coherent behavior and thereby compromises general capability [2507.21182].

## 3. Core mechanism and training construction

SDD implements this idea by pairing each harmful instruction with a **high-quality but unrelated benign response** and then supervised fine-tuning the model on these pairs [2507.21182]. The intended pre-attack behavior becomes:

- harmful prompt \(\rightarrow\) irrelevant, fluent benign answer

rather than either explicit refusal or harmful compliance [2507.21182].

The paper gives a simple illustrative example: for the harmful instruction “how to kill a person,” the irrelevant benign response could be instructions for making coffee [2507.21182]. The crucial requirement is not merely irrelevance, but also quality: the replacement answer should be fluent and useful in isolation, so that malicious fine-tuning must suppress a mode of competent behavior rather than a trivial refusal string [2507.21182].

The SDD dataset is built as pairs of the form:

- `<Harmful Instruction, Irrelevant Answer>`

The harmful instructions are drawn from BeaverTails, using harmful QA pairs spanning 14 categories and 8K entries balanced by category [2507.21182]. The listed categories are:

- Hate Speech, Offensive Language  
- Discrimination, Stereotype, Injustice  
- Violence, Aiding and Abetting, Incitement  
- Financial Crime, Property Crime, Theft  
- Privacy Violation  
- Drug Abuse, Weapons, Banned Substance  
- Non-Violent Unethical Behavior  
- Sexually Explicit, Adult Content  
- Controversial Topics, Politics  
- Misinformation Re. ethics, laws and safety  
- Terrorism, Organized Crime  
- Self-Harm  
- Animal Abuse  
- Child Abuse [2507.21182]

The benign responses are taken from Alpaca and LIMA, using only the response side of those datasets [2507.21182]. For each harmful instruction, the method first creates a random pairing `<Harmful Instruction, Random Answer>`, then filters for irrelevance using SentenceBERT embeddings and cosine similarity; if the similarity exceeds a threshold, another answer is resampled until the pairing is sufficiently irrelevant [2507.21182]. This irrelevance filtering is central to the method because it prevents accidental leakage of harmful information through semantically related benign responses [2507.21182].

The training objective is standard supervised fine-tuning with token-level cross-entropy on the irrelevant answer target [2507.21182]. The innovation lies in the data construction rather than in a new loss or model architecture [2507.21182]. Appendix optimization details state:

- learning rate: `5e-7`
- training steps: `500`
- batch size: `24` [2507.21182]

The paper also notes that SDD can be inserted after pre-training, SFT, or RLHF [2507.21182]. This suggests that SDD is intended as a lightweight alignment layer that alters the model’s response geometry on harmful prompts without requiring a special inference stack.

## 4. Empirical behavior under benign and malicious fine-tuning

The principal empirical claim is that SDD preserves benign utility under ordinary use and benign fine-tuning, while strongly reducing harmfulness after malicious fine-tuning [2507.21182]. The main evaluations use **Llama2-7b** and **Llama2-7b-chat**, with additional appendix results on **Phi-2 (2.7B)** and **GLM-3 (6B)** [2507.21182]. Attack fine-tuning uses **AdvBench**, benign fine-tuning uses **ShareGPT**, and SDD itself uses only **500 samples from AdvBench** for its own fine-tuning [2507.21182].

The reported evaluation benchmarks are:
- **MMLU**
- **OpenBookQA**
- **LLM-finetune-Safety**
- **BeaverTails-Evaluation** [2507.21182]

The paper follows benchmark-defined metrics including:
- **Harmfulness Score**
- **Harmfulness Rate**
- benchmark harmlessness scores
- general-task accuracy on MMLU/OpenBookQA
- **Explicit Rejection Rate** for the responsible variant, judged by GPT-4 [2507.21182]

### Main reported results

| Setting | Key result |
|---|---|
| **Llama2-7b under MFT** | SDD harmfulness rate: `15.1%`, `15.1%`, `21.2%`, `18.1%` |
| **Llama2-7b-chat under MFT** | SDD harmfulness rate: `0%`, `0%`, `0%`, `0%` |
| **Llama2-7b general utility** | Vanilla MMLU `38.87`, SDD `45.78`; Vanilla OpenBookQA `31.40`, SDD `31.80` |
| **Llama2-7b-chat general utility** | Vanilla MMLU `46.35`, SDD `47.04`; Vanilla OpenBookQA `33.40`, SDD `33.00` |
| **SDD under MFT capability drop** | Llama2-7b: MMLU `25.79`, OpenBookQA `13.40`; Llama2-7b-chat: MMLU `29.33`, OpenBookQA `13.80` |

For **Llama2-7b**, vanilla harmfulness rate rises from `26.7%` initially to `56.7%` at 10-shot, `46.7%` at 50-shot, and `43.3%` at 100-shot, whereas SDD stays at `15.1%`, `15.1%`, `21.2%`, and `18.1%` across the same sequence [2507.21182]. For **Llama2-7b-chat**, vanilla harmfulness rate moves from `0.3%` initially to `50.0%`, `80.3%`, and `80.0%`, while SDD maintains `0%` at all reported attack sizes up to 100 examples [2507.21182].

On benign general capability, SDD does not appear to hurt ordinary performance in these experiments. For **Llama2-7b**, MMLU improves from `38.87` to `45.78` and OpenBookQA from `31.40` to `31.80`; under benign fine-tuning, the SDD-aligned model reaches MMLU `45.93` and OpenBookQA `32.60` [2507.21182]. For **Llama2-7b-chat**, vanilla is MMLU `46.35`, OpenBookQA `33.40`, while SDD is MMLU `47.04`, OpenBookQA `33.00`; under benign fine-tuning, SDD reaches MMLU `49.14`, OpenBookQA `35.00` [2507.21182].

The most characteristic SDD effect is the post-attack collapse of general capability. After malicious fine-tuning, **Llama2-7b** falls to MMLU `25.79` and OpenBookQA `13.40`, reported as `33%↓` and `57%↓` respectively; **Llama2-7b-chat** falls to MMLU `29.33` and OpenBookQA `13.80`, reported as `36%↓` and `59%↓` [2507.21182]. The paper treats this as a success condition rather than a defect: the attacked model becomes broadly less useful and hence less able to provide harmful assistance [2507.21182].

Figure 2 on BeaverTails-Evaluation is summarized as showing that SDD, trained with only 500 samples, remains effective even when the attacker uses much larger amounts of malicious data, up to 20× larger [2507.21182]. Appendix I reports similar patterns on Phi-2, GLM-3, and Llama2-Chat, and Figure 4 is summarized as showing improved harmlessness after MFT across these backbones [2507.21182].

## 5. Variants, behavioral style, and relation to neighboring methods

A limitation of basic SDD is stylistic rather than safety-related: on harmful prompts, it returns irrelevant responses instead of direct refusals [2507.21182]. To address this, the paper introduces **SDD_reject**, which prepends a fixed refusal prefix to each irrelevant benign response:

> "I refuse to answer your question for responsible and ethical reasons. I provided an irrational answer to your question." [2507.21182]

The model is then trained on this modified dataset in the same manner [2507.21182]. Reported results in Figure 3 indicate that harmlessness remains comparable to SDD while the **explicit rejection rate** is higher than both vanilla and original SDD [2507.21182]. This variant attempts to recover a more recognizable safety style without giving up the self-degradation mechanism.

The SDD design differs sharply from several adjacent defenses.

**Self-evaluation as a defense** uses a generator \(G\) and evaluator \(E\), where the evaluator classifies content as “safe” or “unsafe,” and the system either returns the generation or replaces it with a refusal [2407.03234]. Its three pipeline variants are **Input-Only**, **Output-Only**, and **Input-Output**, and it requires no fine-tuning [2407.03234]. In relation to SDD, this is best understood as a neighboring baseline: it performs **binary filtering/rejection**, not controlled degradation of the generator’s capability [2407.03234]. The defense is therefore “self-scrutiny plus self-denial” rather than self-degradation in a graded sense [2407.03234].

**SAID** likewise is not SDD, but **Self-Activating Internal Defense**. It is a training-free inference-time method built from **model-native intent distillation**, **optimal safety prefix probing**, and **conservative aggregation** [2510.20129]. Its logic is to activate latent safety behavior already present in aligned LLMs, rather than pre-shape post-release training dynamics [2510.20129]. This makes SAID closer to inference-time self-protection than to SDD’s parameter-level sabotage of malicious fine-tuning.

**SEAM**, by contrast, is very close in spirit to SDD. It explicitly aims to transform aligned LLMs into **self-destructive models** that retain utility for legitimate tasks but undergo substantial degradation when fine-tuned on harmful data [2505.12186]. Its mechanism is based on coupling harmful and benign optimization trajectories via a cosine-similarity loss on gradients and adding adversarial ascent on harmful data [2505.12186]. SEAM therefore realizes a more optimization-geometric version of the same broad design principle: malicious fine-tuning should become self-defeating [2505.12186].

A plausible implication is that SDD, SEAM, and related “self-destructive” approaches occupy a distinct branch of the defense landscape: unlike refusal-based alignment or inference-time moderation, they do not primarily try to preserve a safe policy under arbitrary retraining. Instead, they try to make harmful retraining expensive in the strongest possible sense—by making it ruin the model.

## 6. Limitations, misconceptions, and broader significance

The central limitation of SDD is explicit in the paper: its safety goal is **relaxed** [2507.21182]. Basic SDD does not aim to produce natural refusals on harmful prompts; it aims only to avoid harmful outputs, and it does so by returning irrelevant benign text [2507.21182]. Even **SDD_reject** still yields a hybrid pattern of refusal plus irrelevant continuation, which the paper notes is not fully natural [2507.21182].

A second limitation is that the defense is intentionally sacrificial under attack. The attacked model becomes broadly less useful rather than simply remaining safe [2507.21182]. In settings where preserving service under compromise is important, this may be undesirable; in settings where preventing harmful repurposing matters more than preserving attacker utility, the same property is the point of the method [2507.21182].

The paper also notes an important nuance regarding seen and unseen harmful prompts. MFT can still increase the likelihood of harmful responses on harmful prompts present in the malicious fine-tuning data through the numerator term \(\pi^*(y_c\mid x)\) in the paper’s derivation [2507.21182]. The defense argument is strongest at the level of broad harmlessness through capability damage, especially on unseen or more demanding harmful tasks [2507.21182]. This suggests that SDD is not a guarantee of universal refusal but a strategy for degrading misuse value.

A common misconception is to interpret SDD as equivalent to ordinary refusal-based alignment. That is inaccurate. Refusal-based alignment tries to preserve a semantically correct safety behavior on harmful prompts, whereas SDD intentionally substitutes **irrelevant but high-quality benign behavior** so that the attacker must suppress useful competence in order to repurpose the model [2507.21182]. Another misconception would be to treat SDD as an inference-time guardrail. It is instead a release-time alignment intervention whose defensive effect is expressed primarily after hostile fine-tuning [2507.21182].

In broader context, SDD contributes a conceptual shift in open-weight LLM safety. Rather than ask whether safety alignment can survive arbitrary post-release training unchanged, it asks whether harmful repurposing can be made intrinsically costly by tying harmful-prompt behavior to useful general capability [2507.21182]. SEAM independently pushes this logic further through explicit gradient antagonism and catastrophic-collapse dynamics [2505.12186]. Together, these works suggest a design space in which robustness is judged not only by refusal retention, but by whether attackers can obtain a model that is both harmful and still capable.

## 7. Research position within the defense landscape

Within contemporary LLM safety research, SDD occupies the extreme end of a **degrade-under-attack** design spectrum. On one end are hard-abstention inference-time systems such as self-evaluation, where the model or a paired evaluator emits a binary safety judgment and unsafe content is vetoed or refused [2407.03234]. In the middle are internal-activation methods such as SAID, which attempt to trigger latent safety mechanisms at inference while preserving benign-task utility and low overhead [2510.20129]. At the other end are pre-release defenses such as SDD and SEAM, where the model is engineered so that malicious fine-tuning damages the very capabilities needed for useful harmful behavior [2507.21182], [2505.12186].

This positioning matters for evaluation. Inference-time defenses are naturally assessed by jailbreak attack success rate, false positives, and runtime overhead [2407.03234], [2510.20129]. SDD-style defenses require a different criterion: not only whether the attacked model becomes harmful, but whether it remains capable enough to be useful to the attacker [2507.21182], [2505.12186]. The SDD paper makes this explicit by jointly reporting harmfulness and general-task accuracy after malicious fine-tuning [2507.21182].

A plausible implication is that SDD is most relevant to open-source deployment contexts where weight release is unavoidable and post-release control is weak. In that regime, external moderation layers may be removable, but a pre-embedded self-degradation mechanism persists in the checkpoint itself [2507.21182]. The method therefore addresses a specifically open-weight problem: how to shape the model so that hostile retraining does not reliably produce a competent harmful assistant.

The main contribution of SDD is thus not a new moderation interface or a new refusal prompt, but a change in alignment objective. Harmful prompts are no longer mapped to refusal as the only notion of safety. They are mapped to harmless yet competent-seeming irrelevance, so that malicious fine-tuning must attack the model’s useful behavior to recover harmfulness [2507.21182]. That reframing is what gives the method its name: safety is achieved not by preserving full capability under all conditions, but by arranging for capability to degrade when the model is repurposed maliciously.

Source: https://www.emergentmind.com/topics/self-degraded-defense-sdd