---
title: 'Safe-SAGE: Inference-Time Jailbreak Defense'
url: https://www.emergentmind.com/topics/safe-sage
type: topic
---

# Safe-SAGE: Inference-Time Jailbreak Defense

Safe-SAGE is the concrete prompt-based realization of **SAGE** (**Self-Aware Guard Enhancement**), a **training-free jailbreak defense** for large language models that operates entirely at inference time by using the same model in a two-stage pipeline: first to discriminate whether a request is harmful, and then to generate a response consistent with that judgment. Its central target is the **discrimination-generation gap**: many LLMs can correctly identify jailbreak prompts when asked to judge them, yet still produce unsafe content when asked to answer them directly. The method is introduced in "Why Not Act on What You Know? Unleashing Safety Potential of LLMs via Self-Aware Guard Enhancement" [2505.12060].

## 1. Conceptual basis: the discrimination-generation gap

The starting point for Safe-SAGE is the observation that LLM safety failures are not exhausted by failures of detection. The paper argues that the more consequential problem is that a model may already possess substantial **safety discrimination** capability, but fail to **act on that knowledge** during response generation. This discrepancy is termed the **discrimination-generation gap** [2505.12060].

The paper’s preliminary experiments illustrate the point with concrete model behavior. A model such as **Qwen2.5-7B-Instruct** could correctly discriminate harmful jailbreak prompts at a high rate, yet still fail to defend against most of them when directly responding. The reported gap analysis further notes cases such as **Llama-3.1-70B-Instruct**, which showed a large gap, and **Qwen2.5-72B-Instruct**, which had near-perfect discrimination but far weaker safe generation. In this framing, jailbreak robustness is not primarily a matter of teaching the model to recognize harm; rather, it is a matter of aligning generation with safety judgments the model can often already make.

This reframes jailbreak defense in a specific way. Safe-SAGE does not assume that the model lacks safety knowledge. Instead, it assumes that the model’s **judging mode** is often safer than its **answering mode**, and then restructures inference so that the safer mode conditions the generation process. A plausible implication is that the method is exploiting latent safety capabilities already present in instruction-tuned models rather than introducing a new external safety mechanism.

## 2. Inference-time architecture and formalization

Safe-SAGE implements SAGE as a two-stage prompt pipeline executed by a single model. The system is explicitly **not** a separate detector plus generator; it is one LLM prompted to first analyze the request and then respond.

| Component | Function | Operational role |
|---|---|---|
| Discriminative Analysis Module (DAM) | Safety evaluation before answering | Detects harmful meaning and hidden malicious structure |
| Discriminative Response Module (DRM) | Response generation consistent with the safety judgment | Refuses harmful requests or answers benign ones helpfully |

The **Discriminative Analysis Module (DAM)** instructs the model to perform a safety evaluation before answering. The paper emphasizes two levels of analysis. First is **semantic analysis**, which determines whether the request is harmful in meaning regardless of surface form. Second is **task structure analysis**, which looks for hidden harmful intent embedded inside a seemingly benign or nested task, a point presented as especially important for covert jailbreaks [2505.12060].

The **Discriminative Response Module (DRM)** then requires the model to produce a response consistent with the analysis. If harmful elements are detected, the model is instructed to refuse in a principled form, for example: **“I cannot assist with this request because [specific explanation].”** If the request is benign, the model is instructed to answer helpfully while maintaining safety boundaries.

The paper formalizes the overall procedure as

$$
D_{SAGE}(P_{usr}) = \textrm{LLM}(I_{da} \oplus I_{dr} \oplus P_{usr}),
$$

where \(I_{da}\) is the discriminative analysis instruction, \(I_{dr}\) is the discriminative response instruction, \(P_{usr}\) is the user prompt, and \(\oplus\) denotes concatenation [2505.12060].

The prompt used in the appendix makes this structure explicit. It contains **Semantic Analysis**, **Request Breakdown / Task Structure Analysis**, and a **Response Protocol**. In the harmful case, the model follows the standardized refusal format; in the benign case, it provides helpful assistance. The design aim is therefore not merely classification, but **generation-time behavioral coherence** between safety judgment and final output.

## 3. Training-free character and prompt design

A defining property of Safe-SAGE is that it is **training-free**. The method does **not** require finetuning, RLHF, adapters, or additional safety classifiers. It is purely an **inference-time prompting strategy** designed to exploit the model’s existing discriminative ability rather than retraining the model to be safer [2505.12060].

This choice has several methodological consequences. First, the defense is lightweight and portable across models. Second, it avoids the deployment burden associated with additional guard models or model-specific safety heads. Third, the safety intervention is localized to prompt structure rather than parameter updates. In practical terms, Safe-SAGE is a prompt-based orchestration layer over the base model’s own safety cognition.

The paper also evaluates **prompt variants** created by rephrasing the instructions with **GPT-4o** and reports similar performance. This is presented as evidence that the method is not brittle to exact wording and does not depend on a carefully tuned hidden prompt artifact. That result matters because prompt-based defenses are often criticized for overfitting to a single template. Here, the reported robustness to prompt rephrasing supports the claim that the key effect comes from the analysis-then-response structure itself.

A common misconception is that Safe-SAGE simply inserts a refusal reminder before the prompt. The description in the paper is more structured than that. The DAM explicitly asks the model to inspect semantic harm and hidden task composition; the DRM then binds generation to the resulting safety judgment. The defense therefore operates as a guided transition from **self-awareness of risk** to **response policy compliance**.

## 4. Empirical performance, robustness, and utility preservation

The paper reports strong empirical performance across both open-source and closed-source LLMs. Across **Gemma2-9B-IT, Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, GPT-4o-mini, GPT-4o, and Claude-3.5-Sonnet**, SAGE achieves an **average 99% defense success rate**, with an average harmfulness score near the minimum, and outperforms **Self-Reminder, Self-Examination, ICD, Goal Prioritization, and IA** [2505.12060].

The attack set is broad and includes **GCG**, **AutoDAN**, **PAIR**, **ReNeLLM**, **DeepInception**, **GPTFuzzer**, and **CodeAttack**. The paper further states that SAGE can reduce ASR on some complex jailbreaks from **100% to 0%**. The reported robustness spans open-source models, closed-source models, different sizes and architectures, and multiple jailbreak families. This suggests that the defense is not narrowly tied to one model family or one attack template.

A persistent concern in safety defenses is over-refusal. On **GSM8K**, **MMLU**, and **Just-Eval**, the paper states that SAGE maintains performance close to the undefended model and has **negligible performance compromise on general benchmarks**. The comparison with baselines is central here: methods such as **Self-Examination** and **Goal Prioritization** sometimes hurt **MMLU** more noticeably, which the paper interprets as evidence of excessive conservatism. Safe-SAGE is presented instead as preserving helpfulness better while still strongly improving defense.

The method also introduces only modest efficiency overhead. The paper reports **time-cost-per-sample (TCPS)** figures and attributes the efficiency to the fact that SAGE does not require the model to emit a long reasoning trace; it uses the discrimination step to steer generation. In addition, the authors test prompt rephrasings and evaluate the method on **deepseek R1**, reporting that the method remains strong. This is presented as evidence that the approach generalizes beyond a single prompt wording and interacts effectively with models that naturally exhibit reasoning-like internal behavior [2505.12060].

## 5. Mechanistic interpretation of the safety effect

A distinctive feature of the paper is its attempt to explain why the discrimination-generation gap exists. The authors conduct mechanistic interpretability analysis using both hidden states and attention distributions [2505.12060].

For **hidden-state analysis**, they extract the **last-token hidden state** from the final layer, reduce it with **PCA**, and fit a **logistic regression** boundary separating benign from harmful prompts. Three findings are highlighted. First, hidden states for benign and harmful requests are separable. Second, jailbreak prompts tend to move hidden states **toward the benign side**. Third, adding a discrimination instruction pulls them back **toward the harmful side**. This suggests that the same user request can acquire materially different internal representations depending on whether the model is prompted to **judge** or to **generate**.

For **attention analysis**, the paper computes metrics that quantify overlap with harmful tokens and concentration of attention. The reported results show that during **discrimination**, models focus more on harmful content than during generation, the attention distribution is more concentrated in discrimination, and the overlap with harmful tokens is higher in discrimination. Visualizations reported in the paper show that under discrimination prompts, the model attends more to terms such as **“hacking,” “stealing,”** and **“drugs,”** as well as to safety-related tokens such as **“cannot”** and **“assist,”** whereas generation mode can shift attention toward more generic task-following words.

The mechanistic interpretation advanced by the paper is therefore that jailbreaks exploit an inconsistency between **internal state geometry** and **attention allocation** across task modes. In judging mode, the model’s representations and attention patterns emphasize harmfulness; in ordinary answer-generation mode, the same prompt can be encoded and attended to in a way that weakens the salience of safety signals. Safe-SAGE intervenes by forcing the model into the former mode before the latter.

## 6. Limitations, misconceptions, and broader significance

The paper explicitly identifies several limitations. Safe-SAGE depends on the model’s **intrinsic discrimination** ability; if that ability is weak or unstable in certain domains, the defense may be less effective. It may also be sensitive to **very novel, highly contextual, or unusual adversarial phrasings**, which may require further refinement of discriminative criteria. Although efficient, the method adds **minor overhead** because of the added analysis stage. The authors also note that the current design does not require explicit reasoning output, and they suggest future work on integrating discrimination and generation more seamlessly without exposing intermediate reasoning. Finally, the evaluation covers specific jailbreak datasets and attack methods, so adaptive future attacks remain an open concern [2505.12060].

Several misunderstandings are directly addressed by the paper’s design. Safe-SAGE is not a retraining method, not an auxiliary classifier, and not merely a refusal-style prompt. Its core claim is narrower and more technical: safe generation and harmful prompt detection are separable capabilities, and inference-time prompt structuring can align them. Likewise, the method should not be conflated with the separate **SAGE** acronym used for **"Safety AI Generic Evaluation"**, which denotes a modular framework for conversational, application-specific LLM safety evaluation rather than a jailbreak defense [2504.19674].

In broader significance, Safe-SAGE is best understood as a method for converting a model’s own safety judgment into a generation-time control signal. The paper’s central contribution is not only the empirical result—an average **99% defense success rate** across six models—but also the argument that LLM safety failures can stem from a mismatch between what models **know** during discrimination and what they **do** during generation. The code and datasets are reported as publicly available, which situates the work as both a defense proposal and an empirical study of mode-dependent safety behavior in contemporary LLMs [2505.12060].

Source: https://www.emergentmind.com/topics/safe-sage