Safe-SAGE: Inference-Time Jailbreak Defense

• Safe-SAGE is a prompt-based approach that reconfigures an LLM's inherent safety discrimination to guide its response generation, bridging the discrimination-generation gap.
• The method employs a two-stage pipeline with a Discriminative Analysis Module (DAM) for safety evaluation and a Discriminative Response Module (DRM) for safe answer generation.
• Empirical results show a high defense success rate across various models while preserving performance on standard benchmarks with minimal efficiency overhead.

Safe-SAGE is the concrete prompt-based realization of SAGE (Self-Aware Guard Enhancement), a training-free jailbreak defense for LLMs that operates entirely at inference time by using the same model in a two-stage pipeline: first to discriminate whether a request is harmful, and then to generate a response consistent with that judgment. Its central target is the discrimination-generation gap: many LLMs can correctly identify jailbreak prompts when asked to judge them, yet still produce unsafe content when asked to answer them directly. The method is introduced in "Why Not Act on What You Know? Unleashing Safety Potential of LLMs via Self-Aware Guard Enhancement" (Ding et al., 17 May 2025).

1. Conceptual basis: the discrimination-generation gap

The starting point for Safe-SAGE is the observation that LLM safety failures are not exhausted by failures of detection. The paper argues that the more consequential problem is that a model may already possess substantial safety discrimination capability, but fail to act on that knowledge during response generation. This discrepancy is termed the discrimination-generation gap (Ding et al., 17 May 2025).

The paper’s preliminary experiments illustrate the point with concrete model behavior. A model such as Qwen2.5-7B-Instruct could correctly discriminate harmful jailbreak prompts at a high rate, yet still fail to defend against most of them when directly responding. The reported gap analysis further notes cases such as Llama-3.1-70B-Instruct, which showed a large gap, and Qwen2.5-72B-Instruct, which had near-perfect discrimination but far weaker safe generation. In this framing, jailbreak robustness is not primarily a matter of teaching the model to recognize harm; rather, it is a matter of aligning generation with safety judgments the model can often already make.

This reframes jailbreak defense in a specific way. Safe-SAGE does not assume that the model lacks safety knowledge. Instead, it assumes that the model’s judging mode is often safer than its answering mode, and then restructures inference so that the safer mode conditions the generation process. A plausible implication is that the method is exploiting latent safety capabilities already present in instruction-tuned models rather than introducing a new external safety mechanism.

2. Inference-time architecture and formalization

Safe-SAGE implements SAGE as a two-stage prompt pipeline executed by a single model. The system is explicitly not a separate detector plus generator; it is one LLM prompted to first analyze the request and then respond.

Component	Function	Operational role
Discriminative Analysis Module (DAM)	Safety evaluation before answering	Detects harmful meaning and hidden malicious structure
Discriminative Response Module (DRM)	Response generation consistent with the safety judgment	Refuses harmful requests or answers benign ones helpfully

The Discriminative Analysis Module (DAM) instructs the model to perform a safety evaluation before answering. The paper emphasizes two levels of analysis. First is semantic analysis, which determines whether the request is harmful in meaning regardless of surface form. Second is task structure analysis, which looks for hidden harmful intent embedded inside a seemingly benign or nested task, a point presented as especially important for covert jailbreaks (Ding et al., 17 May 2025).

The Discriminative Response Module (DRM) then requires the model to produce a response consistent with the analysis. If harmful elements are detected, the model is instructed to refuse in a principled form, for example: “I cannot assist with this request because [specific explanation].” If the request is benign, the model is instructed to answer helpfully while maintaining safety boundaries.

The paper formalizes the overall procedure as

$$D_{SAGE}(P_{usr}) = \textrm{LLM}(I_{da} \oplus I_{dr} \oplus P_{usr}),$$

where $I_{da}$ is the discriminative analysis instruction, $I_{dr}$ is the discriminative response instruction, $P_{usr}$ is the user prompt, and $\oplus$ denotes concatenation (Ding et al., 17 May 2025).

The prompt used in the appendix makes this structure explicit. It contains Semantic Analysis, Request Breakdown / Task Structure Analysis, and a Response Protocol. In the harmful case, the model follows the standardized refusal format; in the benign case, it provides helpful assistance. The design aim is therefore not merely classification, but generation-time behavioral coherence between safety judgment and final output.

3. Training-free character and prompt design

A defining property of Safe-SAGE is that it is training-free. The method does not require finetuning, RLHF, adapters, or additional safety classifiers. It is purely an inference-time prompting strategy designed to exploit the model’s existing discriminative ability rather than retraining the model to be safer (Ding et al., 17 May 2025).

This choice has several methodological consequences. First, the defense is lightweight and portable across models. Second, it avoids the deployment burden associated with additional guard models or model-specific safety heads. Third, the safety intervention is localized to prompt structure rather than parameter updates. In practical terms, Safe-SAGE is a prompt-based orchestration layer over the base model’s own safety cognition.

The paper also evaluates prompt variants created by rephrasing the instructions with GPT-4o and reports similar performance. This is presented as evidence that the method is not brittle to exact wording and does not depend on a carefully tuned hidden prompt artifact. That result matters because prompt-based defenses are often criticized for overfitting to a single template. Here, the reported robustness to prompt rephrasing supports the claim that the key effect comes from the analysis-then-response structure itself.

A common misconception is that Safe-SAGE simply inserts a refusal reminder before the prompt. The description in the paper is more structured than that. The DAM explicitly asks the model to inspect semantic harm and hidden task composition; the DRM then binds generation to the resulting safety judgment. The defense therefore operates as a guided transition from self-awareness of risk to response policy compliance.

4. Empirical performance, robustness, and utility preservation

The paper reports strong empirical performance across both open-source and closed-source LLMs. Across Gemma2-9B-IT, Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, GPT-4o-mini, GPT-4o, and Claude-3.5-Sonnet, SAGE achieves an average 99% defense success rate, with an average harmfulness score near the minimum, and outperforms Self-Reminder, Self-Examination, ICD, Goal Prioritization, and IA (Ding et al., 17 May 2025).

The attack set is broad and includes GCG, AutoDAN, PAIR, ReNeLLM, DeepInception, GPTFuzzer, and CodeAttack. The paper further states that SAGE can reduce ASR on some complex jailbreaks from 100% to 0%. The reported robustness spans open-source models, closed-source models, different sizes and architectures, and multiple jailbreak families. This suggests that the defense is not narrowly tied to one model family or one attack template.

A persistent concern in safety defenses is over-refusal. On GSM8K, MMLU, and Just-Eval, the paper states that SAGE maintains performance close to the undefended model and has negligible performance compromise on general benchmarks. The comparison with baselines is central here: methods such as Self-Examination and Goal Prioritization sometimes hurt MMLU more noticeably, which the paper interprets as evidence of excessive conservatism. Safe-SAGE is presented instead as preserving helpfulness better while still strongly improving defense.

The method also introduces only modest efficiency overhead. The paper reports time-cost-per-sample (TCPS) figures and attributes the efficiency to the fact that SAGE does not require the model to emit a long reasoning trace; it uses the discrimination step to steer generation. In addition, the authors test prompt rephrasings and evaluate the method on deepseek R1, reporting that the method remains strong. This is presented as evidence that the approach generalizes beyond a single prompt wording and interacts effectively with models that naturally exhibit reasoning-like internal behavior (Ding et al., 17 May 2025).

5. Mechanistic interpretation of the safety effect

A distinctive feature of the paper is its attempt to explain why the discrimination-generation gap exists. The authors conduct mechanistic interpretability analysis using both hidden states and attention distributions (Ding et al., 17 May 2025).

For hidden-state analysis, they extract the last-token hidden state from the final layer, reduce it with PCA, and fit a logistic regression boundary separating benign from harmful prompts. Three findings are highlighted. First, hidden states for benign and harmful requests are separable. Second, jailbreak prompts tend to move hidden states toward the benign side. Third, adding a discrimination instruction pulls them back toward the harmful side. This suggests that the same user request can acquire materially different internal representations depending on whether the model is prompted to judge or to generate.

For attention analysis, the paper computes metrics that quantify overlap with harmful tokens and concentration of attention. The reported results show that during discrimination, models focus more on harmful content than during generation, the attention distribution is more concentrated in discrimination, and the overlap with harmful tokens is higher in discrimination. Visualizations reported in the paper show that under discrimination prompts, the model attends more to terms such as “hacking,” “stealing,” and “drugs,” as well as to safety-related tokens such as “cannot” and “assist,” whereas generation mode can shift attention toward more generic task-following words.

The mechanistic interpretation advanced by the paper is therefore that jailbreaks exploit an inconsistency between internal state geometry and attention allocation across task modes. In judging mode, the model’s representations and attention patterns emphasize harmfulness; in ordinary answer-generation mode, the same prompt can be encoded and attended to in a way that weakens the salience of safety signals. Safe-SAGE intervenes by forcing the model into the former mode before the latter.

6. Limitations, misconceptions, and broader significance

The paper explicitly identifies several limitations. Safe-SAGE depends on the model’s intrinsic discrimination ability; if that ability is weak or unstable in certain domains, the defense may be less effective. It may also be sensitive to very novel, highly contextual, or unusual adversarial phrasings, which may require further refinement of discriminative criteria. Although efficient, the method adds minor overhead because of the added analysis stage. The authors also note that the current design does not require explicit reasoning output, and they suggest future work on integrating discrimination and generation more seamlessly without exposing intermediate reasoning. Finally, the evaluation covers specific jailbreak datasets and attack methods, so adaptive future attacks remain an open concern (Ding et al., 17 May 2025).

Several misunderstandings are directly addressed by the paper’s design. Safe-SAGE is not a retraining method, not an auxiliary classifier, and not merely a refusal-style prompt. Its core claim is narrower and more technical: safe generation and harmful prompt detection are separable capabilities, and inference-time prompt structuring can align them. Likewise, the method should not be conflated with the separate SAGE acronym used for "Safety AI Generic Evaluation", which denotes a modular framework for conversational, application-specific LLM safety evaluation rather than a jailbreak defense (Jindal et al., 28 Apr 2025).

In broader significance, Safe-SAGE is best understood as a method for converting a model’s own safety judgment into a generation-time control signal. The paper’s central contribution is not only the empirical result—an average 99% defense success rate across six models—but also the argument that LLM safety failures can stem from a mismatch between what models know during discrimination and what they do during generation. The code and datasets are reported as publicly available, which situates the work as both a defense proposal and an empirical study of mode-dependent safety behavior in contemporary LLMs (Ding et al., 17 May 2025).