Super Suffixes: Bypassing Text Generation Alignment and Guard Models Simultaneously (2512.11783v1)

Abstract: The rapid deployment of LLMs has created an urgent need for enhanced security and privacy measures in Machine Learning (ML). LLMs are increasingly being used to process untrusted text inputs and even generate executable code, often while having access to sensitive system controls. To address these security concerns, several companies have introduced guard models, which are smaller, specialized models designed to protect text generation models from adversarial or malicious inputs. In this work, we advance the study of adversarial inputs by introducing Super Suffixes, suffixes capable of overriding multiple alignment objectives across various models with different tokenization schemes. We demonstrate their effectiveness, along with our joint optimization technique, by successfully bypassing the protection mechanisms of Llama Prompt Guard 2 on five different text generation models for malicious text and code generation. To the best of our knowledge, this is the first work to reveal that Llama Prompt Guard 2 can be compromised through joint optimization. Additionally, by analyzing the changing similarity of a model's internal state to specific concept directions during token sequence processing, we propose an effective and lightweight method to detect Super Suffix attacks. We show that the cosine similarity between the residual stream and certain concept directions serves as a distinctive fingerprint of model intent. Our proposed countermeasure, DeltaGuard, significantly improves the detection of malicious prompts generated through Super Suffixes. It increases the non-benign classification rate to nearly 100%, making DeltaGuard a valuable addition to the guard model stack and enhancing robustness against adversarial prompt attacks.

• The paper demonstrates that composite adversarial suffixes can simultaneously break text generation alignment and evade guard models using alternating joint optimization techniques.
• It employs a two-part design, combining a primary suffix to induce misaligned outputs and a secondary suffix to maximize benign scores from guard models.
• The study reveals significant vulnerabilities in current AI safety measures and proposes DeltaGuard as a dynamic defense based on time-series analysis of model internals.

Super Suffixes: Simultaneous Evasion of Text Generation Alignment and Guard Models

Introduction and Motivation

The proliferation of LLMs for generative tasks in production environments has necessitated robust alignment mechanisms to prevent models from aiding in harmful activities such as code generation of malware or dissemination of misinformation. Model alignment, typically achieved via RLHF protocols, enforces normative refusals to unsafe input prompts. Major vendors now deploy secondary guard models—lightweight classifiers operating in parallel with the text generator—to further filter adversarial inputs and supplement alignment. Yet, recent advances in adversarial suffix construction have persistently undermined standalone alignment in text generation models. This work introduces and systematizes Super Suffixes, composite suffix payloads which simultaneously break the alignment of text generation models and evade detection by state-of-the-art guard models. The research demonstrates, for the first time, joint optimization capable of bypassing both objectives, including the widely used Llama Prompt Guard 2.

Threat Model and Terminology

The threat model considers an adversary who can craft arbitrary text inputs to an LLM stack, consisting of a primary text generation model and a secondary guard model. The attacker’s goal is to elicit a misaligned, typically unsafe output while maximizing the probability that the guard model classifies the input as benign and allows completion.

A Super Suffix comprises two optimized components:

• Primary Suffix: Breaks generation alignment, inducing misaligned outputs in the text model.
• Secondary Suffix: Appended to amplify the benign score from the guard model, bypassing its detection heuristics.

The full Super Suffix is the concatenation of both, jointly optimized via an alternating loss minimization strategy to navigate incompatibilities in tokenization and internal objectives.

Figure 1: A Super Suffix combining primary and secondary payloads, yielding a model output flagged as benign by Prompt Guard (over 99\% probability).

Joint Optimization Algorithm for Super Suffixes

Crafting suffixes capable of evading two distinct alignment objectives across disparate tokenization spaces is a non-trivial bi-objective problem. The joint optimization alternates between minimizing the generator’s loss (via production of a target unsafe output) and maximizing the benign probability of the guard classifier. Unlike GCG and IRIS—which target one direction or loss—this work proposes an alternating approach: in each iteration, either the generator or guard loss is targeted depending on current benign confidence and optimization schedule.

Figure 2: Loss trajectories during joint optimization of Super Suffixes for Google Gemma 2B and Prompt Guard 2 86M show oscillations tied to alternation between objectives, converging to high benign scores and misaligned output.

Key insights include:

• Guard models typically converge faster due to smaller capacity, but naive secondary suffix addition often harms generator misalignment.
• Alternating the focus enables recovering generator misalignment after maximizing the benign guard score.
• The procedure is compatible with heterogeneous tokenization and model architectures, as demonstrated on five generator-guard pairs.

Mechanistic Analysis: Concept Directions and Dynamic Tracing

Construction of effective adversarial suffixes leverages mechanistic interpretability of model internals, particularly the Linear Representation Hypothesis (LRH). LRH posits that abstract concepts such as refusal or acceptance are encoded as directions in hidden representations. This work extends LRH, demonstrating through a specifically curated malicious code generation dataset that domain-specific refusal can be accurately characterized, yielding vectors optimized for targeted jailbreaks.

Figure 3: Cosine similarity heatmap for Llama3.2 3B; abstract behaviors are encoded in late layers where refusal and malicious directions are most orthogonal.

Temporal dynamics of these concept directions—traced via cosine similarity over token sequences—reveal that adversarial attacks and benign completions exhibit distinguishable trajectories in the residual stream. Dynamic tracing enables robust, low-cost detection of composite adversarial suffixes.

Figure 4: Cosine similarity traces of concept directions for Gemma 2B, exposing temporal differentiation between benign, primary suffix, and Super Suffix prompts.

DeltaGuard: Dynamic Countermeasures via Cosine Similarity Trajectories

Recognizing the limitations of guard models against Super Suffixes, the authors introduce DeltaGuard, a dynamic defense mechanism based on time-series classification of similarity traces to refusal directions. DeltaGuard leverages non-parametric classifiers (KNN) over these trajectories, trained to distinguish benign, refused, primary suffix, and Super Suffix inputs.

• DeltaGuard attains nearly perfect non-benign classification under multiple attack classes.
• It is capable of differentiating primary versus Super Suffix payloads through trajectory clustering.

  Figure 5: DeltaGuard confusion matrix for Gemma 2B, showing high sensitivity in discriminating suffix types.

The PCA and t-SNE projections further display separability of time-series features in latent space, supporting the efficacy of dynamic tracing for adversarial activity and providing cross-model generalization.

Figure 6: PCA projection visualizing separability of classification features for different prompt types.

Figure 7: Aggregate cosine similarity trajectories over time for various input classes, validating shape-based discriminative power.

Empirical Results and Observations

The proposed Super Suffix pipeline was evaluated over multiple generator-guard model pairs using both HarmBench and custom malicious code datasets. Notable empirical results:

• Refusal rates for malicious prompts dropped by up to 84% after primary suffix inclusion, and Super Suffix attacks restored high benign probabilities (>90%) while maintaining misaligned output.
• Jointly optimized Super Suffixes achieved successful jailbreaks in models where naive attacks failed.
• DeltaGuard consistently outperformed static guard classification, reducing false benign rates to below 7% for successful Super Suffix attacks.

Benign probability sweeps for Super Suffixes illustrate how the combined approach subverts existing Prompt Guard scores.

Figure 8: Benign probabilities for Super Suffixes which bypass both Prompt Guard and text generation alignment.

Optimization steps and runtime profiles demonstrate the practical feasibility of Super Suffix generation on commodity infrastructure, with wall time and resource expenditures manageable for attackers operating at scale.

Figure 9: Optimization timeline for constructing a Super Suffix targeting Gemma 2B for keylogger code generation.

Implications, Limitations, and Future Directions

The efficacy of Super Suffixes in universally bypassing both alignment and guard models exposes significant vulnerabilities in prevailing AI safety architectures. Static guard models and output-only refusal tracing are insufficient; attackers can craft composite suffixes that exploit both alignment and moderation objectives in tandem.

• Deployment of interpretability-driven defenses such as DeltaGuard offers a pathway for more robust detection of dynamic bypasses, though joint optimization approaches may be co-opted to evade such defenses in the future.
• The suffix generation paradigm extends to possible simultaneous multi-modal and multi-task jailbreaks, including combinations of text/image generation and moderation.
• The tradeoff between general alignment and domain-specific sensitivity is highlighted; domain-adapted concept directions significantly improve adversarial and defense performance.

Clustering of similarity traces confirms the existence of higher-order intent signatures in model internals, suggesting that future work can exploit mechanistic signals for both attack and defense.

Figure 10: t-SNE analysis on Gemma 2B demonstrates separable groupings of prompt classes using dynamic concept direction trajectories.

Conclusion

This work provides a systematic method for constructing adversarial suffixes—the Super Suffix attack—that jointly defeat text generation alignment and production guard models. By leveraging concept direction analysis and alternating joint optimization, these attacks reliably achieve misaligned, harmful outputs undetected by current model-based defenses. Dynamic tracing of internal representation evolution via DeltaGuard offers an effective supplementary countermeasure, although adversaries may develop similar joint optimization techniques to bypass it. These results have major implications on the future of AI safety, urging the development of dynamic, interpretability-based, and multi-objective defense strategies for deployed LLMs.

Reference: "Super Suffixes: Bypassing Text Generation Alignment and Guard Models Simultaneously" (2512.11783)