SlotGCG: Exploiting the Positional Vulnerability in LLMs for Jailbreak Attacks

Abstract: As LLMs are widely deployed, identifying their vulnerability through jailbreak attacks becomes increasingly critical. Optimization-based attacks like Greedy Coordinate Gradient (GCG) have focused on inserting adversarial tokens to the end of prompts. However, GCG restricts adversarial tokens to a fixed insertion point (typically the prompt suffix), leaving the effect of inserting tokens at other positions unexplored. In this paper, we empirically investigate \emph{slots}, i.e., candidate positions within a prompt where tokens can be inserted. We find that vulnerability to jailbreaking is highly related to the selection of the \emph{slots}. Based on these findings, we introduce the \textit{Vulnerable Slot Score} (VSS) to quantify the positional vulnerability to jailbreaking. We then propose SlotGCG, which evaluates all slots with VSS, selects the most vulnerable slots for insertion, and runs a targeted optimization attack at those slots. Our approach provides a position-search mechanism that is attack-agnostic and can be plugged into any optimization-based attack, adding only 200ms of preprocessing time. Experiments across multiple models demonstrate that SlotGCG significantly outperforms existing methods. Specifically, it achieves 14\% higher Attack Success Rates (ASR) over GCG-based attacks, converges faster, and shows superior robustness against defense methods with 42\% higher ASR than baseline approaches. Our implementation is available at \href{https://github.com/youai058/SlotGCG}{https://github.com/youai058/SlotGCG}

• The paper introduces SlotGCG, demonstrating that non-suffix token positions in LLMs exhibit significant vulnerability to adversarial perturbations.
• It presents a novel Vulnerable Slot Score (VSS) metric to quantitatively assess slot sensitivity, enhancing attack efficiency and success rates.
• SlotGCG accelerates convergence and bypasses state-of-the-art defenses by distributing adversarial tokens across optimized positions.

SlotGCG: Exploiting the Positional Vulnerability in LLMs for Jailbreak Attacks

Introduction

The positional vulnerability of LLMs to adversarial perturbations presents an under-investigated axis in LLM security. The prevalent Greedy Coordinate Gradient (GCG) attack paradigm restricts adversarial token insertion to the suffix, based on the assumption that the end-of-prompt is universally most susceptible. This work rigorously contests that convention by empirically studying the spatial sensitivity of token injection and proposes SlotGCG, an attack framework that systematizes the identification and optimization of vulnerable insertion positions across entire prompts, guided by a newly introduced Vulnerable Slot Score (VSS). Experiments demonstrate that leveraging slot vulnerability yields substantial gains in attack success rate (ASR), convergence speed, and robustness to input filtering defenses across a range of LLMs.

Analysis of Positional Vulnerability in Adversarial Prompts

Two primary exploratory studies—exhaustive slot scanning and random multi-position insertion—reveal that vulnerability to adversarial tokens is not maximized at the suffix for many prompts. The slot yielding minimal adversarial loss varies considerably between prompts and rarely aligns with the prompt end. Further, token positions exhibiting higher model attention demonstrate higher vulnerability, as characterized by lower achievable adversarial loss following targeted optimization. Adversarial token attention patterns are persistent and strongly prompt-dependent, showing little drift across GCG optimization steps.

Figure 1: Pilot experiments delineate how slot position impacts adversarial effectiveness; both exhaustive scans and randomized distributions illustrate clear positional dependencies.

Figure 2: Adversarial loss landscapes across normalized insertion slots for select prompts highlight that minimum loss is rarely achieved at the suffix.

Figure 3: VSS distributions for successful and failed attacks (right) underscore that higher VSS leads to more effective jailbreaking.

Comprehensive empirical auditing confirms that multiple, non-suffix slots can be simultaneously vulnerable to adversarial payloads, facilitating attacks that reduce both the number of required optimization steps and the resulting loss.

The Vulnerable Slot Score (VSS): Quantifying Slot Sensitivity

To operationalize slot-wise vulnerability, the authors formalize the Vulnerable Slot Score. For a probing prompt (i.e., prompt with a unique probe token in every slot), VSS for slot $s$ is computed as a weighted sum of attention scores—specifically, attention from after-chat template positions (token set $\mathcal{C}$) to adversarial tokens at $s$, across upper transformer layers $\mathcal{L}_{UH}$ and all heads. The VSS metric robustly predicts which insertion slots are most susceptible to adversarial modifications, and its inter-slot distribution persists over optimization runs.

Figure 4: VSS and adversarial loss across insertion slots for ten AdvBench prompts—high-VSS slots accurately forecast low-loss regions.

SlotGCG: Attack Algorithm and Pipeline

Building on these findings, SlotGCG introduces a four-stage attack workflow:

1. Slot Probing: Insert explicit probe tokens into all candidate slots of the prompt.
2. VSS Computation: Measure attention-based VSS at each slot; derive a slot vulnerability probability distribution via softmax over VSS.
3. Token Allocation: Allocate adversarial tokens proportionally to the VSS-based probabilities, applying right-to-left position semantics to preserve slot indices.
4. Coordinate Gradient Optimization: Optimize tokens across the allocated slots using standard GCG or its variants.

   Figure 5: The SlotGCG framework: from slot probing to VSS-driven allocation and coordinated token optimization.

SlotGCG is attack-agnostic and introduces a negligible preprocessing latency (~200 ms). It generalizes directly to alternative optimization-based attacks, and seamlessly integrates into existing pipelines.

Results: Superiority Over Traditional Suffix-Based Attacks

Extensive experimentation across Llama-2, Llama-3, Mistral, Vicuna, and Qwen demonstrates that SlotGCG achieves higher ASR than classic suffix-based approaches. On robust models such as Llama-2-13B, SlotGCG applied to I-GCG yields a 94% ASR, with +38 percentage point improvement. The gains are systematic, with average ASR increase of 14% across attack-model pairs. For models where baseline defense performance is already strong, SlotGCG exposes new input vulnerabilities otherwise missed by suffix-restricted methods.

Figure 6: Contrasting GCG (suffix-only) and SlotGCG (arbitrary-slot) strategies; the latter unlocks attacks across dispersed positions, sidestepping narrow suffix defense pipelines.

Figure 3: Attention heatmaps and VSS statistics reveal more evenly exploited vulnerability across tokens for SlotGCG than GCG, confirming broader adversarial influence.

SlotGCG accelerates convergence, reducing optimization steps required for jailbreak by 2×–10× depending on model and baseline. This improved efficiency is attributable to better initial placement of adversarial tokens per slot-specific vulnerabilities.

Robustness Against State-of-the-Art Defenses

Application of SlotGCG to prominent GCG-variants under a battery of defenses (Erase-and-Check, Perplexity Filter, SmoothLLM, RPO, SafeDecoding, Llama-Guard-3) confirms that the attack sustains high ASR, even where all baseline attacks fail. With Erase-and-Check in suffix mode, baseline GCG achieves 0% ASR, but GCG + SlotGCG nets 52%. For defense mechanisms using random perturbations (SmoothLLM), SlotGCG consistently achieves up to 96% ASR (+42% relative).

Dispersed allocation—by breaking up suffix-centric adversarial patches—improves robustness to slot-based filtering, token removal, and permutation-based defenses, as adversarial functionality cannot be neutralized by single-point modifications.

Universal SlotGCG: Extending to Transferable Multi-Behavior Attacks

SlotGCG extends with universal optimization for high-transfer adversarial sequences. Aggregation of VSS distributions across multi-behavior datasets produces a universal vulnerability profile, allowing for a global token allocation template that is mapped proportional to prompt length per behavior. Training SlotGCG adversarials on 50 behaviors and evaluating zero-shot on a transfer set of 388 harmful prompts yields strong cross-prompt and cross-model transfer, including on closed LLMs such as GPT-3.5 and Google Gemini.

Figure 7: Universal AggregationSlots algorithm computes global VSS distributions for transferable adversarial token insertions.

Figure 8: AttackInput algorithm bridges universal slot indices to behavior-specific slot sets for scalable transfer.

Ablations and Analysis

Temperature Sensitivity

Temperature parameterization in the VSS softmax trades off allocation sharpness and generalization. Empirically, $T=8$ achieves optimal transferability/robustness balance, with higher ASR in the presence of defenses.

Figure 9: ASR as a function of VSS softmax temperature $T$; performance dips at both extremes, supporting moderate smoothing.

Output Distribution Impact

SlotGCG's VSS-based allocation induces larger perturbations to the LLM’s output distribution—measured by KL-divergence and L2 metrics—than suffix-only approaches, substantiating that slot-sensitive payloads alter crucial context more effectively and reduce reliance on any single positional exploit.

Practical and Theoretical Implications

The findings demonstrate that LLM positional sensitivity is much richer than previously acknowledged, with non-suffix slots presenting powerful (but model- and prompt-specific) attack vectors. SlotGCG both enables stronger red-teaming and offers theoretical guidance for model architecture, defense design (by suggesting robustification of all positions, not just suffixes), and robust optimization of generative models under adversarial constraints.

Mechanistically, the stability of VSS across optimization episodes and its attention-based definition connect LLM jailbreaking vulnerability to information routing and context gating within Transformer decoder stacks—implicating attention masking, template hardening, and distributed defenses rather than local suffix cleaning.

Future extensions of SlotGCG may encompass prompt and context-sensitive dynamic adversarial token generators, black-box approximations leveraging surrogate attention models, and adversarial training that regularizes positional vulnerabilities.

Conclusion

This work shows that positional vulnerability is a critical, previously underexploited axis for adversarial attacks against LLMs. The SlotGCG algorithm achieves enhanced attack success, improved convergence, and superior robustness over suffix-constrained attacks by leveraging attention-based slot vulnerability quantification. SlotGCG's slot-agnostic, attack-agnostic framework exposes persistent security limitations, necessitating holistic defense systems that account for positionally diverse exploits. The formalization of VSS, empirical dissecting of prompt-level vulnerabilities, and universal transferability analysis together establish SlotGCG as a comprehensive methodology for both future red-teaming evaluation and defensive research in LLMs.