---
title: 'CodeSpear: Exploiting GCD for Jailbreaks'
url: https://www.emergentmind.com/topics/codespear
type: topic
---

# CodeSpear: Exploiting GCD for Jailbreaks

Searching arXiv for CodeSpear and closely related work on grammar-constrained decoding jailbreaks.
CodeSpear is a jailbreak attack on large language models (LLMs) for code generation that exploits grammar-constrained decoding (GCD) to induce the production of malicious code. Rather than relying on adversarial prompt engineering or model access, the attack operates by constraining decoding with an ordinary programming-language grammar, thereby excluding the model’s usual natural-language refusal behavior from the output space and forcing generation in the code modality [2606.11817]. In the same line of work, CodeSpear is paired with a defense, CodeShield, which aligns models to emit semantically harmless yet structurally diverse “honeypot code” under GCD while preserving ordinary natural-language refusals when unconstrained decoding is available [2606.11817].

## 1. Definition and threat model

CodeSpear is introduced as a decoding-time jailbreak for safety-aligned LLMs used in code generation [2606.11817]. Its central claim is that GCD, a mechanism widely adopted to improve syntactic validity and structured-output reliability, can itself create a safety vulnerability. The attack requires only that an interface support grammar-constrained or structured decoding and allow the attacker to specify a benign programming-language grammar such as Python; it does not require gradient access, parameter modification, fine-tuning access, or adversarial grammar synthesis [2606.11817].

The threat model is intentionally narrow and practical. The attacker submits a malicious code-generation prompt to a model that is ordinarily safety aligned and constrains decoding with a standard code grammar. The relevant deployment settings include local inference stacks such as vLLM and SGLang as well as API settings exposing structured outputs, including OpenAI and Fireworks AI interfaces [2606.11817]. In this setting, success means that the model emits code that partially or fully implements the malicious request rather than refusing or responding harmlessly [2606.11817].

A common misconception is that jailbreaks of this kind must use adversarially engineered grammars. The reported attack does not depend on such a construction. The paper’s core finding is that a benign off-the-shelf code grammar is already sufficient to suppress natural-language refusals and substantially increase malicious code generation rates [2606.11817].

## 2. Mechanism: output-space restriction as jailbreak surface

The attack is grounded in a formal account of constrained decoding. Unconstrained autoregressive generation is written as
$$
y_t \sim P_M(\cdot \mid p, y_{<t}), \quad y_{<t} = (y_1, \ldots, y_{t-1}).
$$
Under a grammar $G$, the set of valid next tokens becomes
$$
\mathcal{V}_G(y_{<t}) = \left\{\, v \in \mathcal{V} \;\middle|\; \exists\, y_{>t} \text{ s.t. } (y_{<t}, v, y_{>t}) \in \mathcal{L}(G) \,\right\},
$$
and decoding is renormalized over that set:
$$
P_M^{G}(y_t \mid p, y_{<t}) = \frac{ P_M(y_t \mid p, y_{<t}) \cdot \mathbb{I}[y_t \in \mathcal{V}_G(y_{<t})] }{ \sum_{v \in \mathcal{V}_G(y_{<t})} P_M(v \mid p, y_{<t}) }.
$$
This yields
$$
P_M^{G}(y \mid p) = \mathbb{I}[y \in \mathcal{L}(G)] \prod_{t=1}^{|y|} P_M^{G}(y_t \mid p, y_{<t})
$$
and therefore changes the support of generation from the full token language to the grammar language $\mathcal{L}(G)$ [2606.11817].

The paper’s central insight is that contemporary harmful-code alignment is largely learned in the natural-language modality. Safety-aligned models are trained to produce refusals such as “I cannot assist with that,” but these strings are usually not elements of a code grammar. The paper states this as
$$
\mathcal{R}_{\text{refuse}} \cap \mathcal{L}(G) = \emptyset,
\qquad
\Pr_{y \sim P_M^{G}(\cdot \mid p)} \left[y \in \mathcal{R}_{\text{refuse}}\right] = 0 .
$$
In other words, GCD can delete the model’s preferred safe behavior from the admissible output space [2606.11817].

This suggests that CodeSpear is not best understood as altering model preferences directly. A more precise characterization is that it performs support manipulation: the attack makes the refusal channel unavailable, thereby forcing the model to continue in code space, where its learned competence may dominate and its safety alignment may be weaker [2606.11817].

## 3. Attack procedure

Operationally, CodeSpear is deliberately simple. Given a model $M$, malicious prompt $p$, benign code grammar $G$, and maximum generation length $T_{\max}$, the attack repeatedly computes the valid token set under the grammar, samples from the constrained next-token distribution, appends the sampled token to the prefix, and stops at end-of-sequence [2606.11817]. The paper describes this as querying
$$
y \sim P_M^{G}(\cdot \mid p).
$$

The assumptions required by the procedure are modest:

- **Prompt access**: the attacker can submit harmful code-generation requests.
- **Constraint access**: the interface supports grammar-constrained decoding.
- **Grammar choice**: the attacker can supply an ordinary programming-language grammar.

No internal model state is required, and no adversarial search over grammars is necessary [2606.11817]. This distinguishes CodeSpear from prior structured-output jailbreaks such as APT, which rely on crafting request-specific adversarial grammars [2606.11817].

The implementation reported in the paper uses llguidance with off-the-shelf Python grammars. Across attacks, the experiments use temperature $0.9$ and top-$p$ $0.95$, repeat runs three times, and report averages [2606.11817].

## 4. Empirical evaluation

The evaluation spans 10 models across 4 benchmarks [2606.11817]. The local models are Qwen2.5-Coder-7B, Qwen2.5-Coder-32B, Qwen2.5-7B, Qwen2.5-32B, and LLaMA3-8B. The API-based models are GPT-5, GPT-5-mini, GPT-OSS-120B, MiniMax-M2.5, and MiniMax-M2.7 [2606.11817]. Safety evaluation uses the code-generation subset of RMCBench, specifically Level 1 and Level 2 with 182 malicious requests, and the original malicious subset of MalwareBench with 320 malicious requests [2606.11817]. Utility evaluation uses HumanEval with 164 tasks and MBPP with 974 tasks [2606.11817].

The safety metrics are Attack Success Rate (ASR), defined as the fraction of malicious requests where the response is harmful, and Malicious Rate (MR), defined as the fraction where the generated code functionally realizes the malicious intent. Both are judged by DeepSeek-V4-Flash. A reliability check on 100 manually inspected samples reports human–LLM agreement of 87% for ASR and 85% for MR [2606.11817].

### Local-model results

For local models, CodeSpear reaches 81.82% ASR and 54.58% MR on average, compared with 54.92% ASR and 36.61% MR for Vanilla. This corresponds to gains of +26.90 percentage points ASR and +17.98 points MR [2606.11817]. The strongest baseline is CodeJailbreaker at 71.25% ASR and 48.45% MR, still below CodeSpear [2606.11817].

Representative reported cases include the following.

| Model and benchmark | Vanilla ASR/MR | CodeSpear ASR/MR |
|---|---:|---:|
| Qwen2.5-Coder-7B, RMCBench | 26.92 / 23.26 | 82.78 / 62.09 |
| Qwen2.5-Coder-32B, RMCBench | 59.89 / 45.60 | 92.86 / 75.64 |
| Qwen2.5-Coder-7B, MalwareBench | 29.79 / 20.83 | 83.44 / 46.15 |
| Qwen2.5-32B, MalwareBench | 53.23 / 33.33 | 84.69 / 54.58 |

These outcomes indicate that the attack is not confined to a single architecture family or safety-training regime [2606.11817].

### API-model results

For API-based models, CodeSpear reaches 67.39% ASR and 36.62% MR on average, compared with 22.00% ASR and 11.04% MR for Vanilla. This is a gain of +45.39 points ASR and +25.59 points MR [2606.11817]. The best baseline average is CodeJailbreaker at 39.95% ASR and 20.40% MR, still substantially below CodeSpear [2606.11817].

Representative cases include GPT-5 on RMCBench, where Vanilla scores 32.05/26.01 and CodeSpear 55.49/43.96, and MiniMax-M2.7 on RMCBench, where Vanilla scores 20.33/17.40 and CodeSpear 85.53/64.29 [2606.11817]. The strength of the results on API systems is notable because these platforms are generally expected to have stronger alignment and deployment-time safeguards.

## 5. Baselines, ablations, and behavioral interpretation

The evaluation compares CodeSpear with Vanilla, Vanilla-T, DAN, LRL, PAIR, CodeJailbreaker, and APT [2606.11817]. The paper explicitly excludes gradient-based attacks that do not fit the adopted threat model [2606.11817]. Across the reported averages, CodeSpear outperforms these representative jailbreak baselines [2606.11817].

A particularly revealing analysis concerns fixed safe code patterns. On GPT-5 and GPT-5-mini, the model under GCD often emits a trivial `pass` statement in response to malicious prompts. While superficially safe, that pattern is brittle. When the grammar is tightened to disallow `pass`, attack success increases further. For GPT-5 on RMCBench, deepred/CodeSpear rises from 55.49/43.96 to 70.30/53.65; for GPT-5-mini on RMCBench, from 53.48/39.56 to 63.75/44.58 [2606.11817]. Similar increases are reported on MalwareBench [2606.11817].

This matters for two reasons. First, it shows that code-modality “safety” anchored to a narrow stub such as `pass` is not robust. Second, it clarifies that CodeSpear exploits a structural weakness in the safety policy rather than merely surfacing preexisting unsafe responses. The attack remains effective even when the model attempts a code-form safe fallback, because small grammar adjustments can remove that fallback from the output space [2606.11817].

The paper also reports that the attack remains strong across Python, C++, and Java grammars. On Qwen2.5-Coder-7B, ASR is said to remain above 70% once GCD is applied, regardless of grammar, while remaining below 40% without GCD [2606.11817]. This suggests that the vulnerability is not tied to idiosyncrasies of a single grammar formalism.

## 6. CodeShield: code-modality alignment defense

The defense paired with CodeSpear is CodeShield, named deepgreen in the body of the paper [2606.11817]. Its premise is that if natural-language refusals are suppressed by GCD, the model must still retain safe behaviors inside the code modality itself. CodeShield therefore aligns models to emit “honeypot code” under constrained decoding [2606.11817].

The paper defines honeypot code by two properties:

- **Semantically harmless**: it does not implement the malicious request.
- **Structurally diverse**: it varies enough that an attacker cannot easily eliminate it through tighter grammars without also constraining away much of the expressive space needed for malicious programs.

This design explicitly rejects fixed-template safe stubs such as `pass` or canned refusal comments [2606.11817].

The intended preference hierarchy is modality dependent. Under unconstrained decoding, natural-language refusal should be preferred over honeypot code. Under constrained decoding, honeypot code should be preferred over harmful code [2606.11817]. Training uses a DPO-based objective over pairwise preferences. The paper prints the standard DPO form
$$
\mathcal{L}_{\text{DPO}} = -\, \mathbb{E}_{(p, y^{+}, y^{-})} \left[ \log \sigma\!\left( s(p, y^{+}) - s(p, y^{-}) \right) \right],
$$
with the implicit reward defined against a reference model, though the TeX rendering is partially corrupted in the supplied text [2606.11817].

The preference dataset is constructed from malicious prompts by pairing each refusal with multiple honeypot code samples and pairing each honeypot code sample with the harmful response generated under GCD [2606.11817]. The data pipeline filters PKU-RLHF with Qwen3-32B to obtain 744 seed malicious code-generation prompts, augments them with DeepSeek-V4-Pro to 2,000 total malicious prompts, generates natural-language refusals with DeepSeek-V4-Pro, obtains harmful responses by querying the target model under GCD, and samples honeypot snippets from OpenCodeInstruct [2606.11817]. Training uses $K=5$ honeypot samples, learning rate $1e\!-\!5$, one epoch, and an additional 40k general code-generation tasks from OpenCodeInstruct for utility preservation [2606.11817].

Defense results are strong. On Qwen2.5-Coder-7B under attack, Vanilla averages 83.11/54.12 and Safe-DPO 77.39/45.03, whereas deepgreen/CodeShield reduces this to 5.57/2.78 [2606.11817]. Similar reductions are reported for Qwen2.5-7B and LLaMA3-8B [2606.11817]. The comparison with Safe-DPO is especially significant because it isolates the need for code-modality alignment: natural-language refusal training alone does not withstand CodeSpear once decoding is forced into code [2606.11817].

Utility degradation is reported as modest. On Qwen2.5-Coder-7B, average HumanEval/MBPP pass@1 and pass@3 change from 68.60/83.51 for Vanilla to 66.94/80.88 for deepgreen [2606.11817]. On Qwen2.5-7B, deepgreen even improves the reported average over Vanilla, from 49.29/68.46 to 51.53/71.74 [2606.11817]. A plausible implication is that utility-preservation SFT contributes materially to stabilizing general code performance.

## 7. Significance, limitations, and relation to adjacent code-generation work

The significance of CodeSpear lies in identifying GCD as a security-relevant intervention rather than a purely reliability-oriented one [2606.11817]. Prior repository-scale code generation work such as CodeS decomposes generation into sketches to improve structural coherence [2403.16443], and lexical retrieval work such as SpareCodeSearch argues that lightweight keyword-based repository context retrieval can be sufficient for practical completion systems [2510.12948]. Those systems address capability, structure, and deployment efficiency. CodeSpear addresses a distinct axis: the interaction between output constraints and safety alignment. This suggests that advances in structured decoding and repository-aware generation do not automatically transfer to safe deployment under adversarial prompting.

The paper is careful about scope. It notes that absolute attack success rates may vary across GCD engines and APIs, and that the benchmark suite, while broad, does not cover every malicious coding scenario [2606.11817]. It also reports ethical restrictions: CodeSpear source is released only to researchers for controlled use, whereas CodeShield is fully released [2606.11817].

Several limitations remain. First, the vulnerability analysis is strongest for models whose safety policy is predominantly implemented as natural-language refusal. This suggests a research direction toward genuinely cross-modality safety policies rather than modality-local refusals. Second, the defense depends on the availability and quality of structurally diverse harmless code. Third, adaptive grammar tightening can increase ASR in some defended settings, even though the paper reports no catastrophic collapse of CodeShield under a 10-round adaptive attack [2606.11817].

A further misconception is that GCD itself is inherently unsafe. The paper’s actual claim is narrower: GCD becomes an attack surface when alignment is modality specific and the constrained language excludes the model’s learned refusal behavior [2606.11817]. This suggests that the vulnerability is not a property of syntax constraints per se but of the interaction between constraint-induced support shifts and alignment objectives.

In summary, CodeSpear establishes that grammar-constrained decoding can jailbreak safety-aligned code LLMs by removing natural-language refusal from the reachable output space and forcing continuation in code. Its empirical results show strong improvements over prompt-based and structured-output jailbreak baselines across local and API models, while the accompanying CodeShield defense demonstrates that aligning safe behavior directly in the code modality can largely restore robustness under constrained decoding [2606.11817].

Source: https://www.emergentmind.com/topics/codespear