Latent Fusion Jailbreak in AI Safety

• Latent Fusion Jailbreak is a latent-manipulation paradigm that blends harmful and safe hidden representations to bypass refusal behavior in language models.
• It employs hidden-state interpolation, gradient-guided layer selection, and token-level influence scoring to fuse benign and malignant signals.
• Empirical studies reveal high attack success rates across modalities, highlighting critical vulnerabilities in current AI safety mechanisms.

Searching arXiv for the cited papers to ground the article and verify metadata. Latent Fusion Jailbreak (LFJ) denotes a class of jailbreak mechanisms in which harmful and harmless, or safety-suppressing and task-preserving, representations are fused so that a model produces non-refusal, unsafe, or policy-violating outputs. In the most explicit usage, LFJ is a representation-space attack that interpolates hidden states from harmful and benign query pairs at influential layers and tokens (Xing et al., 8 Aug 2025). Closely related work uses the same underlying logic in other forms: gradient-based image attacks against early-fusion multimodal models (Rando et al., 2024), residual-stream steering with jailbreak vectors and harmfulness vectors (Ball et al., 2024), disguise-based latent-space attacks and defenses (Yi et al., 18 Jan 2025), text prompts that combine an explicit benign task with an embedded malicious instruction (Qiu et al., 2023), acoustic latent semantic interference in large audio LLMs (Wang et al., 18 May 2026), and defenses that target upstream refusal trajectories rather than terminal refusal vectors (Hu et al., 2 May 2026). Taken together, the sources suggest that LFJ is best understood as a latent-manipulation paradigm rather than a single modality-specific algorithm.

1. Terminology and scope

The literature does not use the term uniformly. One strand treats LFJ as a named hidden-state interpolation attack; another uses the term to describe equivalent latent fusion mechanisms in multimodal, audio, or defense-oriented settings. A plausible implication is that the common denominator is not prompt form, but controlled movement of internal representations toward a region that preserves task coherence while attenuating refusal behavior.

Paper	LFJ usage	Core mechanism
"Latent Fusion Jailbreak: Blending Harmful and Harmless Representations to Elicit Unsafe LLM Outputs" (Xing et al., 8 Aug 2025)	Explicit definition	Hidden State Interpolation between harmful and benign query pairs
"Gradient-based Jailbreak Images for Multimodal Fusion Models" (Rando et al., 2024)	Early-fusion multimodal formulation	Differentiable tokenizer shortcut enabling universal adversarial image optimization
"Understanding Jailbreak Success: A Study of Latent Space Dynamics in LLMs" (Ball et al., 2024)	Equivalent mechanism	Jailbreak vectors and harmfulness vectors in the residual stream
"Latent-space adversarial training with post-aware calibration for defending LLMs against jailbreak attacks" (Yi et al., 18 Jan 2025)	Disguise-based latent-space framing	Safety-critical latent dimensions, MRFA, and PAC
"Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of LLMs" (Qiu et al., 2023)	Closely related text construction	$P \oplus C \oplus I$ latent jailbreak prompts
"Acoustic Interference: A New Paradigm Weaponizing Acoustic Latent Semantic for Universal Jailbreak against Large Audio LLMs" (Wang et al., 18 May 2026)	Audio latent-fusion style jailbreak	Acoustic Latent Semantics and instruction-neutral interference audio
"Tracing the Dynamics of Refusal: Exploiting Latent Refusal Trajectories for Robust Jailbreak Detection" (Hu et al., 2 May 2026)	LFJ as defense threat model	Suppression of terminal refusal while harmful intent persists upstream

In the explicit LFJ formulation, the attack assumes white-box access to hidden activations, gradients of a refusal-oriented loss with respect to hidden states, and the ability to edit or replace activations at inference time, while masking parameters so that no weight updates occur (Xing et al., 8 Aug 2025). In the multimodal image formulation, the attacker is also assumed to have white-box access to the fusion model to compute gradients, because image tokenization is otherwise non-differentiable (Rando et al., 2024). In the residual-stream steering literature, the same family of phenomena is operationalized via measured latent directions rather than direct hidden-state interpolation (Ball et al., 2024).

2. Hidden-state interpolation as the canonical LFJ formulation

The explicit LFJ pipeline begins with query pair selection. For each harmful query $q_h$, a benign query $q_b$ is generated or retrieved, and the pair is accepted when the cosine similarity of mean-pooled BERT embeddings satisfies

$$\mathrm{sim}(q_h, q_b)=\frac{e(q_h)\cdot e(q_b)}{\|e(q_h)\|\|e(q_b)\|}\ge 0.8.$$

When scalable adversarial data generation is required, the benign query is produced by a pre-trained LLM and may be further filtered by dependency overlap; in the defense data construction, dependency tree overlap is explicitly required to be at least $70\%$ (Xing et al., 8 Aug 2025).

Forward passes on $q_h$ and $q_b$ produce hidden states $h_i^{(l)}(q_h)$ and $h_i^{(l)}(q_b)$. LFJ defines a refusal-oriented loss

$\mathcal{L}=-\sum_{w\in W_r}\log P(w\mid q_h),$

then computes token-level influence scores

$q_h$0

and layer scores

$q_h$1

Layers are selected when $q_h$2, and within each selected layer, interpolation is applied to tokens satisfying $q_h$3. The interpolation itself is layer-wise: $q_h$4 After each edit, the model is sequentially forward propagated from the edited layer to downstream layers (Xing et al., 8 Aug 2025).

Optimization is governed by a composite objective

$q_h$5

with

$q_h$6

The reported validation setting uses $q_h$7 and $q_h$8. In practice, $q_h$9 is initialized at $q_b$0 and incremented by $q_b$1 up to $q_b$2 after failed attempts to suppress refusal. The paper reports top-$q_b$3 with refusal-token resampling up to $q_b$4 times in the token-generation section, and top-$q_b$5, temperature $q_b$6, FP16 computation, maximum output length $q_b$7, and interpolation only for the final token’s hidden state in the implementation details (Xing et al., 8 Aug 2025).

This formulation locates jailbreak efficacy in the internal computation rather than in surface-form token perturbation. Thematically similar benign queries preserve fluency, while gradient-guided layer and token selection concentrates edits on representations that most strongly modulate refusal. The paper contrasts this with prompt-based jailbreaks and with representation-space methods that require explicit neuron subspace navigation, arguing that LFJ uses paired-query interpolation rather than overt prompt artifacts or costly subspace search (Xing et al., 8 Aug 2025).

3. Multimodal and cross-modal latent fusion variants

In early-fusion multimodal models, the same principle is implemented at the input-to-latent interface rather than directly in decoder hidden states. For Chameleon, the image tokenization pipeline is

$q_b$8

where $q_b$9 is a VQ-VAE encoder, $$\mathrm{sim}(q_h, q_b)=\frac{e(q_h)\cdot e(q_b)}{\|e(q_h)\|\|e(q_b)\|}\ge 0.8.$$0 assigns each of $$\mathrm{sim}(q_h, q_b)=\frac{e(q_h)\cdot e(q_b)}{\|e(q_h)\|\|e(q_b)\|}\ge 0.8.$$1 latent vectors to a codebook index, $$\mathrm{sim}(q_h, q_b)=\frac{e(q_h)\cdot e(q_b)}{\|e(q_h)\|\|e(q_b)\|}\ge 0.8.$$2 is the token embedding matrix, $$\mathrm{sim}(q_h, q_b)=\frac{e(q_h)\cdot e(q_b)}{\|e(q_h)\|\|e(q_b)\|}\ge 0.8.$$3 is vocabulary size, and $$\mathrm{sim}(q_h, q_b)=\frac{e(q_h)\cdot e(q_b)}{\|e(q_h)\|\|e(q_b)\|}\ge 0.8.$$4 is embedding dimension (Rando et al., 2024). Because $$\mathrm{sim}(q_h, q_b)=\frac{e(q_h)\cdot e(q_b)}{\|e(q_h)\|\|e(q_b)\|}\ge 0.8.$$5 is non-differentiable, LFJ introduces a differentiable tokenizer shortcut $$\mathrm{sim}(q_h, q_b)=\frac{e(q_h)\cdot e(q_b)}{\|e(q_h)\|\|e(q_b)\|}\ge 0.8.$$6 that approximates or bypasses tokenization. Two variants are defined. The embedding shortcut maps each $$\mathrm{sim}(q_h, q_b)=\frac{e(q_h)\cdot e(q_b)}{\|e(q_h)\|\|e(q_b)\|}\ge 0.8.$$7-dimensional latent directly to the decoder embedding space, $$\mathrm{sim}(q_h, q_b)=\frac{e(q_h)\cdot e(q_b)}{\|e(q_h)\|\|e(q_b)\|}\ge 0.8.$$8, with $$\mathrm{sim}(q_h, q_b)=\frac{e(q_h)\cdot e(q_b)}{\|e(q_h)\|\|e(q_b)\|}\ge 0.8.$$9. The 1-hot shortcut maps $70\%$0, forms $70\%$1, and computes $70\%$2. A softmax temperature $70\%$3 is reported as critical to avoid vanishing gradients and enable smooth optimization (Rando et al., 2024).

The attack then optimizes a universal adversarial image $70\%$4 against a prompt set $70\%$5: $70\%$6 The best-performing variant sets $70\%$7 typically to $70\%$8, because maximizing contextual non-refusal prefixes yields the best results and adding refusal suppression provides no benefit and can hurt optimization. Pixel updates follow

$70\%$9

with $q_h$0, halving every $q_h$1 steps to a minimum $q_h$2, for $q_h$3 total steps; no projection to an $q_h$4-ball is used, and no patch constraints or perceptual bounds are enforced (Rando et al., 2024).

A second multimodal generalization appears in large audio LLMs. The Acoustic Interference Attack (AIA) uses instruction-neutral audio containing specific Acoustic Latent Semantics (ALS) to induce “inference path drift” when paired with malicious text queries. The audio content is benign and instruction-neutral; the malicious payload resides only in the text; and the interference set is precomputed and ranked universally rather than optimized per instance (Wang et al., 18 May 2026). The paper frames this as a decoupled attack in which audio functions as a “safety alignment interferer” rather than a payload carrier. Its refusal-logit analysis defines

$q_h$5

and reports that acoustic interference systematically yields $q_h$6, suppressing refusal token probabilities (Wang et al., 18 May 2026). A plausible implication is that LFJ generalizes naturally to any modality that can contribute latent features to a shared safety-relevant computation.

Text-only “latent jailbreak” provides an earlier, prompt-space analogue. In that construction, prompts have the form $q_h$7, where $q_h$8 is an explicit normal instruction, $q_h$9 is an optional cue word, and $q_b$0 is an implicit malicious instruction embedded in the text to be processed. The attack works when the model follows the embedded instruction rather than the explicit task instruction, especially under suffix placement of the explicit instruction (Qiu et al., 2023). This is not hidden-state interpolation, but it is a closely related fusion of benign-task structure and malicious intent.

4. Empirical results and benchmark evidence

Across studies, LFJ-style mechanisms are evaluated with different observables: attack success rate, harmfulness or refusal judgments, robustness of benign task completion, latent cosine similarity, and refusal-logit or refusal-trajectory measures. The combined record indicates that latent fusion can be highly effective, but that its efficacy is sensitive to the quality of the fused representation, the intervention layer, and the modality through which the fusion is introduced.

Paper	Setting	Reported result
"Latent Fusion Jailbreak: Blending Harmful and Harmless Representations to Elicit Unsafe LLM Outputs" (Xing et al., 8 Aug 2025)	Vicuna, LLaMA-2, Guanaco, LLaMA-3, Mistral on AdvBench, MaliciousInstruct, PKU, ToxicChat	Average ASR $q_b$1
"Gradient-based Jailbreak Images for Multimodal Fusion Models" (Rando et al., 2024)	Chameleon multimodal fusion models	Harmful information for $q_b$2 of prompts
"Understanding Jailbreak Success: A Study of Latent Space Dynamics in LLMs" (Ball et al., 2024)	Vicuna 13B v1.5, 24 jailbreak types, steering at layer 20	Most entries drop to $q_b$3 ASR after steering
"Latent-space adversarial training with post-aware calibration for defending LLMs against jailbreak attacks" (Yi et al., 18 Jan 2025)	Llama3-8B-Instruct under GCG, Adaptive, PAIR, HumanJailbreak, GPTFUZZER	GCG $q_b$4, PAIR $q_b$5, Adaptive $q_b$6, OR-Bench $q_b$7
"Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of LLMs" (Qiu et al., 2023)	416 latent jailbreak prompts across three LLMs	ChatGLM2-6B $q_b$8
"Acoustic Interference: A New Paradigm Weaponizing Acoustic Latent Semantic for Universal Jailbreak against Large Audio LLMs" (Wang et al., 18 May 2026)	10 LALMs across JBB, WildJailbreak, HH-RLHF	Qwen2.5-Omni JBB ASR-M $q_b$9
"Tracing the Dynamics of Refusal: Exploiting Latent Refusal Trajectories for Robust Jailbreak Detection" (Hu et al., 2 May 2026)	Qwen2.5-7B, Mistral-7B, Llama-3.1-8B against Prefilling, GCG, AutoDAN	Detection rates improve from $h_i^{(l)}(q_h)$0 to $h_i^{(l)}(q_h)$1 in settings where terminal-state methods fail

Within the explicit LFJ paper, ablations isolate the dependence on pair quality and edit placement. Random pairing reduces average ASR from $h_i^{(l)}(q_h)$2 to $h_i^{(l)}(q_h)$3; uniform interpolation with $h_i^{(l)}(q_h)$4 across all layers yields $h_i^{(l)}(q_h)$5; random layers yield $h_i^{(l)}(q_h)$6; fixed shallow layers yield $h_i^{(l)}(q_h)$7; fixed deep layers yield $h_i^{(l)}(q_h)$8; removing token selection yields $h_i^{(l)}(q_h)$9; removing sequential propagation yields $h_i^{(l)}(q_b)$0; setting a fixed $h_i^{(l)}(q_b)$1 yields $h_i^{(l)}(q_b)$2; and removing the fluency term increases average perplexity from $h_i^{(l)}(q_b)$3 to $h_i^{(l)}(q_b)$4 while reducing ASR to $h_i^{(l)}(q_b)$5 (Xing et al., 8 Aug 2025). These numbers identify thematic similarity, gradient-guided layer/token selection, sequential propagation, and fluency-aware optimization as central components rather than incidental engineering.

The multimodal image study shows a different but related pattern. Universal jailbreak images outperform text jailbreaks optimized with the same objective and require “3x lower compute budget to optimize 50x more input tokens.” When the shortcut is removed at inference, 1-hot-optimized images retain substantial success at approximately $h_i^{(l)}(q_b)$6 ASR, whereas embedding-optimized images collapse to approximately $h_i^{(l)}(q_b)$7 ASR, indicating that the 1-hot shortcut aligns more closely with the original tokenization pipeline (Rando et al., 2024). In the latent-dynamics study, cosine similarities between jailbreak vectors are positive, approximately $h_i^{(l)}(q_b)$8–$h_i^{(l)}(q_b)$9, across most potent and moderately effective types, and harmful prompts without jailbreak have an end-of-instruction harmfulness cosine of approximately $\mathcal{L}=-\sum_{w\in W_r}\log P(w\mid q_h),$0; effective jailbreaks substantially reduce this score, although some successful classes such as wikipedia_with_title and style_injection_json do not do so as strongly, suggesting additional mechanisms beyond straightforward harmfulness suppression (Ball et al., 2024).

The text benchmark contributes a complementary safety–robustness view. It defines jailbreak rate $\mathcal{L}=-\sum_{w\in W_r}\log P(w\mid q_h),$1, robustness rate $\mathcal{L}=-\sum_{w\in W_r}\log P(w\mid q_h),$2, and trustworthiness $\mathcal{L}=-\sum_{w\in W_r}\log P(w\mid q_h),$3, and reports strong position sensitivity: models are safer when the explicit normal instruction is a prefix, whereas suffix placement increases unsafe outputs across all three evaluated models (Qiu et al., 2023). This supports the broader LFJ interpretation that fusion attacks are often effective because the model misallocates control authority between surface instructions and latent task content.

5. Defensive methods and mechanistic countermeasures

One defensive line treats LFJ as a representation-steering problem. In the latent-dynamics study, jailbreak vectors are computed from activation differences

$\mathcal{L}=-\sum_{w\in W_r}\log P(w\mid q_h),$4

while harmfulness vectors are computed from matched harmful–harmless pairs

$\mathcal{L}=-\sum_{w\in W_r}\log P(w\mid q_h),$5

Mitigation is performed by subtracting the normalized jailbreak vector at layer 20 after the end of the instruction: $\mathcal{L}=-\sum_{w\in W_r}\log P(w\mid q_h),$6 The paper reports no induced refusal on harmless questions and no degradation of appropriate refusals on harmful questions without jailbreak, with a minor trade-off of occasional repetitiveness in refusal text (Ball et al., 2024). It also references projection-based removal of a feature direction,

$\mathcal{L}=-\sum_{w\in W_r}\log P(w\mid q_h),$7

as a related defense.

A second line is adversarial hardening of the manipulated subspace. LATPC identifies safety-critical latent dimensions by computing harmful–harmless differences $\mathcal{L}=-\sum_{w\in W_r}\log P(w\mid q_h),$8, then selecting low-variance dimensions via $\mathcal{L}=-\sum_{w\in W_r}\log P(w\mid q_h),$9 and a Top-$q_h$00 mask $q_h$01. It instantiates a mask-based refusal features attack

$q_h$02

and adversarially trains the model to maintain refusal under such corruption (Yi et al., 18 Jan 2025). During inference, post-aware calibration uses

$q_h$03

to detect pseudo-harmful states, then applies a minimal linear correction when $q_h$04. The paper reports nearly $q_h$05 classification accuracy per layer, default Top-$q_h$06, $q_h$07, and for Llama3-8B a reduction of ORR by $q_h$08 points and ASR under Adaptive by $q_h$09 points when PAC is added (Yi et al., 18 Jan 2025).

The explicit LFJ paper proposes an adversarial training defense built directly on interpolated examples. It constructs $q_h$10 interpolated examples and $q_h$11 benign examples, and trains with

$q_h$12

Fine-tuning is performed on LLaMA-2-7B via LoRA on higher-layer attention $q_h$13, with rank $q_h$14, $q_h$15, AdamW learning rate $q_h$16, weight decay $q_h$17, batch size $q_h$18, and $q_h$19 epochs. The reported effect is a reduction in average LFJ ASR from $q_h$20 to $q_h$21, described as “over 80% reduction,” with benign refusal rate $q_h$22 and preserved perplexity on standard benchmarks (Xing et al., 8 Aug 2025).

A third line targets the temporal dynamics of refusal rather than a static refusal vector. SALO is trained on a latent volume $q_h$23 extracted from middle-layer residual streams, processes it with multi-scale $q_h$24D convolutions of sizes $q_h$25, $q_h$26, and $q_h$27, and uses global max-pooling as a sparse selector (Hu et al., 2 May 2026). Its central claim is that refusal is a dynamic, sparse “Refusal Trajectory,” with strongest causal anchors in intermediate layers at tokens immediately following the harmful semantic anchor. Because suffix-based attacks cannot retroactively rewrite upstream states under causal self-attention, terminal-only detectors are structurally vulnerable, whereas trajectory-based detectors remain effective. The paper reports AUROC values of $q_h$28, $q_h$29, and $q_h$30 on XSTest for Qwen, Mistral, and Llama respectively, and a Detection Success Rate of $q_h$31 even under an adaptive white-box GCG attack with evasion weight $q_h$32, while the attack success rate itself collapses to $q_h$33 (Hu et al., 2 May 2026).

In multimodal settings, defenses may transfer across modalities. For Chameleon, representation engineering defenses such as Circuit Breakers, when trained only on text attacks, “can effectively transfer to adversarial image inputs” (Rando et al., 2024). This suggests that at least part of the safety-relevant geometry is shared between prompt-induced and image-induced latent perturbations.

6. Limitations, ambiguities, and open problems

A persistent limitation is access assumptions. The explicit LFJ attack requires white-box access to hidden activations, gradients, and hidden-state editing at inference time (Xing et al., 8 Aug 2025). The gradient-based image attack similarly assumes white-box access to the multimodal fusion model to compute pixel gradients through the tokenizer shortcut (Rando et al., 2024). These assumptions bound direct deployability against closed models, although they do not preclude transfer-based or surrogate-based variants.

A second limitation is model and mechanism specificity. The latent-dynamics study is conducted on Vicuna 13B v1.5; the image attack is evaluated on Chameleon; LATPC is centered on Llama3-8B-Instruct with transfer experiments to Mistral-7B, Gemma-7B, and Qwen2-7B; the text benchmark evaluates ChatGLM2-6B, BELLE-7B-2M, and ChatGPT; and SALO studies Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3, and Llama-3.1-8B-Instruct (Ball et al., 2024). Generality therefore remains partly empirical rather than universal.

Several papers also identify mechanistic diversity. In the latent-dynamics work, style_injection_json and wikipedia_with_title can achieve high ASR without equally strong harmfulness suppression at the trigger point, implying “alternative or additional mechanisms” such as mismatched generalization to new styles or overriding instruction-following objectives despite perceived harmfulness (Ball et al., 2024). In LATPC, extracting too many features, such as above approximately $q_h$34 of dimensions, or using too large $q_h$35, can degrade generation quality and even increase ASR (Yi et al., 18 Jan 2025). In the explicit LFJ paper, random pairing, removal of the fluency term, and omission of sequential propagation all produce large performance drops (Xing et al., 8 Aug 2025). In SALO, if an attacker fully obfuscates malicious semantics so that the model never recognizes harm, the refusal trajectory does not form and the detector cannot detect it (Hu et al., 2 May 2026).

The benchmark and multimodal studies add further gaps. The text latent jailbreak benchmark is limited to three LLMs, translation as the primary task exemplar, and does not report inter-annotator agreement (Qiu et al., 2023). The acoustic interference paper does not report over-the-air, playback, compression, SNR, sampling rate, or environmental robustness experiments, and does not experimentally evaluate defense mechanisms such as audio sanitization or adversarial hardening (Wang et al., 18 May 2026). The image attack paper does not use projection to an $q_h$36-ball, patch constraints, or perceptual bounds, so its optimized images “need not be human-interpretable” (Rando et al., 2024).

A broader interpretive ambiguity remains around the term itself. Some sources reserve LFJ for hidden-state interpolation between harmful and benign queries; others use it as a more general label for latent direction fusion, multimodal interference, or defense threat models. This suggests that LFJ functions less as a single canonical algorithm than as a family resemblance term for attacks that blend internal features so that harmful intent is preserved while refusal features are diluted, displaced, or bypassed. The open research problem is to determine which aspects of this family are genuinely shared across model families and modalities, and which are artifacts of specific architectures, tokenization schemes, or alignment procedures.