Defending Jailbreak Attacks on Large Language Models via Manifold Trajectory Kinetics

Abstract: Jailbreak prompts can bypass alignment guardrails in LLMs and elicit unsafe outputs, making reliable deployment-time detection critical. Prior detection approaches largely rely on a fixed metric space, e.g., raw inputs, gradients, or hidden features, in which benign and jailbreak prompts are linearly separable. We show this assumption breaks under (i) pseudo-malicious prompts that are benign by intent but contain safety-related keywords, and (ii) adaptive attacks that explicitly optimize against the deployed detector. To overcome this limitation, we shift our focus from identifying a universal metric space to analyzing the more robust neighborhood structure of the underlying data manifold. We present Manifold Trajectory Kinetics (MTK), which treats an LLM as a kinetic system transforming inputs into outputs and detects jailbreaks by tracking how a prompt's neighborhood structure evolves across layers. Benign prompts remain close to benign neighborhoods throughout inference, whereas jailbreak prompts exhibit a characteristic trajectory that begins near malicious seeds and later strategically shifts toward benign neighborhoods to evade refusal.Across four LLMs and ten jailbreak attacks, MTK achieves strong robustness to both failure modes: on pseudo-malicious prompts, it attains a jailbreak true positive rate of 95% at a false positive rate of 5% on benign prompts and 2% on pseudo-malicious prompts, and under adaptive attacks, it maintains a true positive rate of 85%. We further demonstrate the superior performance of MTK for jailbreak detection in vision-LLMs. Our code is available at https://github.com/Rookie143/mtk.

• The paper introduces Manifold Trajectory Kinetics (Mtk) to dynamically analyze layerwise activations for detecting LLM jailbreak attacks.
• Mtk generates rank trajectories using benign and malicious anchors, achieving high AUROC and low false positive rates even against adaptive and pseudo-malicious prompts.
• The study demonstrates that dynamic manifold analysis outperforms static detection methods across both LLMs and Vision-Language Models while reducing user friction.

Defending Jailbreak Attacks on LLMs via Manifold Trajectory Kinetics

Introduction and Motivation

This work addresses the critical problem of deployment-time detection of jailbreak attacks on LLMs. Jailbreak attacks are input prompts engineered to bypass alignment guardrails and elicit noncompliant, unsafe outputs from models that nominally follow safety protocols. The authors argue that prior defenses—primarily based on fixed metric spaces such as raw prompt features, gradients, or single-layer representations—fail in two important scenarios: (1) pseudo-malicious prompts (PMPs), which merely resemble malicious content by surface form but are benign in intent, leading to over-refusal and high false positive rates (FPR); and (2) adaptive attacks, where the attacker explicitly optimizes prompt features to evade the detector, thus collapsing the assumed latent separability between benign and adversarial samples.

Figure 1: Overview of the four prompt scenarios—benign, pseudo-malicious, harmful, and jailbreak—and the fundamental requirement for effective detectors: accept benign/PMPs and flag harmful/jailbreak prompts.

Manifold Trajectory Kinetics (Mtk): Methodology

Dynamic Manifold Analysis

The central contribution of the paper is a method—Manifold Trajectory Kinetics (Mtk)—which reconceptualizes the problem from static detection in a fixed feature space to dynamic analysis over the entire activation manifold as a sample propagates through the LLM. The key observation is that benign prompts remain in the benign region of the activation manifold throughout the layerwise forward pass, whereas jailbreak prompts traverse a characteristic trajectory: they are initially close to malicious regions (as they originate from malicious seeds) but are engineered to converge towards benign neighborhoods in later layers, thereby fooling both the alignment policy and conventional detectors.

Figure 2: Layer-wise rank differences between proximity to benign and malicious clusters, illustrating oscillatory and non-monotonic trajectories of jailbreak prompts, unlike the more stable benign/malicious classes.

This trajectory is not well-captured by fixed, single-layer statistics. Mtk models this temporal evolution by recording the rank of the $k$-nearest benign neighbors at each layer for every input. Using only benign and malicious prompts as anchors (no jailbreak samples), Mtk constructs a manifold trajectory for each input, which encodes semantic transitions and is robust to superficial lexical obfuscations and adaptive loss-based attacks.

Figure 3: Rank distribution of benign and jailbreak prompts relative to the benign cluster, demonstrating clear separability through trajectory-aware features.

Detection Framework

Mtk comprises three main stages:

1. Activation Trajectory Extraction: At each transformer layer, extract the hidden state for the final token of each reference anchor (benign and known malicious).
2. Manifold Trajectory Generation: For any test input, compute its distances to all anchors per layer and extract the neighbor-rank sequence (typically using the average over the $k$ nearest neighbors).
3. Kinetics Anomaly Detection: Fit an anomaly detector (e.g., Isolation Forest) to the rank trajectories of benign prompts. Test inputs that significantly deviate from the empirical benign pattern are flagged as potential jailbreaks.

   Figure 4: Mtk’s three-phase framework: activation extraction, trajectory generation, and anomaly-based detection via rank kinetics.

This approach is both lightweight (simple anomaly detection on fixed-size feature vectors) and zero-shot with respect to jailbreak attack data.

Experimental Results

Evasion and Over-Refusal Robustness

Quantitative evaluations span four LLMs and two VLMs, over ten jailbreak attack classes including AutoDAN, GCG, PAIR, SAA, TAP, and Zulu. Mtk demonstrates robust detection across traditional and adaptive attacks. On Llama2-7B, Mtk achieves a mean AUROC of 0.940 across all attacks, outperforming competitors such as GradSafe, SaP, GradCuff, and SelfDefend—including in settings where these methods receive explicit jailbreak prompts or augmented training.

Critically, on adaptive white-box attacks that optimize evasion objectives tailored to each baseline (including specialized surrogate losses matching Mtk’s non-differentiable trajectory signature), Mtk maintains an average true positive rate (TPR) of 0.85 and never exceeds a joint evasion rate (eASR) of 0.19. Competing defenses exhibit catastrophic drops in TPR under these conditions, demonstrating low robustness to adaptive evasion.

With respect to PMPs (drawn from OR-Bench for text and USB for multimodal), Mtk maintains an FPR of 0.02 (text) and demonstrates very low FPRs in VLM settings, outperforming all tested detectors (Table: see text, and Table 4). Mere augmentation of prior detectors with PMPs as additional benign anchors does not rescue their performance—the underlying representations are insufficiently structured.

Transferability to Vision-LLMs

The manifold kinetics perspective generalizes naturally to VLMs. Results show strong AUROC scores for Mtk on LLaVA-7B and Qwen-VL against multimodal jailbreak datasets such as MM-SafetyBench and JailBreakV-28K, with clear wins on PMP FPR in the multimodal setting.

Efficiency and Ablations

Mtk’s computational overhead is minimal, requiring less inference time per prompt than other accurate detectors using gradient analysis or perturbed outputs. Ablations (Figure 5 and 6) indicate robustness to reference bank size and composition, $k$-nearest neighbor choice, anomaly detector architecture, and layer selection (deep and all layers preferred over shallow).

Figure 5: UMAP visualization confirms structured separation in Mtk’s trajectory feature space and benchmark inference run time.

Figure 6: Ablation analyses for detector hyperparameters and the benign/malicious anchor ratio, showing stable performance over varied settings.

Figure 7: Influence of LLM layer selection on AUROC—using all or deep layers consistently yields optimal performance.

Theoretical and Practical Implications

The evidence provided undermines the robustness of static, single-layer, or output-based LLM detectors, particularly for PMPs and defense-aware adaptive attacks. Mtk’s design, grounded in manifold regularities and temporal processing, yields features that are resistant to evasion and do not rely on explicit knowledge of future attack forms. Practically, this substantially reduces both harmful content pass-through rates (by robustly capturing jailbreaks) and user-facing friction from false positives (by accepting PMPs where intent is benign).

The approach also suggests that manifold-based perspectives should receive greater attention in LLM robustness—fixed metric spaces fail under adversarial optimization, but the evolution of neighborhood structure is intrinsically harder to spoof without sacrificing attack utility.

Limitations and Future Work

Mtk’s core claim is the separability of manifold trajectories for benign and non-benign queries; should future adversaries learn to generate activations following entirely benign-like trajectories while producing unsafe outputs, the efficacy of the approach could be compromised. Current adaptive attacks, even when explicitly targeting Mtk’s signature, have not achieved this. Future research might explore richer manifold proximity measures, alternative anomaly detection methods, and implications for detection layer localization.

Conclusion

This work advances the state of LLM security in deployment by reframing jailbreak detection as a dynamic, trajectory-based anomaly identification problem over the semantic manifold. Mtk introduces a highly robust, data-efficient, and model-agnostic scheme that is demonstrably superior to metric-based or static-feature prior art in the presence of PMPs and adaptive adversaries. These results both deepen understanding of LLM/jailbreak dynamics and provide actionable methodologies for practical and robust alignment-layer defenses.