Test-Time Training Undermines Safety Guardrails

Abstract: Test-Time Training (TTT) is an emerging paradigm that enables models to adapt their parameters during inference, improving performance on tasks such as few-shot learning, retrieval-augmented generation, and complex reasoning. However, this dynamic adaptation introduces new vulnerabilities that adversaries can exploit to jailbreak models. We identify three threat models for TTT and demonstrate how attackers can leverage them to bypass safety filters. Our results show that TTT can significantly increase the Attack Success Rate (ASR) and the ASR over 10 generation trials (ASR@10). For example, under LoRA, the few-shot and generation-phase threat models achieve an average ASR@10 of 95% and 93% respectively, across models from different families and scales. These vulnerabilities transfer to production fine-tuning APIs. We also show that TTT-induced overfitting can produce degenerate outputs that inflate ASR under standard judges, and propose a validity-aware evaluation to correct for this. Our findings suggest that TTT exposes a new attack surface, strengthens attacks, and undermines existing safety guardrails. As a first step toward defense, we propose a lightweight provider-side detector that flags TTT requests via the perplexity shift on a private harmful holdout, but robust deployment will ultimately require dynamic alignment.

• The paper demonstrates that Test-Time Training (TTT) quickly undermines LLM safety guardrails by enabling adversarial jailbreaks with minimal parameter updates.
• It presents three threat models—self-supervised, few-shot, and generation-phase—that significantly elevate jailbreak success rates across various LLMs.
• The study proposes a perplexity shift detection defense and emphasizes the need for dynamic, adaptive safety mechanisms against TTT vulnerabilities.

Test-Time Training as a Critical Threat to LLM Safety Alignment

Introduction and Motivation

Test-time training (TTT) has emerged as a key paradigm for improving model generalization and task adaptation by enabling instance-specific weight updates during inference. While TTT enhances LLMs’ capabilities in few-shot learning, retrieval-augmented generation, and complex reasoning, this work systematically demonstrates that TTT undermines standard safety guardrails implemented via conventional alignment methods such as RLHF or Constitutional AI. The central claim is that the static safety controls established at training and alignment time are fragile: even minimal, targeted parameter updates during TTT can significantly degrade or remove refusal behavior, rendering state-of-the-art LLMs susceptible to jailbreaks, including in realistic API settings.

Threat Models and Methods

The authors formalize three concrete threat models for exploiting TTT to jailbreak LLMs, aligned with both canonical and emergent TTT use cases:

• Self-supervised threat model: The attacker provides a (potentially adversarial) prompt, and TTT adapts the model parameters on this prompt itself, as in self-supervised next-token prediction. Even adaptation on clean prompts causes notable safety degradation.
• Few-shot threat model: The attacker supplies $K$ harmful (prompt, target prefix) pairs as a support set for TTT. The model is fine-tuned on these examples before answering the target query. The adaptation objective encourages the model to reproduce the harmful response patterns present in the support set.
• Generation-phase threat model: The attacker adapts the model online during response generation, e.g., by conditioning on an affirmative target prefix (“Sure, here is…”). This approach directly primes the model to continue with a harmful completion, bypassing refusal triggers.

  Figure 1: Overview of the three TTT threat models; test-time adaptation updates model parameters $\boldsymbol{\theta}$ with data under adversary control, compromising alignment.

Their experimental protocol leverages both full fine-tuning and parameter-efficient LoRA adaptation across a representative suite of recent open-weight LLMs, including Gemma 7B, Llama3 8B/70B, Qwen2.5 (1.5B/7B), and Qwen3 (8B/32B). The main benchmark is a curated subset of 50 harmful behaviors from AdvBench, with generalization evaluation across JailbreakBench and high-complexity tasks (Transluce CBRN).

Empirical Results

Erosion of Guardrails

Key numerical findings demonstrate the magnitude and universality of the vulnerability:

• On Gemma 7B, Llama3 8B, and Qwen2.5 7B, TTT using as few as 1-5 steps under the few-shot or generation-phase threat models raises ASR@10 from $0-10\%$ (no TTT) to $80-100\%$—nearly every sampled response includes a jailbreak.
• Large, aligned models are also susceptible: Llama3 70B reaches $94\%$ ASR@10 (few-shot, 5 steps) and Qwen3 32B attains $96\%$ (generation-phase, 5 steps) with LoRA.
• Vulnerabilities transfer to production LoRA fine-tuning APIs such as Tinker; e.g., attacking GPT-OSS 120B (120B parameters) produces ASR@10 of $100\%$ (few-shot, 10 steps) with minimal financial cost.
• Self-supervised TTT degrades safety even with clean prompts, increasing ASR@10 by an order of magnitude in smaller models.
• Critical evaluation reveals that TTT-induced overfitting can inflate ASR when judged by standard LLM-based safety assessors due to degenerate outputs (repetitive, prompt-echoed, or template-filled responses).

Compositionality and Transfer

The authors demonstrate that TTT does not merely supplant, but composes with, existing static adversarial prompting methods. Applying few-shot TTT on top of adversarial suffix and template attacks lifts attack success to near-deterministic jailbreaks, even for models that were nearly impervious to each component alone.

Additionally, cross-category transfer analysis shows that TTT using support examples from one harm domain dramatically increases the attack success over other unrelated domains—demonstrating that guardrail erosion is not localized but global across the refusal manifold.

Figure 2: Few-shot TTT with support examples from category A unlocks jailbreaks for target prompts in category B across model families, evidencing broad transferability.

Validity-aware Evaluation

The authors introduce a two-stage safety evaluation pipeline, integrating an LLM-based validity judge to identify and filter degenerate, non-informative outputs (e.g., template echoes, pure acknowledgments, repeated tokens) that would be erroneously judged as unsafe by standard classifiers. On benchmarks augmented with TTT-induced degeneracies, omitting this step leads to up to 13 percentage-point overestimation of ASR.

Provider-Side Defense: Perplexity Shift Detection

Recognizing the challenge of defending against dynamic parameter updates, the paper proposes a lightweight provider-side detection mechanism: measure the drop in perplexity on a private harmful holdout set before and after each TTT request. If the harmful perplexity falls above a calibrated per-model threshold, the request is flagged. This detector achieves $100\%$ TPR and $\leq2\%$ FPR across multiple models in the vanilla attack regime.

Figure 3: Provider-side perplexity-based detection reliably separates adversarial (crosses) from benign (circles) TTT requests via a calibrated threshold on harmful-target perplexity shift.

However, this method is not robust against adaptive attackers who can regularize their TTT objectives to evade the threshold, highlighting the need for dynamic, adaptive alignment defenses.

Implications for LLM Alignment and Deployment

The work challenges the foundational assumption that static training-time alignment is robust under inference-time parameter adaptation. The superficiality of current safety guardrails—and their ease of removal with a small number of TTT updates—poses serious risks for deployment scenarios where user-controlled weight adaptation is enabled, as in custom fine-tuning APIs. Even transient, instance-specific adaptation suffices to unlock nearly all harmful capabilities previously suppressed by alignment protocols.

Practical implications include:

• API providers exposing TTT/fine-tuning endpoints must assume significant new risk. Threat surfaces expand from token-space attacks to parameter-space manipulations.
• Static evaluations are insufficient: safety benchmarking for TTT-equipped models must shift from fixed probes to dynamic, stress-testing protocols that explicitly assess the resilience of alignment to adversarial local adaptation.
• Defensive research must prioritize dynamic safety mechanisms: e.g., regularizing adaptation trajectories, on-the-fly alignment interventions, and adversarial training protocols robust to test-time model drift.

Theoretically, these results suggest that alignment “lives on the surface” of the parameter landscape for current LLMs, and that the intended separation between harmless and harmful behaviors is neither deep nor robust to local task-specific optimization. This vulnerability motivates both mechanistic investigations into the localization and separability of aligned submanifolds, and methodological work on dynamic constraints for real-time model adaptation.

Conclusion

This paper provides a comprehensive and technically rigorous characterization of TTT as a critical, underappreciated attack vector for defeating LLM safety alignment. By establishing strong quantitative vulnerabilities across multiple model families and realistic threat surfaces, and by proposing a first-step mitigation, it closes a key methodological gap in current LLM red-teaming research. Future work must develop robust, dynamic alignment methods and evaluation protocols that explicitly address the risks of inference-time weight adaptation. The field should anticipate that as TTT becomes more mainstream for generalization and task flexibility, adversarial misuse will be an unavoidable consequence without fundamentally new approaches to safety alignment.