SafeTuning: Preserving Model Safety

Updated 10 July 2026

SafeTuning is a broad term for techniques that preserve safety properties during model tuning, ensuring that alignment and refusal behaviors remain intact.
Researchers apply methods such as mechanistic constraints (e.g., ProCon, AsFT) and inference-time hyperparameter tuning (e.g., SafeTune) to reduce harmful response rates and maintain task performance.
Studies using benchmarking frameworks like SafeTuneBed reveal trade-offs between safety and utility, with approaches refined for both LLM fine-tuning and domain-specific system tuning.

Searching arXiv for papers on “SafeTuning” and closely related fine-tuning safety methods to ground the article. SafeTuning is an umbrella term for methods that improve or at least preserve safety properties during model tuning or fine-tuning, especially when downstream adaptation would otherwise erode refusal behavior, increase harmful-response rates, or induce architecture-specific failure modes. In the LLM literature, the term spans training-time alignment-preserving regularization, post-hoc safety transfer, configurable preference tuning, inference-time hyperparameter and guardrail search, and benchmarking frameworks for safety–utility trade-offs; in a broader systems literature, it also denotes tuning under explicit safety constraints for controllers, accelerators, and cloud databases (Du et al., 8 Sep 2025, Hossain et al., 31 May 2025, Zhang et al., 2022).

1. Scope and terminology

The term does not denote a single algorithm. A useful synthesis is to distinguish SafeTuning by where safety is enforced: internal representations, parameter updates, routing behavior, prompt/configuration control, training-data sanitization, or evaluation protocol. Several papers use the label directly, while others present concrete methods that the paper itself describes as a “SafeTuning” approach or as “safe fine-tuning” in the broader sense (Du et al., 8 Sep 2025, d'Aloisio et al., 8 May 2026, Rezakhani et al., 29 Apr 2026, Gallego, 2024, Wu et al., 13 Nov 2025, Abdulkadir, 14 Dec 2025).

Family	Core mechanism	Representative work
Mechanistic constraint-based	Preserve safety-related internal directions or routing	ProCon (Du et al., 8 Sep 2025), AsFT (Yang et al., 10 Jun 2025), SafeMoE (Kim et al., 26 Sep 2025)
Preference/configuration-based	Condition safety behavior on system prompts	CST (Gallego, 2024)
Search-based inference-time tuning	Optimize hyperparameters, prompts, or filter modes	SafeTune (d'Aloisio et al., 8 May 2026), guardrail auto-tuning (Abdulkadir, 14 Dec 2025)
Domain-specific data sanitization	Filter poisoned inputs and rewrite prompts	RTL SafeTune (Rezakhani et al., 29 Apr 2026)
Post-hoc alignment transfer	Distill and merge a reusable safety vector	EnchTable (Wu et al., 13 Nov 2025)
Benchmarking and evaluation	Standardize datasets, defenses, and metrics	SafeTuneBed (Hossain et al., 31 May 2025)

This plurality matters because “SafeTuning” is sometimes used for training-time preservation of an aligned model, sometimes for black-box tuning of a frozen model, and sometimes for safe optimization of non-LLM systems. A common thread is that safety is treated as a property that must be preserved under adaptation rather than assumed to survive it.

2. Safety degradation under adaptation

A central motivation for SafeTuning is that ordinary instruction fine-tuning is safety-destructive even when the downstream objective is benign. In the standard formulation, instruction fine-tuning minimizes autoregressive cross-entropy,

$\mathcal{L}_{\text{IFT}}=\mathcal{L}_{\text{CE}}=-\mathbb{E}_{s}\sum_{j=1}^{m}\log p_\theta(s_{n+j}\mid s_{1:n+j-1}),$

which improves task capability but can “undo” RLHF safety; the paper on ProCon ties this effect to drift in a refusal direction in hidden-state space, especially early in training and in deeper layers (Du et al., 8 Sep 2025). SafeTuneBed generalizes the same concern as alignment erosion under benign fine-tuning and formalizes harmful fine-tuning as mixing a benign fine-tuning corpus with a harmful corpus at poison ratios such as $0\%$ , $5\%$ , and $30\%$ , then evaluating ASR and Harmfulness Score on AdvBench and PolicyEval (Hossain et al., 31 May 2025).

Mechanistic evidence for why fine-tuning “breaks” safety is provided by the study of safety fine-tuning itself. That work reports that supervised safety fine-tuning, DPO, and unlearning minimally transform MLP weights to specifically align unsafe inputs into the weights’ null space, yielding a clustering of inputs based on whether the model deems them safe or not; correspondingly, jailbreak activations are closer to safer samples, so the model processes them as if they were safe (Jain et al., 2024). A related post-hoc transfer perspective is offered by EnchTable, which states that unsafe rate increases as large as $0.772$ after fine-tuning and frames the resulting problem as safety alignment transfer into specialized downstream models without retraining them (Wu et al., 13 Nov 2025).

These results jointly suggest that SafeTuning is not merely a matter of adding more refusal data. The literature repeatedly treats safety degradation as a representational or routing drift problem induced by adaptation itself.

3. Mechanistic SafeTuning inside the model

The most explicitly mechanistic LLM SafeTuning line constrains internal quantities with known causal relevance to safety. ProCon begins from the refusal direction $\mathbf{r}^{(l)}$ , estimated by difference-in-means between malicious and benign prompts at each layer, and penalizes changes in each hidden state’s projection onto that direction. Its objective is

$\mathcal{L}_{\text{overall}}=\mathcal{L}_{\text{CE}}+\alpha \times \mathcal{L}_{\text{ProCon}},$

with

$\mathcal{L}_{\text{ProCon}}=\mathbb{E}_{s}\left[\sum_{l=1}^{L}\sum_{j=1}^{m}\left\|z_t^{(l)}(s_{(n+j)})-z_0^{(l)}(s_{(n+j)})\right\|_2\right].$

A warm-up strategy applies strong constraint only in the early stage, where the paper observes “early-stage sharp drift,” and additional safety-oriented data broaden the training distribution. On LLaMA2 benign IFT, the paper reports Vanilla $\text{AVG(HS)}=1.20$ , ASR $=6.12\%$ , Task $0\%$ 0; after IFT, HS $0\%$ 1, ASR $0\%$ 2, Task $0\%$ 3; and for $0\%$ 4, HS $0\%$ 5, ASR $0\%$ 6, Task $0\%$ 7 (Du et al., 8 Sep 2025).

AsFT anchors updates in parameter space rather than representation space. It defines an alignment direction

$0\%$ 8

shows that harmful directions are nearly orthogonal to it, and characterizes parameter space as a narrow safety basin. The regularized objective is

$0\%$ 9

where $5\%$ 0 projects onto the subspace orthogonal to the alignment direction. The paper states that AsFT outperforms Safe LoRA, reducing harmful behavior by $5\%$ 1 percent, improving model performance by $5\%$ 2 percent, and maintaining robust performance across various experimental settings (Yang et al., 10 Jun 2025).

For MoE LLMs, SafeMoE argues that safety is mediated by routing harmful inputs to safety-critical experts and that harmful fine-tuning causes routing drift. Its regularizer matches routing distributions of the fine-tuned model to those of the initial safety-aligned model:

$5\%$ 3

The paper reports that SafeMoE reduces the harmfulness score of OLMoE from $5\%$ 4 to $5\%$ 5, while maintaining task utility within $5\%$ 6 degradation and incurring only $5\%$ 7 overhead (Kim et al., 26 Sep 2025).

A neuron-level variant identifies refined safety knowledge neurons $5\%$ 8, constructs refusal and conformity directions in the embedding space spanned by those neurons, and uses them to generate a refusal corpus before fine-tuning only the corresponding neuron parameters. That paper reports that adjusting the activation of safety-related neurons can effectively control the model’s behavior with a mean ASR higher than $5\%$ 9, and that its SafeTuning consistently reduces attack success rates across multiple LLMs and outperforms all four baseline defenses (Zhao et al., 1 Sep 2025). EnchTable addresses the same problem post hoc by distilling a safety vector from a surrogate model in an NTK-linearized regime and merging it into downstream fine-tuned models with interference-aware scaling, thereby making safety transfer a tuning-free deployment step rather than a retraining phase (Wu et al., 13 Nov 2025).

4. Configurable and inference-time SafeTuning

Not all SafeTuning modifies weights. Configurable Safety Tuning extends DPO by conditioning preferences on a system prompt $30\%$ 0, so that the same response pair has opposite preference labels under opposite safety configurations. Its configuration-aware preference probability is

$30\%$ 1

For OpenHermes-2.5-Mistral-7B, the paper reports $30\%$ 2 for the original model, $30\%$ 3 for DPO, and $30\%$ 4 for CST; for SOLAR-Instruct-10.7B it reports $30\%$ 5 for CST (Gallego, 2024).

SafeTune formulates safety tuning as a multi-objective search problem over decoding hyperparameters and system prompt templates. For a configuration $30\%$ 6, it optimizes

$30\%$ 7

using NSGA-II over temperature, top- $30\%$ 8, top- $30\%$ 9, repetition penalty, max new tokens, and one of three safety-oriented system prompts. In its baseline study over 137 harmful-leading prompts, Qwen3.5 $0.772$0B exhibits prompt-level harmfulness $0.772$1 ($0.772$2) and response-level harmfulness $0.772$3; the paper then reports that SafeTune significantly reduces harmfulness and increases relevance, and that repetition penalty is the most important parameter for both harmfulness and relevance, with Pareto-optimal settings always choosing repetition penalty less than $0.772$4 (d'Aloisio et al., 8 May 2026).

A closely related black-box variant treats guardrail design itself as hyperparameter optimization over a frozen model. The system wraps Mistral-7B-Instruct with modular jailbreak and malware system prompts plus a ModernBERT-based harmfulness classifier, scores each configuration on malware and jailbreak attack success rate, benign harmful-response rate, and latency, and then searches a $0.772$5-point design space of prompt combinations and filter modes. The paper reports that a black-box Optuna study reliably rediscovers the best grid configurations while requiring an order of magnitude fewer evaluations and roughly $0.772$6 less wall-clock time (Abdulkadir, 14 Dec 2025).

5. Domain-specific and broader safe tuning

In domain-specific LLM applications, SafeTuning often means protecting the fine-tuning data path rather than the model internals. The RTL code-generation SafeTune framework hardens fine-tuning against data poisoning and hardware Trojan insertion by combining a GNN-based structural filter over Data-Flow Graphs with a semantic verification module using GTE-large embeddings and an XGBoost risk regressor, then adding runtime paraphrasing to neutralize residual triggers. The paper reports that for Qwen2.5-Coder-14B, Baseline ASR $0.772$7 and SafeTune $0.772$8, while VerilogEval Pass@k remains identical at $0.772$9; for CodeLlama-13B, Baseline ASR $\mathbf{r}^{(l)}$ 0 and SafeTune $\mathbf{r}^{(l)}$ 1, with Pass@k $\mathbf{r}^{(l)}$ 2 versus $\mathbf{r}^{(l)}$ 3 (Rezakhani et al., 29 Apr 2026).

Outside LLM alignment, the older systems literature uses “safe tuning” for online optimization under explicit safety constraints. A safety-aware cascade-controller method formulates PID tuning as constrained Bayesian optimization with a GP-modeled objective and a data-driven constraint capturing stability requirements, then adds automatic detection of critical gains and a penalty based on proximity to those gains (König et al., 2020). A later automotive controller method develops Lipschitz Safe Bayesian Optimization with multiple safety constraints and reports learning tracking controllers without leaving the track or violating any other safety constraints (Menn et al., 22 Jan 2025). In accelerator control, a step-size limited variant of safe Bayesian optimization tunes up to $\mathbf{r}^{(l)}$ 4 parameters subject to $\mathbf{r}^{(l)}$ 5 constraints (Kirschner et al., 2022). For cloud databases, OnlineTune treats configuration tuning as contextual Bayesian optimization with context-space partition and safe exploration via subspace adaptation, and reports $\mathbf{r}^{(l)}$ 6 improvement on cumulative performance while reducing $\mathbf{r}^{(l)}$ 7 unsafe configuration recommendations (Zhang et al., 2022). This broader usage suggests that SafeTuning, in its most general sense, denotes adaptation under explicitly modeled safety envelopes.

6. Evaluation protocols, trade-offs, and open problems

SafeTuneBed systematizes evaluation by defining SafeTuning as maintaining low harmful-response rate on harmful prompts while preserving task performance on benign targets, then unifying datasets, threat regimes, defenses, and metrics in a single benchmark and toolkit. It covers sentiment analysis, question-answering, multi-step reasoning, dialogue summarization, code/SQL generation, and open-ended instruction tasks; supports benign, low-harm, and high-harm variants; and evaluates both safety and utility with ASR, Harmfulness Score, MMLU, MT-Bench, and task-specific metrics. Its benchmarked results highlight a recurring trade-off: LoRA and Vaccine can be safer under benign training, whereas LiSA is more robust under heavy poisoning but weaker on benign safety (Hossain et al., 31 May 2025).

The main open problems are consistent across the literature. ProCon explicitly notes residual attacks and imperfect safety, dependence on a good refusal-direction estimate, hyperparameter sensitivity, and evaluation limited to chat LLMs and LoRA tuning (Du et al., 8 Sep 2025). SafeTune emphasizes limited evaluation scope, reliance on automated evaluation, computational cost, prompt-specific tuning, and no formal theoretical guarantees (d'Aloisio et al., 8 May 2026). The RTL SafeTune framework depends on labeled Trojan data and remains only a partial mitigation (Rezakhani et al., 29 Apr 2026). EnchTable requires a well-aligned surrogate with the same architecture and access to its weights, while SafeMoE requires a safety-aligned checkpoint and a representative harmful set for routing alignment (Wu et al., 13 Nov 2025, Kim et al., 26 Sep 2025).

Taken together, this suggests that SafeTuning is converging toward a multi-layered research program rather than a single recipe. One layer constrains safety-critical geometry inside the model; another treats prompts, decoders, and guardrails as tunable control surfaces around frozen models; a third standardizes threat models and measurement. The shared premise is that safety is not static after alignment, and that robust adaptation requires explicit mechanisms to preserve, transfer, or verify the structures that alignment originally created.