SafeMERGE: Safeguarding LLM Merging

• SafeMERGE is a framework that preserves safety alignment in large language models by selectively merging safety-relevant LoRA layers post-finetuning.
• It employs per-layer safety subspace projections and cosine similarity metrics to identify layers where merging can restore alignment without compromising utility.
• Empirical results show SafeMERGE dramatically reduces harmfulness metrics (e.g., DirectHarm, HexPhi) while retaining strong downstream performance on benchmarks like GSM8K and PubMedQA.

SafeMERGE denotes a line of work on preserving or enforcing safety during model merging, especially in LLMs whose downstream fine-tuning or expert composition can erode refusal behavior and other alignment properties. In the most specific sense, SafeMERGE is a post-fine-tuning framework that selectively merges safety-relevant LoRA layers from a safety model into a task-tuned model when those layers deviate from a safety-aligned subspace (Djuhera et al., 21 Mar 2025). In a broader sense, recent literature uses the term for safety-aware merging strategies that combine domain utility with alignment, or for defenses that make merging safe by preventing unauthorized or adversarial fusion (Thakkar et al., 2024).

1. Definition, scope, and disambiguation

In current LLM research, the label SafeMERGE is used in at least two closely related ways. The first is the named framework introduced in “SafeMERGE: Preserving Safety Alignment in Fine-Tuned LLMs via Selective Layer-Wise Model Merging,” which targets post-hoc restoration of safety after benign downstream fine-tuning (Djuhera et al., 21 Mar 2025). The second is a broader usage in which SafeMERGE denotes safety-aware model-merging practice more generally, including dense task-vector composition, benchmark-driven safe merging, and application-specific safety realignment in domains such as telecom (Thakkar et al., 2024, 2505.10833, Djuhera et al., 29 May 2025).

A distinct and older use of the near-identical name “SafeMerge” refers to a program-analysis system for verifying semantic conflict-freedom in three-way software merges rather than neural model fusion. That system reasons over base, left, right, and candidate programs and was evaluated on 52 real-world Java merge scenarios (Sousa et al., 2018). This older usage is technically unrelated to LLM safety alignment, although both share the idea that a merge should preserve intended behavior.

Within the LLM setting, SafeMERGE arose from a recurrent empirical finding: fine-tuning on benign downstream tasks can degrade safety, and naïve parameter merging can propagate misalignment rather than neutralize it. “Model Merging and Safety Alignment: One Bad Model Spoils the Bunch” formalized this concern by treating alignment as a skill that must be optimized during merging, rather than assumed to survive it (Hammoud et al., 2024).

2. Core mechanism of the named SafeMERGE framework

The named SafeMERGE framework is a post-fine-tuning method built around four checkpoints: an unaligned base model $B$, a safety-aligned instruct/chat model $A$, a utility fine-tuned model $F$, and a safety fine-tuned model $S$ (Djuhera et al., 21 Mar 2025). In the reported experiments, the target architectures were Llama-2-7B-Chat and Qwen-2-7B-Instruct; LoRA was applied to attention $q\_proj$ and $v\_proj$ modules only, with LoRA rank $r = 8$, LoRA alpha $= 16$, and dropout $= 0$.

The method first constructs a per-layer, per-module safety direction from the difference between aligned and unaligned weights:

$$v_{l,m} = \mathrm{vec}\!\big(W_{A,l,m} - W_{B,l,m}\big), \qquad P_{s,l,m} = \frac{v_{l,m} v_{l,m}^\top}{\|v_{l,m}\|_2^2}.$$

This follows the same base-to-chat difference view of safety direction used by SafeLoRA, but SafeMERGE uses that direction for selection and merging rather than direct projection alone (Djuhera et al., 21 Mar 2025).

For the utility adapter update $A$0, SafeMERGE measures alignment with the safety subspace by cosine similarity:

$A$1

aggregates this across $A$2 to obtain $A$3, and marks a layer for intervention when $A$4. The merge set is therefore

$A$5

Only layers in this set are altered; all others keep the utility adapter unchanged (Djuhera et al., 21 Mar 2025).

The primary merge rule is linear weighted adapter interpolation:

$A$6

The paper reports that fixed $A$7 values in ranges $A$8 to $A$9 gave strong utility–safety trade-offs, with linear weighted merging as the main strategy. A subspace-guided variant,

$F$0

is conceptually consistent but was not the primary reported setting (Djuhera et al., 21 Mar 2025).

The full pipeline is strictly post hoc. It computes base-to-aligned safety projections once, scores LoRA deviations, merges only the selected layers, and returns a model with the same inference-time structure and no additional runtime modules. The reported implementation used Llama-Factory and FSDP on $F$1A100 80GB GPUs for fine-tuning and lm-eval-harness v0.4.3 for GSM8K and PubMedQA evaluation; harmfulness was measured on DirectHarm and HexPhi with Llama-Guard-3-8B (Djuhera et al., 21 Mar 2025).

3. Empirical behavior and safety–utility trade-offs

The main SafeMERGE experiments evaluate utility on GSM8K and PubMedQA and harmfulness on DirectHarm and HexPhi for Llama-2-7B-Chat and Qwen-2-7B-Instruct (Djuhera et al., 21 Mar 2025). Across all four settings, the fine-tuned models became markedly more harmful than the original instruction-tuned bases, and SafeMERGE reduced harmfulness while retaining most or all downstream utility.

Setting	Fine-tuned	SafeMERGE
Llama-2-7B-Chat, GSM8K	27.37 acc; 27.80 DirectHarm; 16.40 HexPhi	26.96 acc; 7.50 DirectHarm; 5.70 HexPhi
Llama-2-7B-Chat, PubMedQA	72.60 acc; 12.50 DirectHarm; 6.20 HexPhi	72.20 acc; 8.10 DirectHarm; 4.30 HexPhi
Qwen-2-7B-Instruct, GSM8K	70.13 acc; 25.30 DirectHarm; 16.80 HexPhi	72.90 acc; 8.20 DirectHarm; 7.50 HexPhi
Qwen-2-7B-Instruct, PubMedQA	79.60 acc; 26.00 DirectHarm; 13.20 HexPhi	80.30 acc; 8.50 DirectHarm; 5.90 HexPhi

These results place SafeMERGE in a different regime from projection-only post hoc defenses. On Qwen-2-7B-Instruct, SafeMERGE achieved the lowest harmfulness among post-fine-tuning defenses and high utility, with 72.90% GSM8K and 80.30% PubMedQA. On Llama-2-7B-Chat, it reached near-best utility on GSM8K and the highest utility on PubMedQA while substantially reducing harmfulness (Djuhera et al., 21 Mar 2025).

The selectivity mechanism is central. The paper reports that SafeMERGE merged only a subset of LoRA components, including 34 Q/V components on Qwen-2 GSM8K and 28 on Llama-2 PubMedQA. Threshold ablations showed that as $F$2 increases, more layers are merged and the behavior converges to full linear model merging; however, merging as few as 8 layers already significantly decreases harmfulness. Weighting ablations showed that weights summing to 1.0 provide a “sweet spot,” whereas TIES merging was inconsistent across settings (Djuhera et al., 21 Mar 2025).

The same basic strategy was applied to telecom-tuned models in SafeCOMM, where safety degradation was observed after both supervised fine-tuning and continual pretraining on benign telecom corpora (Djuhera et al., 29 May 2025). On TeleData for Llama-2-7B-Chat, accuracy changed from 38.70 to 38.50 under SafeMERGE, while DirectHarm dropped from 36.70 to 6.90 and HexPhi from 20.10 to 5.10. On severely unsafe CPT models, SafeMERGE also restored safety: for Llama-3-8B-Tele-it, DirectHarm fell from 78.20 to 14.30 and HexPhi from 73.00 to 11.10 while TeleData accuracy moved from 34.50 to 33.90 (Djuhera et al., 29 May 2025).

A denser full-parameter counterpart appears in MergeAlign, which merges domain and alignment task vectors with

$F$3

using $F$4 and $F$5 in its default configuration (Thakkar et al., 2024). On Llama-3-8B medicine and finance settings, MergeAlign achieved 98.33 and 97.87 safe responses on HH-Red team, 99.67 and 99.70 on BeaverTails, while preserving near-domain-expert averages: 61.33 versus 61.37 in medicine and 74.07 versus 74.47 in finance (Thakkar et al., 2024). SafeMERGE and MergeAlign therefore occupy neighboring points in the design space: both are post hoc and safety-preserving, but one is selective LoRA layer merging and the other is dense full-parameter task arithmetic.

4. Methodological relatives and extensions

SafeMERGE belongs to a broader safety-preserving merging taxonomy. One branch works in dense parameter space. MergeAlign computes full task vectors with respect to a common base, adds domain and alignment vectors without normalization, and relies on linear mode connectivity among checkpoints sharing the same base (Thakkar et al., 2024). Another dense but more geometry-aware branch is AlignMerge, which estimates an alignment subspace with projector $F$6 in a local Fisher chart around an instruction-tuned base and optimizes

$F$7

In the reported LLaMA-3 8B setting, AlignMerge improved over SafeMerge on AQI, toxicity, and alignment-subspace drift: AQI 0.77 versus 0.72, Tox-Mean 0.038 versus 0.051, Tox-Rate 8.3% versus 11.8%, and drift $F$8 0.16 versus 0.19 (Roy et al., 18 Dec 2025).

A second branch operates through structured sparsity or neuron isolation. LED-Merging identifies task-specific neurons with SNIP, elects neurons important in both base and task-tuned models by intersection, and disjoints overlaps across tasks before summing masked task vectors. On Llama3-8B safety+math+code, LED-Merging reported ASR 20.75%, GSM8K 52.39%, MBPP 44.6%, and HumanEvalPack 36.59%; on Mistral-7B safety+math, removing Disjoint caused ASR 63.0%, while the full method reduced it to 16.0% (Ma et al., 24 Feb 2025). This explicitly targets cross-task neuron interference rather than subspace deviation from a safety anchor.

A third branch is distribution-aware sparse fusion. SCF-RKL constructs a binary mask $F$9 and fuses

$S$0

where importance is driven by reverse KL on parameter-induced distributions and thresholded by Tukey’s rule (Lin et al., 12 Feb 2026). The reported sparsity is roughly 5–15% per layer in practice. On GSM8K, baseline dense merges exhibited repetition rates approaching 100% on Qwen2.5-14B and Qwen2.5-32B, versus 1% for the base model and 0.2% for SCF-RKL. The same paper reports safety gains across S-Eval, HarmBench, JBB-Behaviors, WildJailbreak, and StrongREJECT when fusing safety-hardened and reasoning-specialized models (Lin et al., 12 Feb 2026).

These variants differ in what they regard as the primary failure mode. SafeMERGE treats harmfulness as layer-local deviation from a safety-aligned subspace (Djuhera et al., 21 Mar 2025). MergeAlign treats safety as an additive task vector anchored at a shared base (Thakkar et al., 2024). LED-Merging treats safety–utility conflict as neuron misidentification and cross-task collision (Ma et al., 24 Feb 2025). AlignMerge treats alignment as a geometric invariant in Fisher space (Roy et al., 18 Dec 2025). SCF-RKL treats instability and harmfulness as functional interference induced by dense parameter arithmetic (Lin et al., 12 Feb 2026).

5. Security, misuse, and adversarial interpretations

A separate body of work redefines safe merging as protection against unauthorized or malicious fusion. MergeGuard addresses a white-box free-rider who merges a released fine-tuned model with another homologous model using weight averaging, task arithmetic, TIES, DARE, or AdaMerging. Its two stages are Density-Aware Finetuning and Adversarial Weight Negation, with reported hyperparameters $S$1, $S$2, $S$3, and $S$4 (Chen et al., 14 Nov 2025). The goal is that the protected model remains faithful on its own task while any merged version collapses on that task. Reported results include GSM8K on Gemma2 dropping from 69.6% to 1.52% post-merge and Mistral with TA dropping from 75.7% to 0% (Chen et al., 14 Nov 2025).

MergeBarrier targets the same threat through disruption of Linear Mode Connectivity. It applies a shared orthogonal transformation to attention query and key matrices and reparameterizes the FFN by polynomial expansion. The protected model remains useful, but merged models fail across Task Arithmetic, TIES-Merge, and DARE. For example, under Task Arithmetic, LM + Math on LLaMA-2-13B dropped from Alpaca 26.87 and GSM8K 62.17 to 0.14 and 0.00 after protection; the aggregate merged-model accuracy relative to MergeBarrier was 8.27× for unprotected models and 1.00× for MergeBarrier (Li et al., 13 Nov 2025).

TrojanMerge exposes a complementary threat: source models can remain individually benign yet become severely misaligned when merged because latent perturbations are engineered to reconstruct a safety-critical attack vector under the merge operator (Li et al., 1 Apr 2026). The paper reports Harmful Score increases from 1.9% to 71.9% on Llama 2, from 3.1% to 81.0% on Llama 3, and from 24.0% to 85.4% on Mistral when TrojanMerge-modified sources are fused. The effect persisted across TA, DARE, TIES, and KnOTS, with average HS 79.4%, 77.1%, 79.1%, and 73.6%, respectively (Li et al., 1 Apr 2026).

These papers shift the meaning of SafeMERGE away from alignment retention alone. In this security-centered interpretation, a safe merge is one that either cannot be stolen, cannot be weaponized by latent perturbations, or can be validated against merge-induced attack surfaces. This suggests that future SafeMERGE systems may need both alignment-preserving objectives and adversarial provenance checks.

6. Evaluation protocols, practical criteria, and open problems

Benchmarking work has made SafeMERGE more operational. MergeBench standardized five domains—instruction following, mathematics, multilingual understanding, coding, and safety—across Llama and Gemma families from 2B to 9B and evaluated eight representative merging methods (2505.10833). Safety evaluation used WildGuardTest, HarmBench, DoAnythingNow, and XSTest, with Refuse-To-Answer and XSTest Accuracy as primary metrics. The benchmark reports that stronger base models help: on 8B–9B pretrained models, methods consistently recover more than 90% normalized performance overall, and instruction-tuned bases consistently exceed 90% normalized performance across methods. Localize-and-Stitch and Dataless LAS achieved the highest average normalized multi-task performance, 89.7% and 89.1%, while Task Arithmetic achieved 87.2% and TIES 84.8% (2505.10833).

Predictive work has also reframed safe merging as a measurement problem. MergeProbe defines mergeability as utility retention after merging and forecasts it from signals collected in the first 10% of LoRA training using a 256-example calibration batch per domain (Tang et al., 17 Jun 2026). It uses update alignment, gradient alignment, Fisher-weighted overlap, and activation disturbance to decide whether to MERGE, REWEIGHT, PRUNE, or ROUTE. On MERGE-PEFT, MergeProbe reported Avg 91.4% and Worst 88.7%, compared with 86.7% and 81.9% for FlyLoRA and 84.0% and 77.6% for OSRM (Tang et al., 17 Jun 2026). Because safety adapters are often the harmed party when merged with capability adapters, the paper explicitly optimizes worst-task retention rather than mean retention.

Several limitations recur across the literature. SafeMERGE’s single-direction safety subspace may under- or over-flag layers if the aligned model’s safety is brittle or domain-specific (Djuhera et al., 21 Mar 2025). MergeAlign depends on the quality of the aligned model and on base consistency; if the aligned checkpoint is weaker or derives from a different base, safety gains can be smaller or merges can fail (Thakkar et al., 2024). SafeCOMM emphasizes that safety restored on DirectHarm and HexPhi may still miss niche telecom-specific adversarial prompts unless safety data covers them (Djuhera et al., 29 May 2025). MergeBench notes a persistent in-domain gap between merging and well-trained multi-task models, as well as the computational cost of tuning sparsity and coefficients on large models (2505.10833).

A common misconception is that safety alignment is automatically preserved if all component models are individually useful or even individually safe. The literature repeatedly rejects this assumption. One line shows that one misaligned expert can spoil a merged model (Hammoud et al., 2024); another shows that individually benign source models can be engineered to merge into danger (Li et al., 1 Apr 2026). The cumulative implication is that SafeMERGE is best understood not as a single algorithm but as a design problem spanning layer selection, task-vector geometry, sparse interference control, validation protocol, and threat modeling.