Safety-Preserving Adaptation (SPA)
- Safety Preserving Adaptation (SPA) is a design pattern that enables systems to adapt to new tasks by constraining updates to preserve established safety properties, such as forward invariance of safe sets.
- SPA techniques are applied across robotics, reinforcement learning, and LLM fine-tuning, employing methods like runtime shielding, gradient projection, and post-hoc repair to prevent safety breaches.
- Key strategies include projecting adaptations away from unsafe directions, freezing safety-critical components, and filtering high-risk actions to maintain overall system integrity.
Searching arXiv for recent and foundational papers on Safety Preserving Adaptation and closely related formulations. Safety Preserving Adaptation (SPA) denotes a family of methods that adapt a model, controller, or policy to new tasks, environments, or data distributions while preserving pre-existing safety properties. Across the literature, the term is used in multiple but structurally related senses. In robotics and control, SPA typically means wrapping adaptation inside a certified safety mechanism so that learning does not violate forward invariance of a safe set under uncertainty (Noren et al., 2019). In LLMs, SPA usually refers to constraining fine-tuning, low-rank adaptation, or continual domain adaptation so that downstream capability gains do not erode refusal behavior or other safety alignment properties (Zhang et al., 15 Jan 2026, Qi et al., 9 Apr 2026, Guo et al., 20 Apr 2026). In reinforcement learning, SPA appears as safe policy updating under certified parameter-space or barrier-based constraints, so that downstream policy improvement preserves previously verified safety guarantees (Anisimov et al., 10 Apr 2026, Xiao et al., 2023). Despite this diversity, the common principle is stable: adaptation is permitted only within a mechanism that either certifies safety-preserving updates, projects away safety-conflicting directions, freezes safety-critical components, or filters unsafe actions at execution time.
1. Conceptual scope and recurring formulation
SPA arises from a recurring failure mode: adaptation can improve utility while degrading safety. In the adaptive-control setting of robotic manipulation, standard adaptive control can produce inputs that are safe for the estimated model but unsafe for the true plant when parameters are uncertain (Noren et al., 2019). In LLM post-training, fine-tuning can erode refusal behavior even on benign data, and small amounts of adversarial or unsafe data can sharply increase harmful compliance (Zhang et al., 15 Jan 2026, Ao et al., 21 Jun 2025, Wang et al., 8 Mar 2026, Breneur et al., 28 May 2026). In continual RL, downstream policy updates can catastrophically forget source-task safety unless adaptation is restricted to a certified safe parameter region (Anisimov et al., 10 Apr 2026).
This suggests a unifying view: SPA addresses the tension between plasticity and stability. The model or controller must change enough to acquire new task competence, but not along directions, states, parameters, or actions that would break safety (Sun et al., 8 Feb 2026, Alssum et al., 10 Dec 2025). A plausible implication is that SPA is best understood not as a single algorithm, but as a design pattern for constrained adaptation.
Within that pattern, the literature repeatedly instantiates four mechanisms. One is runtime shielding, where the learned adaptive controller or policy proposes an action but a safety filter enforces a barrier or safe-set constraint before execution (Noren et al., 2019, Xiao et al., 2023). Another is update-space restriction, where optimization steps are projected away from safety-sensitive subspaces or frozen safety-critical parameters (Sun et al., 8 Feb 2026, Zhang et al., 15 Jan 2026, Qi et al., 9 Apr 2026). A third is post-hoc repair, where an already trained adapter is pruned, translated, or geometrically corrected to restore safety with minimal loss of utility (Ao et al., 21 Jun 2025, Arazzi et al., 6 May 2026, Breneur et al., 28 May 2026, Ao et al., 20 Aug 2025). A fourth is continual-learning style preservation, where safety alignment is treated as an earlier task that must not be forgotten during later fine-tuning (Sun et al., 8 Feb 2026, Alssum et al., 10 Dec 2025, Guo et al., 20 Apr 2026).
2. Safe-set and barrier-based SPA in adaptive control and robotics
A foundational control-theoretic formulation appears in "Safe Adaptation with Multiplicative Uncertainties Using Robust Safe Set Algorithm" (Noren et al., 2019). There, SPA is defined for a robot manipulation system with multiplicative or parametric uncertainty in control-affine dynamics
where the true control effectiveness belongs to an uncertainty set . The core idea is that an adaptive controller may estimate unknown parameters online, but every applied control must still satisfy a robust safety condition valid for all (Noren et al., 2019).
Safety is expressed through a scalar energy-like safety index augmented by a time-varying term , giving the composite index
The safe set is
The robust safe-set condition requires that whenever the boundary is active,
for some (Noren et al., 2019). Lemma 1 shows that this derivative condition yields forward invariance of the safe set, and Theorem 1 extends the guarantee to the true uncertain plant provided the true dynamics lie in the uncertainty family (Noren et al., 2019).
A central technical contribution is the robust admissible control set, denoted in the paper as something like 0, defined through the condition that there exists a control satisfying
1
when safety is active (Noren et al., 2019). The optimization-based synthesis chooses the minimum-effort safe action by aligning control with the worst-case Lie-derivative direction. In the minimum-effort lemma, the control takes the form
2
with smallest admissible
3
This yields the least corrective robustly safe input (Noren et al., 2019).
The adaptive component is a Slotine–Li controller,
4
with parameter update law
5
and
6
SPA is realized by placing the robust safe-set filter on top of this adaptive tracker, so that inaccurate transient parameter estimates cannot directly generate unsafe inputs (Noren et al., 2019).
Related robotics work preserves safety under adaptation in different forms. "Safe Deep Policy Adaptation" (Xiao et al., 2023) combines latent environment adaptation with a Control Barrier Function (CBF) safety filter. The deployment-time filter solves a quadratic program that keeps the executed action close to the adaptive policy while satisfying a discrete-time forward-invariance condition for a safe set 7 (Xiao et al., 2023). "Domain Adaptation for Outdoor Robot Traversability Estimation from RGB data with Safety-Preserving Loss" (Palazzo et al., 2020) uses a different notion of SPA: unsupervised domain adaptation plus an asymmetric regression loss
8
which penalizes dangerous overestimation of traversability more than conservative underestimation (Palazzo et al., 2020). This shows that, in robotics, SPA can refer either to formal set invariance during adaptation or to loss shaping that biases adaptation toward safer errors.
A further generalization appears in "Generalizations of Backup Control Barrier Functions: Expansion and Adaptation for Input-Bounded Safety-Critical Control" (Wijk et al., 19 Mar 2026). There, the controller used to expand the implicit safe set is decoupled from the verified backup controller that certifies safety. The parameterized expansion controller can then be adapted online in an augmented-state formulation, while forward invariance is preserved by the generalized backup-CBF construction (Wijk et al., 19 Mar 2026). This suggests a broader control-theoretic interpretation of SPA: separate the mechanism that improves performance from the mechanism that certifies recoverability.
3. Gradient-, parameter-, and token-level SPA in LLM fine-tuning
In LLMs, SPA typically targets the safety degradation that accompanies downstream fine-tuning. One of the clearest geometric formulations is "Understanding and Preserving Safety in Fine-Tuned LLMs" (Zhang et al., 15 Jan 2026), which introduces Safety-Preserving Fine-Tuning (SPF). The paper reports three empirical insights: safety gradients lie in a low-rank subspace; utility gradients span a broader space; and the dominant safety direction can be estimated from a single sample (Zhang et al., 15 Jan 2026). The update rule computes a utility gradient 9, a safety gradient 0, checks whether they conflict through 1, and if so projects the utility update away from the safety subspace: 2 The parameters are then updated with the projected gradient (Zhang et al., 15 Jan 2026). The paper states that this preserves downstream utility while bounding safety drift, and empirically restores ASR close to the initial aligned model on Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, and Qwen2.5-7B-Instruct (Zhang et al., 15 Jan 2026).
A closely related continual-learning interpretation is developed in "Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projection" (Sun et al., 8 Feb 2026). There, the alignment tax is defined as
3
and attributed to heterogeneous continual learning, where sequential SFT and DPO overwrite pretrained capabilities (Sun et al., 8 Feb 2026). OGPSA estimates a low-rank capability subspace from gradients on small reference datasets,
4
and projects the safety gradient onto the orthogonal complement: 5 This constrains safety alignment updates not to move along directions important for general capability (Sun et al., 8 Feb 2026).
Other LLM SPA methods localize safety at different granularities. "Towards Identification and Intervention of Safety-Critical Parameters in LLMs" (Qi et al., 9 Apr 2026) introduces Expected Safety Impact (ESI),
6
to identify safety-critical parameters. In SPA mode, the top-ranked safety-critical parameters are frozen during downstream task fine-tuning, while only non-critical parameters are updated (Qi et al., 9 Apr 2026). The paper reports that this limits the safety degradation of aligned LLMs within 7 after a 8-iteration instruction fine-tuning on different tasks (Qi et al., 9 Apr 2026).
At a finer granularity, "Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning" (Wang et al., 8 Mar 2026) argues that refusal behavior is concentrated in a small set of safety-related output tokens. It identifies a top-9 safety token set 0 by discrepancy between aligned and base next-token distributions on harmful prompts and regularizes only the safety-token distribution with a weighted KL term: 1 The method also calibrates the safety reference using a mixture of full-context and response-only reference logits to reduce harmful-prefix contamination (Wang et al., 8 Mar 2026). This is a token-level SPA mechanism: preserve only the safety-relevant slice of the output distribution and leave the rest free for task adaptation.
A different structural decomposition appears in "A Guardrail for Safety Preservation: When Safety-Sensitive Subspace Meets Harmful-Resistant Null-Space" (Zhang et al., 16 Oct 2025). GuardSpace computes a covariance-preconditioned SVD of 2 using harmful-prompt activations, freezes the large-singular-value safety-sensitive subspace, initializes low-rank adapters from the safety-irrelevant tail,
3
and constrains the effective update through a null-space projector 4 derived from the harmful-prompt covariance (Zhang et al., 16 Oct 2025). The key invariance relation
5
is used to preserve the original behavior on harmful prompts throughout fine-tuning (Zhang et al., 16 Oct 2025).
These methods differ in where they locate safety—gradient directions, parameter subsets, token distributions, or harmful-input null spaces—but they share the same adaptation logic: preserve a safety-relevant structure and route learning elsewhere.
4. Low-rank adaptation, post-hoc repair, and adapter-space SPA
A large subliterature studies SPA specifically for LoRA and related parameter-efficient fine-tuning. "SaLoRA: Safety-Alignment Preserved Low-Rank Adaptation" (Li et al., 3 Jan 2025) is an early formulation that inserts a fixed safety module
6
to project trainable low-rank updates away from a harmful-feature subspace estimated from safety data (Li et al., 3 Jan 2025). It pairs this with task-specific initialization of the adapters using downstream task features. The reparameterized layer is
7
The paper reports that SaLoRA sharply reduces harmful rate after Alpaca fine-tuning relative to LoRA, DoRA, and PiSSA while maintaining or improving utility on commonsense reasoning benchmarks (Li et al., 3 Jan 2025).
Pruning-based variants remove parts of a trained LoRA update deemed most responsible for safety degradation. "Safe Pruning LoRA: Robust Distance-Guided Pruning for Safety Alignment in Adaptation of LLMs" (Ao et al., 21 Jun 2025) introduces Empirical-DIEM (E-DIEM) based on the discrepancy between a LoRA update 8 and its projection into an aligned subspace constructed from an aligned/unaligned model pair. Layers with large discrepancy are pruned entirely according to
9
(Ao et al., 21 Jun 2025). The paper reports strong ASR reductions on Dialog Summary + PureBad, Alpaca + PureBad, and pure benign Alpaca settings, often with maintained or improved utility and reduced inference time (Ao et al., 21 Jun 2025).
"S3LoRA: Safe Spectral Sharpness-Guided Pruning in Adaptation of Agent Planner" (Ao et al., 20 Aug 2025) removes a different class of risky layers. It analyzes only the LoRA update 0, performs Magnitude-Aware Spherically Normalized SVD, and computes the Spectral Sharpness Index
1
Layers with top-2 SSI are pruned post-hoc, without requiring base/aligned checkpoint pairs (Ao et al., 20 Aug 2025). This suggests a more deployment-oriented SPA criterion: structurally sharp and concentrated updates are treated as potential safety risks even in the absence of a reference aligned subspace.
Another direction repairs rather than deletes unsafe update components. "CSULoRA: Closest Safe Update Low-Rank Adaptation" (Breneur et al., 28 May 2026) estimates a safety-aligned subspace from the displacement between a safety-aligned checkpoint and its base checkpoint,
3
builds double-sided projectors 4, decomposes each LoRA update into four orthogonal blocks 5, 6, 7, and 8, and solves a closed-form penalized minimum-change problem (Breneur et al., 28 May 2026). The corrected update is
9
with penalties set from relative block energy (Breneur et al., 28 May 2026). This is a geometric SPA: preserve the fully aligned part exactly and attenuate the rest rather than removing it.
The same post-hoc philosophy is extended beyond linear correction in "You Snooze, You Lose: Automatic Safety Alignment Restoration through Neural Weight Translation" (Arazzi et al., 6 May 2026). NeWTral learns a non-linear parameter-space map from unsafe adapters to safe aligned adapters,
0
using unsafe-to-safe adapter pairs and, in its main variant, a Mixture of Experts routing mechanism that interpolates between a safety-aggressive expert and a utility-preserving surgical expert (Arazzi et al., 6 May 2026). The paper reports average ASR reduction from about 1 in unsafe experts to about 2 with the MoE translator while maintaining around 3 average knowledge fidelity (Arazzi et al., 6 May 2026).
The cumulative picture from these LoRA papers is that SPA in adapter space can be preventive, as in SaLoRA; selective, as in SPLoRA and S3LoRA; corrective, as in CSULoRA; or translational, as in NeWTral. This suggests that the adapter itself is treated as the main locus of safety erosion and therefore as the main intervention target.
5. Continual adaptation, forgetting, and multi-stage safety preservation
Several works explicitly frame SPA as a continual-learning problem. "Unforgotten Safety: Preserving Safety Alignment of LLMs with Continual Learning" (Alssum et al., 10 Dec 2025) formalizes a two-stage pipeline: safety alignment first, user adaptation second. The goal is to keep
4
after downstream fine-tuning (Alssum et al., 10 Dec 2025). The paper evaluates regularization-based methods such as EWC and LwF, memory-based methods such as A-GEM and DER, and a merging method MagMax. Among these, DER is reported as the strongest overall method, drastically reducing ASR relative to standard fine-tuning while maintaining utility across GSM8K, SST2, and Code, in both benign and poisoned settings (Alssum et al., 10 Dec 2025). The paper’s central claim is that safety compromise is catastrophic forgetting of alignment rather than merely a one-off optimization artifact.
The same theme is sharpened in sequential multi-domain adaptation by "SafeAnchor: Preventing Cumulative Safety Erosion in Continual Domain Adaptation of LLMs" (Guo et al., 20 Apr 2026). SafeAnchor has three components: Safety Subspace Identification (SSI) via empirical Fisher eigendecomposition in LoRA parameter space; Orthogonal Safety-Constrained Adaptation (OSCA), which projects each domain-task gradient away from the safety subspace,
5
and Cumulative Safety Monitoring (CSM), which triggers corrective replay if the refusal rate falls below a threshold
6
(Guo et al., 20 Apr 2026). Across a Medical 7 Legal 8 Code pipeline on Llama-2-7B-Chat and Mistral-7B-Instruct, the paper reports retention of 9 and 0 of original safety alignment, respectively, while staying within about 1 points of unconstrained LoRA on domain tasks (Guo et al., 20 Apr 2026).
In continual RL, "SafeAdapt: Provably Safe Policy Updates in Deep Reinforcement Learning" (Anisimov et al., 10 Apr 2026) provides a parameter-space certificate rather than a replay-based or gradient-projection heuristic. It defines a Rashomon set
2
a center-symmetric orthotope in policy parameter space such that every policy inside it is certified safe on the source task (Anisimov et al., 10 Apr 2026). Downstream policy updates are projected back into this region by element-wise clipping after each gradient step. The paper proves that if the certified lower bound on the surrogate critical-state safety rate exceeds a threshold, then source-task safety is preserved for every iteration of projected adaptation (Anisimov et al., 10 Apr 2026). This is a stronger notion of SPA than most LLM work: safety is preserved a priori by parameter-space certification rather than statistically or geometrically encouraged.
This family of works indicates that SPA becomes more demanding when adaptation is repeated. A plausible implication is that one-shot safety-preserving fine-tuning methods may not suffice in long adaptation chains, because safety-relevant subspaces can drift and indirect erosion can accumulate across stages.
6. Self-triggered and preference-based uses of the term in LLM alignment
The term SPA is not used uniformly across all LLM alignment papers. In some works it denotes a broad category of safety-preserving adaptation rather than the name of the specific method. "Adaptive and Explicit safe: Triggering Latent Safety Awareness in Large Reasoning Models" (Miao et al., 15 Jun 2026) presents Safe Trigger, which the paper describes as a safety-preserving adaptation method for large reasoning models. Its key observation is Latent Safety Awareness: when the model reviews the original risky query together with its own reasoning trace, its Risk Identification Success Rate (RISR) is much higher than its direct-attack ASR, ranging from 3 to 4 (Miao et al., 15 Jun 2026). The method uses Safe Trigger SFT with structured tags <safe> ... </safe> and Safe Trigger DPO on self-generated data. Across models, the aggregated results show Base / ST-S / ST-D harmful rates of 5, 6, and 7, jailbreak rates of 8, 9, and 0, and nearly unchanged general performance and over-refusal (Miao et al., 15 Jun 2026). Here, SPA denotes selective activation of a safety-analysis module only on risky inputs, rather than geometric protection of safety parameters.
Another distinct usage appears in "SPA: Achieving Consensus in LLM Alignment via Self-Priority Optimization" (Huang et al., 9 Nov 2025), where SPA stands specifically for Self-Priority Alignment rather than Safety Preserving Adaptation. The method imposes a lexicographic trustworthy-before-helpful ordering,
1
constructs lexicographically ordered preference pairs from self-generated samples, and optimizes an uncertainty-weighted SimPO-style loss (Huang et al., 9 Nov 2025). Although the acronym overlaps, this is a distinct concept. The paper reports improved harmlessness or honesty together with improved helpfulness on Llama-3.1-8B-Instruct and Mistral-7B-Instruct across SafeRLHF, WildGuard, and HoneSet (Huang et al., 9 Nov 2025).
This lexical ambiguity matters. In the broader literature, “SPA” may refer to a method category—adapt safely while preserving existing behavior—or to the specific alignment paradigm of Self-Priority Alignment. A common misconception is that every “SPA” paper studies the same mechanism. The data instead indicate that the acronym spans at least two conceptually different traditions: safety-preserving model or controller adaptation, and trustworthy-before-helpful preference optimization.
7. Comparative structure, assumptions, and limitations
The SPA literature varies substantially in what safety means, what is preserved, and what guarantees are available.
| Regime | Preservation target | Main mechanism |
|---|---|---|
| Adaptive control / robotics | Forward invariance of safe set | Robust safe set, CBF, backup-set certification |
| LLM fine-tuning | Refusal behavior / safety alignment | Gradient projection, parameter freezing, token constraints |
| LoRA post-hoc repair | Safe behavior of adapted checkpoint | Pruning, subspace correction, parameter translation |
| Continual RL | Source-task certified safety | Projection into certified Rashomon set |
Control-theoretic SPA generally offers the strongest guarantees. The robust safe-set method in (Noren et al., 2019), the CBF-shielded SafeDPA in (Xiao et al., 2023), the Rashomon-set projection of SafeAdapt (Anisimov et al., 10 Apr 2026), and the generalized backup-CBF adaptation framework (Wijk et al., 19 Mar 2026) all provide formal forward-invariance or certified-safety statements under explicit assumptions. Those assumptions include bounded model error, Lipschitz continuity, nonempty admissible safe-control sets, or correct uncertainty families. The guarantees are therefore rigorous but model-dependent.
LLM SPA papers more often provide empirical safety-utility trade-offs with lighter theory. SPF proves utility convergence with bounded safety drift under a low-rank projection model (Zhang et al., 15 Jan 2026). OGPSA proves steepest feasible descent under first-order capability-preservation constraints (Sun et al., 8 Feb 2026). GuardSpace provides a harmful-input invariance argument through its null-space projector (Zhang et al., 16 Oct 2025). Yet most LLM methods explicitly do not claim formal absolute safety. CSULoRA, for example, states that its safety subspace is only a proxy and “it is not a formal guarantee of safety” (Breneur et al., 28 May 2026). NeWTral likewise reduces ASR sharply but does not fully eliminate unsafe outputs (Arazzi et al., 6 May 2026).
The preservation target also differs. Some methods preserve capability during safety alignment, as in OGPSA (Sun et al., 8 Feb 2026). Others preserve safety during capability tuning, as in SPF (Zhang et al., 15 Jan 2026), ESI-based SPA (Qi et al., 9 Apr 2026), PACT (Wang et al., 8 Mar 2026), or GuardSpace (Zhang et al., 16 Oct 2025). LoRA repair methods preserve both the added specialization and the original safety alignment, but only relative to the chosen geometric or reference-model proxy (Ao et al., 21 Jun 2025, Breneur et al., 28 May 2026, Arazzi et al., 6 May 2026).
Several limitations recur across papers. Many LLM methods depend on a reference aligned/base model pair or a safety calibration set, as in SPLoRA (Ao et al., 21 Jun 2025), SaLoRA (Li et al., 3 Jan 2025), CSULoRA (Breneur et al., 28 May 2026), and GuardSpace (Zhang et al., 16 Oct 2025). Others assume that safety is low-rank or localized, as in SPF (Zhang et al., 15 Jan 2026), SafeAnchor (Guo et al., 20 Apr 2026), and ESI-based SPA (Qi et al., 9 Apr 2026). Post-hoc methods can lose some utility when unsafe and useful directions overlap (Breneur et al., 28 May 2026). Continual methods must manage evolving safety geometry and cumulative drift (Guo et al., 20 Apr 2026). These recurring assumptions suggest that the central open question is not whether safety can be preserved at all, but how robustly one can identify the correct safety structure under varying architectures, tasks, and adaptation regimes.
Overall, the literature portrays Safety Preserving Adaptation as a general solution strategy for a persistent systems problem: adaptation tends to move a model or controller into regions that improve immediate utility but destabilize previously aligned safe behavior. SPA methods differ in whether they act on actions, gradients, parameters, tokens, adapters, or certified parameter regions, but they converge on the same principle: learning is acceptable only when safety-preserving structure remains invariant, recoverable, or explicitly protected.