Asymmetric Safety Alignment

• Asymmetric safety alignment is a framework where safety measures are unevenly encoded across low-dimensional subspaces, attention heads, and layers, leading to dominant refusal directions and auxiliary components.
• It highlights how safety tuning concentrates in specific model regions, such as singular vectors, token positions, and specialized experts, affecting both safety performance and reasoning capabilities.
• The concept informs the design of mitigation strategies that redistribute safety signals and enhance fine-tuning robustness while preserving overall model utility.

Searching arXiv for papers on asymmetric safety alignment and closely related alignment geometry/mechanistic work. Asymmetric safety alignment is a family of observations, mechanisms, and design principles in which safety behavior is not uniformly encoded, not activated symmetrically across contexts, and not neutral with respect to other capabilities. Across recent work, the term refers to several recurring patterns: safety may concentrate in a low-dimensional residual subspace with one dominant refusal direction and multiple auxiliary directions (Pan et al., 13 Feb 2025); in a low-dimensional, high-curvature subspace that is dynamically fragile under benign fine-tuning (Springer et al., 17 Feb 2026); in a small number of attention heads (Huang et al., 27 Aug 2025) or experts (Zhang et al., 28 May 2026); at particular generation depths (Zhang et al., 20 Oct 2025, Lyu et al., 2 Jun 2026); in particular tasks such as QA rather than summarization (Fu et al., 2023); or in particular token positions such as the middle of a diffusion-model response rather than the prefix (Xie et al., 17 Aug 2025). A plausible implication is that “alignment” is often better described as a structured, uneven control system than as a single global property of a model.

1. Conceptual definitions and recurring asymmetries

One influential mechanistic definition states that safety fine-tuning does not just add one linear refusal axis. Instead, it induces a low-rank, structured set of directions in activation space: a dominant component for refusing harmful or jailbreak prompts and non-dominant orthogonal components for “indirect” safety-relevant features such as hypothetical narrative, role-playing / assistant persona triggers, and patterns like “Sure, I’m happy to help” in PAIR-style attacks (Pan et al., 13 Feb 2025). In that usage, asymmetry means that one direction is dominant and behavior-defining, while other directions are auxiliary, context-dependent, and can either support or undermine refusal.

A second line of work uses the term geometrically. There, safety-critical behavior is concentrated in a low-dimensional, high-curvature subspace, and the usual high-dimensional orthogonality intuition is said to offer false reassurance because curvature can rotate an apparently harmless fine-tuning trajectory into alignment-sensitive regions (Springer et al., 17 Feb 2026). In this formulation, asymmetry is dynamic: initial gradients may avoid the safety subspace, while second-order effects later steer optimization into it.

Other papers define the asymmetry operationally. In large reasoning models, post hoc safety tuning can restore refusal behavior but reduce reasoning capability, a trade-off called Safety Tax (Huang et al., 1 Mar 2025). In task-conditioned NLP, safety alignment is task-dependent, not uniform: summarization may be weakly aligned while translation or QA is comparatively robust, and the weaker task can act as an in-context attack on the stronger one (Fu et al., 2023). In diffusion LLMs, the defender and attacker have different leverage over token positions, because the defender can align the middle tokens while the attacker is constrained by a practical sequential generation bias (Xie et al., 17 Aug 2025). In context-invariance work, the asymmetry lies in the trustworthiness of supervision: verifiable prompts are treated as anchors, while open-ended variants are judged by noisier proxies (Wang et al., 20 May 2026).

Taken together, these definitions suggest that “asymmetric safety alignment” is not a single theorem or benchmark label. It is an umbrella description for settings in which safety is localized, depth-dependent, context-sensitive, or coupled unevenly to utility and optimization.

2. Internal representations: subspaces, heads, experts, and layers

The most explicit representation-space account is the safety residual space. Let $\mathcal{T}$ denote the learned transformation from unaligned to aligned representations. The paper defines the safety residual space as the best affine approximation

$$V_\mathcal{T} \;=\; \underset{\hat{\mathbf W},\hat{\mathbf b}}{\arg\min}\; \mathbb{E}_{\mathbf x\sim \mathcal X_u} \left\| \mathcal T(\mathbf x)-(\hat{\mathbf W}\mathbf x+\hat{\mathbf b}) \right\|^2,$$

then analyzes $\mathbf W-\mathbf I$ with SVD and interprets the right singular vectors as orthogonal components (Pan et al., 13 Feb 2025). The first SVD component, LN-C1, is described as the main refusal axis, while secondary components identified with PLRP correspond to semantically distinct jailbreak-related features. The paper highlights L14-C2 for “Imagine”, “fictional”, “hypothetical”; L14-C5 for “Chat”, “G”, “PT”; and L14-C6 for “Sure, I’m happy to help” together with “Imagine” (Pan et al., 13 Feb 2025). This framing treats alignment features as learned, interpretable directions rather than arbitrary geometric artifacts.

A closely related architectural observation is that existing safety mechanisms can depend on a limited subset of attention heads. Using RDSHA, safety-critical heads are scored by projecting head outputs onto a refusal direction,

$$s_h^{(p)} = \frac{|\mathbf{O}_h^{(p)} \cdot \mathbf{r}|}{\|\mathbf{r}\|},$$

and head ablation shows that safety can collapse after removing only a relatively small number of top-ranked heads (Huang et al., 27 Aug 2025). The paper reports recurrent concentration in heads such as Head12.0, Head12.1, Head16.0, and Head16.30 in Llama-2, mostly in the middle to upper layers. Its interpretation is that safety is strong in ordinary settings but weak against targeted adversarial prompts because the relevant behavior is over-concentrated.

Mixture-of-Experts models show a different but related asymmetry. The MoE study argues that routing is largely topic-driven, while safety behavior can be altered by modifying a small subset of experts with little change to the intrinsic routing path (Zhang et al., 28 May 2026). For expert $i$ in layer $l$, it defines an average accumulated activation $\mathcal{A}(l,i;\mathcal{D})$ and a Safety Sensitivity Score

$$S_{l,i} = \mathcal{A}(l, i; \mathcal{D}_{\text{harm}}) - \lambda \cdot \mathcal{A}(l, i; \mathcal{D}_{\text{norm}}),$$

with $\lambda = 0.5$, then tunes only the top-$\mathcal{K}$ experts (Zhang et al., 28 May 2026). The reported result is that harmful behavior can be changed while routing remains largely preserved, which implies that the router is not the sole or primary safety control surface.

An analogous unevenness appears in vision-LLMs. ICET shows that safety is not evenly distributed across image encoder layers: replacing the final-layer vision representation with earlier or middle-layer features can substantially increase harmful outputs (Bachu et al., 2024). The paper reports, for example, that LLaVA-1.5 has AASR $$V_\mathcal{T} \;=\; \underset{\hat{\mathbf W},\hat{\mathbf b}}{\arg\min}\; \mathbb{E}_{\mathbf x\sim \mathcal X_u} \left\| \mathcal T(\mathbf x)-(\hat{\mathbf W}\mathbf x+\hat{\mathbf b}) \right\|^2,$$0 in early layers, $$V_\mathcal{T} \;=\; \underset{\hat{\mathbf W},\hat{\mathbf b}}{\arg\min}\; \mathbb{E}_{\mathbf x\sim \mathcal X_u} \left\| \mathcal T(\mathbf x)-(\hat{\mathbf W}\mathbf x+\hat{\mathbf b}) \right\|^2,$$1 in middle layers, and $$V_\mathcal{T} \;=\; \underset{\hat{\mathbf W},\hat{\mathbf b}}{\arg\min}\; \mathbb{E}_{\mathbf x\sim \mathcal X_u} \left\| \mathcal T(\mathbf x)-(\hat{\mathbf W}\mathbf x+\hat{\mathbf b}) \right\|^2,$$2 in late layers, while Llama 3.2 shows MASR $$V_\mathcal{T} \;=\; \underset{\hat{\mathbf W},\hat{\mathbf b}}{\arg\min}\; \mathbb{E}_{\mathbf x\sim \mathcal X_u} \left\| \mathcal T(\mathbf x)-(\hat{\mathbf W}\mathbf x+\hat{\mathbf b}) \right\|^2,$$3 in early layers but near-zero in middle and late layers (Bachu et al., 2024). A plausible implication is that “where alignment lives” can be layer-specific even within a single subsystem.

3. Training dynamics, curvature, and collapse under adaptation

The geometric fragility account formalizes aligned behavior in terms of skill utilities

$$V_\mathcal{T} \;=\; \underset{\hat{\mathbf W},\hat{\mathbf b}}{\arg\min}\; \mathbb{E}_{\mathbf x\sim \mathcal X_u} \left\| \mathcal T(\mathbf x)-(\hat{\mathbf W}\mathbf x+\hat{\mathbf b}) \right\|^2,$$4

alignment loss $$V_\mathcal{T} \;=\; \underset{\hat{\mathbf W},\hat{\mathbf b}}{\arg\min}\; \mathbb{E}_{\mathbf x\sim \mathcal X_u} \left\| \mathcal T(\mathbf x)-(\hat{\mathbf W}\mathbf x+\hat{\mathbf b}) \right\|^2,$$5, and the Fisher Information Matrix $$V_\mathcal{T} \;=\; \underset{\hat{\mathbf W},\hat{\mathbf b}}{\arg\min}\; \mathbb{E}_{\mathbf x\sim \mathcal X_u} \left\| \mathcal T(\mathbf x)-(\hat{\mathbf W}\mathbf x+\hat{\mathbf b}) \right\|^2,$$6 (Springer et al., 17 Feb 2026). Under the paper’s “skill optimality” assumption, the degradation is exactly an expected KL divergence between the aligned base policy and the fine-tuned policy on the safety distribution. Locally,

$$V_\mathcal{T} \;=\; \underset{\hat{\mathbf W},\hat{\mathbf b}}{\arg\min}\; \mathbb{E}_{\mathbf x\sim \mathcal X_u} \left\| \mathcal T(\mathbf x)-(\hat{\mathbf W}\mathbf x+\hat{\mathbf b}) \right\|^2,$$7

which motivates the alignment sensitivity subspace $$V_\mathcal{T} \;=\; \underset{\hat{\mathbf W},\hat{\mathbf b}}{\arg\min}\; \mathbb{E}_{\mathbf x\sim \mathcal X_u} \left\| \mathcal T(\mathbf x)-(\hat{\mathbf W}\mathbf x+\hat{\mathbf b}) \right\|^2,$$8 as the span of the leading eigenvectors of $$V_\mathcal{T} \;=\; \underset{\hat{\mathbf W},\hat{\mathbf b}}{\arg\min}\; \mathbb{E}_{\mathbf x\sim \mathcal X_u} \left\| \mathcal T(\mathbf x)-(\hat{\mathbf W}\mathbf x+\hat{\mathbf b}) \right\|^2,$$9 (Springer et al., 17 Feb 2026).

The critical claim is that the standard orthogonality story fails because fine-tuning is not a one-shot random perturbation in a flat space. With gradient flow

$\mathbf W-\mathbf I$0

the expansion

$\mathbf W-\mathbf I$1

introduces a second-order acceleration term that can point into $\mathbf W-\mathbf I$2 even when the initial gradient has little projection onto it (Springer et al., 17 Feb 2026). This is formalized through the Alignment Instability Condition (AIC): Low-Rank Sensitivity, Initial Orthogonality, and Curvature Coupling. The resulting corollary states that for sufficiently small $\mathbf W-\mathbf I$3,

$\mathbf W-\mathbf I$4

described as a quartic onset of alignment loss (Springer et al., 17 Feb 2026).

A separate but complementary dynamics account links shallow safety alignment to autoregressive consistency. That paper assumes that after some critical prefix length $\mathbf W-\mathbf I$5, the next token is almost deterministic under the base model, and then shows that the softmax Jacobian and token-level gradients are suppressed at late positions (Lyu et al., 2 Jun 2026). The consequence is that SFT or DPO updates concentrate on the first few refusal tokens, leaving the later part of the trajectory close to the base model. The paper identifies this as a mechanistic reason that refusal is often strong at the beginning of an answer but weak once a harmful continuation state is established.

These two accounts differ in object—parameter-space curvature versus token-level autoregressive dynamics—but they converge on a shared conclusion: safety alignment is often localized in a way that makes it vulnerable to later perturbations, whether by benign fine-tuning or by attacks that enter the trajectory after the initial refusal region.

4. Behavioral asymmetry across reasoning, utility, and capability

In large reasoning models, the asymmetry is framed as a safety–reasoning trade-off. The Safety Tax paper studies a two-stage pipeline,

$\mathbf W-\mathbf I$6

and reports that safety alignment can restore safety capability while degrading reasoning capability (Huang et al., 1 Mar 2025). On s1.1-32B, the harmful score rises from 16.70 in the base model to 60.40 after reasoning training, then drops to 0.80 with DirectRefusal and 30.80 with SafeChain. Over the same transitions, average reasoning accuracy changes from 63.40 in the LRM to 32.49 with DirectRefusal and 56.31 with SafeChain (Huang et al., 1 Mar 2025). The paper interprets this as an asymmetric intervention: safety gains are large, but the side effects on reasoning are not neutral.

A more surgical account argues that safety–utility conflicts are not global. CAST constructs a head-level conflict score

$\mathbf W-\mathbf I$7

where $\mathbf W-\mathbf I$8 is gradient opposition between safety and utility objectives and $\mathbf W-\mathbf I$9 is a functional sensitivity term derived from head ablation (Cai et al., 7 Jan 2026). The paper reports that utility loss mainly comes from updating a small group of “high-conflict” heads, and that skipping those heads during training reduces capability degradation without compromising safety. On Llama, it reports that CAST improves MMLU by about +9.45% while maintaining equivalent safety (Cai et al., 7 Jan 2026). This suggests that part of the apparent “tax” is localization-dependent rather than uniformly distributed through the network.

A distinct approach to the same trade-off is ASCL, which argues that safety rules and reasoning are often too entangled in context-distilled CoT training (Wang et al., 14 Feb 2026). The framework treats safety alignment as a multi-turn tool-use process in which the model chooses whether to consult external safety policies, and then uses IFPO to correct RL’s tendency to over-prefer rule consultation. On Qwen3-8B, the paper reports Safety avg 97.20 and Over-refusal avg 14.98 for BC, versus 98.98 and 4.33 for IFPO; similar reductions in over-refusal are reported for 4B and 14B variants (Wang et al., 14 Feb 2026). The central claim is not that safety disappears, but that selective rule retrieval preserves the intended asymmetry: strong intervention for harmful cases without universal conservatism.

Reasoning-based alignment pursues a related goal by changing the internal decision process rather than merely adding refusal data. SaRO combines Reasoning-style Warmup with Safety-oriented Reasoning Process Optimization, and the preference ranking explicitly values reasoning chains in which safety-oriented reflection occurs earlier (Mou et al., 13 Apr 2025). On LLaMA3-8B, RW + SRPO reports AutoDAN 22.57, PAIR 27.81, and XSTest ERR 7.39, compared with 50.61, 69.55, and 8.91 for SafetySFT + DPO; on Qwen2-7B, RW + SRPO reports AutoDAN 11.67, PAIR 23.20, and XSTest ERR 5.22 (Mou et al., 13 Apr 2025). This suggests that some instances of asymmetry can be improved by teaching the model to reason about policy violations rather than only memorizing refusal templates.

5. Context, task, position, and depth as attack surfaces

Several papers show that safety is asymmetric across contexts even when the underlying harmful intent is unchanged. In NLP task chains, summarization is identified as the weakest-aligned task and can function as an in-context attack on translation or QA (Fu et al., 2023). For Llama2-7B, the paper reports that Translation on Most-Harmful goes from 7.10% to 16.30% process rate under summarization attack, and QA on Full rises from 12.68% to 26.08% (Fu et al., 2023). The key claim is that weak tasks can attack strong tasks, but not vice versa.

Prompt context can also matter because not all variants are supervised equally well. AIR addresses this by privileging a verifiable anchor context with a stop-gradient target,

$$s_h^{(p)} = \frac{|\mathbf{O}_h^{(p)} \cdot \mathbf{r}|}{\|\mathbf{r}\|},$$0

thereby regularizing only open-ended variants toward the anchor (Wang et al., 20 May 2026). The paper reports 12.71% in-distribution group accuracy gain and 33.49% out-of-distribution consistency gain, while also showing that the symmetric V-REx baseline can sharply reduce ID group accuracy on Safety and Moral tasks (Wang et al., 20 May 2026). The asymmetry here lies in signal quality rather than model internals.

Generation depth is another major asymmetry. Any-Depth Alignment argues that aligned models are strong at $$s_h^{(p)} = \frac{|\mathbf{O}_h^{(p)} \cdot \mathbf{r}|}{\|\mathbf{r}\|},$$1, immediately after the prompt, but weak once a harmful continuation is underway (Zhang et al., 20 Oct 2025). The paper attributes this to strong alignment priors in assistant-header “Safety Tokens,” which can be reintroduced mid-stream. Across model families, a logistic probe on assistant-header hidden states reaches >99.5% validation accuracy for harmful-versus-benign continuations, and ADA (LP) maintains 99.7%–100.0% refusal at $$s_h^{(p)} = \frac{|\mathbf{O}_h^{(p)} \cdot \mathbf{r}|}{\|\mathbf{r}\|},$$2 on harmful prefill attacks where base models are near zero (Zhang et al., 20 Oct 2025). A related account, When Autoregressive Consistency Hurts Safety Alignment, argues that a short harmful insertion can redirect a generation even after a long refusal prefix, because continuation consistency stabilizes the new harmful branch (Lyu et al., 2 Jun 2026).

Diffusion LLMs move the positional asymmetry away from the prefix. MOSA argues that the critical positions are 20 to 60, not the first few tokens, and reports that forcing an affirmative phrase at the first token succeeds at about 33%, whereas forcing a procedural phrase at a middle token succeeds at only 2% (Xie et al., 17 Aug 2025). On AdvBench, MOSA reduces TAP from 79.1 / 77.2 in the original model and 29.6 / 28.1 after initial alignment to 4.5 / 3.7; comparable reductions are reported across eight attacks and two benchmarks (Xie et al., 17 Aug 2025). The defender–attacker asymmetry is explicit: the defender can target the middle, while the attacker is constrained by practical sequential generation behavior.

Trigger-based vulnerabilities in autoregressive models occupy a similar conceptual space. In the safety residual-space study, removing trigger tokens changes projection onto the dominant refusal component and can bypass learned safety capability (Pan et al., 13 Feb 2025). The paper states that Trigger Removal remains effective much longer than other jailbreaks on Llama 3.1 8B, implying that some safety behavior is conditioned on shortcut correlations to particular lexical patterns rather than only on harmful intent (Pan et al., 13 Feb 2025).

6. Mitigation strategies and broader implications

The mitigation literature largely consists of methods that either redistribute safety, localize optimization more carefully, or anchor safety to more reliable signals. AHD redistributes refusal behavior across many attention heads by training with head-level dropout and a joint objective over harmful and benign data; after training, harmfulness under head ablation rises much more slowly, and jailbreak harmfulness is driven near zero in several model families while OR-Bench refusal rates remain similar to baseline (Huang et al., 27 Aug 2025). L-PPO generalizes safety over image encoder depth rather than only the default final layer, explicitly addressing cross-layer OOD in VLMs (Bachu et al., 2024). Staged-Competence reorders DPO preference learning by difficulty and updates the reference model between stages; averaged across three model families, it reduces OOD harmful response rates by 16% and jailbreak attack success rates by 20%, while preserving general capabilities with near-zero over-refusal (Kumar et al., 25 May 2026).

Some proposals intervene at inference time rather than during post-training. SafeAligner uses disagreement between a Sentinel Model and an Intruder Model to bias decoding toward beneficial tokens and away from harmful ones: $$s_h^{(p)} = \frac{|\mathbf{O}_h^{(p)} \cdot \mathbf{r}|}{\|\mathbf{r}\|},$$3 It reports average safety scores such as 4.92 on Llama-3-8B-Instruct, 4.24 on Qwen1.5-7B-Chat, and 4.82 on Phi-3-small-8k-instruct, with modest general-score changes and low ATGR overheads of 1.06×, 1.18×, and 1.07× respectively (Huang et al., 2024). ADA, by contrast, accesses latent safety signals already present in assistant-header states and stops or redirects generation at arbitrary depth without changing model parameters (Zhang et al., 20 Oct 2025).

A different class of work questions whether refusal rate is the right abstraction at all. In autonomous security agents, aligned and less-restricted models behave differently at the system level, and the effects are not cleanly reducible to single-turn refusals (David et al., 19 May 2026). On Gemma 4 31B, the aligned condition has 0.7% success and the less-restricted derivative 14.0%, with grounding 3.27 versus 3.91, yet both have 0.0% refusal and 0.0% suppressed-action rates (David et al., 19 May 2026). The authors therefore argue that safety alignment effects should be separated into refusal, unsafe action, tool reliability, grounding, and deterministic task success.

A broader implication of this literature is that safety alignment is repeatedly observed to be directionally uneven and compositionally fragile. Where safety is concentrated—in a subspace, head set, expert subset, token region, or prompt context—often predicts how it fails. Another plausible implication is methodological: evaluations limited to direct harmful prompts and refusal rates can miss the mechanisms that actually govern safety robustness, including context contamination, curvature coupling, harmful continuation states, and system-level tool behavior.

The most general interpretation appears in the contrast between “symmetric” and “asymmetric” objectives. AIR explicitly rejects symmetric invariance penalties because they can lower performance on reliable contexts instead of improving unreliable ones (Wang et al., 20 May 2026). CAST rejects uniform network-wide updates because conflicts are localized (Cai et al., 7 Jan 2026). ASCL rejects universal rule memorization because safety should be activated only when warranted (Wang et al., 14 Feb 2026). Across these works, asymmetric safety alignment is less a single method than a research stance: safety should not be assumed to be globally encoded, uniformly trainable, or equivalently supervised across all contexts.