Conflict-Aware Sparse Tuning (CAST)

• The paper introduces CAST, a framework that selectively updates low-conflict attention heads to reduce utility loss during safety alignment.
• CAST leverages a pre-alignment conflict map computed from gradient opposition and functional sensitivity to identify high-conflict heads.
• Experimental results on models like Llama show that CAST’s surgical updates recover over 9 MMLU points while maintaining stringent safety metrics.

Searching arXiv for the specified paper to ground the article in current metadata and citation details. Conflict-Aware Sparse Tuning (CAST) is a framework for safety alignment in LLMs that combines head-level conflict diagnosis with sparse fine-tuning. It is introduced in "Safety-Utility Conflicts Are Not Global: Surgical Alignment via Head-Level Diagnosis" (Cai et al., 7 Jan 2026), which argues that safety alignment in LLMs inherently presents a multi-objective optimization conflict, often accompanied by an unintended degradation of general capabilities. CAST is designed around the claim that this conflict is not uniformly distributed across Transformer parameters: instead, the functional sensitivity and degree of conflict vary substantially across different attention heads. The method therefore constructs a pre-alignment conflict map and uses it to guide selective parameter updates, with the stated goal of reducing utility loss without compromising safety (Cai et al., 7 Jan 2026).

1. Conceptual basis and problem setting

CAST is motivated by a limitation attributed to existing mitigation strategies: they typically rely on global gradient geometry to resolve safety-utility conflicts, but they overlook Modular Heterogeneity within Transformers. In the formulation of the paper, global approaches impose uniform update rules across all parameters, often resulting in suboptimal trade-offs by indiscriminately updating utility sensitive heads that exhibit intense gradient conflicts (Cai et al., 7 Jan 2026).

Within this framework, the central empirical claim is that alignment conflicts in LLMs are not uniformly distributed. The reported finding is that the drop in general capabilities mainly comes from updating a small group of "high-conflict" heads. CAST operationalizes this observation by identifying those heads before alignment and excluding them from updates during safety tuning. This suggests a shift from parameter-global alignment to what the paper characterizes as a surgical alignment regime, in which the granularity of diagnosis and intervention is the attention head rather than the entire model (Cai et al., 7 Jan 2026).

A common misconception addressed by the CAST formulation is that safety-utility conflict should be treated as a global property of the model or training objective. The reported results instead support the narrower claim that conflict is concentrated in a small subset of heads, while other heads can be updated to improve safety with minimal utility cost (Cai et al., 7 Jan 2026).

2. Head-level diagnosis: optimization conflict and functional sensitivity

CAST hinges on a per-head "Conflict Score" $C(h)$ that combines two diagnostic metrics: Optimization Conflict $O(h)$ and Functional Sensitivity $S(h)$ (Cai et al., 7 Jan 2026).

Optimization Conflict measures geometric disagreement between the safety and utility gradient signals at head $h$. The notation is defined as follows: $\theta_h$ denotes the parameters of head $h$, with the analysis focusing on its query-projection submatrix; $D_{\rm safe}$ is a small set of harmful prompts used to induce refusal gradients; $D_{\rm util}$ is a small set of general-capability benchmarks used to induce utility gradients; and $$g_{\rm safe}(h)=\nabla_{\theta_h}\ell_{\rm safe}(\theta)$$ and $$g_{\rm util}(h)=\nabla_{\theta_h}\ell_{\rm util}(\theta)$$. The paper gives the metric as

$O(h)$0

This metric is intended to identify heads where safety and utility gradients are strongly opposed (Cai et al., 7 Jan 2026).

Functional Sensitivity measures how "causally critical" head $O(h)$1 is for utility tasks versus safety tasks via zero-shot ablation. Let $O(h)$2 denote the model with head $O(h)$3 masked, $O(h)$4 denote accuracy on $O(h)$5, and $O(h)$6 denote refusal rate or defense success on $O(h)$7. The raw shifts are

$O(h)$8

$O(h)$9

These shifts are converted to percentile ranks $S(h)$0, after which the sensitivity metric is defined as

$S(h)$1

The interpretation supplied in the paper is explicit: $S(h)$2 if head $S(h)$3 is more critical to utility than safety, making it bad to update; $S(h)$4 if head $S(h)$5 is more critical to safety, so updating it is "safer" (Cai et al., 7 Jan 2026).

The unified score is then

$S(h)$6

Under this definition, only heads with both large gradient opposition and high utility-dependency become "high conflict" (Cai et al., 7 Jan 2026).

3. CAST algorithm and sparse update rule

The CAST procedure consists of two stages: head-level conflict diagnosis and budget-matched sparse fine-tuning (Cai et al., 7 Jan 2026).

In the diagnosis stage, the method computes $S(h)$7 and $S(h)$8 for each attention head on a frozen pretrained model, uses them to obtain $S(h)$9, then performs zero-shot ablation to obtain $h$0 and $h$1, converts these to percentile ranks, and computes $h$2 and finally $h$3. Heads are then sorted by $h$4 in descending order and partitioned into $h$5 equal buckets from top to bottom. The paper gives an example with $h$6, where one selects a bucket to update, such as the "safe zone" corresponding to the bottom-$h$7 bucket (Cai et al., 7 Jan 2026).

In the fine-tuning stage, all heads outside the selected bucket are frozen, and LoRA or full SFT is applied only to the chosen heads. Training then proceeds on the full safety-alignment dataset $h$8 for a fixed budget, after which safety and utility are evaluated. The update rule is summarized as

$h$9

with $\theta_h$0 remaining frozen for heads outside the safe zone (Cai et al., 7 Jan 2026).

The criteria attached to the bucket structure are central to the method’s interpretation. Heads in the top-$\theta_h$1 buckets, described as the "risky zone," are reported to cause large utility drops if updated, whereas heads in the bottom-$\theta_h$2 buckets, described as the "safe zone," improve safety with minimal utility cost (Cai et al., 7 Jan 2026). A plausible implication is that CAST treats sparse tuning not merely as a parameter-efficiency device but as a diagnostic intervention guided by measured conflict structure.

4. Experimental configuration and evaluation protocol

The reported experiments cover three instruction-tuned base models: Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Mistral-7B-v0.2 (Cai et al., 7 Jan 2026). Diagnostics and LoRA updates are applied only on the query-projection $\theta_h$3. The LoRA configuration is rank $\theta_h$4, $\theta_h$5, learning rate $\theta_h$6, with $\theta_h$7 seeds $\theta_h$8 (Cai et al., 7 Jan 2026).

For pre-alignment calibration, the utility set $\theta_h$9 consists of $h$0 MMLU examples, and the safety set $h$1 consists of $h$2 harmful prompts from WildJailbreak (Cai et al., 7 Jan 2026). The alignment dataset $h$3 contains $h$4 samples, with $h$5 each of vanilla/adversarial $h$6 harmful/benign (Cai et al., 7 Jan 2026).

The evaluation protocol separates safety and utility. Safety is measured by Defense Success Rate on WildJailbreak test, WildGuard, and DAN. Utility is evaluated on knowledge benchmarks MMLU and CSQA using multiple-choice accuracy, and on reasoning benchmarks GSM8K and MATH using chain-of-thought generation (Cai et al., 7 Jan 2026).

The principal trade-off metric is the Utility Cost Ratio (UCR), together with MMLU-CR, both defined in equation 8 of the paper. The provided formula is

$h$7

with the interpretation that lower values indicate better trade-off efficiency (Cai et al., 7 Jan 2026). This metric formalizes the claim that safety gains should be evaluated jointly with any degradation in general capability.

5. Empirical findings

The main quantitative result is a safety-versus-utility comparison between full fine-tuning, random sparse updates, and CAST-guided sparse updates. On Llama, Full-SFT over all heads drops MMLU from $h$8 (a decrease of $h$9 percentage points) while achieving $D_{\rm safe}$0 safety. Random $D_{\rm safe}$1 head selection recovers some utility, with MMLU approximately $D_{\rm safe}$2. CAST-SFT on the bottom-$D_{\rm safe}$3 safe-zone heads yields $D_{\rm safe}$4, which is reported as $D_{\rm safe}$5 percentage points relative to Full-SFT, at matching safety of $D_{\rm safe}$6 (Cai et al., 7 Jan 2026). Similar patterns are reported on Qwen and Mistral (Cai et al., 7 Jan 2026).

The paper also reports synergy with PCGrad. Even when applying global PCGrad projection, updating only safe-zone heads through CAST+PCG strictly dominates full PCGrad and full-SFT in Pareto plots (Cai et al., 7 Jan 2026). This result is presented as evidence that head-level diagnosis and global gradient projection are not mutually exclusive; rather, the former can constrain where the latter is allowed to act.

Predictive validity is another prominent empirical claim. Aggregate bucket scores $D_{\rm safe}$7 strongly correlate with realized MMLU-CR, with Pearson $D_{\rm safe}$8, and with UCR, with Pearson $D_{\rm safe}$9 (Cai et al., 7 Jan 2026). The intended interpretation is that the pre-alignment conflict map is not merely descriptive but predictive of post-alignment safety-utility outcomes.

A concise summary of the principal reported results is useful:

Setting	Reported outcome	Interpretation in paper
Full-SFT on Llama	MMLU $D_{\rm util}$0; safety $D_{\rm util}$1	Global updates incur substantial utility loss
Random $D_{\rm util}$2 heads	MMLU $D_{\rm util}$3	Some utility can be recovered by sparsity alone
CAST-SFT bottom-$D_{\rm util}$4	MMLU $D_{\rm util}$5; safety $D_{\rm util}$6	Selective low-conflict updates improve the trade-off
CAST+PCG	Strictly dominates full PCGrad and full-SFT in Pareto plots	Head-level selection complements global projection

The conclusion drawn in the source is that CAST achieves safety gains equal to global fine-tuning while recovering over $D_{\rm util}$7 points of MMLU and preserving chain-of-thought abilities (Cai et al., 7 Jan 2026). Because this language appears in the conclusion rather than the benchmark table, it is best read as the paper’s overall synthesis of the reported experiments.

6. Ablations, interpretability, and limitations

The ablation studies are designed to test whether CAST’s conflict score requires both constituent metrics. The reported result is that $D_{\rm util}$8 alone or $D_{\rm util}$9 alone often fail, including cases with negative correlations, whereas the unified score $$g_{\rm safe}(h)=\nabla_{\theta_h}\ell_{\rm safe}(\theta)$$0 maintains strong positive correlations across all models (Cai et al., 7 Jan 2026). This directly supports the claim that gradient opposition and functional importance encode distinct but jointly necessary aspects of conflict.

The sparsity-ratio ablation reports that the bottom-$$g_{\rm safe}(h)=\nabla_{\theta_h}\ell_{\rm safe}(\theta)$$1 safe zone achieves the best utility/safety trade-off. Expanding the updated set to the bottom-$$g_{\rm safe}(h)=\nabla_{\theta_h}\ell_{\rm safe}(\theta)$$2 or $$g_{\rm safe}(h)=\nabla_{\theta_h}\ell_{\rm safe}(\theta)$$3 degrades utility without improving safety, which the paper interprets as confirming that most benign heads live in that $$g_{\rm safe}(h)=\nabla_{\theta_h}\ell_{\rm safe}(\theta)$$4 tail. Updating the top-$$g_{\rm safe}(h)=\nabla_{\theta_h}\ell_{\rm safe}(\theta)$$5 yields the worst utility collapse (Cai et al., 7 Jan 2026). This suggests that the relevant structure is sharply concentrated rather than smoothly distributed.

Data-efficiency experiments indicate that using only $$g_{\rm safe}(h)=\nabla_{\theta_h}\ell_{\rm safe}(\theta)$$6 MMLU calibration samples still identifies the same safe-zone heads and preserves the utility peak (Cai et al., 7 Jan 2026). Domain-transfer experiments further show that calibrating on $$g_{\rm safe}(h)=\nabla_{\theta_h}\ell_{\rm safe}(\theta)$$7 GSM8K reasoning samples shifts the "safe zone" to an intermediate bucket rather than the extreme tail, revealing that domain-specific conflict topologies can differ, described in the paper as an "inverted U-shape" (Cai et al., 7 Jan 2026). A plausible implication is that CAST’s head ranking is not universal across task families, even within the same model.

Conflict map visualization is reported to show that high-conflict heads are sparse and cluster in middle-to-deep layers, while other heads remain low-conflict (Cai et al., 7 Jan 2026). This is the basis for the paper’s interpretability claim: the head-level map provides concrete insights into which layers and heads mediate refusal versus reasoning.

The paper’s limitations are stated explicitly. First, scope is limited to query projections, and future work should include MLP layers. Second, static one-time diagnosis may miss "conflict drift" during long training. Third, calibration is task-dependent, so cross-domain alignment, such as code generation, may require bespoke calibration sets (Cai et al., 7 Jan 2026). These limitations qualify the method’s generality and indicate that CAST is presently a diagnosis-and-tuning scheme for a restricted subspace of Transformer parameters rather than a complete account of alignment conflict.

7. Position within safety-utility alignment research

CAST is presented as a response to a particular framing of the alignment problem: safety alignment as a multi-objective optimization conflict whose costs are not globally homogeneous across model components (Cai et al., 7 Jan 2026). In this framing, the key contribution is not merely sparse fine-tuning, but sparse fine-tuning driven by a pre-alignment diagnostic map that integrates optimization conflict and functional sensitivity.

The paper attributes three broader properties to this approach. First, under the heading of Modular Heterogeneity, it argues that the safety-utility conflict is concentrated in a small subset of heads rather than being a global phenomenon. Second, under the heading of Interpretability, it claims that the head-level map provides concrete insights into the functional roles of layers and heads. Third, under the heading of Pareto Efficiency, it states that CAST consistently pushes out the safety-utility frontier, including in combination with optimization-centric methods like PCGrad (Cai et al., 7 Jan 2026).

Taken together, these claims position CAST as a diagnostic-driven sparse tuning paradigm for LLM safety alignment. Its central thesis is that selective skipping of high-conflict heads during alignment can preserve general capabilities while maintaining safety performance, provided that head selection is based on the coupled score

$$g_{\rm safe}(h)=\nabla_{\theta_h}\ell_{\rm safe}(\theta)$$8

This suggests a broader methodological lesson: when alignment costs are heterogeneous across modules, parameter selection informed by causal and geometric diagnostics may be more effective than uniform update rules applied across the entire model (Cai et al., 7 Jan 2026).