Papers
Topics
Authors
Recent
Search
2000 character limit reached

The Surprising Effectiveness of Deleting Weights in LLM Post-Training

Published 7 Nov 2026 in cs.LG and cs.CL | (2611.10866v1)

Abstract: How little of a pretrained LLM has to change for it to acquire new capabilities? We find that zeroing fewer than 0.05% of its weights, with no other modification, matches full fine-tuning (FFT) at scale across three LLM post-training regimes: reinforcement learning with verifiable rewards, supervised fine-tuning, and on-policy distillation. Our method, Deletion Tuning, learns a binary mask over the gate projection of transformer blocks. Two practical advantages follow: the trained mask is hundreds to thousands of times smaller than the corresponding FFT checkpoint, and Deletion Tuning forgets far less, which addresses a central challenge in continual learning. Under reinforcement learning, its prior-task accuracy stays near the base level while FFT, LoRA, and random sparse fine-tuning lose up to 5.4 points. To understand why masking alone suffices, we analyze the geometry of the learned weight changes under reinforcement learning using stable rank, spectrum drift, and principal-weight overlap. Compared with LoRA and random sparse fine-tuning, Deletion Tuning is closest to FFT across the three measures.

Summary

  • The paper introduces Deletion Tuning, demonstrating that freezing predated model parameters and deleting fewer than $0.05$% of a pretrained model's parameters can achieve similar performance as full fine-tuning across multiple benchmarks.
  • Evaluation on large LLM models including Llama and Qwen shows that Deletion Tuning narrows the gap to full fine-tuning results as the model scale grows, notably achieving parity or increased accuracy on benchmarks such as GSM8K and SimpleRL-Zoo in GRPO.
  • Deletion Tuning retains valuable knowledge from pre-existing benchmarks and improves capability acquisition, markedly reducing policy drift and priortask forgetting compared to leading sparsity adaptation methods, facilitating a promising avenue for continual learning with minimum memory footprints.

Problem formulation and central claim

The paper studies whether post-training an LLM requires learning new numerical parameter values, or whether it is sufficient to select a sparse subset of pretrained weights for deletion. It introduces Deletion Tuning, a parameter-efficient adaptation method that freezes the pretrained model and learns only a binary mask over the gate-projection matrices in the transformer MLP blocks. At deployment, retained weights preserve their pretrained values, while selected weights are set exactly to zero. Thus, the method performs no additive update and writes no new nonzero values (2611.10866).

The central claim is unusually restrictive: zeroing fewer than 0.05%0.05\% of the model’s total weights can match full fine-tuning (FFT) across reinforcement learning with verifiable rewards (RLVR), supervised fine-tuning (SFT), and on-policy distillation. The experiments cover Llama and Qwen models from $0.5$B to $30$B total parameters, including the Qwen3-30B-A3B mixture-of-experts model. The claim is not that arbitrary pruning preserves post-training performance. Rather, the result depends on learning a task-specific deletion pattern through gradient-based optimization.

The distinction between one-shot pruning and learned deletion is essential. A Wanda-initialized mask, used without post-training optimization, remains close to base-model performance. In contrast, the trained mask improves accuracy by as much as $53.2$ points over the corresponding base model in the reported GRPO experiments. The paper therefore treats deletion not merely as a compression operation, but as the adaptation mechanism itself.

Deletion Tuning mechanism

For each selected pretrained gate-projection matrix WW, Deletion Tuning introduces a trainable logit matrix GG of identical shape. The pretrained matrix remains frozen. During optimization, the effective matrix is

W~(t)=W⊙σ ⁣(α(t)G),\widetilde{W}(t) = W \odot \sigma\!\left(\alpha(t)G\right),

where σ\sigma is the logistic sigmoid and α(t)\alpha(t) is an increasing sharpness schedule. At evaluation and deployment, the soft mask is replaced with the hard mask 1[G>0]\mathbf{1}[G>0]:

W^=W⊙1[G>0].\widehat{W} = W \odot \mathbf{1}[G>0].

Positive logits retain the associated pretrained weights; negative logits delete them. The method therefore requires one trainable scalar per candidate weight during training but only one bit per masked weight for deployment.

The method masks only the gate projection in each transformer block. In SwiGLU architectures, the gate projection controls one factor in the multiplicative MLP pathway,

down⁡(SiLU⁡(gate⁡(x))⊙up⁡(x)),\operatorname{down}\left(\operatorname{SiLU}(\operatorname{gate}(x))\odot \operatorname{up}(x)\right),

which provides a structurally plausible location for selective deletion. The restriction is consequential: the paper does not establish that deletion works equally well when applied to arbitrary projections, attention matrices, channels, or structured groups.

The sigmoid is sharpened linearly during training. For the main GRPO experiments, α\alpha increases from 10310^3 to 10410^4; SFT and SDFT use related schedules. Hard masks are used for evaluation and for on-policy rollouts, while gradients are computed through the soft relaxation. The discrepancy between training and deployment is small in the reported experiments: replacing the soft mask with the hard mask changes accuracy by at most $0.08$ points.

Figure 1

Figure 1: Deletion Tuning freezes pretrained weights and learns only the binary decisions determining which gate-projection weights are retained or deleted.

Initialization uses Wanda scores, which combine pretrained weight magnitude with the norm of the corresponding input activation. High-scoring weights receive logits sampled near +0.03+0.03, while the remaining weights receive logits sampled near −0.03-0.03. With the default quantile parameter η=1/2048\eta=1/2048, the initial hard mask deletes approximately 0.18%0.18\% of gate-projection weights. A sparse-weight regularizer mildly favors retaining larger weights in GRPO and SFT, while SDFT omits the regularizer because the EMA teacher provides an anchoring mechanism.

The paper explicitly tests whether the Wanda prior accounts for the gains. It does not. Across six GRPO backbones, the best one-shot Wanda mask remains within $2.4$ points of the base model, whereas trained Deletion Tuning improves performance by up to $53.2$ points. This establishes that the learned mask, rather than the initialization alone, is responsible for adaptation.

Experimental design

The evaluation spans three post-training objectives with different sources of supervision. GRPO supplies response-level verifier rewards from model-generated samples; SFT supplies fixed token-level targets; and SDFT uses on-policy samples together with a teacher distribution generated by an exponential moving average of the student. This design tests whether deletion is limited to sparse RL updates or can also support denser supervised and distillation signals.

For GRPO, the in-distribution setting uses GSM8K with Llama 1B, 3B, and 8B models. The out-of-distribution setting trains on the SimpleRL-Zoo mathematical corpus and evaluates the mean over GSM8K, MATH500, Minerva Math, OlympiadBench, AIME24, and AMC23 using Qwen models from $0.5$B to $7$B, with additional experiments at Qwen14B and Qwen3-30B-A3B. SFT uses 14 MMLU-Pro subjects and 23 BBEH subtasks with Llama 1B and 3B. SDFT evaluates tool use on ToolAlpaca and chemistry questions from SciKnowEval using Qwen7B.

The principal baselines are FFT, LoRA with ranks one and four, and random sparse fine-tuning through MTH at 99%99\% and 99.95%99.95\% sparsity. The comparison is therefore not only against dense adaptation, but also against methods that modify a similarly small number of parameters through additive updates.

GRPO performance and adapter compactness

Deletion Tuning recovers most of the FFT gain at smaller scales and reaches parity or slightly higher accuracy at larger scales. On Llama8B GSM8K, it obtains 83.55%83.55\%, compared with 83.24%83.24\% for FFT. On Qwen7B SimpleRL-Zoo, it obtains 52.32%52.32\%, compared with 51.89%51.89\% for FFT. At the larger scales, it reaches 55.43%55.43\% on Qwen14B versus 54.87%54.87\% for FFT, and 58.54%58.54\% on Qwen3-30B-A3B versus 58.43%58.43\% for FFT.

The result is more qualified at smaller scales. On Llama1B, Deletion Tuning reaches 54.44%54.44\% on GSM8K, below FFT’s 60.50%60.50\% but above the base model’s 1.21%1.21\%. On Qwen0.5B, it reaches 17.64%17.64\% on SimpleRL-Zoo, below FFT’s 19.01%19.01\%. The paper reports that the method recovers approximately 90%90\% to 99%99\% of FFT’s accuracy gain at these smaller scales. Thus, the evidence supports a scale-dependent claim: deletion becomes increasingly competitive with FFT as model capacity grows.

Model and task Base FFT Deletion Tuning Deletion adapter
Llama8B, GSM8K 60.27 83.24 83.55 8.4 MB
Qwen7B, SimpleRL-Zoo 40.18 51.89 52.32 6.9 MB
Qwen14B, SimpleRL-Zoo 44.37 54.87 55.43 11.4 MB
Qwen3-30B-A3B, SimpleRL-Zoo 49.41 58.43 58.54 29.6 MB

The storage advantage is substantial. Across the six primary GRPO runs, the trained mask deletes only 0.0347%0.0347\% to 0.0459%0.0459\% of all model weights. The resulting adapters range from $0.5$ MB for Qwen0.5B to $8.4$ MB for Llama8B, while corresponding FFT checkpoints range from $1,885$ MB to $30,633$ MB. The paper reports compression ratios of approximately $3,600$ to $4,700$ relative to FFT in these experiments. The mask is also smaller than rank-one LoRA in every reported GRPO comparison.

This comparison should be interpreted as deployment storage rather than training memory. Deletion Tuning maintains a trainable logit for every masked pretrained weight, so its optimization state is much larger than its final adapter. The compactness claim applies to the stored task-specific update, not to the parameterization required during training.

Capability acquisition under sampling

The method does not merely preserve base behavior under pruning. It improves the model’s sampling distribution in a way that remains visible at high pass rates. On Llama8B GSM8K, Deletion Tuning increases pass@1 from the base model’s 53.20%53.20\% to 81.75%81.75\%, and pass@128 from 95.29%95.29\% to 99.62%99.62\%. On Qwen7B SimpleRL-Zoo, it increases pass@1 from 27.35%27.35\% to 49.39%49.39\% and pass@128 from 74.01%74.01\% to 80.08%80.08\%.

The high-kk result is important because it rules out a narrow interpretation in which the method merely increases the probability of answers that the base model already occasionally produces. At k=128k=128, the trained model still exceeds the base by $4.3$ points on Llama8B and $6.1$ points on Qwen7B. The paper’s interpretation is that deletion can alter the model’s capabilities sufficiently to solve problems absent from all 128 sampled base-model attempts in the evaluation protocol.

Retention and catastrophic forgetting

The strongest practical result concerns retention during RLVR. The paper evaluates forward KL between the adapted and base policies on the new task, together with accuracy on a six-benchmark prior-task suite comprising HellaSwag, MMLU, TruthfulQA, WinoGrande, IFEval, and HumanEval. Forward KL is used because prior work identifies it as a predictor of forgetting.

Deletion Tuning maintains forward KL at or below $0.05$ across all three Llama scales. Its prior-task accuracy remains within one point of the base model throughout training. By contrast, FFT reaches forward KL values of approximately $1.0$ on Llama1B and $1.5$ on Llama8B by step 348, with prior-task accuracy declining by $1.3$ to $3.1$ points. LoRA and MTH also exhibit substantially larger policy drift and lose as much as $5.4$ points of prior-task accuracy.

Figure 2

Figure 2: Deletion Tuning maintains low forward KL and near-base prior-task accuracy across Llama scales, while FFT, LoRA, and random sparse fine-tuning drift from the pretrained policy.

The comparison with MTH is particularly informative. MTH at 99.95%99.95\% sparsity updates only 0.05%0.05\% of all parameters, a quantity comparable to the fraction deleted by Deletion Tuning. Nevertheless, MTH forgets more. This weakens the explanation that retention follows solely from changing very few parameters. The relevant constraint appears to be the form of the update: setting selected pretrained values to zero rather than learning arbitrary floating-point deltas.

The implication is specific rather than universal. Deletion Tuning is a promising continual-learning primitive because it combines task acquisition with low measured policy drift, but the paper does not perform sequential multi-task continual-learning experiments. Its retention result is demonstrated for a single GRPO adaptation relative to a fixed prior-task suite.

Geometry of the learned update

To explain how deletion can approximate FFT, the paper compares the effective weight changes across FFT, LoRA, MTH, and Deletion Tuning using three geometric measures: stable rank of ΔW\Delta W, spectrum drift relative to the pretrained matrix, and overlap between changed positions and the principal weights of the pretrained matrix.

Deletion Tuning produces an intermediate stable rank. At the analyzed checkpoints, its stable rank is between $66$ and $93$, compared with $410$ to $650$ for MTH and only $1$ to $2$ for LoRA rank one. This indicates that its update is neither constrained to a nearly one-dimensional low-rank subspace nor distributed like a random sparse update. The method can select many coordinates, but its update structure is still much more concentrated than random sparse fine-tuning.

Its singular-value behavior is also close to FFT. Deletion Tuning’s spectrum drift remains within 1.4×1.4\times FFT’s drift, whereas LoRA rank one produces as much as 84×84\times more drift than FFT on Llama8B. This is notable because LoRA is often treated as a conservative adaptation method solely because it has few trainable parameters. The result shows that parameter count does not determine spectral proximity to the base model.

Finally, the principal-weight overlap of Deletion Tuning is $0.43$ to $0.46$, closely matching FFT’s $0.41$ to $0.46$ and remaining below the random baseline of $0.5$. LoRA remains near random at $0.48$ to $0.50, while MTH is approximately$0.50$. The sub-random overlap reproduces a reported RLVR signature: effective RL updates avoid the dominant principal weights while largely preserving the pretrained spectrum.

Figure 3

Figure 3: Deletion Tuning most closely matches FFT in stable rank, spectrum drift, and sub-random overlap with principal pretrained weights.

The geometric evidence supports the paper’s explanation that the learned mask has two freedoms absent from the baselines: it can choose which coordinates to alter, and it has no explicit rank constraint. Although each alteration is only a deletion, the aggregate coordinate selection can approximate the geometry of the dense RL update. This is an explanation of correlation, not a causal proof. The experiments do not identify which individual deletions implement particular reasoning changes, nor do they establish that the same geometry explains the SFT and SDFT results.

Extension to SFT and on-policy distillation

Deletion Tuning also works outside RLVR, although the scale and benchmark coverage differ. Under SFT, it matches or exceeds FFT at Llama3B. On MMLU-Pro, it reaches 22.28%22.28\% compared with FFT’s 21.19%21.19\%; on BBEH, both Deletion Tuning and FFT reach 11.39%11.39\%. At Llama1B, Deletion Tuning improves over the base model but does not consistently match FFT: it reaches 18.62%18.62\% on MMLU-Pro versus FFT’s 16.97%16.97\%, and 5.65%5.65\% on BBEH versus FFT’s 8.35%8.35\%.

Under SDFT with Qwen7B, it reaches 69.07%69.07\% on ToolAlpaca versus FFT’s 68.04%68.04\%, and 70.61%70.61\% on Science versus FFT’s 71.20%71.20\%. The corresponding adapters are $27.1$ MB and $47.1$ MB, compared with $29,051$ MB for the FFT checkpoint. Across SFT and SDFT, the paper reports adapters between $600$ and $4,700$ times smaller than FFT.

These results broaden the claim beyond sparse RL updates, but they do not remove the dependence on architectural and experimental choices. All experiments mask gate projections, use task-specific Wanda calibration data, and select the best validation checkpoint. The paper therefore demonstrates transfer across training objectives, not objective-independent universality.

Limitations and open questions

The principal limitation is the asymmetry between training and deployment. The final mask is extremely small, but training still requires one logit per candidate gate-projection weight, together with optimizer state. The authors explicitly note that Deletion Tuning uses less training memory than FFT but more than LoRA. Restricting optimization to a small candidate set could reduce this overhead, but that modification is not evaluated in the main experiments.

The second limitation is structural. The method is tested on gate projections, and the strongest claims rely on Llama and Qwen architectures with SwiGLU-style MLPs. It remains open whether equivalent results hold for attention projections, non-gated MLPs, other activation mechanisms, or models with substantially different architectures. The mixture-of-experts experiment is encouraging, but it masks each expert’s gate projection while leaving the router unchanged; it does not test router adaptation.

The evaluation also uses best-checkpoint validation accuracy and relatively narrow task suites. Although the reported benchmarks include both in-distribution and out-of-distribution mathematics, MMLU-Pro, BBEH, tool use, and chemistry, the paper does not provide a systematic study of calibration, generation quality, safety behavior, or interference across long sequences of tasks. The continual-learning implication is therefore a measured hypothesis rather than an established result.

Finally, the paper leaves unresolved why zeroing weights is preferable to learning small signed perturbations at the same coordinates. MTH provides evidence that random coordinate selection with trainable values is less retentive, but a controlled comparison between learned deletion and learned sparse additive updates using identical coordinate-selection mechanisms would more directly isolate the effect of deletion. Likewise, the geometric analysis is performed at selected GRPO checkpoints and does not demonstrate that FFT-like geometry is necessary for performance.

Conclusion

The paper presents a tightly constrained form of LLM post-training in which the model learns only which pretrained gate-projection weights to delete. Across RLVR, SFT, and SDFT, the method obtains FFT-level performance at sufficiently large model scales while storing binary masks that are hundreds to thousands of times smaller than dense checkpoints. In GRPO, it combines capability acquisition with markedly lower policy drift and prior-task forgetting: forward KL remains at or below $0.05$, while competing methods lose up to $5.4$ points on the prior-task suite.

The central technical result is that learned coordinate selection can produce an effective update whose stable rank, spectral drift, and principal-weight overlap resemble FFT more closely than either low-rank or random sparse baselines. The work therefore identifies weight deletion as a viable adaptation mechanism under specific architectural and training conditions, while leaving open whether its retention advantages and FFT-like geometry persist under broader continual-learning sequences, alternative target modules, and training-memory-constrained implementations.

Whiteboard

Explain it Like I'm 14

1. What is this paper about?

This paper studies a surprising way to teach a LLM new skills.

Usually, when researchers want to improve an LLM, they change many of its numbers, called weights. These numbers control how the model processes information. Changing all of them is called full fine-tuning, but it requires a lot of storage and can cause the model to forget what it already knew.

The paper asks:

Can a model learn a new skill by changing almost none of its weights—only by deciding which old weights to turn off?

The researchers find that the answer is often yes. Their method, called deletion tuning, learns which pretrained weights to set to zero while leaving all the other weights exactly as they were.

2. What questions did the researchers ask?

The main research questions were:

  • Can deleting a tiny number of weights give an LLM new abilities?
  • Can this work in different kinds of training?
  • Can the method perform as well as full fine-tuning?
  • Does deleting weights help the model remember its old abilities?
  • Why can such a small change work so well?

The researchers tested three kinds of training:

  1. Reinforcement learning with checkable rewards: the model tries answers, and a program or evaluator gives them scores.
  2. Supervised fine-tuning: the model studies examples of correct answers.
  3. On-policy distillation: one version of the model teaches another version how to respond.

3. How did the method work?

A simple analogy

Imagine an enormous control panel with billions of switches. Each switch represents a model weight.

Ordinary fine-tuning changes the settings of many switches. Other methods add new control panels. Deletion tuning does something much simpler: it keeps almost every original switch unchanged and learns which few switches should be turned off.

The method focuses on one important part of each Transformer block called the gate projection. This part helps decide which information should pass through the model.

For every selected weight, the researchers added a small trainable value called a logit. The logit eventually makes a yes-or-no decision:

  • Positive logit: keep the original weight.
  • Negative logit: delete the weight by replacing it with zero.

During training, the decision is softened slightly so that normal learning algorithms can adjust it. This is similar to using a dimmer switch before finally choosing whether a light should be completely on or off.

The original model weights remain frozen. Only the mask—the collection of keep/delete decisions—is trained.

How did they evaluate it?

The researchers tested different model families and sizes, from about 0.5 billion to 30 billion parameters. They used tasks involving:

  • Mathematical reasoning
  • General knowledge
  • Complex problem solving
  • Tool use
  • Science questions

They compared deletion tuning with:

  • Full fine-tuning (FFT): changes every model weight.
  • LoRA: adds small, trainable low-rank updates.
  • Random sparse fine-tuning: changes a randomly selected small group of weights.

They measured both task accuracy and how much storage each method needed.

4. What did they find?

Deleting very few weights can match full fine-tuning

The most important result is that deletion tuning often performed about as well as full fine-tuning.

In the largest experiments, it matched or slightly exceeded full fine-tuning on:

  • Mathematics with Llama and Qwen models
  • MMLU-Pro and BBEH knowledge and reasoning tasks
  • Tool-use tasks
  • Science questions

For example, on one Llama 8B mathematics test:

  • Full fine-tuning scored 83.24%.
  • Deletion tuning scored 83.55%.

On a large Qwen mixture-of-experts model:

  • Full fine-tuning scored 58.43%.
  • Deletion tuning scored 58.54%.

The method became more effective as the models became larger.

It changed less than 0.05% of all weights

In the main reinforcement-learning experiments, deletion tuning removed fewer than 0.05% of all model weights.

That is less than 1 weight out of every 2,000. Almost the entire original model was left untouched.

This is surprising because the model still learned new abilities, especially better mathematical reasoning.

The trained mask was extremely small

Because the method stores only keep/delete decisions, its adapter—the extra information needed to modify the model—was tiny.

For example:

  • Some adapters were less than 1 megabyte.
  • An 8-billion-parameter model used an adapter of about 8.4 MB.
  • The corresponding full fine-tuned checkpoint required about 30,633 MB.

So deletion tuning used an adapter thousands of times smaller than full fine-tuning.

It forgot less

A major problem with training an AI on a new task is catastrophic forgetting. This happens when the model becomes better at the new task but worse at older tasks.

Deletion tuning preserved the model’s earlier abilities much better than the comparison methods.

During reinforcement learning:

  • Deletion tuning kept previous-task accuracy within about one percentage point of the original model.
  • Full fine-tuning, LoRA, and random sparse fine-tuning sometimes lost as much as 5.4 percentage points.

This suggests that turning off a few carefully chosen weights is less disruptive than changing many weights.

The gains came from learning, not just from choosing weights intelligently at the start

The researchers first used a technique called Wanda to guess which weights were safe to delete. However, simply deleting weights based on that initial guess did not greatly improve the model.

The large improvements happened only after the mask itself was trained.

For example, on one task, the initial pruning method improved the model by only a small amount, while trained deletion tuning improved its score by more than 50 percentage points.

This shows that the important idea is not merely “delete unimportant weights.” It is:

Train the model to discover which deletions help it learn the new skill.

Its changes looked more like full fine-tuning

The researchers also studied the mathematical shape of the model’s changes. They used measures with technical names such as:

  • Stable rank: roughly, how many important directions the change uses.
  • Spectrum drift: how much the overall structure of a weight matrix moves.
  • Principal-weight overlap: whether the changed weights are among the model’s most important original weights.

Deletion tuning looked more similar to full fine-tuning than LoRA or random sparse fine-tuning did.

This may explain why it could achieve similar accuracy even though it only deleted weights. It was free to choose which weights to delete instead of being forced to use a particular type of update.

5. Why are these results important?

The research suggests that adapting an LLM may require far fewer changes than people previously thought.

Instead of rewriting the model, it may be possible to preserve nearly all of its original knowledge and make a small number of carefully selected deletions. This could have several benefits:

  • Lower storage costs: Tiny adapters are easier to save and share.
  • Faster task switching: A system could load a small mask instead of a whole new model.
  • Less forgetting: The model may keep its old skills while learning new ones.
  • Simpler deployment: The original model stays the same, while the mask tells it which weights to ignore.
  • Better understanding: Researchers could compare masks to see which parts of a model are involved in different skills.

This could be especially useful for continual learning, where an AI must learn many new tasks over time without losing its earlier knowledge.

Conclusion

The paper shows that an LLM can sometimes learn important new abilities by changing almost nothing. The researchers train a tiny binary mask that decides which pretrained weights to keep and which to delete. Despite making changes to fewer than 0.05% of the weights, the method often matches full fine-tuning.

The main lesson is that improving an AI may not require rewriting its entire “brain.” Sometimes, carefully removing a very small number of connections is enough to help it learn—while allowing it to remember what it already knows.

Practical Applications

Immediate Applications

The paper’s results support several applications that can be implemented with existing LLM infrastructure, although most should initially be treated as engineering pilots rather than universally reliable replacements for full fine-tuning.

  • Compact task-specific adapters for enterprise LLMs — Industry / Software
    • Train a binary deletion mask for a base model instead of storing a full fine-tuned checkpoint or floating-point adapter.
    • A model-serving platform could maintain one shared base model and load small, task-specific masks for customer support, legal drafting, coding, finance, or internal knowledge tasks.
    • The reported masks are substantially smaller than full fine-tuning checkpoints and, in the GRPO experiments, were smaller than rank-1 LoRA adapters.
    • Potential workflow: calibration-data collection → Wanda-style initialization → mask training with SFT, reinforcement learning, or distillation → bit-packing and compression → runtime mask switching.
    • Dependencies: the base checkpoint must remain available; the inference stack must efficiently apply or materialize zeros in the targeted gate-projection layers; performance must be validated on the organization’s own distribution.
  • Low-cost adapter switching in multi-tenant model serving — Cloud AI / Software
    • A provider could serve many specialized model variants without storing or transferring complete model copies.
    • Small masks could reduce model-distribution bandwidth, cold-start latency, and storage costs when switching between domains or customers.
    • This is especially relevant for large models where a full checkpoint may occupy tens or hundreds of gigabytes, while the reported adapters occupy megabytes.
    • Dependencies: the base model still dominates GPU memory and compute; mask loading only reduces adaptation-storage costs, not the cost of hosting the underlying model.
  • Task specialization for reasoning and verifiable-reward systems — Education, coding, mathematics, and agents
    • Reinforcement learning with verifiable rewards could use deletion tuning to specialize an LLM for mathematical problem solving, code generation with unit tests, theorem proving, structured planning, or tool-use tasks.
    • The paper shows improvements in pass@1 through pass@128 and reports that the method can solve some problems missed by the base model even after many sampling attempts.
    • Potential products: compact “reasoning skill packs,” mathematics-specific adapters, coding-test adapters, or agent masks optimized for tool execution.
    • Dependencies: rewards or verifiers must be reliable. Incorrect, exploitable, or incomplete reward functions could cause the mask to optimize undesirable behavior.
  • Continual-learning prototypes with reduced catastrophic forgetting — Research / Industry
    • Teams can train new task masks while preserving the original base weights, then measure whether prior-task performance remains closer to the base model than under full fine-tuning or LoRA.
    • The reported reinforcement-learning experiments kept prior-task accuracy within approximately one point of the base model, whereas some baselines lost substantially more.
    • Potential workflow: maintain a frozen foundation model → train one mask per new capability → evaluate new-task and prior-task suites → activate masks selectively according to the application.
    • Dependencies: the paper evaluates retention primarily in limited task sequences rather than long, sequential learning histories. Mask interference across many tasks remains unresolved.
  • Specialized adapters for tool-use and domain knowledge — Healthcare, science, and enterprise automation
    • On-policy distillation and supervised deletion tuning can produce compact variants for tool-use, scientific question answering, chemistry, or domain-specific instruction following.
    • A hospital, laboratory, or company could distribute a specialized mask alongside a centrally managed base model rather than distributing a modified model checkpoint.
    • Dependencies: domain data must be representative and legally usable; medical, scientific, and financial deployments require independent accuracy, safety, and compliance testing. The reported results do not establish clinical or regulatory reliability.
  • Privacy- and governance-oriented model distribution — Policy / Enterprise IT
    • Organizations could distribute a base model plus signed binary masks, making task-specific model changes easier to audit, version, revoke, and compare.
    • Since the adapter contains only keep/delete decisions and no newly learned floating-point values, it may be easier to inspect than a dense checkpoint, although the mask itself can still encode information about the training task.
    • Potential tools: mask registries, cryptographically signed adapter packages, provenance dashboards, and automated regression testing across old and new tasks.
    • Dependencies: binary masks are not automatically interpretable or privacy-preserving. Membership-inference, model-extraction, and sensitive-information leakage analyses are still required.
  • Efficient academic experimentation on post-training mechanisms — Academia
    • Researchers can use deletion tuning as a controlled baseline for studying which pretrained weights are necessary for acquiring new capabilities.
    • Comparing masks across tasks can reveal whether different tasks delete overlapping weights and whether task-specific changes concentrate in particular transformer blocks or MLP pathways.
    • The method provides a useful comparison point against full fine-tuning, LoRA, random sparse updates, pruning, and distillation.
    • Dependencies: conclusions may depend on the specific model families, gate projections, datasets, reward functions, and training schedules used in the paper. Broader replication is needed.
  • Task-specific variants for personal or edge applications — Daily life / Consumer software
    • A personal assistant could use separate compact masks for writing, coding, travel planning, household budgeting, or study assistance while retaining one shared base model.
    • Masks could be downloaded or exchanged more cheaply than full model updates, which may benefit intermittent-connectivity or storage-constrained environments.
    • Dependencies: the method does not by itself make inference inexpensive enough for phones or embedded devices. Hardware support for sparse or masked computation, quantization, and memory constraints remain important.
  • Incremental model-update distribution for software products — Software engineering
    • Vendors could ship a stable base model once and distribute small task or version updates as masks.
    • This could simplify rollback: disabling a mask restores the original base model, subject to the runtime preserving the base checkpoint unchanged.
    • Dependencies: masks must be tested for compatibility with the exact base-model hash, tokenizer, quantization format, and inference engine.

Long-Term Applications

These applications follow naturally from the paper but require additional research, hardware support, or large-scale validation before dependable deployment.

  • Large-scale continual learning through composable skill masks — Industry / Academia
    • One mask could represent each acquired capability, with a routing system selecting masks based on the task or combining them for multi-skill behavior.
    • The paper specifically suggests combining masks, for example through their union, to merge skills without retraining.
    • Potential products: skill marketplaces, modular agent architectures, and lifelong-learning systems that accumulate domain capabilities over time.
    • Dependencies: mask union may produce conflicting deletions, destroy useful shared behavior, or cause task interference. Composition rules, conflict resolution, and capacity limits need systematic study.
  • Structured sparse adapters that both adapt and accelerate inference — Hardware / Robotics / Edge AI
    • The current method uses unstructured deletion in gate projections. Future versions could constrain masks to N:MN{:}M patterns, channels, blocks, or expert structures so that the same update both adds a capability and reduces computation.
    • This could support compact robotics, autonomous systems, industrial controllers, and edge assistants.
    • Dependencies: the reported accuracy results do not demonstrate inference-speed gains. Hardware and compiler support for the selected sparsity pattern is essential, and unstructured masks may provide little practical acceleration.
  • Composable expert libraries for mixture-of-experts models — Cloud AI / Robotics / Enterprise
    • The paper demonstrates deletion tuning on a mixture-of-experts model by masking gate projections inside experts while leaving the router unchanged.
    • A future system could maintain specialized expert masks for coding, vision-language interaction, scientific reasoning, or robotic control while sharing the original router and base weights.
    • Dependencies: expert routing may need to adapt as masks change. The reported mixture-of-experts experiment is limited, so scalability, routing stability, and cross-expert interference remain open questions.
  • Model personalization without dense parameter updates — Consumer AI / Education
    • Personal assistants or tutoring systems could acquire user-specific preferences, writing styles, or study strategies through compact masks rather than modifying the entire model.
    • Separate masks could represent a user’s preferred tone, language, curriculum, or accessibility settings.
    • Dependencies: user data may be sensitive, and the mask could still encode personal information. Safe deletion, consent, reversibility, and protection against malicious personalization would be required.
  • Safety-preserving specialization and policy enforcement — Policy / High-assurance AI
    • Because deletion tuning tends to keep the base policy close to its original behavior, it could be investigated as a way to add domain capabilities while reducing unintended changes to safety, factuality, or general-purpose behavior.
    • Organizations might maintain regression-tested safety masks or compare the forward KL divergence between specialized and base models as a deployment criterion.
    • Dependencies: low forward KL and prior-task retention do not guarantee safety. A specialized mask could still introduce harmful behavior on rare or adversarial inputs, so formal safety evaluations and adversarial testing are necessary.
  • Automated mask interpretability and capability localization — Academia / AI governance
    • Comparing which gate-projection weights are deleted across tasks could support tools that map model changes to capabilities, domains, or behaviors.
    • Researchers could build “mask similarity” analyses to identify shared mechanisms between mathematical reasoning, tool use, scientific knowledge, and language skills.
    • Dependencies: individual weight positions may not have stable semantic meaning, and deletion patterns may vary with initialization, calibration data, model scale, or random seed. Interpretability claims require causal ablations rather than correlations alone.
  • Federated or bandwidth-efficient model adaptation — Telecommunications / Public-sector AI
    • Clients could train or select local masks and transmit only compressed binary updates to a central server.
    • This could support federated domain adaptation for schools, hospitals, enterprises, or regional language systems without transferring full model checkpoints.
    • Dependencies: the paper does not evaluate federated optimization, secure aggregation, differential privacy, or robustness to malicious masks. Binary updates may be compact but can still leak task or user information.
  • Adaptive model marketplaces and reversible capability licensing — Policy / Commercial AI
    • A vendor might license capabilities as signed masks rather than distributing separate model copies: for example, a coding mask, financial-analysis mask, or translation mask.
    • Masks could be activated, revoked, audited, or combined under software or hardware access controls.
    • Dependencies: capability isolation is not guaranteed. A mask may depend on other masks or on latent capabilities already present in the base model, complicating licensing, attribution, and enforcement.
  • Deletion-based adaptation beyond LLMs — Vision, multimodal systems, and robotics
    • The principle could be tested on vision transformers, multimodal models, speech systems, and robotic policies, particularly in architectures with multiplicative gates or expert pathways.
    • Potential uses include adapting a robot to a new environment, specializing a vision model for manufacturing defects, or adding a new speech domain while preserving existing recognition abilities.
    • Dependencies: the paper’s evidence is concentrated on LLMs and gate projections. Other architectures may not exhibit the same geometry or may require different mask targets and training relaxations.
  • Training-memory reduction for very large models — High-performance computing
    • Since the base weights remain frozen and only mask logits are optimized, deletion tuning could reduce trainable-state requirements compared with full fine-tuning, especially when optimizer states for all model parameters are otherwise required.
    • This may make post-training of large models more accessible to smaller research groups or reduce multi-GPU requirements.
    • Dependencies: the paper notes that deletion tuning still uses more training memory than LoRA because it maintains one trainable logit per targeted weight. The method therefore may not be advantageous for memory-constrained training unless candidate-weight restriction or block-level parameterization is developed.

Glossary

  • AdamW: An optimizer that combines Adam with decoupled weight decay for training neural networks. “All runs use AdamW with fp32 master weights, bf16 mixed precision, and gradient checkpointing.”
  • Annealing: Gradually changing a parameter during training, often to make an approximation increasingly discrete or sharp. “During training, a sigmoid of each logit is annealed toward a step function”
  • Attention layer: A transformer component that computes relationships between token representations to determine which tokens should influence one another. “each averaged over all attention and MLP linear layers”
  • Binary mask: A mask whose elements have only two possible values, typically indicating whether a parameter is retained or removed. “We propose , which learns only a binary mask over the gate-projection weights of transformer blocks.”
  • bf16: The bfloat16 numerical format, commonly used for memory-efficient mixed-precision deep-learning computation. “All runs use AdamW with fp32 master weights, bf16 mixed precision, and gradient checkpointing.”
  • Catastrophic forgetting: The loss of previously learned capabilities when a model is trained on new data or tasks. “SDFT uses it to reduce catastrophic forgetting”
  • Calibration prompts: Input prompts used to measure model activations or other statistics for initializing or configuring a model method. “calibration prompts from the task's training distribution”
  • Continual learning: Learning new tasks while preserving performance on previously learned tasks. “Learning new tasks while keeping old ones makes a promising candidate for continual learning.”
  • Cross-entropy: A loss function that measures the discrepancy between a predicted probability distribution and a target distribution. “SFT. We minimize cross-entropy with a constant learning rate”
  • Exponential moving average (EMA): A running weighted average that gives greater weight to recent values, often used to stabilize model parameters. “The teacher's weights θT\theta_T are an exponential moving average of the student's weights θ\theta”
  • Forward KL divergence: A directional measure of difference between two probability distributions, used here to compare a fine-tuned policy with a base policy. “Forward KL on the new task predicts how much a fine-tuned model forgets”
  • Fully sharded data parallelism (FSDP): A distributed-training technique that shards model parameters, gradients, and optimizer states across multiple devices. “using one GPU for backbones up to $3$B and multiple GPUs with FSDP”
  • Gate projection: The transformer MLP projection that controls which intermediate channels are activated in a gated linear unit. “In all our experiments, the target is the gate projection of transformer blocks”
  • Gradient checkpointing: A memory-saving technique that recomputes intermediate activations during backpropagation instead of storing all of them. “All runs use AdamW with fp32 master weights, bf16 mixed precision, and gradient checkpointing.”
  • Group Relative Policy Optimization (GRPO): A reinforcement-learning method that estimates response advantages by comparing rewards within groups of sampled responses rather than learning a value function. “For RLVR, we use [PERSON_NAME]}.”
  • Hard mask: A discrete mask that makes a definitive retain-or-delete decision for each parameter. “We call σ(α(t) G)\sigma(\alpha(t)\,G) the soft mask and 1[G>0]\mathbf{1}[G > 0] the hard mask.”
  • Importance ratio: The ratio between the probability assigned to an action by the updated policy and the previous policy in policy-gradient optimization. “where $\rho^i_j = \pi_\theta(y^i_j \mid x, y^i_{<j})\,/\,\pi_{\theta_\text{old}(y^i_j \mid x, y^i_{<j})$ is the importance ratio for token jj.”
  • Mixture-of-experts (MoE): A model architecture containing multiple expert subnetworks, with a router selecting which experts process each input. “the Qwen3-30B-A3B mixture-of-experts model”
  • Negative log-likelihood: The negative logarithm of the probability assigned to observed target outcomes, commonly used as a probabilistic training loss. “SFT minimizes the negative log-likelihood of the target tokens”
  • On-policy distillation: Knowledge distillation in which the student generates its own training samples and learns from a teacher on those samples. “For on-policy distillation, we use SDFT~\citep{sdft}, in which the model is its own teacher.”
  • Principal-weight overlap: The proportion of changed parameters that belong to the highest-magnitude weights identified from a low-rank approximation of the original model. “Principal-weight overlap measures the fraction of changed weight positions that fall on what~\citet{pathnottaken} call the base model's principal weights.”
  • Quantile: A value that divides a distribution into specified proportions. “whether the weight is above or below the η\eta-quantile of ss”
  • Reinforcement learning with verifiable rewards (RLVR): Reinforcement learning in which generated outputs receive rewards from rule-based checkers or verifiers. “We push this observation to its limit across reinforcement learning with verifiable rewards (RLVR), supervised fine-tuning (SFT), and on-policy distillation (OPD)”
  • Singular value decomposition (SVD): A factorization of a matrix into orthogonal directions and associated nonnegative singular values. “the top-magnitude entries of the rank-kk SVD reconstruction of W0W_0”
  • Soft mask: A differentiable, continuous approximation to a binary mask, typically containing values between zero and one. “We call σ(α(t) G)\sigma(\alpha(t)\,G) the soft mask and 1[G>0]\mathbf{1}[G > 0] the hard mask.”
  • Spectrum drift: A normalized measure of how much a matrix’s singular values change after model training. “Spectrum drift is the normalized singular-value shift”
  • Stable rank: A continuous analogue of matrix rank defined as the squared Frobenius norm divided by the squared spectral norm. “Stable rank of ΔW\Delta W is $\lVert \Delta W \rVert_F2 / \lVert \Delta W \rVert_22”
  • Straight-through or sigmoid relaxation: A differentiable approximation that permits gradient-based optimization of an otherwise discrete masking decision. “During training, a sigmoid of each logit is annealed toward a step function”
  • Supervised fine-tuning (SFT): Training a pretrained model to imitate fixed, labeled target outputs. “Supervised fine-tuning (SFT) imitates fixed labeled targets”
  • Token-level distribution: A probability distribution over possible next tokens at a particular sequence position. “the teacher's per-token distribution”
  • Unstructured pruning: Removing individual weights without imposing a regular structure on their locations. “In pruning terms, performs learned unstructured pruning to fine-tune the model.”
  • Verifier: A rule-based system or LLM that evaluates generated outputs and assigns rewards. “With verifiable rewards, rule-based checkers or LLM verifiers score each sample”
  • Wanda score: A pruning-importance score based on a weight’s magnitude multiplied by the norm of its input activation. “The [PERSON_NAME] score of a weight is its magnitude times the norm of its input activation”
  • Weight delta: The elementwise difference between a trained parameter matrix and its pretrained counterpart. “and ΔW=W+−W0\Delta W = W_+ - W_0 the weight delta”
  • Zero-shot pruning: Removing parameters using an importance criterion without training the model after pruning. “We apply [PERSON_NAME] one-shot pruning without GRPO training at $8$ sparsity levels”

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Tweets

Sign up for free to view the 15 tweets with 294 likes about this paper.