The Surprising Effectiveness of Deleting Weights in LLM Post-Training
Abstract: How little of a pretrained LLM has to change for it to acquire new capabilities? We find that zeroing fewer than 0.05% of its weights, with no other modification, matches full fine-tuning (FFT) at scale across three LLM post-training regimes: reinforcement learning with verifiable rewards, supervised fine-tuning, and on-policy distillation. Our method, Deletion Tuning, learns a binary mask over the gate projection of transformer blocks. Two practical advantages follow: the trained mask is hundreds to thousands of times smaller than the corresponding FFT checkpoint, and Deletion Tuning forgets far less, which addresses a central challenge in continual learning. Under reinforcement learning, its prior-task accuracy stays near the base level while FFT, LoRA, and random sparse fine-tuning lose up to 5.4 points. To understand why masking alone suffices, we analyze the geometry of the learned weight changes under reinforcement learning using stable rank, spectrum drift, and principal-weight overlap. Compared with LoRA and random sparse fine-tuning, Deletion Tuning is closest to FFT across the three measures.
Paper Prompts
Sign up for free to create and run prompts on this paper.
Explain it Like I'm 14
1. What is this paper about?
This paper studies a surprising way to teach a LLM new skills.
Usually, when researchers want to improve an LLM, they change many of its numbers, called weights. These numbers control how the model processes information. Changing all of them is called full fine-tuning, but it requires a lot of storage and can cause the model to forget what it already knew.
The paper asks:
Can a model learn a new skill by changing almost none of its weights—only by deciding which old weights to turn off?
The researchers find that the answer is often yes. Their method, called deletion tuning, learns which pretrained weights to set to zero while leaving all the other weights exactly as they were.
2. What questions did the researchers ask?
The main research questions were:
- Can deleting a tiny number of weights give an LLM new abilities?
- Can this work in different kinds of training?
- Can the method perform as well as full fine-tuning?
- Does deleting weights help the model remember its old abilities?
- Why can such a small change work so well?
The researchers tested three kinds of training:
- Reinforcement learning with checkable rewards: the model tries answers, and a program or evaluator gives them scores.
- Supervised fine-tuning: the model studies examples of correct answers.
- On-policy distillation: one version of the model teaches another version how to respond.
3. How did the method work?
A simple analogy
Imagine an enormous control panel with billions of switches. Each switch represents a model weight.
Ordinary fine-tuning changes the settings of many switches. Other methods add new control panels. Deletion tuning does something much simpler: it keeps almost every original switch unchanged and learns which few switches should be turned off.
The method focuses on one important part of each Transformer block called the gate projection. This part helps decide which information should pass through the model.
For every selected weight, the researchers added a small trainable value called a logit. The logit eventually makes a yes-or-no decision:
- Positive logit: keep the original weight.
- Negative logit: delete the weight by replacing it with zero.
During training, the decision is softened slightly so that normal learning algorithms can adjust it. This is similar to using a dimmer switch before finally choosing whether a light should be completely on or off.
The original model weights remain frozen. Only the mask—the collection of keep/delete decisions—is trained.
How did they evaluate it?
The researchers tested different model families and sizes, from about 0.5 billion to 30 billion parameters. They used tasks involving:
- Mathematical reasoning
- General knowledge
- Complex problem solving
- Tool use
- Science questions
They compared deletion tuning with:
- Full fine-tuning (FFT): changes every model weight.
- LoRA: adds small, trainable low-rank updates.
- Random sparse fine-tuning: changes a randomly selected small group of weights.
They measured both task accuracy and how much storage each method needed.
4. What did they find?
Deleting very few weights can match full fine-tuning
The most important result is that deletion tuning often performed about as well as full fine-tuning.
In the largest experiments, it matched or slightly exceeded full fine-tuning on:
- Mathematics with Llama and Qwen models
- MMLU-Pro and BBEH knowledge and reasoning tasks
- Tool-use tasks
- Science questions
For example, on one Llama 8B mathematics test:
- Full fine-tuning scored 83.24%.
- Deletion tuning scored 83.55%.
On a large Qwen mixture-of-experts model:
- Full fine-tuning scored 58.43%.
- Deletion tuning scored 58.54%.
The method became more effective as the models became larger.
It changed less than 0.05% of all weights
In the main reinforcement-learning experiments, deletion tuning removed fewer than 0.05% of all model weights.
That is less than 1 weight out of every 2,000. Almost the entire original model was left untouched.
This is surprising because the model still learned new abilities, especially better mathematical reasoning.
The trained mask was extremely small
Because the method stores only keep/delete decisions, its adapter—the extra information needed to modify the model—was tiny.
For example:
- Some adapters were less than 1 megabyte.
- An 8-billion-parameter model used an adapter of about 8.4 MB.
- The corresponding full fine-tuned checkpoint required about 30,633 MB.
So deletion tuning used an adapter thousands of times smaller than full fine-tuning.
It forgot less
A major problem with training an AI on a new task is catastrophic forgetting. This happens when the model becomes better at the new task but worse at older tasks.
Deletion tuning preserved the model’s earlier abilities much better than the comparison methods.
During reinforcement learning:
- Deletion tuning kept previous-task accuracy within about one percentage point of the original model.
- Full fine-tuning, LoRA, and random sparse fine-tuning sometimes lost as much as 5.4 percentage points.
This suggests that turning off a few carefully chosen weights is less disruptive than changing many weights.
The gains came from learning, not just from choosing weights intelligently at the start
The researchers first used a technique called Wanda to guess which weights were safe to delete. However, simply deleting weights based on that initial guess did not greatly improve the model.
The large improvements happened only after the mask itself was trained.
For example, on one task, the initial pruning method improved the model by only a small amount, while trained deletion tuning improved its score by more than 50 percentage points.
This shows that the important idea is not merely “delete unimportant weights.” It is:
Train the model to discover which deletions help it learn the new skill.
Its changes looked more like full fine-tuning
The researchers also studied the mathematical shape of the model’s changes. They used measures with technical names such as:
- Stable rank: roughly, how many important directions the change uses.
- Spectrum drift: how much the overall structure of a weight matrix moves.
- Principal-weight overlap: whether the changed weights are among the model’s most important original weights.
Deletion tuning looked more similar to full fine-tuning than LoRA or random sparse fine-tuning did.
This may explain why it could achieve similar accuracy even though it only deleted weights. It was free to choose which weights to delete instead of being forced to use a particular type of update.
5. Why are these results important?
The research suggests that adapting an LLM may require far fewer changes than people previously thought.
Instead of rewriting the model, it may be possible to preserve nearly all of its original knowledge and make a small number of carefully selected deletions. This could have several benefits:
- Lower storage costs: Tiny adapters are easier to save and share.
- Faster task switching: A system could load a small mask instead of a whole new model.
- Less forgetting: The model may keep its old skills while learning new ones.
- Simpler deployment: The original model stays the same, while the mask tells it which weights to ignore.
- Better understanding: Researchers could compare masks to see which parts of a model are involved in different skills.
This could be especially useful for continual learning, where an AI must learn many new tasks over time without losing its earlier knowledge.
Conclusion
The paper shows that an LLM can sometimes learn important new abilities by changing almost nothing. The researchers train a tiny binary mask that decides which pretrained weights to keep and which to delete. Despite making changes to fewer than 0.05% of the weights, the method often matches full fine-tuning.
The main lesson is that improving an AI may not require rewriting its entire “brain.” Sometimes, carefully removing a very small number of connections is enough to help it learn—while allowing it to remember what it already knows.
Practical Applications
Immediate Applications
The paper’s results support several applications that can be implemented with existing LLM infrastructure, although most should initially be treated as engineering pilots rather than universally reliable replacements for full fine-tuning.
- Compact task-specific adapters for enterprise LLMs — Industry / Software
- Train a binary deletion mask for a base model instead of storing a full fine-tuned checkpoint or floating-point adapter.
- A model-serving platform could maintain one shared base model and load small, task-specific masks for customer support, legal drafting, coding, finance, or internal knowledge tasks.
- The reported masks are substantially smaller than full fine-tuning checkpoints and, in the GRPO experiments, were smaller than rank-1 LoRA adapters.
- Potential workflow: calibration-data collection → Wanda-style initialization → mask training with SFT, reinforcement learning, or distillation → bit-packing and compression → runtime mask switching.
- Dependencies: the base checkpoint must remain available; the inference stack must efficiently apply or materialize zeros in the targeted gate-projection layers; performance must be validated on the organization’s own distribution.
- Low-cost adapter switching in multi-tenant model serving — Cloud AI / Software
- A provider could serve many specialized model variants without storing or transferring complete model copies.
- Small masks could reduce model-distribution bandwidth, cold-start latency, and storage costs when switching between domains or customers.
- This is especially relevant for large models where a full checkpoint may occupy tens or hundreds of gigabytes, while the reported adapters occupy megabytes.
- Dependencies: the base model still dominates GPU memory and compute; mask loading only reduces adaptation-storage costs, not the cost of hosting the underlying model.
- Task specialization for reasoning and verifiable-reward systems — Education, coding, mathematics, and agents
- Reinforcement learning with verifiable rewards could use deletion tuning to specialize an LLM for mathematical problem solving, code generation with unit tests, theorem proving, structured planning, or tool-use tasks.
- The paper shows improvements in pass@1 through pass@128 and reports that the method can solve some problems missed by the base model even after many sampling attempts.
- Potential products: compact “reasoning skill packs,” mathematics-specific adapters, coding-test adapters, or agent masks optimized for tool execution.
- Dependencies: rewards or verifiers must be reliable. Incorrect, exploitable, or incomplete reward functions could cause the mask to optimize undesirable behavior.
- Continual-learning prototypes with reduced catastrophic forgetting — Research / Industry
- Teams can train new task masks while preserving the original base weights, then measure whether prior-task performance remains closer to the base model than under full fine-tuning or LoRA.
- The reported reinforcement-learning experiments kept prior-task accuracy within approximately one point of the base model, whereas some baselines lost substantially more.
- Potential workflow: maintain a frozen foundation model → train one mask per new capability → evaluate new-task and prior-task suites → activate masks selectively according to the application.
- Dependencies: the paper evaluates retention primarily in limited task sequences rather than long, sequential learning histories. Mask interference across many tasks remains unresolved.
- Specialized adapters for tool-use and domain knowledge — Healthcare, science, and enterprise automation
- On-policy distillation and supervised deletion tuning can produce compact variants for tool-use, scientific question answering, chemistry, or domain-specific instruction following.
- A hospital, laboratory, or company could distribute a specialized mask alongside a centrally managed base model rather than distributing a modified model checkpoint.
- Dependencies: domain data must be representative and legally usable; medical, scientific, and financial deployments require independent accuracy, safety, and compliance testing. The reported results do not establish clinical or regulatory reliability.
- Privacy- and governance-oriented model distribution — Policy / Enterprise IT
- Organizations could distribute a base model plus signed binary masks, making task-specific model changes easier to audit, version, revoke, and compare.
- Since the adapter contains only keep/delete decisions and no newly learned floating-point values, it may be easier to inspect than a dense checkpoint, although the mask itself can still encode information about the training task.
- Potential tools: mask registries, cryptographically signed adapter packages, provenance dashboards, and automated regression testing across old and new tasks.
- Dependencies: binary masks are not automatically interpretable or privacy-preserving. Membership-inference, model-extraction, and sensitive-information leakage analyses are still required.
- Efficient academic experimentation on post-training mechanisms — Academia
- Researchers can use deletion tuning as a controlled baseline for studying which pretrained weights are necessary for acquiring new capabilities.
- Comparing masks across tasks can reveal whether different tasks delete overlapping weights and whether task-specific changes concentrate in particular transformer blocks or MLP pathways.
- The method provides a useful comparison point against full fine-tuning, LoRA, random sparse updates, pruning, and distillation.
- Dependencies: conclusions may depend on the specific model families, gate projections, datasets, reward functions, and training schedules used in the paper. Broader replication is needed.
- Task-specific variants for personal or edge applications — Daily life / Consumer software
- A personal assistant could use separate compact masks for writing, coding, travel planning, household budgeting, or study assistance while retaining one shared base model.
- Masks could be downloaded or exchanged more cheaply than full model updates, which may benefit intermittent-connectivity or storage-constrained environments.
- Dependencies: the method does not by itself make inference inexpensive enough for phones or embedded devices. Hardware support for sparse or masked computation, quantization, and memory constraints remain important.
- Incremental model-update distribution for software products — Software engineering
- Vendors could ship a stable base model once and distribute small task or version updates as masks.
- This could simplify rollback: disabling a mask restores the original base model, subject to the runtime preserving the base checkpoint unchanged.
- Dependencies: masks must be tested for compatibility with the exact base-model hash, tokenizer, quantization format, and inference engine.
Long-Term Applications
These applications follow naturally from the paper but require additional research, hardware support, or large-scale validation before dependable deployment.
- Large-scale continual learning through composable skill masks — Industry / Academia
- One mask could represent each acquired capability, with a routing system selecting masks based on the task or combining them for multi-skill behavior.
- The paper specifically suggests combining masks, for example through their union, to merge skills without retraining.
- Potential products: skill marketplaces, modular agent architectures, and lifelong-learning systems that accumulate domain capabilities over time.
- Dependencies: mask union may produce conflicting deletions, destroy useful shared behavior, or cause task interference. Composition rules, conflict resolution, and capacity limits need systematic study.
- Structured sparse adapters that both adapt and accelerate inference — Hardware / Robotics / Edge AI
- The current method uses unstructured deletion in gate projections. Future versions could constrain masks to patterns, channels, blocks, or expert structures so that the same update both adds a capability and reduces computation.
- This could support compact robotics, autonomous systems, industrial controllers, and edge assistants.
- Dependencies: the reported accuracy results do not demonstrate inference-speed gains. Hardware and compiler support for the selected sparsity pattern is essential, and unstructured masks may provide little practical acceleration.
- Composable expert libraries for mixture-of-experts models — Cloud AI / Robotics / Enterprise
- The paper demonstrates deletion tuning on a mixture-of-experts model by masking gate projections inside experts while leaving the router unchanged.
- A future system could maintain specialized expert masks for coding, vision-language interaction, scientific reasoning, or robotic control while sharing the original router and base weights.
- Dependencies: expert routing may need to adapt as masks change. The reported mixture-of-experts experiment is limited, so scalability, routing stability, and cross-expert interference remain open questions.
- Model personalization without dense parameter updates — Consumer AI / Education
- Personal assistants or tutoring systems could acquire user-specific preferences, writing styles, or study strategies through compact masks rather than modifying the entire model.
- Separate masks could represent a user’s preferred tone, language, curriculum, or accessibility settings.
- Dependencies: user data may be sensitive, and the mask could still encode personal information. Safe deletion, consent, reversibility, and protection against malicious personalization would be required.
- Safety-preserving specialization and policy enforcement — Policy / High-assurance AI
- Because deletion tuning tends to keep the base policy close to its original behavior, it could be investigated as a way to add domain capabilities while reducing unintended changes to safety, factuality, or general-purpose behavior.
- Organizations might maintain regression-tested safety masks or compare the forward KL divergence between specialized and base models as a deployment criterion.
- Dependencies: low forward KL and prior-task retention do not guarantee safety. A specialized mask could still introduce harmful behavior on rare or adversarial inputs, so formal safety evaluations and adversarial testing are necessary.
- Automated mask interpretability and capability localization — Academia / AI governance
- Comparing which gate-projection weights are deleted across tasks could support tools that map model changes to capabilities, domains, or behaviors.
- Researchers could build “mask similarity” analyses to identify shared mechanisms between mathematical reasoning, tool use, scientific knowledge, and language skills.
- Dependencies: individual weight positions may not have stable semantic meaning, and deletion patterns may vary with initialization, calibration data, model scale, or random seed. Interpretability claims require causal ablations rather than correlations alone.
- Federated or bandwidth-efficient model adaptation — Telecommunications / Public-sector AI
- Clients could train or select local masks and transmit only compressed binary updates to a central server.
- This could support federated domain adaptation for schools, hospitals, enterprises, or regional language systems without transferring full model checkpoints.
- Dependencies: the paper does not evaluate federated optimization, secure aggregation, differential privacy, or robustness to malicious masks. Binary updates may be compact but can still leak task or user information.
- Adaptive model marketplaces and reversible capability licensing — Policy / Commercial AI
- A vendor might license capabilities as signed masks rather than distributing separate model copies: for example, a coding mask, financial-analysis mask, or translation mask.
- Masks could be activated, revoked, audited, or combined under software or hardware access controls.
- Dependencies: capability isolation is not guaranteed. A mask may depend on other masks or on latent capabilities already present in the base model, complicating licensing, attribution, and enforcement.
- Deletion-based adaptation beyond LLMs — Vision, multimodal systems, and robotics
- The principle could be tested on vision transformers, multimodal models, speech systems, and robotic policies, particularly in architectures with multiplicative gates or expert pathways.
- Potential uses include adapting a robot to a new environment, specializing a vision model for manufacturing defects, or adding a new speech domain while preserving existing recognition abilities.
- Dependencies: the paper’s evidence is concentrated on LLMs and gate projections. Other architectures may not exhibit the same geometry or may require different mask targets and training relaxations.
- Training-memory reduction for very large models — High-performance computing
- Since the base weights remain frozen and only mask logits are optimized, deletion tuning could reduce trainable-state requirements compared with full fine-tuning, especially when optimizer states for all model parameters are otherwise required.
- This may make post-training of large models more accessible to smaller research groups or reduce multi-GPU requirements.
- Dependencies: the paper notes that deletion tuning still uses more training memory than LoRA because it maintains one trainable logit per targeted weight. The method therefore may not be advantageous for memory-constrained training unless candidate-weight restriction or block-level parameterization is developed.
Glossary
- AdamW: An optimizer that combines Adam with decoupled weight decay for training neural networks. “All runs use AdamW with fp32 master weights, bf16 mixed precision, and gradient checkpointing.”
- Annealing: Gradually changing a parameter during training, often to make an approximation increasingly discrete or sharp. “During training, a sigmoid of each logit is annealed toward a step function”
- Attention layer: A transformer component that computes relationships between token representations to determine which tokens should influence one another. “each averaged over all attention and MLP linear layers”
- Binary mask: A mask whose elements have only two possible values, typically indicating whether a parameter is retained or removed. “We propose , which learns only a binary mask over the gate-projection weights of transformer blocks.”
- bf16: The bfloat16 numerical format, commonly used for memory-efficient mixed-precision deep-learning computation. “All runs use AdamW with fp32 master weights, bf16 mixed precision, and gradient checkpointing.”
- Catastrophic forgetting: The loss of previously learned capabilities when a model is trained on new data or tasks. “SDFT uses it to reduce catastrophic forgetting”
- Calibration prompts: Input prompts used to measure model activations or other statistics for initializing or configuring a model method. “calibration prompts from the task's training distribution”
- Continual learning: Learning new tasks while preserving performance on previously learned tasks. “Learning new tasks while keeping old ones makes a promising candidate for continual learning.”
- Cross-entropy: A loss function that measures the discrepancy between a predicted probability distribution and a target distribution. “SFT. We minimize cross-entropy with a constant learning rate”
- Exponential moving average (EMA): A running weighted average that gives greater weight to recent values, often used to stabilize model parameters. “The teacher's weights are an exponential moving average of the student's weights ”
- Forward KL divergence: A directional measure of difference between two probability distributions, used here to compare a fine-tuned policy with a base policy. “Forward KL on the new task predicts how much a fine-tuned model forgets”
- Fully sharded data parallelism (FSDP): A distributed-training technique that shards model parameters, gradients, and optimizer states across multiple devices. “using one GPU for backbones up to $3$B and multiple GPUs with FSDP”
- Gate projection: The transformer MLP projection that controls which intermediate channels are activated in a gated linear unit. “In all our experiments, the target is the gate projection of transformer blocks”
- Gradient checkpointing: A memory-saving technique that recomputes intermediate activations during backpropagation instead of storing all of them. “All runs use AdamW with fp32 master weights, bf16 mixed precision, and gradient checkpointing.”
- Group Relative Policy Optimization (GRPO): A reinforcement-learning method that estimates response advantages by comparing rewards within groups of sampled responses rather than learning a value function. “For RLVR, we use [PERSON_NAME]}.”
- Hard mask: A discrete mask that makes a definitive retain-or-delete decision for each parameter. “We call the soft mask and the hard mask.”
- Importance ratio: The ratio between the probability assigned to an action by the updated policy and the previous policy in policy-gradient optimization. “where $\rho^i_j = \pi_\theta(y^i_j \mid x, y^i_{<j})\,/\,\pi_{\theta_\text{old}(y^i_j \mid x, y^i_{<j})$ is the importance ratio for token .”
- Mixture-of-experts (MoE): A model architecture containing multiple expert subnetworks, with a router selecting which experts process each input. “the Qwen3-30B-A3B mixture-of-experts model”
- Negative log-likelihood: The negative logarithm of the probability assigned to observed target outcomes, commonly used as a probabilistic training loss. “SFT minimizes the negative log-likelihood of the target tokens”
- On-policy distillation: Knowledge distillation in which the student generates its own training samples and learns from a teacher on those samples. “For on-policy distillation, we use SDFT~\citep{sdft}, in which the model is its own teacher.”
- Principal-weight overlap: The proportion of changed parameters that belong to the highest-magnitude weights identified from a low-rank approximation of the original model. “Principal-weight overlap measures the fraction of changed weight positions that fall on what~\citet{pathnottaken} call the base model's principal weights.”
- Quantile: A value that divides a distribution into specified proportions. “whether the weight is above or below the -quantile of ”
- Reinforcement learning with verifiable rewards (RLVR): Reinforcement learning in which generated outputs receive rewards from rule-based checkers or verifiers. “We push this observation to its limit across reinforcement learning with verifiable rewards (RLVR), supervised fine-tuning (SFT), and on-policy distillation (OPD)”
- Singular value decomposition (SVD): A factorization of a matrix into orthogonal directions and associated nonnegative singular values. “the top-magnitude entries of the rank- SVD reconstruction of ”
- Soft mask: A differentiable, continuous approximation to a binary mask, typically containing values between zero and one. “We call the soft mask and the hard mask.”
- Spectrum drift: A normalized measure of how much a matrix’s singular values change after model training. “Spectrum drift is the normalized singular-value shift”
- Stable rank: A continuous analogue of matrix rank defined as the squared Frobenius norm divided by the squared spectral norm. “Stable rank of is $\lVert \Delta W \rVert_F2 / \lVert \Delta W \rVert_22”
- Straight-through or sigmoid relaxation: A differentiable approximation that permits gradient-based optimization of an otherwise discrete masking decision. “During training, a sigmoid of each logit is annealed toward a step function”
- Supervised fine-tuning (SFT): Training a pretrained model to imitate fixed, labeled target outputs. “Supervised fine-tuning (SFT) imitates fixed labeled targets”
- Token-level distribution: A probability distribution over possible next tokens at a particular sequence position. “the teacher's per-token distribution”
- Unstructured pruning: Removing individual weights without imposing a regular structure on their locations. “In pruning terms, performs learned unstructured pruning to fine-tune the model.”
- Verifier: A rule-based system or LLM that evaluates generated outputs and assigns rewards. “With verifiable rewards, rule-based checkers or LLM verifiers score each sample”
- Wanda score: A pruning-importance score based on a weight’s magnitude multiplied by the norm of its input activation. “The [PERSON_NAME] score of a weight is its magnitude times the norm of its input activation”
- Weight delta: The elementwise difference between a trained parameter matrix and its pretrained counterpart. “and the weight delta”
- Zero-shot pruning: Removing parameters using an importance criterion without training the model after pruning. “We apply [PERSON_NAME] one-shot pruning without GRPO training at $8$ sparsity levels”


