Gradient Suppression Fine Tuning

• Gradient Suppression Fine Tuning is a collection of methods that control gradient updates through masking, projection, or attenuation to prevent destructive changes.
• These approaches balance safety, alignment, and robust task performance by selectively limiting gradients, ensuring retention of pretrained priors.
• Empirical results demonstrate improved out-of-distribution performance and alignment by mitigating harmful gradient influences during fine-tuning.

Searching arXiv for papers on gradient suppression and closely related fine-tuning methods. First, I’ll look up the main SafeGrad paper and then a broader set of related methods on gradient masking, projection, and selective updating. Searching for: (Yi et al., 10 Aug 2025) Gradient Surgery for Safe LLM Fine-Tuning Gradient suppression fine tuning can be understood as an umbrella label for fine-tuning methods that deliberately constrain how gradient information changes a pretrained model. Across the recent literature, suppression may be direct—by masking low-magnitude gradients, projecting away conflicting components, or downweighting harmful samples—or indirect, by projecting parameter updates back toward the pretrained initialization, intervening on hidden representations only behind learned gates, or removing adaptation-time gradients altogether. The recurring goal is to preserve utility, safety alignment, robustness, or pretrained priors while avoiding destructive updates during adaptation (Yi et al., 10 Aug 2025, Li et al., 2024, Tian et al., 2023, Nguyen et al., 28 Feb 2026, Kim et al., 21 Apr 2025).

1. Scope and taxonomy

The label is not standardized in a single paper. Across the cited work, it covers several technically distinct interventions that all reduce the effect of updates judged harmful, low-value, or misaligned with the intended adaptation objective. This suggests that the topic is best treated as a family of gradient-control strategies rather than a single algorithmic recipe.

Mechanism family	Characteristic operation	Representative papers
Projection and conflict resolution	Remove or reorient conflicting gradient components	SafeGrad (Yi et al., 10 Aug 2025); Bucket-Level MOO (Hoang et al., 4 Jun 2026)
Elementwise masking and sparsification	Zero or retain only selected gradient entries	GMT (Li et al., 2024); GradDrop (Neill et al., 2023); SGC (Yang et al., 1 Feb 2025)
Structured group selection	Update only chosen rows, columns, blocks, or layers	GPS (Zhang et al., 2023); GRFT (Liu et al., 14 Jun 2025); AdaGradSelect (Kumar et al., 12 Dec 2025)
Example- or loss-level attenuation	Downweight some sample gradients or reshape logit gradients	Antibody (Nguyen et al., 28 Feb 2026); DSGD (Bui et al., 30 Apr 2026); DSPT (Li et al., 1 May 2026)
Projected-update or representation-level suppression	Constrain parameter drift or gate hidden-state edits	TPGM (Tian et al., 2023); GRUN (Ren et al., 25 Feb 2025); DGS-Net (Yan et al., 17 Nov 2025)
Gradient-free adaptation-time emulation	Replace test-time gradient updates with learned dynamics	HyperFlow (Kim et al., 21 Apr 2025)

A central distinction is between raw-gradient suppression and effective-update suppression. Methods such as GMT or GradDrop explicitly zero or mask gradient entries before the optimizer step. By contrast, TPGM leaves the task gradient intact but projects the resulting parameter iterate back into a learned trust region around the pretrained model, thereby suppressing realized parameter drift rather than the raw gradient itself. A second distinction is between static and dynamic suppression. GPS and GRFT choose a sparse trainable structure once before or at the start of fine-tuning, whereas SafeGrad, Antibody, DSGD, and GradDrop recompute the relevant suppression signal during training.

The granularity also varies. Some methods operate on coordinates or tensors, some on layers or blocks, some on samples within a batch, and some on feature-space directions. This heterogeneity is important: the literature does not support reducing the topic to “gradient clipping” or “smaller learning rates.” In several papers, the key object being suppressed is directional conflict, not gradient norm per se.

2. Projection-based suppression and gradient conflict resolution

A prominent line of work treats fine-tuning as a multi-objective problem in which destructive updates arise from gradient conflict. SafeGrad formulates safe LLM fine-tuning under poisoned user data as

$$\min_{\theta} \left( \mathcal{L}_{\text{user}}(\theta), \mathcal{L}_{\text{align}}(\theta) \right),$$

with standard practice approximated by a weighted sum

$$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$

It diagnoses failure through negative cosine similarity between

$$g_{\text{user}}=\nabla_\theta \mathcal{L}_{\text{user}}(\theta), \qquad g_{\text{align}}=\nabla_\theta \mathcal{L}_{\text{align}}(\theta),$$

namely

$$\cos(g_{\text{user}},g_{\text{align}})=\frac{g_{\text{user}}\cdot g_{\text{align}}}{\|g_{\text{user}}\|\,\|g_{\text{align}}\|}<0.$$

The reported cosine similarities shift from $0.02$ at harmful ratio $0.00$ to $-0.05$, $-0.09$, $-0.13$, and $-0.16$ as the harmful ratio rises to $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$0, $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$1, $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$2, and $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$3, respectively. SafeGrad then performs conditional projection: $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$4 when $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$5, so that $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$6 in the conflict case. The final update is

$$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$7

The paper is explicit that this is not global shrinking of the user gradient; it removes only the anti-alignment component and reorients the update into the orthogonal plane of the alignment gradient (Yi et al., 10 Aug 2025).

A distributed multilingual analogue appears in Bucket-Level MOO, which reformulates multilingual fine-tuning as a multi-objective optimization problem over per-language losses and resolves gradient conflict locally on parameter buckets rather than on full gradients. For bucket-level PCGrad, when

$$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$8

the local task update is replaced by

$$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$9

The method also instantiates local MGDA and CAGrad, and argues theoretically that bucket-local conflict resolution enforces Refined Pareto Stationarity, a stricter necessary condition than standard Pareto Stationarity. Empirically, conventional global MOO on Qwen3-4B-Base with 8 H200 GPUs required 123 GB peak VRAM, whereas Bucket-Level MOO stayed within the 72 GB footprint of vanilla SFT (Hoang et al., 4 Jun 2026).

TPGM occupies a related but distinct position. It does not suppress raw gradients directly; instead it suppresses the effective parameter update by projecting each layer back toward the pretrained initialization. Its L2 projection is

$$g_{\text{user}}=\nabla_\theta \mathcal{L}_{\text{user}}(\theta), \qquad g_{\text{align}}=\nabla_\theta \mathcal{L}_{\text{align}}(\theta),$$0

with layer-specific radii $$g_{\text{user}}=\nabla_\theta \mathcal{L}_{\text{user}}(\theta), \qquad g_{\text{align}}=\nabla_\theta \mathcal{L}_{\text{align}}(\theta),$$1 learned by bilevel optimization. This turns update suppression into a learned trust-region constraint around $$g_{\text{user}}=\nabla_\theta \mathcal{L}_{\text{user}}(\theta), \qquad g_{\text{align}}=\nabla_\theta \mathcal{L}_{\text{align}}(\theta),$$2. The reported result is $$g_{\text{user}}=\nabla_\theta \mathcal{L}_{\text{user}}(\theta), \qquad g_{\text{align}}=\nabla_\theta \mathcal{L}_{\text{align}}(\theta),$$3 and $$g_{\text{user}}=\nabla_\theta \mathcal{L}_{\text{user}}(\theta), \qquad g_{\text{align}}=\nabla_\theta \mathcal{L}_{\text{align}}(\theta),$$4 relative OOD improvement, respectively, when fine-tuned on DomainNet-Real and ImageNet and evaluated on their sketch counterparts (Tian et al., 2023).

3. Masking, sparsification, and selective update regimes

The most literal form of gradient suppression is dynamic masking. GMT computes accumulated gradients over an interval of $$g_{\text{user}}=\nabla_\theta \mathcal{L}_{\text{user}}(\theta), \qquad g_{\text{align}}=\nabla_\theta \mathcal{L}_{\text{align}}(\theta),$$5 mini-batches,

$$g_{\text{user}}=\nabla_\theta \mathcal{L}_{\text{user}}(\theta), \qquad g_{\text{align}}=\nabla_\theta \mathcal{L}_{\text{align}}(\theta),$$6

constructs saliency from $$g_{\text{user}}=\nabla_\theta \mathcal{L}_{\text{user}}(\theta), \qquad g_{\text{align}}=\nabla_\theta \mathcal{L}_{\text{align}}(\theta),$$7, keeps only a top percentile, and updates

$$g_{\text{user}}=\nabla_\theta \mathcal{L}_{\text{user}}(\theta), \qquad g_{\text{align}}=\nabla_\theta \mathcal{L}_{\text{align}}(\theta),$$8

Its defining property is that the mask is recomputed during training from task-specific gradient magnitude, so the suppressed subset is temporary rather than permanent (Li et al., 2024).

GradDrop introduces stochastic masked fine-tuning for Transformers by replacing each gradient tensor with

$$g_{\text{user}}=\nabla_\theta \mathcal{L}_{\text{user}}(\theta), \qquad g_{\text{align}}=\nabla_\theta \mathcal{L}_{\text{align}}(\theta),$$9

before the optimizer step, using either elementwise Bernoulli masks or layerwise binary masks. Layer-GradDrop and GradDrop-Epoch are the strongest variants in the reported XGLUE experiments with XLM-R-Large, reaching average scores of $$\cos(g_{\text{user}},g_{\text{align}})=\frac{g_{\text{user}}\cdot g_{\text{align}}}{\|g_{\text{user}}\|\,\|g_{\text{align}}\|}<0.$$0 and $$\cos(g_{\text{user}},g_{\text{align}})=\frac{g_{\text{user}}\cdot g_{\text{align}}}{\|g_{\text{user}}\|\,\|g_{\text{align}}\|}<0.$$1, versus $$\cos(g_{\text{user}},g_{\text{align}})=\frac{g_{\text{user}}\cdot g_{\text{align}}}{\|g_{\text{user}}\|\,\|g_{\text{align}}\|}<0.$$2 for standard fine-tuning. The method is framed as backward-pass regularization: sparse and stochastic, unlike deterministic gradual unfreezing (Neill et al., 2023).

Other methods impose sparsity through trainable-structure selection rather than per-step masking. GPS computes task-specific gradients from supervised contrastive loss, selects the top-$$\cos(g_{\text{user}},g_{\text{align}})=\frac{g_{\text{user}}\cdot g_{\text{align}}}{\|g_{\text{user}}\|\,\|g_{\text{align}}\|}<0.$$3 input connections per neuron, freezes the rest, and then fine-tunes with

$$\cos(g_{\text{user}},g_{\text{align}})=\frac{g_{\text{user}}\cdot g_{\text{align}}}{\|g_{\text{user}}\|\,\|g_{\text{align}}\|}<0.$$4

Across 24 FGVC and VTAB tasks, it tunes about $$\cos(g_{\text{user}},g_{\text{align}})=\frac{g_{\text{user}}\cdot g_{\text{align}}}{\|g_{\text{user}}\|\,\|g_{\text{align}}\|}<0.$$5 of parameters on average and reports $$\cos(g_{\text{user}},g_{\text{align}})=\frac{g_{\text{user}}\cdot g_{\text{align}}}{\|g_{\text{user}}\|\,\|g_{\text{align}}\|}<0.$$6 versus $$\cos(g_{\text{user}},g_{\text{align}})=\frac{g_{\text{user}}\cdot g_{\text{align}}}{\|g_{\text{user}}\|\,\|g_{\text{align}}\|}<0.$$7 on FGVC and $$\cos(g_{\text{user}},g_{\text{align}})=\frac{g_{\text{user}}\cdot g_{\text{align}}}{\|g_{\text{user}}\|\,\|g_{\text{align}}\|}<0.$$8 versus $$\cos(g_{\text{user}},g_{\text{align}})=\frac{g_{\text{user}}\cdot g_{\text{align}}}{\|g_{\text{user}}\|\,\|g_{\text{align}}\|}<0.$$9 on VTAB (Zhang et al., 2023).

GRFT makes the same basic move at row or column granularity. For a gradient matrix $0.02$0, it scores row $0.02$1 by

$0.02$2

and updates only the selected rows or columns. The paper reports updating only $0.02$3 and $0.02$4 of total parameters on FGVC and VTAB, respectively, and $0.02$5 on GLUE with LLaMA3-1B (Liu et al., 14 Jun 2025).

AdaGradSelect uses gradient norms to select transformer blocks during an exploration phase, then replaces repeated norm ranking with a Dirichlet-based sampling policy over historical block-selection frequencies. The exploration schedule decays as

$0.02$6

after which exploitation is driven by $0.02$7 and $0.02$8. The reported effect is about $0.02$9 faster training and $0.00$0 less GPU memory while remaining close to full fine-tuning (Kumar et al., 12 Dec 2025).

SGC is a stronger sparsification regime. It first keeps only the top-$0.00$1 gradient coordinates,

$0.00$2

then projects them into a $0.00$3-dimensional subspace, updates compressed Adam moments there, and reconstructs sparse full-dimensional updates with OMP. This couples coordinate-level suppression with optimizer-state compression and yields a state dimension that is independent of the original layer width (Yang et al., 1 Feb 2025).

GradPruner is adjacent but conceptually distinct. It uses early accumulated gradients to build an IGIA-Matrix, rank layers, prune the least important ones, and merge sparse sign-consistent remnants into retained layers. The paper explicitly frames this as gradient-guided structured pruning rather than as direct per-step gradient suppression, although pruning subsequently suppresses all future updates to removed layers (Huang et al., 27 Jan 2026).

4. Example-level attenuation and loss-level intrinsic suppression

Another major family operates not on tensor coordinates but on which examples are allowed to dominate the batch gradient. Antibody does this in two stages. First, before user fine-tuning, it optimizes the model into a flat harmful-loss region by minimizing

$0.00$4

with

$0.00$5

Then, during fine-tuning, it computes per-sample scores

$0.00$6

relative to a refusal completion $0.00$7, turns them into normalized weights

$0.00$8

and updates with weighted sample gradients. Harmful samples therefore receive smaller coefficients in the batch update. On Llama-2-7B across datasets, the reported average Harmful Score is $0.00$9, versus $-0.05$0 for Lisa, with component ablations showing large gains from both sharpness alignment and weighted fine-tuning (Nguyen et al., 28 Feb 2026).

DSGD suppresses gradients from correctly classified examples rather than from harmful ones. With a batch split into misclassified and correctly classified subsets, it replaces the standard batch gradient by

$-0.05$1

and updates

$-0.05$2

where the default schedule is

$-0.05$3

The paper’s claim is that correctly classified examples can cancel the corrective signal from misclassified ones, especially on sparse or imbalanced classification data. On RoBERTa-large across six NLP tasks, DSGD reports average $-0.05$4 ACC and $-0.05$5 STD, versus $-0.05$6 ACC and $-0.05$7 STD for full fine-tuning (Bui et al., 30 Apr 2026).

DSPT moves suppression into the loss geometry itself. Standard prompt tuning on CLIP uses

$-0.05$8

whereas DSPT uses

$-0.05$9

For a single sample, with $-0.09$0 and $-0.09$1, the paper derives

$-0.09$2

The multiplicative factor $-0.09$3 creates what the paper calls a self-adaptive saturation zone. Theorem 3.2 states that when the model makes a high-confidence prediction inconsistent with the noisy label, the gradient magnitude converges to zero for all logits. Empirically, DSPT reports, for example, $-0.09$4 on Caltech101 under symmetric $-0.09$5 noise (Li et al., 1 May 2026).

5. Representation-level suppression, gated interventions, and gradient-free boundaries

A different branch of the literature treats suppression as a representation-space intervention rather than a modification of the raw optimizer gradient. Structured Gradient Guidance for few-shot LLM adaptation adds three gradient-aware regularizers: a direction-consistency term, a magnitude-control term, and a source-target gradient-alignment term. The paper frames this as suppressing noisy, unstable, cross-task conflicting, or abnormally large updates rather than shrinking all gradients uniformly. On SuperGLUE few-shot tasks, it reports $-0.09$6 accuracy, gradient stability $-0.09$7, and directional alignment $-0.09$8, versus $-0.09$9, $-0.13$0, and $-0.13$1 for prompt-tuning (Zheng et al., 31 May 2025).

GRUN makes selective suppression explicit in LLM unlearning. It argues that both gradient-ascent-based and suppression-based unlearning succeed by distinguishing target data and suppressing related generations, then implements this mechanism with a gated ReFT intervention: $-0.13$2 trained under

$-0.13$3

The base model is frozen, and the gate is trained to open on target data and close on retain data. On TOFU, GRUN+NPO improves both forgetting and utility; for Llama with $-0.13$4 target data, vanilla NPO reports unlearn ROUGE-L Recall $-0.13$5 and utility $-0.13$6, whereas GRUN+NPO reports $-0.13$7 and $-0.13$8 (Ren et al., 25 Feb 2025).

DGS-Net applies gradient surgery at the feature level for CLIP fine-tuning. It decomposes the text-branch gradient into a harmful component

$-0.13$9

and the frozen teacher-image gradient into a beneficial component

$-0.16$0

The student image-task gradient is projected off the harmful direction,

$-0.16$1

then combined with prior alignment,

$-0.16$2

This is explicit suppression of a harmful direction rather than generic regularization. The main ablation reports GenImage $-0.16$3 and AIGIBench $-0.16$4 for baseline, Prior Alignment only, Orthogonal Suppression only, and full DGS-Net, respectively (Yan et al., 17 Nov 2025).

HyperFlow marks the outer boundary of the topic. It does not modify gradients during adaptation; instead, it removes them at adaptation time by learning a task-conditional drift network

$-0.16$5

and then integrates

$-0.16$6

without target-model backpropagation. The paper explicitly states that the method is gradient-free only at adaptation time, not during offline training. This suggests a limiting case of gradient suppression in which adaptation-time gradients are eliminated rather than merely attenuated (Kim et al., 21 Apr 2025).

6. Empirical regularities, trade-offs, and conceptual boundaries

Taken together, the papers suggest several recurring empirical regularities. First, harmful or unhelpful gradients are often detectable by structure: by negative cosine similarity in multi-objective settings, by low magnitude, by example correctness, by target-versus-refusal likelihood, by representation source, or by sign-split feature gradients. SafeGrad reports cosine similarity moving from $-0.16$7 to $-0.16$8 as harmful ratio increases, and its post-surgery cosine is effectively clamped to zero (Yi et al., 10 Aug 2025). Antibody reports lower harmful gradient norms after flatness alignment and clear separation between harmful and benign weighting scores (Nguyen et al., 28 Feb 2026). DSPT shows noisy-sample logit gradients compressed toward zero under double softmax (Li et al., 1 May 2026). DGS-Net finds that using exactly the negative half of teacher-image gradients and the positive half of text gradients gives the best sign-split variant (Yan et al., 17 Nov 2025). This suggests that fine-tuning failures often arise not from gradients being large in the abstract, but from specific subspaces or samples being allowed to dominate updates.

Second, suppression rarely comes for free. SafeGrad’s reported GPU memory-time is $-0.16$9 GB·Hour for SFT, $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$00 for Lisa, $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$01 for SafeGrad with SFT alignment, and $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$02 for SafeGrad with KL alignment, with the extra forward pass through a frozen reference model identified as a major cost (Yi et al., 10 Aug 2025). SGC can drastically reduce optimizer-state size, but MESGC reports $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$03 seconds per iteration versus $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$04 for GaLore, $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$05 for LoRA, and $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$06 for full fine-tuning, so aggressive sparsification can exchange memory savings for runtime overhead (Yang et al., 1 Feb 2025). HyperFlow moves cost out of deployment and into meta-training: on a 40-way, 5-shot episode, the reported peak memory is $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$07 MB for 50-step fine-tuning and $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$08 MB for HyperFlow ODE solving, while the table supports about $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$09 rather than $$\min_{\theta} \mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{\text{user}}(\theta)+\rho\cdot \mathcal{L}_{\text{align}}(\theta).$$10 of the 50-step fine-tuning time, a discrepancy the paper itself effectively acknowledges (Kim et al., 21 Apr 2025).

Third, not every gradient-guided method is a direct instance of suppression. TPGM is best described as projected-update suppression, because it constrains layerwise distance from the pretrained model rather than modifying the raw gradient vector (Tian et al., 2023). GradPruner is best described as gradient-guided structured pruning, because it uses early gradients to decide what to remove, after which the removed layers no longer receive updates (Huang et al., 27 Jan 2026). These cases are adjacent to the topic, but they occupy the boundary between gradient suppression, selective updating, and structural model reduction.

A common misconception is that suppression is equivalent to globally shrinking learning rates or clipping norms. The cited work does not support that simplification. SafeGrad removes only the anti-alignment component; Antibody changes per-sample coefficients; DSGD rescales correctly classified examples but not misclassified ones; DSPT reshapes the loss so that certain high-confidence noisy cases enter a saturation zone; GRUN and DGS-Net intervene at the representation level; HyperFlow removes adaptation-time gradients entirely. The literature therefore points to a broader principle: effective fine-tuning often depends less on whether gradients exist, and more on which gradients are allowed to matter, at what granularity, and in what geometric form.