Dynamic Entropy Fine-Tuning (DEFT)

• Dynamic Entropy Fine-Tuning (DEFT) is a supervised fine-tuning method that dynamically scales token gradients based on the model’s predictive concentration, mitigating harmful updates from noisy labels.
• It uses Rényi-2 entropy as a proxy to assess model confidence, allowing a smooth transition from broad exploratory learning at low certainty to focused refinement when predictions are confident.
• Empirical evaluations demonstrate that DEFT enhances performance and stability across various regimes by effectively balancing exploratory coverage and sharpening of predictions.

Dynamic Entropy Fine-Tuning (DEFT) is a supervised fine-tuning objective for pretrained LLMs that replaces the uniform token weighting of standard negative log-likelihood with a dynamic, prediction-state-dependent trust gate. In the formulation introduced in “Gradients Must Earn Their Influence: Unifying SFT with Generalized Entropic Objectives” (Wang et al., 11 Feb 2026), DEFT is motivated by a two-fold failure mode of conventional NLL-based SFT: it overemphasizes low-probability targets, which can amplify harmful updates under noisy or conflicting supervision, and it provides weak sharpening when the model is already confident. DEFT addresses this by modulating token gradients according to the concentration of the model’s full predictive distribution, using Rényi-2 entropy as a practical proxy for predictive state. In this sense, DEFT is neither a generic name for any entropy-aware training scheme nor a synonym for other uses of the acronym “DEFT” in diffusion, sparsity, or edge systems; it refers specifically to a dynamic entropic objective for LLM supervised fine-tuning (Wang et al., 11 Feb 2026).

1. Definition and conceptual scope

DEFT is defined as a token-level SFT objective in which the gradient on each supervised target token is multiplicatively gated by a dynamic function of the model’s current predictive concentration (Wang et al., 11 Feb 2026). The underlying problem is the plasticity–stability dilemma in post-pretraining adaptation. Standard NLL remains maximally responsive to low target probability, which is beneficial when the model lacks knowledge, but it is also vulnerable to noisy labels, atypical formatting artifacts, and confident conflicts between pretrained priors and downstream supervision. At the same time, NLL weakens linearly as target probability approaches one, which limits sharpening of already-correct predictions (Wang et al., 11 Feb 2026).

The DEFT paper situates this problem within a broader family of generalized entropic objectives. Rather than treating cross-entropy as uniquely canonical, it views token-level SFT losses as members of a family indexed by a scalar trust gate controlling how much the model should trust its current predictive state. This suggests that the main distinction among such objectives is not their gradient direction, which remains the same under softmax parameterization, but the confidence-dependent scalar that scales that direction (Wang et al., 11 Feb 2026).

A common misconception is that DEFT means adding an entropy bonus to the loss. In the paper’s formulation, DEFT does not directly maximize predictive entropy. Instead, it uses distribution concentration to decide whether learning should behave more like broad-coverage NLL or more like sharpening-oriented probability weighting. This suggests that its entropy dependence is indirect but structural: entropy informs gradient trust rather than serving as an optimization target in its own right (Wang et al., 11 Feb 2026).

2. Mathematical formulation

The paper defines a general token-level SFT objective by applying a differentiable nonincreasing scalar function $f$ to the target-token probability $p = p_\theta(\tilde y \mid c)$, where $c$ is the token context and $\tilde y$ the supervised target token:

$$\mathcal{L}_f(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[f\!\left(p_\theta(\tilde y\mid c)\right)\right].$$

Standard NLL is the special case

$$\mathcal{L}_{\mathrm{NLL}}(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[-\log p_\theta(\tilde y\mid c)\right].$$

A central result is that all such objectives induce the same softmax-gradient structure up to a scalar gate:

$$\frac{\partial \mathcal{L}_f}{\partial z_i(c)} = s_f(p)\,\bigl(p_\theta(i\mid c)-\delta_{i,\tilde y}\bigr),$$

with

$s_f(p)\triangleq -f'(p)\,p \ge 0.$

For the target logit, the gradient magnitude is

$W_f(p) = s_f(p)(1-p).$

The paper interprets this universally as a “gate $\times$ error” decomposition, with $p = p_\theta(\tilde y \mid c)$0 as the prediction error and $p = p_\theta(\tilde y \mid c)$1 as the trust gate (Wang et al., 11 Feb 2026). Under NLL, $p = p_\theta(\tilde y \mid c)$2, so the gate is permanently open.

DEFT is built on a deformed-log family parameterized by $p = p_\theta(\tilde y \mid c)$3, where the token loss is

$p = p_\theta(\tilde y \mid c)$4

with gradient

$p = p_\theta(\tilde y \mid c)$5

Here the gate is $p = p_\theta(\tilde y \mid c)$6. Small $p = p_\theta(\tilde y \mid c)$7 recovers NLL-like behavior; $p = p_\theta(\tilde y \mid c)$8 yields linear probability loss; intermediate values interpolate between them (Wang et al., 11 Feb 2026).

The paper derives an “ideal” confidence-to-focus trajectory using the Cayley transform. With uncertainty radius

$p = p_\theta(\tilde y \mid c)$9

the focus index is

$c$0

This satisfies the desired anchors $c$1 and $c$2, suggesting a continuous path from exploratory, NLL-like learning at low confidence to sharpening-oriented learning at high confidence (Wang et al., 11 Feb 2026).

The practical DEFT objective then replaces target-probability-only focus with a distribution-level proxy. For predictive distribution $c$3, DEFT defines

$c$4

This is the concentration of the predictive distribution and equals the exponential of negative Rényi-2 entropy:

$c$5

The resulting gate is

$c$6

and the target-logit learning signal becomes

$c$7

The paper states that $c$8 is treated as a stop-gradient quantity during optimization (Wang et al., 11 Feb 2026).

3. Entropic interpretation and gradient behavior

The theoretical motivation for DEFT is tied to an optimization–entropy duality. In the deformed-log family, the loss index $c$9 corresponds to a Tsallis entropy index $\tilde y$0, with the relation

$\tilde y$1

This means that tuning the gate changes not only gradient magnitudes but also the implied entropy geometry of the learning objective (Wang et al., 11 Feb 2026). Small $\tilde y$2 is associated with Shannon-like coverage; large $\tilde y$3 with concentration and sharpening.

The decisive practical move in DEFT is to use full-distribution concentration rather than target probability alone. Target probability is ambiguous: low $\tilde y$4 may indicate genuine ignorance, in which case learning should remain strong, or confident disagreement, in which case aggressive correction may damage useful priors. By using

$\tilde y$5

the method distinguishes these cases at the level of predictive state. A diffuse distribution yields small $\tilde y$6, so the gate behaves close to NLL; a sharply concentrated distribution yields large $\tilde y$7, so the gate suppresses low-probability targets and emphasizes refinement (Wang et al., 11 Feb 2026).

The appendix result highlighted in the paper makes this explicit. If a non-target token $\tilde y$8 has probability at least $\tilde y$9, then

$$\mathcal{L}_f(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[f\!\left(p_\theta(\tilde y\mid c)\right)\right].$$0

and therefore

$$\mathcal{L}_f(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[f\!\left(p_\theta(\tilde y\mid c)\right)\right].$$1

which vanishes as $$\mathcal{L}_f(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[f\!\left(p_\theta(\tilde y\mid c)\right)\right].$$2. This means DEFT can strongly suppress updates under confident misalignment. By contrast, the Cayley target-probability trajectory alone remains fully open in the $$\mathcal{L}_f(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[f\!\left(p_\theta(\tilde y\mid c)\right)\right].$$3 limit, which does not distinguish ignorance from confident conflict (Wang et al., 11 Feb 2026).

This suggests that the distinctive contribution of DEFT is not merely dynamic interpolation, but dynamic interpolation anchored in whole-distribution predictive structure.

4. Empirical evidence and benchmark behavior

The DEFT paper evaluates the method across four regimes: model-strong, model-intermediate, model-weak, and mixed SFT data (Wang et al., 11 Feb 2026). The reported baselines are standard NLL $$\mathcal{L}_f(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[f\!\left(p_\theta(\tilde y\mid c)\right)\right].$$4, linear probability loss $$\mathcal{L}_f(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[f\!\left(p_\theta(\tilde y\mid c)\right)\right].$$5, EAFT, Cayley-Trans, and DEFT.

In the model-strong regime, where pretrained priors are already competent, DEFT outperforms NLL and typically also static alternatives. For example, on LLaMA-3.1-8B the reported average rises from $$\mathcal{L}_f(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[f\!\left(p_\theta(\tilde y\mid c)\right)\right].$$6 under $$\mathcal{L}_f(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[f\!\left(p_\theta(\tilde y\mid c)\right)\right].$$7 to $$\mathcal{L}_f(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[f\!\left(p_\theta(\tilde y\mid c)\right)\right].$$8 under DEFT, and on Qwen2.5-Math-7B from $$\mathcal{L}_f(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[f\!\left(p_\theta(\tilde y\mid c)\right)\right].$$9 to $$\mathcal{L}_{\mathrm{NLL}}(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[-\log p_\theta(\tilde y\mid c)\right].$$0 (Wang et al., 11 Feb 2026). The paper interprets this as evidence that confident-state sharpening and conflict suppression matter when the model already has substantial relevant knowledge.

In the model-intermediate regime, DEFT remains competitive and is often best on average. One reported example gives LLaMA-3.1-8B average performance of $$\mathcal{L}_{\mathrm{NLL}}(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[-\log p_\theta(\tilde y\mid c)\right].$$1 for NLL, $$\mathcal{L}_{\mathrm{NLL}}(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[-\log p_\theta(\tilde y\mid c)\right].$$2 for Cayley-Trans, and $$\mathcal{L}_{\mathrm{NLL}}(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[-\log p_\theta(\tilde y\mid c)\right].$$3 for DEFT (Wang et al., 11 Feb 2026). This suggests that heterogeneous-prior settings benefit from state-dependent gating.

The model-weak regime is where DEFT most sharply separates itself from static probability-weighted alternatives. On synthetic FigFont puzzles, $$\mathcal{L}_{\mathrm{NLL}}(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[-\log p_\theta(\tilde y\mid c)\right].$$4 collapses badly because it suppresses low-probability signals too strongly, while DEFT remains sufficiently NLL-like in uncertain states. For LLaMA-3.2-3B, the reported Jaro-Winkler similarity is $$\mathcal{L}_{\mathrm{NLL}}(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[-\log p_\theta(\tilde y\mid c)\right].$$5 for NLL, $$\mathcal{L}_{\mathrm{NLL}}(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[-\log p_\theta(\tilde y\mid c)\right].$$6 for $$\mathcal{L}_{\mathrm{NLL}}(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[-\log p_\theta(\tilde y\mid c)\right].$$7, $$\mathcal{L}_{\mathrm{NLL}}(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[-\log p_\theta(\tilde y\mid c)\right].$$8 for Cayley-Trans, and $$\mathcal{L}_{\mathrm{NLL}}(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[-\log p_\theta(\tilde y\mid c)\right].$$9 for DEFT (Wang et al., 11 Feb 2026). The paper treats this as evidence that DEFT preserves plasticity without reverting fully to NLL.

In the mixed regime, using a subset of Tulu3-SFT, DEFT also produces the best overall average among compared objectives. For Qwen2.5-1.5B, the reported Total-Avg is $$\frac{\partial \mathcal{L}_f}{\partial z_i(c)} = s_f(p)\,\bigl(p_\theta(i\mid c)-\delta_{i,\tilde y}\bigr),$$0 for NLL, $$\frac{\partial \mathcal{L}_f}{\partial z_i(c)} = s_f(p)\,\bigl(p_\theta(i\mid c)-\delta_{i,\tilde y}\bigr),$$1 for $$\frac{\partial \mathcal{L}_f}{\partial z_i(c)} = s_f(p)\,\bigl(p_\theta(i\mid c)-\delta_{i,\tilde y}\bigr),$$2, $$\frac{\partial \mathcal{L}_f}{\partial z_i(c)} = s_f(p)\,\bigl(p_\theta(i\mid c)-\delta_{i,\tilde y}\bigr),$$3 for Cayley-Trans, and $$\frac{\partial \mathcal{L}_f}{\partial z_i(c)} = s_f(p)\,\bigl(p_\theta(i\mid c)-\delta_{i,\tilde y}\bigr),$$4 for DEFT (Wang et al., 11 Feb 2026). This is presented as evidence that DEFT achieves a stronger exploration–exploitation balance across mixed supervision types.

The paper also reports out-of-domain preservation. A LLaMA-3.1-8B model fine-tuned on MATH and tested zero-shot on medical benchmarks achieves average scores of $$\frac{\partial \mathcal{L}_f}{\partial z_i(c)} = s_f(p)\,\bigl(p_\theta(i\mid c)-\delta_{i,\tilde y}\bigr),$$5 under NLL, $$\frac{\partial \mathcal{L}_f}{\partial z_i(c)} = s_f(p)\,\bigl(p_\theta(i\mid c)-\delta_{i,\tilde y}\bigr),$$6 under Cayley-Trans, and $$\frac{\partial \mathcal{L}_f}{\partial z_i(c)} = s_f(p)\,\bigl(p_\theta(i\mid c)-\delta_{i,\tilde y}\bigr),$$7 under DEFT (Wang et al., 11 Feb 2026). This suggests better retention of pretrained competencies and reduced catastrophic forgetting.

5. Relationship to neighboring entropy-based fine-tuning methods

DEFT sits within a broader landscape of entropy-aware training methods, but its mechanism differs from several neighboring directions.

The closest neighboring work in supervised fine-tuning is “InstructDiff: Domain-Adaptive Data Selection via Differential Entropy for Efficient LLM Fine-Tuning” (Su et al., 30 Jan 2026). InstructDiff does not modify the token loss. Instead, it uses entropy differences between a base model and a minimally instruction-tuned calibration model to select a training subset. Its scoring signal is

$$\frac{\partial \mathcal{L}_f}{\partial z_i(c)} = s_f(p)\,\bigl(p_\theta(i\mid c)-\delta_{i,\tilde y}\bigr),$$8

combined with a bi-directional $$\frac{\partial \mathcal{L}_f}{\partial z_i(c)} = s_f(p)\,\bigl(p_\theta(i\mid c)-\delta_{i,\tilde y}\bigr),$$9 learnability filter. The method shows that selecting the lowest signed differential entropy within the learnable range outperforms full-data training while using only 10–20% of the data, with reported relative improvements of $s_f(p)\triangleq -f'(p)\,p \ge 0.$0 for math, $s_f(p)\triangleq -f'(p)\,p \ge 0.$1 for general instruction following, $s_f(p)\triangleq -f'(p)\,p \ge 0.$2 for medical QA, and $s_f(p)\triangleq -f'(p)\,p \ge 0.$3 for code (Su et al., 30 Jan 2026). Conceptually, InstructDiff is dynamic at the data-selection level, whereas DEFT is dynamic at the token-objective level.

A second nearby framework is EDCO, “Dynamic Curriculum Orchestration for Domain-specific LLM Fine-tuning” (Pang et al., 7 Jan 2026). EDCO rescored training samples under the current model using inference entropy,

$s_f(p)\triangleq -f'(p)\,p \ge 0.$4

approximated by prefix negative log-probabilities under a quick-answer prompt. It then selects the top-$s_f(p)\triangleq -f'(p)\,p \ge 0.$5 highest-entropy samples for the next training phase. The paper reports that its prefix estimator reduces entropy-scoring time by $s_f(p)\triangleq -f'(p)\,p \ge 0.$6 while maintaining a Pearson correlation of $s_f(p)\triangleq -f'(p)\,p \ge 0.$7 with full-sequence entropy, and that EDCO outperforms random sampling and static curricula in communication, medicine, and law domains (Pang et al., 7 Jan 2026). This suggests a complementary perspective: entropy can control which examples are trained on, not only how token gradients are scaled.

In reinforcement fine-tuning, the most relevant theoretical neighbors are “On the Entropy Dynamics in Reinforcement Fine-Tuning of LLMs” (Wang et al., 3 Feb 2026) and “Entropy Polarity in Reinforcement Fine-Tuning: Direction, Asymmetry, and Control” (Zhang et al., 12 May 2026). These papers derive first-order token-level entropy-change predictors for RL updates. In (Wang et al., 3 Feb 2026), the key discriminator is

$s_f(p)\triangleq -f'(p)\,p \ge 0.$8

which predicts whether a GRPO update will increase or decrease policy entropy. In (Zhang et al., 12 May 2026), the corresponding quantity is entropy polarity,

$s_f(p)\triangleq -f'(p)\,p \ge 0.$9

which classifies token updates into entropy-expanding and entropy-contracting branches. These works are not SFT objectives, but they reinforce a broader principle DEFT embodies: entropy control can be made local, signed, and state dependent rather than globally regularized.

A plausible implication is that DEFT could be understood as the SFT analogue of this trend: instead of a global entropy bonus or static token weighting, it uses predictive state to modulate token influence online.

6. Distinctions from unrelated “DEFT” acronyms

The acronym “DEFT” has been used for several unrelated methods, and these should not be conflated.

In diffusion modeling, “DEFT: Efficient Fine-Tuning of Diffusion Models by Learning the Generalised $W_f(p) = s_f(p)(1-p).$0-transform” (Denker et al., 2024) defines DEFT as Doob’s $W_f(p) = s_f(p)(1-p).$1-transform Efficient Fine-Tuning. There the objective is to freeze an unconditional diffusion model and learn a small conditional correction network $W_f(p) = s_f(p)(1-p).$2 such that

$W_f(p) = s_f(p)(1-p).$3

That DEFT concerns conditional diffusion adaptation, not entropy-based SFT (Denker et al., 2024). Relatedly, “DEFT-VTON” (Xu et al., 16 Sep 2025) uses the same acronym for a virtual try-on diffusion framework that trains only a small $W_f(p) = s_f(p)(1-p).$4-transform adapter, about $W_f(p) = s_f(p)(1-p).$5 of the backbone-equivalent parameter count (Xu et al., 16 Sep 2025).

In transformer efficiency, “From PEFT to DEFT” (Runwal et al., 2024) defines DEFT as Density-Efficient Fine-Tuning, adding a density regularizer to PEFT objectives to reduce activation density in MLP blocks. Its central loss is

$W_f(p) = s_f(p)(1-p).$6

and it targets activation sparsity rather than predictive entropy (Runwal et al., 2024).

In wireless systems, “Device-Edge Cooperative Fine-Tuning of Foundation Models as a 6G Service” (Wu et al., 2023) and “Resource Management for Low-latency Cooperative Fine-tuning of Foundation Models at the Network Edge” (Wu et al., 2024) use DEFT to mean device-edge fine-tuning or device-edge cooperative fine-tuning, concerned with block allocation, communication, and latency optimization, not entropy-aware learning objectives (Wu et al., 2023, Wu et al., 2024).

These distinctions matter because the DEFT of (Wang et al., 11 Feb 2026) is a very specific proposal: a dynamic entropic SFT objective based on Rényi-2 concentration.

7. Limitations, open questions, and broader significance

The DEFT paper is explicit that its theory centers on gradient shaping and entropy duality rather than a full convergence theory (Wang et al., 11 Feb 2026). It proves boundedness, monotonicity, and endpoint-consistency properties for the Cayley trajectory and establishes robustness behavior under confident misalignment, but it does not provide a global optimization guarantee for DEFT itself.

A second limitation is the choice of uncertainty proxy. DEFT uses predictive concentration,

$W_f(p) = s_f(p)(1-p).$7

equivalently Rényi-2 entropy, as its state estimator. The paper motivates this strongly, but it does not exhaustively compare alternative proxies such as Shannon entropy, max probability, or margin-based uncertainty in the visible results (Wang et al., 11 Feb 2026). This suggests that future work may investigate whether other concentration measures yield different plasticity–stability tradeoffs.

A third caveat is that DEFT suppresses low-probability targets more strongly when the model is confidently concentrated elsewhere. The paper interprets this as beneficial in the presence of conflicting or noisy supervision. A plausible implication is that if a model is systematically, confidently wrong for domain-mismatch reasons, DEFT could slow correction unless the predictive distribution first becomes less concentrated. The empirical results suggest this does not dominate in the tested settings, but the tradeoff remains structurally present (Wang et al., 11 Feb 2026).

The broader significance of DEFT is that it reframes SFT objective design around gradient trust rather than fixed likelihood maximization. This aligns with a wider movement in the 2025–2026 literature toward entropy as a dynamic control signal: entropy differences for data curation (Su et al., 30 Jan 2026), entropy-driven curricula (Pang et al., 7 Jan 2026), and token-level entropy mechanics in RL fine-tuning (Wang et al., 3 Feb 2026, Zhang et al., 12 May 2026). DEFT contributes the supervised fine-tuning analogue of that movement by making token influence depend on predictive state.

In that sense, Dynamic Entropy Fine-Tuning is best understood as a state-aware objective that interpolates between exploratory coverage and confidence-preserving sharpening. It uses entropy not as a scalar target to maximize, but as a lens for deciding when gradients have earned the right to be large (Wang et al., 11 Feb 2026).