---
title: Dynamic Entropy Fine-Tuning (DEFT)
url: https://www.emergentmind.com/topics/dynamic-entropy-fine-tuning-deft
type: topic
---

# Dynamic Entropy Fine-Tuning (DEFT)

Dynamic Entropy Fine-Tuning (DEFT) is a supervised fine-tuning objective for pretrained language models that replaces the uniform token weighting of standard negative log-likelihood with a dynamic, prediction-state-dependent trust gate. In the formulation introduced in “Gradients Must Earn Their Influence: Unifying SFT with Generalized Entropic Objectives” [2602.11424], DEFT is motivated by a two-fold failure mode of conventional NLL-based SFT: it overemphasizes low-probability targets, which can amplify harmful updates under noisy or conflicting supervision, and it provides weak sharpening when the model is already confident. DEFT addresses this by modulating token gradients according to the concentration of the model’s full predictive distribution, using Rényi-2 entropy as a practical proxy for predictive state. In this sense, DEFT is neither a generic name for any entropy-aware training scheme nor a synonym for other uses of the acronym “DEFT” in diffusion, sparsity, or edge systems; it refers specifically to a dynamic entropic objective for LLM supervised fine-tuning [2602.11424].

## 1. Definition and conceptual scope

DEFT is defined as a token-level SFT objective in which the gradient on each supervised target token is multiplicatively gated by a dynamic function of the model’s current predictive concentration [2602.11424]. The underlying problem is the plasticity–stability dilemma in post-pretraining adaptation. Standard NLL remains maximally responsive to low target probability, which is beneficial when the model lacks knowledge, but it is also vulnerable to noisy labels, atypical formatting artifacts, and confident conflicts between pretrained priors and downstream supervision. At the same time, NLL weakens linearly as target probability approaches one, which limits sharpening of already-correct predictions [2602.11424].

The DEFT paper situates this problem within a broader family of generalized entropic objectives. Rather than treating cross-entropy as uniquely canonical, it views token-level SFT losses as members of a family indexed by a scalar trust gate controlling how much the model should trust its current predictive state. This suggests that the main distinction among such objectives is not their gradient direction, which remains the same under softmax parameterization, but the confidence-dependent scalar that scales that direction [2602.11424].

A common misconception is that DEFT means adding an entropy bonus to the loss. In the paper’s formulation, DEFT does not directly maximize predictive entropy. Instead, it uses distribution concentration to decide whether learning should behave more like broad-coverage NLL or more like sharpening-oriented probability weighting. This suggests that its entropy dependence is indirect but structural: entropy informs gradient trust rather than serving as an optimization target in its own right [2602.11424].

## 2. Mathematical formulation

The paper defines a general token-level SFT objective by applying a differentiable nonincreasing scalar function \(f\) to the target-token probability \(p = p_\theta(\tilde y \mid c)\), where \(c\) is the token context and \(\tilde y\) the supervised target token:

\[
\mathcal{L}_f(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[f\!\left(p_\theta(\tilde y\mid c)\right)\right].
\]

Standard NLL is the special case

\[
\mathcal{L}_{\mathrm{NLL}}(\theta) = \mathbb{E}_{(c,\tilde y)\sim T}\!\left[-\log p_\theta(\tilde y\mid c)\right].
\]

A central result is that all such objectives induce the same softmax-gradient structure up to a scalar gate:

\[
\frac{\partial \mathcal{L}_f}{\partial z_i(c)} = s_f(p)\,\bigl(p_\theta(i\mid c)-\delta_{i,\tilde y}\bigr),
\]

with

\[
s_f(p)\triangleq -f'(p)\,p \ge 0.
\]

For the target logit, the gradient magnitude is

\[
W_f(p) = s_f(p)(1-p).
\]

The paper interprets this universally as a “gate \(\times\) error” decomposition, with \(1-p\) as the prediction error and \(s_f(p)\) as the trust gate [2602.11424]. Under NLL, \(s_f(p)=1\), so the gate is permanently open.

DEFT is built on a deformed-log family parameterized by \(\alpha\), where the token loss is

\[
\mathcal{L}_\alpha(p)=\frac{1-p^\alpha}{\alpha},
\]

with gradient

\[
\frac{\partial \mathcal{L}_\alpha(p)}{\partial z_{\tilde y}} = -\,p^\alpha(1-p).
\]

Here the gate is \(p^\alpha\). Small \(\alpha\) recovers NLL-like behavior; \(\alpha=1\) yields linear probability loss; intermediate values interpolate between them [2602.11424].

The paper derives an “ideal” confidence-to-focus trajectory using the Cayley transform. With uncertainty radius

\[
z=\sqrt{1-p},
\]

the focus index is

\[
\alpha^*(p)=\frac{1-\sqrt{1-p}}{1+\sqrt{1-p}}.
\]

This satisfies the desired anchors \(\alpha(0)=0\) and \(\alpha(1)=1\), suggesting a continuous path from exploratory, NLL-like learning at low confidence to sharpening-oriented learning at high confidence [2602.11424].

The practical DEFT objective then replaces target-probability-only focus with a distribution-level proxy. For predictive distribution \(P_\theta(\cdot\mid c)\), DEFT defines

\[
\alpha_{\mathrm{DEFT}}(c) = \sum_{v\in\mathcal V} P_\theta(v\mid c)^2.
\]

This is the concentration of the predictive distribution and equals the exponential of negative Rényi-2 entropy:

\[
H_2(P)\triangleq -\ln \sum_{v\in\mathcal V} P(v)^2,
\qquad
\alpha_{\mathrm{DEFT}}(c)=\exp\!\bigl(-H_2(P_\theta(\cdot\mid c))\bigr).
\]

The resulting gate is

\[
\mathcal G_{\mathrm{DEFT}}(p,c)=p^{\alpha_{\mathrm{DEFT}}(c)},
\]

and the target-logit learning signal becomes

\[
W_{\mathrm{DEFT}}(p,c)=p^{\alpha_{\mathrm{DEFT}}(c)}(1-p).
\]

The paper states that \(\alpha_{\mathrm{DEFT}}(c)\) is treated as a stop-gradient quantity during optimization [2602.11424].

## 3. Entropic interpretation and gradient behavior

The theoretical motivation for DEFT is tied to an optimization–entropy duality. In the deformed-log family, the loss index \(1-\alpha\) corresponds to a Tsallis entropy index \(1+\alpha\), with the relation

\[
q_{\mathrm{loss}}+q_{\mathrm{ent}}=2.
\]

This means that tuning the gate changes not only gradient magnitudes but also the implied entropy geometry of the learning objective [2602.11424]. Small \(\alpha\) is associated with Shannon-like coverage; large \(\alpha\) with concentration and sharpening.

The decisive practical move in DEFT is to use full-distribution concentration rather than target probability alone. Target probability is ambiguous: low \(p\) may indicate genuine ignorance, in which case learning should remain strong, or confident disagreement, in which case aggressive correction may damage useful priors. By using

\[
\alpha_{\mathrm{DEFT}}(c)=\sum_v P_\theta(v\mid c)^2,
\]

the method distinguishes these cases at the level of predictive state. A diffuse distribution yields small \(\alpha_{\mathrm{DEFT}}\), so the gate behaves close to NLL; a sharply concentrated distribution yields large \(\alpha_{\mathrm{DEFT}}\), so the gate suppresses low-probability targets and emphasizes refinement [2602.11424].

The appendix result highlighted in the paper makes this explicit. If a non-target token \(v^\star\neq \tilde y\) has probability at least \(1-\varepsilon\), then

\[
\alpha_{\mathrm{DEFT}}(c)\ge (1-\varepsilon)^2,
\]

and therefore

\[
W_{\mathrm{DEFT}}(p,c) \le p^{(1-\varepsilon)^2}(1-p),
\]

which vanishes as \(p\to 0\). This means DEFT can strongly suppress updates under confident misalignment. By contrast, the Cayley target-probability trajectory alone remains fully open in the \(p\to 0\) limit, which does not distinguish ignorance from confident conflict [2602.11424].

This suggests that the distinctive contribution of DEFT is not merely dynamic interpolation, but dynamic interpolation anchored in whole-distribution predictive structure.

## 4. Empirical evidence and benchmark behavior

The DEFT paper evaluates the method across four regimes: model-strong, model-intermediate, model-weak, and mixed SFT data [2602.11424]. The reported baselines are standard NLL \((-\log p)\), linear probability loss \(( -p )\), EAFT, Cayley-Trans, and DEFT.

In the model-strong regime, where pretrained priors are already competent, DEFT outperforms NLL and typically also static alternatives. For example, on LLaMA-3.1-8B the reported average rises from \(6.49\) under \(-\log p\) to \(11.35\) under DEFT, and on Qwen2.5-Math-7B from \(23.46\) to \(36.55\) [2602.11424]. The paper interprets this as evidence that confident-state sharpening and conflict suppression matter when the model already has substantial relevant knowledge.

In the model-intermediate regime, DEFT remains competitive and is often best on average. One reported example gives LLaMA-3.1-8B average performance of \(50.21\) for NLL, \(51.48\) for Cayley-Trans, and \(52.00\) for DEFT [2602.11424]. This suggests that heterogeneous-prior settings benefit from state-dependent gating.

The model-weak regime is where DEFT most sharply separates itself from static probability-weighted alternatives. On synthetic FigFont puzzles, \(-p\) collapses badly because it suppresses low-probability signals too strongly, while DEFT remains sufficiently NLL-like in uncertain states. For LLaMA-3.2-3B, the reported Jaro-Winkler similarity is \(40.37\) for NLL, \(7.68\) for \(-p\), \(46.55\) for Cayley-Trans, and \(60.22\) for DEFT [2602.11424]. The paper treats this as evidence that DEFT preserves plasticity without reverting fully to NLL.

In the mixed regime, using a subset of Tulu3-SFT, DEFT also produces the best overall average among compared objectives. For Qwen2.5-1.5B, the reported Total-Avg is \(22.26\) for NLL, \(23.23\) for \(-p\), \(22.68\) for Cayley-Trans, and \(24.04\) for DEFT [2602.11424]. This is presented as evidence that DEFT achieves a stronger exploration–exploitation balance across mixed supervision types.

The paper also reports out-of-domain preservation. A LLaMA-3.1-8B model fine-tuned on MATH and tested zero-shot on medical benchmarks achieves average scores of \(43.14\) under NLL, \(44.86\) under Cayley-Trans, and \(45.35\) under DEFT [2602.11424]. This suggests better retention of pretrained competencies and reduced catastrophic forgetting.

## 5. Relationship to neighboring entropy-based fine-tuning methods

DEFT sits within a broader landscape of entropy-aware training methods, but its mechanism differs from several neighboring directions.

The closest neighboring work in supervised fine-tuning is “InstructDiff: Domain-Adaptive Data Selection via Differential Entropy for Efficient LLM Fine-Tuning” [2601.23006]. InstructDiff does not modify the token loss. Instead, it uses entropy differences between a base model and a minimally instruction-tuned calibration model to select a training subset. Its scoring signal is

\[
\Delta H_i = H_{\text{base}}(x_i,y_i) - H_{\text{inst}}(x_i,y_i),
\]

combined with a bi-directional \(\Delta \mathrm{NLL}\) learnability filter. The method shows that selecting the lowest signed differential entropy within the learnable range outperforms full-data training while using only 10–20% of the data, with reported relative improvements of \(+17\%\) for math, \(+52\%\) for general instruction following, \(+6.2\%\) for medical QA, and \(+4.9\%\) for code [2601.23006]. Conceptually, InstructDiff is dynamic at the data-selection level, whereas DEFT is dynamic at the token-objective level.

A second nearby framework is EDCO, “Dynamic Curriculum Orchestration for Domain-specific Large Language Model Fine-tuning” [2601.03725]. EDCO rescored training samples under the current model using inference entropy,

\[
H(y|x; \theta_k) = -\mathbb{E}_{y \sim \pi_{\theta_k}(\cdot|x)} \left[ \log \pi_{\theta_k}(y|x) \right],
\]

approximated by prefix negative log-probabilities under a quick-answer prompt. It then selects the top-\(N\) highest-entropy samples for the next training phase. The paper reports that its prefix estimator reduces entropy-scoring time by \(83.5\%\) while maintaining a Pearson correlation of \(0.63\) with full-sequence entropy, and that EDCO outperforms random sampling and static curricula in communication, medicine, and law domains [2601.03725]. This suggests a complementary perspective: entropy can control which examples are trained on, not only how token gradients are scaled.

In reinforcement fine-tuning, the most relevant theoretical neighbors are “On the Entropy Dynamics in Reinforcement Fine-Tuning of Large Language Models” [2602.03392] and “Entropy Polarity in Reinforcement Fine-Tuning: Direction, Asymmetry, and Control” [2605.11775]. These papers derive first-order token-level entropy-change predictors for RL updates. In [2602.03392], the key discriminator is

\[
S^\*-\mathbb{E}_{i\sim p}[S_i],
\]

which predicts whether a GRPO update will increase or decrease policy entropy. In [2605.11775], the corresponding quantity is entropy polarity,

\[
\mathcal P(s_t,y_t,A)=A\bigl[-t_1(s_t,y_t)+t_2(s_t)\bigr],
\]

which classifies token updates into entropy-expanding and entropy-contracting branches. These works are not SFT objectives, but they reinforce a broader principle DEFT embodies: entropy control can be made local, signed, and state dependent rather than globally regularized.

A plausible implication is that DEFT could be understood as the SFT analogue of this trend: instead of a global entropy bonus or static token weighting, it uses predictive state to modulate token influence online.

## 6. Distinctions from unrelated “DEFT” acronyms

The acronym “DEFT” has been used for several unrelated methods, and these should not be conflated.

In diffusion modeling, “DEFT: Efficient Fine-Tuning of Diffusion Models by Learning the Generalised \(h\)-transform” [2406.01781] defines DEFT as Doob’s \(h\)-transform Efficient Fine-Tuning. There the objective is to freeze an unconditional diffusion model and learn a small conditional correction network \(h_t^\phi(x_t,y)\) such that

\[
\nabla_{x_t}\ln p_t(x_t\mid y) = \nabla_{x_t}\ln p_t(x_t) + \nabla_{x_t}\ln p_t(y\mid x_t).
\]

That DEFT concerns conditional diffusion adaptation, not entropy-based SFT [2406.01781]. Relatedly, “DEFT-VTON” [2509.13506] uses the same acronym for a virtual try-on diffusion framework that trains only a small \(h\)-transform adapter, about \(1.42\%\) of the backbone-equivalent parameter count [2509.13506].

In transformer efficiency, “From PEFT to DEFT” [2402.01911] defines DEFT as Density-Efficient Fine-Tuning, adding a density regularizer to PEFT objectives to reduce activation density in MLP blocks. Its central loss is

\[
\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{T}} + \alpha \cdot \mathcal{L}_{\text{density}},
\]

and it targets activation sparsity rather than predictive entropy [2402.01911].

In wireless systems, “Device-Edge Cooperative Fine-Tuning of Foundation Models as a 6G Service” [2310.18602] and “Resource Management for Low-latency Cooperative Fine-tuning of Foundation Models at the Network Edge” [2407.09873] use DEFT to mean device-edge fine-tuning or device-edge cooperative fine-tuning, concerned with block allocation, communication, and latency optimization, not entropy-aware learning objectives [2310.18602; 2407.09873].

These distinctions matter because the DEFT of [2602.11424] is a very specific proposal: a dynamic entropic SFT objective based on Rényi-2 concentration.

## 7. Limitations, open questions, and broader significance

The DEFT paper is explicit that its theory centers on gradient shaping and entropy duality rather than a full convergence theory [2602.11424]. It proves boundedness, monotonicity, and endpoint-consistency properties for the Cayley trajectory and establishes robustness behavior under confident misalignment, but it does not provide a global optimization guarantee for DEFT itself.

A second limitation is the choice of uncertainty proxy. DEFT uses predictive concentration,

\[
\sum_v P_\theta(v\mid c)^2,
\]

equivalently Rényi-2 entropy, as its state estimator. The paper motivates this strongly, but it does not exhaustively compare alternative proxies such as Shannon entropy, max probability, or margin-based uncertainty in the visible results [2602.11424]. This suggests that future work may investigate whether other concentration measures yield different plasticity–stability tradeoffs.

A third caveat is that DEFT suppresses low-probability targets more strongly when the model is confidently concentrated elsewhere. The paper interprets this as beneficial in the presence of conflicting or noisy supervision. A plausible implication is that if a model is systematically, confidently wrong for domain-mismatch reasons, DEFT could slow correction unless the predictive distribution first becomes less concentrated. The empirical results suggest this does not dominate in the tested settings, but the tradeoff remains structurally present [2602.11424].

The broader significance of DEFT is that it reframes SFT objective design around gradient trust rather than fixed likelihood maximization. This aligns with a wider movement in the 2025–2026 literature toward entropy as a dynamic control signal: entropy differences for data curation [2601.23006], entropy-driven curricula [2601.03725], and token-level entropy mechanics in RL fine-tuning [2602.03392; 2605.11775]. DEFT contributes the supervised fine-tuning analogue of that movement by making token influence depend on predictive state.

In that sense, Dynamic Entropy Fine-Tuning is best understood as a state-aware objective that interpolates between exploratory coverage and confidence-preserving sharpening. It uses entropy not as a scalar target to maximize, but as a lens for deciding when gradients have earned the right to be large [2602.11424].

Source: https://www.emergentmind.com/topics/dynamic-entropy-fine-tuning-deft