Entropy-Adaptive Loss Modulation

• Entropy-adaptive loss modulation is a design pattern that adjusts training signals by weighting examples based on entropy measures, differentiating between easy, hard, and noisy cases.
• It spans diverse applications including classification, segmentation, self-training, test-time adaptation, and reinforcement learning, balancing exploration and exploitation.
• Empirical studies show that these methods can reduce error rates and stabilize adaptation with minimal computational overhead.

Entropy-adaptive loss modulation denotes a class of training strategies in which an objective is reshaped by an entropy-derived signal—predictive entropy, confidence, answer-cluster entropy, response-level uncertainty, or local free entropy—so that optimization assigns different emphasis to easy, hard, noisy, conflicting, or geometrically fragile cases. Recent instantiations range from modified cross-entropy for one-hot classification and adaptive pseudo-target construction, to robust entropy minimization for test-time adaptation, token-level gating in supervised fine-tuning, response-level advantage rescaling in reinforcement learning, and anisotropic local-entropic smoothing in parameter space (Shim, 10 Jul 2025, Xu et al., 2023, Seto et al., 2023, Wang et al., 31 Mar 2025, Diao et al., 5 Jan 2026, Zhao et al., 1 May 2026, Musso, 2020).

1. Conceptual scope and recurring design patterns

Across the literature, entropy-adaptive modulation is not a single loss but a design pattern. The modulating quantity may be the probability of the true class, Shannon entropy over model outputs, entropy over clustered self-generated answers, response-level uncertainty proxies in policy optimization, or a local free-energy construction over parameter perturbations. What unifies these methods is that entropy is not merely measured; it directly alters the contribution of samples, tokens, pixels, responses, or neighborhoods in weight space to the training signal (Shim, 10 Jul 2025, Seto et al., 2023, Wang et al., 31 Mar 2025, Diao et al., 5 Jan 2026, Zhao et al., 1 May 2026, Musso, 2020).

A second recurring pattern is selective emphasis. In some formulations, high-uncertainty instances are up-weighted because they are assumed to be informative, as in self-training and pixel-boundary weighting. In others, high-entropy instances are down-weighted because they are treated as noisy or unreliable, as in single-sample test-time adaptation. A third pattern is entropy scheduling: several reinforcement-learning and local-entropy methods explicitly use entropy dynamics to move from exploration to exploitation or from coarse smoothing to fine-grained refinement (Seto et al., 2023, Wang et al., 31 Mar 2025, Zhao et al., 1 May 2026, Zhang et al., 15 Feb 2026, Musso, 2021).

Formulation	Modulation signal	Reported setting
AdpLoss	$(1-q_c)$ multiplying $-\log q_c$	one-hot classification
REALM	$\rho(\mathcal L_{\mathrm{ent}};\alpha,\lambda)$	single-sample online TTA
EAST	normalized $h_i^a$	self-training
EAFT	token gate $g(H_t)$	supervised fine-tuning
AEM	response coefficient $\alpha(s,a)$	agentic RL
Partial local entropy	covariance $\Sigma$ or mask $U$	weight-space smoothing

The literature also distinguishes several meanings of entropy. Output-space entropy concerns uncertainty over labels or tokens. Grouped-answer entropy summarizes disagreement among multiple sampled reasoning paths. Response-level entropy in RL aggregates uncertainty at the level of complete actions rather than individual tokens. Local entropy in weight space instead smooths the original loss over neighborhoods of parameters. These are mathematically distinct objects, even when they serve analogous modulation roles (Wang et al., 31 Mar 2025, Zhao et al., 1 May 2026, Musso, 2020, Musso, 2021).

2. Confidence-adaptive cross-entropy and supervised prediction losses

A direct supervised formulation appears in "Enhancing Cross Entropy with a Linearly Adaptive Loss Function for Optimized Classification Performance" (Shim, 10 Jul 2025). For one-hot labels with true class index $c$ and softmax probability $q_c$, standard cross-entropy is

$-\log q_c$0

while the proposed Linearly Adaptive Cross Entropy Loss is

$-\log q_c$1

The paper interprets this as an information-theoretic approximation to Jeffreys divergence for one-hot $-\log q_c$2 and softmax $-\log q_c$3, stating that $-\log q_c$4 and $-\log q_c$5, so that $-\log q_c$6 (Shim, 10 Jul 2025). Operationally, when $-\log q_c$7 is small, $-\log q_c$8 and the loss behaves similarly to cross-entropy; when $-\log q_c$9, the factor tends to zero, down-weighting very confident correct predictions. The paper explicitly contrasts this with focal loss, noting that focal loss uses $\rho(\mathcal L_{\mathrm{ent}};\alpha,\lambda)$0 with $\rho(\mathcal L_{\mathrm{ent}};\alpha,\lambda)$1, whereas AdpLoss uses the linear factor $\rho(\mathcal L_{\mathrm{ent}};\alpha,\lambda)$2, described as preserving simplicity and interpretability (Shim, 10 Jul 2025).

The reported experiment uses ResNet-18 adapted for CIFAR-100, with per-pixel mean subtraction, random horizontal flips, random $\rho(\mathcal L_{\mathrm{ent}};\alpha,\lambda)$3 crops with padding, SGD with momentum $\rho(\mathcal L_{\mathrm{ent}};\alpha,\lambda)$4, weight decay $\rho(\mathcal L_{\mathrm{ent}};\alpha,\lambda)$5, initial learning rate $\rho(\mathcal L_{\mathrm{ent}};\alpha,\lambda)$6, StepLR decay by factor $\rho(\mathcal L_{\mathrm{ent}};\alpha,\lambda)$7 at epochs $\rho(\mathcal L_{\mathrm{ent}};\alpha,\lambda)$8 and $\rho(\mathcal L_{\mathrm{ent}};\alpha,\lambda)$9, batch size $h_i^a$0, and $h_i^a$1 total epochs. The metric is top-5 error rate averaged over five independent runs over epochs $h_i^a$2–$h_i^a$3. Cross-entropy yields $h_i^a$4, whereas AdpLoss yields $h_i^a$5, corresponding to a $h_i^a$6 percentage-point lower error. The paper also states that AdpLoss requires exactly two extra arithmetic operations per sample, namely one subtraction and one multiplication (Shim, 10 Jul 2025).

Entropy-adaptive supervision also appears in structured prediction. In stereo matching, "Adaptive Multi-Modal Cross-Entropy Loss for Stereo Matching" constructs a per-pixel pseudo-ground-truth as a mixture of Laplacians whose number of modes $h_i^a$7 is determined by DBSCAN over local disparity geometry (Xu et al., 2023). For pixel $h_i^a$8,

$h_i^a$9

and the training loss is a standard cross-entropy between this adaptive target and the predicted disparity distribution. The paper makes the entropy-adaptive mechanism explicit: when $g(H_t)$0, the target is uni-modal; when $g(H_t)$1, typically at depth discontinuities, the target becomes multi-modal and its Shannon entropy increases. The scalars $g(H_t)$2 and $g(H_t)$3 control peak sharpness and central-mode dominance, with reported settings $g(H_t)$4, $g(H_t)$5, $g(H_t)$6 on SceneFlow, $g(H_t)$7 on KITTI, and $g(H_t)$8 (Xu et al., 2023). Reported gains include SceneFlow test EPE $g(H_t)$9, KITTI 2015 D1-all $\alpha(s,a)$0, and strong synthetic-to-real generalization improvements (Xu et al., 2023).

A related but distinct supervised construction is the Adaptive Hybrid Focal-Entropy Loss (AHFE) for diabetic retinopathy detection (Malarvannan et al., 2024). For a batch of $\alpha(s,a)$1 samples and $\alpha(s,a)$2 classes, the paper defines

$\alpha(s,a)$3

with adaptive class weights $\alpha(s,a)$4. Here entropy is combined with focal modulation and inverse-frequency weighting. The report gives example hyperparameters $\alpha(s,a)$5, $\alpha(s,a)$6, and $\alpha(s,a)$7, and frames the entropy term as penalizing high-entropy predictions while the focal term emphasizes hard examples (Malarvannan et al., 2024). This suggests that entropy-adaptive modulation in supervised classification can serve either sample-difficulty shaping, class-imbalance correction, or both.

In binary segmentation, "Uncertainty-Guided Attention and Entropy-Weighted Loss for Precise Plant Seedling Segmentation" computes per-pixel entropy

$\alpha(s,a)$8

and converts it into a weight

$\alpha(s,a)$9

The weighted binary cross-entropy is then combined with Dice loss using $\Sigma$0 and $\Sigma$1 (Ehab et al., 12 Apr 2026). Because $\Sigma$2 peaks at $\Sigma$3, uncertain boundary pixels receive the largest penalty. The paper reports Dice gains for both U-Net and LinkNet under a “Loss-only” setting and under the full UGDA-Net configuration, indicating that entropy weighting alone already contributes the largest single gain in the ablation sequence (Ehab et al., 12 Apr 2026).

3. Adaptive sample weighting in self-training and test-time adaptation

In self-training for mathematical reasoning, "Entropy-Based Adaptive Weighting for Self-Training" defines uncertainty not over class logits but over the empirical distribution of self-generated answer clusters (Wang et al., 31 Mar 2025). For question $\Sigma$4, if $\Sigma$5 generated solutions are partitioned into $\Sigma$6 clusters with proportions $\Sigma$7, the entropy is

$\Sigma$8

Writing $\Sigma$9, the paper maps entropy to a nonnegative normalized weight

$U$0

so that the mean weight is $U$1. The training objective becomes

$U$2

The exponent $U$3 controls sharpness: $U$4 amplifies entropy differences, $U$5 compresses them, and $U$6 inverts the ranking (Wang et al., 31 Mar 2025). The paper evaluates this on GSM8K and MATH using LLaMA-3.2-1B as backbone, reporting that plain self-training gives virtually no improvement on MATH, whereas EAST yields around a $U$7 gain over the backbone model; on GSM8K, EAST gives a further $U$8–$U$9 boost compared with the vanilla method (Wang et al., 31 Mar 2025).

In online fully test-time adaptation, the entropy-adaptive role is reversed. "REALM: Robust Entropy Adaptive Loss Minimization for Improved Single-Sample Test-Time Adaptation" starts from predictive entropy

$c$0

and replaces it with a robustified penalty

$c$1

The final objective may also include a diversity mask $c$2:

$c$3

The paper explicitly frames this as a self-paced-learning and robust-loss alternative to hard thresholding. Instead of skipping high-entropy samples, REALM smoothly down-weights them (Seto et al., 2023).

The empirical motivation is instability in batch-size-$c$4 adaptation. The paper states that naïve entropy minimization in the single-sample setting often collapses to a degenerate solution in which the model predicts one class for everything. EATA and SAR address this by skipping high-entropy samples or adding sharpness-aware updates, but the paper argues that skipping can discard much of the adaptation signal and bias the update procedure (Seto et al., 2023). On CIFAR-10-C with ResNet-26+GN at severity $c$5, REALM reports $c$6 average accuracy versus $c$7 for EATA, $c$8 for SAR, and instability for TENT. On ImageNet-C with ResNet-50+GN at severity $c$9, it reports $q_c$0 versus $q_c$1 for EATA and $q_c$2 for SAR. The paper further reports adaptation on roughly $q_c$3 more samples than EATA and faster attainment of peak adaptation accuracy in the first few thousand samples (Seto et al., 2023).

Taken together, EAST and REALM illustrate a central ambiguity in entropy-adaptive design: high entropy may be treated as an indicator of informational value or as an indicator of unreliability. Which interpretation is appropriate depends on the learning regime. This suggests that entropy itself is rarely the objective; rather, the task is to determine how entropy should alter trust in a training signal.

4. Token-level entropy gating and the mitigation of forgetting

Entropy-adaptive modulation has also been used to suppress destructive supervised updates in LLMs. "Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting" identifies a category of tokens called “Confident Conflicts,” defined by low probability for the ground-truth token together with low token-level entropy (Diao et al., 5 Jan 2026). At decoding step $q_c$4, with vocabulary distribution

$q_c$5

the Shannon entropy is

$q_c$6

and the probability of the gold token is $q_c$7. Tokens with $q_c$8 and $q_c$9 are treated as confident conflicts (Diao et al., 5 Jan 2026).

EAFT replaces standard cross-entropy with a gated version. Using a top-$-\log q_c$00 approximation,

$-\log q_c$01

and the loss becomes

$-\log q_c$02

In practice, $-\log q_c$03 is treated as a constant during backpropagation. The paper reports that with $-\log q_c$04, the top-$-\log q_c$05 entropy approximation has Pearson correlation $-\log q_c$06 with full-vocabulary entropy at negligible extra memory of approximately $-\log q_c$07 KB (Diao et al., 5 Jan 2026).

The key claim is that probability alone is insufficient to distinguish uncertainty from conflict with stored knowledge. Under standard supervised fine-tuning, tokens with low $-\log q_c$08 and low $-\log q_c$09 yield very large gradients of size approximately $-\log q_c$10, which the paper interprets as destructive shifts away from the pretrained basin. By driving $-\log q_c$11 when entropy is low, EAFT suppresses those updates while preserving learning from genuinely uncertain tokens (Diao et al., 5 Jan 2026). Variants explored in the paper include polynomial, sigmoid, and hard-mask gates, with the soft variants reported to occupy the Pareto-optimal frontier of target adaptation versus general capability retention (Diao et al., 5 Jan 2026).

The experimental evidence spans Qwen and GLM families from $-\log q_c$12B to $-\log q_c$13B parameters across math, medical, and agentic domains. The paper reports that EAFT consistently matches or slightly trails standard SFT by less than $-\log q_c$14 point on target tasks in math and medical settings while substantially reducing general-capability degradation. For example, average general-capability drop is reported as $-\log q_c$15 points for SFT versus $-\log q_c$16 for EAFT on Qwen3-4B-Instruct, and $-\log q_c$17 versus $-\log q_c$18 on Qwen2.5-32B-Instruct (Diao et al., 5 Jan 2026). A common misconception addressed by this work is that low ground-truth probability always indicates a useful hard example; the paper’s claim is that some low-probability cases instead represent confident internal disagreement and should be de-emphasized.

5. Entropy modulation in reinforcement learning

Recent reinforcement-learning work extends entropy-adaptive modulation from supervised losses to policy updates. "AEM: Adaptive Entropy Modulation for Multi-Turn Agentic Reinforcement Learning" argues that the relevant action granularity for LLM agents is the complete response rather than the token (Zhao et al., 1 May 2026). For state $-\log q_c$19 and response action $-\log q_c$20, response-level entropy is

$-\log q_c$21

and the paper derives that the directional derivative of response entropy along the policy-gradient update induced by a sampled $-\log q_c$22 is

$-\log q_c$23

where $-\log q_c$24 is response-level surprisal. Aggregated over states, the entropy change is written as a covariance between advantage and surprisal (Zhao et al., 1 May 2026). This motivates a practical uncertainty proxy based on per-token entropies:

$-\log q_c$25

followed by within-group min-max normalization to produce $-\log q_c$26, and then

$-\log q_c$27

so that the modulated advantage is $-\log q_c$28 (Zhao et al., 1 May 2026). The paper reports an extra cost of less than $-\log q_c$29 of iteration time and gains of $-\log q_c$30 on ALFWorld, $-\log q_c$31 on WebShop, and $-\log q_c$32 on SWE-bench-Verified when AEM is integrated into strong baselines (Zhao et al., 1 May 2026).

A different RL perspective appears in "Policy Gradient with Adaptive Entropy Annealing for Continual Fine-Tuning" (Zhang et al., 15 Feb 2026). This work formulates classification as a one-step MDP and contrasts cross-entropy with Expected Policy Gradient (EPG). The paper writes

$-\log q_c$33

whereas

$-\log q_c$34

The contrast is interpreted as a sample-weighting difference: cross-entropy carries a factor $-\log q_c$35 and thus emphasizes low-confidence examples, while EPG prioritizes high-confidence ones. To interpolate between these behaviors, the paper defines

$-\log q_c$36

and uses the mixed update

$-\log q_c$37

Reported results across Split-ImageNet-R, Split-Food101, Split-CUB200, and CLRS25 with LoRA, Adapter, and Prefix modules indicate that aEPG outperforms CE and fixed entropy regularizers by $-\log q_c$38–$-\log q_c$39, with a specific ImageNet-R LoRA example of $-\log q_c$40 for CE, $-\log q_c$41 for EPG, and $-\log q_c$42 for aEPG (Zhang et al., 15 Feb 2026).

"Rethinking Exploration in RLVR: From Entropy Regularization to Refinement via Bidirectional Entropy Modulation" pushes the argument further by distinguishing informative from spurious entropy (Gu et al., 6 Apr 2026). The paper defines entropy on successful rollouts $-\log q_c$43 and failing rollouts $-\log q_c$44 as

$-\log q_c$45

$-\log q_c$46

Its AsymGRPO objective introduces separate coefficients $-\log q_c$47 and $-\log q_c$48 for these two components, together with asymmetric positive and negative rollout advantages parameterized by $-\log q_c$49 and $-\log q_c$50 (Gu et al., 6 Apr 2026). The central claim is not blanket entropy maximization but entropy refinement: preserve informative entropy on positives while suppressing spurious entropy on negatives. On five mathematical reasoning benchmarks with Qwen3-4B, AsymGRPO reports average accuracy $-\log q_c$51 versus $-\log q_c$52 for GRPO, and $-\log q_c$53 when combined with Clip-higher (Gu et al., 6 Apr 2026).

These RL papers collectively challenge a common simplification that entropy regularization is inherently equivalent to exploration. The reported results instead treat entropy as structured: response-level uncertainty can rescale advantages, annealed entropy can mediate exploration–exploitation trade-offs, and positive versus negative trajectories can require opposite entropy updates.

6. Local entropic smoothing, anisotropy, and layer-wise information shaping

A distinct lineage of entropy-adaptive loss modulation operates in parameter space rather than output space. In "Partial local entropy and anisotropy in deep weight spaces," the starting point is the local free-entropy

$-\log q_c$54

or, more generally, a Gaussian-smoothed form with covariance $-\log q_c$55 (Musso, 2020). The paper introduces partial local entropy by restricting the smoothing to a subset of directions or by using an anisotropic covariance, including block-diagonal, layer-wise temperatures. In the Gaussian formulation,

$-\log q_c$56

The resulting gradient is an average of perturbed-loss gradients under a tilted density $-\log q_c$57 (Musso, 2020).

The paper’s reported findings emphasize anisotropy. On fully connected networks for MNIST and Fashion-MNIST, partial layer-wise entropy regularization is said to outperform isotropic smoothing by a large margin, with the best results often obtained by smoothing only one suitably chosen layer, typically the deepest hidden layer. In a 3-layer MLP, isotropic smoothing is reported as always worst once the smoothing radius grows beyond a tiny regime. In convolutional networks for CIFAR10 and STL10, smoothing convolutional kernels tends to hurt, whereas smoothing only the fully connected head still yields gains, especially with early stopping (Musso, 2020). This is an explicit warning against the misconception that more entropy smoothing is necessarily better.

"Entropic alternatives to initialization" develops a related local free-energy formalism and emphasizes scheduled fading of the smoothing covariance $-\log q_c$58 or radius $-\log q_c$59 over training time (Musso, 2021). The paper writes

$-\log q_c$60

with an anisotropic Gaussian kernel $-\log q_c$61 of covariance $-\log q_c$62, and recommends schedules such as

$-\log q_c$63

Reported experiments include a small CNN on MNIST, where scheduled anisotropic entropic smoothing matched Kaiming initialization at approximately $-\log q_c$64 after $-\log q_c$65 epochs, and a deeper network on STL-10, where scheduled fading protocols outperformed random initialization by $-\log q_c$66–$-\log q_c$67 and matched or slightly exceeded Kaiming initialization at approximately $-\log q_c$68 (Musso, 2021). The scoping interpretation is that strong early smoothing acts as a coarse-grained search that progressively fades into low-entropy refinement.

A third information-theoretic line is "Entropy-based Guidance of Deep Neural Networks for Accelerated Convergence and Improved Performance" (Meni et al., 2023). Rather than smoothing the loss over perturbations, this paper derives layer-wise entropy-change quantities from network parameters themselves. For dense layers,

$-\log q_c$69

and for convolutional filters,

$-\log q_c$70

These are added to the task loss through regularizers $-\log q_c$71 and $-\log q_c$72 with layer-wise coefficients. The reported results include up to $-\log q_c$73 faster convergence in dense autoencoders, statistically significant $-\log q_c$74–$-\log q_c$75 validation gains when applying the convolutional entropy loss only to the first layer of small CNNs on CIFAR-10, and $-\log q_c$76–$-\log q_c$77 percentage-point top-1 validation gains in $-\log q_c$78 Imagenette experiments for VGG-16 and ResNet-50 when entropy placement is chosen appropriately (Meni et al., 2023). The paper also warns that applying the entropy loss to all layers generally gives no reliable improvement.

This parameter-space literature broadens the meaning of entropy-adaptive modulation. Here entropy does not weight examples but instead changes the geometry of the optimization landscape. A plausible implication is that “entropy-adaptive” methods should be understood at two levels: output-space mechanisms that redistribute gradient mass across data instances, and weight-space mechanisms that reshape the effective loss surface itself.