Gain-Based Loss Weighting (GLOW)

• GLOW is an adaptive per-sample loss weighting method that uses the reduction in loss between epochs to prioritize hard or under-learned examples.
• It employs a sigmoid-rescaled weighting function controlled by hyperparameters α and β to distinguish easy from challenging samples without additional normalization.
• Empirical results demonstrate that GLOW enhances out-of-domain generalization and reasoning accuracy while promoting model exploration and robustness.

Gain-based Loss Weighting (GLOW) is an adaptive per-sample loss weighting methodology that dynamically reallocates learning emphasis based on empirical training progress, particularly targeting hard or under-learned examples within large-scale reasoning or noisy datasets. Initially developed to address the limitations of supervised fine-tuning (SFT) in chain-of-thought (CoT) reasoning for LLMs, GLOW systematically reconciles against overfitting and promotes out-of-domain (OOD) generalization by leveraging the temporal evolution of each sample's loss reduction across epochs. The approach draws theoretical roots from the broader Derivative Manipulation (DM) framework, enabling GLOW to be realized as a flexible, gain-based instantiation of general example weighting schemes (Tian et al., 8 Jan 2026, &&&1&&&).

1. Formal Definition and Underlying Metric

GLOW assigns adaptive training weights to each sample based on its inter-epoch "gain," defined as the reduction in loss between two successive epochs. Specifically, for sample $i$ at the conclusion of epoch $t$:

• Let $\ell_i^{(t-1)}$ denote its loss at epoch $t-1$
• Let $\ell_i^{(t)}$ denote its loss at epoch $t$
• The sample's gain is $\Delta L_i^{(t)} = \ell_i^{(t-1)} - \ell_i^{(t)}$

This metric operationalizes the extent to which the model has already "mastered" a sample (large $\Delta L$) versus those that remain under-covered (small $\Delta L$). Thus, GLOW reweights subsequent training attention toward samples exhibiting small gains, dynamically shifting focus over SFT iterations (Tian et al., 8 Jan 2026).

2. Adaptive Weighting Function and Update Rule

The core adaptive weighting mechanism in GLOW is a sigmoid-rescaled function of each sample's gain, parameterized by two hyperparameters, $\alpha$ (scaling) and $\beta$ (sharpening):

$$w_i^{(t+1)} = \alpha \cdot [1 - \sigma(\beta \cdot \Delta L_i^{(t)})]$$

where $\sigma(x) = \frac{1}{1 + e^{-x}}$ is the sigmoid activation. For intuition:

• Samples with large gains ($\Delta L$) get $\sigma(\beta \Delta L) \approx 1$, leading to $w_i^{(t+1)} \approx 0$ (downweighted as “easy”)
• Samples with small gains yield $w_i^{(t+1)} \approx \alpha$ (maximally upweighted as “hard” or under-learned)

No normalization across samples is required; weights directly scale each loss in the batch objective. For the initial epoch, $w_i^{(1)}=1$ for all $i$. This formulation may optionally be clipped to desired bounds, though empirical results in (Tian et al., 8 Jan 2026) did not require it.

3. GLOW Training Workflow

Implementation of GLOW requires only minimal modification to standard training pipelines. The canonical procedure is:

1. Data Preparation: Aggregate both positive (correct) and negative (incorrect outcome but potentially valid intermediate reasoning) CoT trajectories.
2. Initialization: Initialize model parameters $\theta^{(0)}$; set $w_i^{(1)}=1$.
3. Epoch Loop ($t = 1,\ldots,T$):
   • Minimize the weighted loss: $$L^{(t)}(\theta) = \sum_i w_i^{(t)} \cdot \ell_i(\theta)$$
   • At epoch’s end, record $\ell_i^{(t)}$ for all $i$.
   • Compute $\Delta L_i^{(t)} = \ell_i^{(t-1)} - \ell_i^{(t)}$.
   • Assign next epoch’s weights as above.
4. Empirical Storage: Maintain two $N$-dimensional vectors to record per-sample losses for successive epochs (Tian et al., 8 Jan 2026).

In terms of overhead, the primary requirement is $O(N)$ memory for loss histories and one sigmoid per sample per epoch, negligible relative to model operations.

4. GLOW within Derivative Manipulation (DM) and Generalizations

GLOW is a specific instantiation of the Derivative Manipulation (DM) framework, which generalizes loss weighting by directly specifying a per-example gradient magnitude function $\varphi'(r): \mathbb{R}_+ \to \mathbb{R}_+$. In DM, the emphasis function $\varphi'(r)$ determines the effective weighting applied at loss value $r$:

• Assign $w_i = \varphi'(L_i)$, typically normalized in-batch (Wang et al., 2019).

GLOW’s gain-based weighting corresponds to particular $\varphi'$ choices. Two canonical gain-based forms emerge:

• Polynomial gain: $\varphi'(r) = r^\alpha$, emphasizing large-loss samples for $\alpha>1$, or softening emphasis with $\alpha<1$.
• Exponential gain: $\varphi'(r) = \exp(\eta r)$, shifting sensitivity between hard (large $\eta$) and easy (negative $\eta$) examples.

Theoretical analysis within DM shows that appropriate tuning of $\alpha$ or $\eta$ enables controlled trade-offs between robustness to noise and hard-sample focus, maintaining SGD convergence guarantees (Wang et al., 2019).

5. Empirical Performance and Observed Effects

On Qwen2.5-7B fine-tuned for mixed Math reasoning, GLOW outperforms both positive-only and naive-mixed SFT conditions. Key results include:

Training Scheme	In-domain Avg.	OOD Avg.	MMLU (RL Init.)
Positives Only	53.51	51.44	72.82
Mixed (No Weighting)	52.66	65.60	—
GLOW	55.18	67.25	76.47

• GLOW delivers a +5.51 point average OOD improvement over positive-only SFT on QA tasks, and +3.65 on MMLU as RL-initialization (Tian et al., 8 Jan 2026).
• On Qwen2.5-14B, GLOW further increases OOD Math (64.33, +2.56) and Reasoning (75.02, +2.27).
• GLOW increases policy entropy by 35.67% during inference, thereby promoting exploration.

Within the DM framework, GLOW and related weighting schemes yield significant robustness and generalization gains across vision and language tasks under label noise, class imbalance, and anomaly conditions, including improvements over Focal Loss, MentorNet, and self-paced learning (Wang et al., 2019).

6. Practical Considerations and Hyperparameters

Key configuration values as instantiated in (Tian et al., 8 Jan 2026):

• Peak learning rate: $5 \times 10^{-5}$ (cosine decay, 10% warm-up)
• Batch size: $128$
• Number of epochs: $T = 20$
• Weighting parameters: default $\alpha=1.0$, $\beta=12$ (robust to $\alpha \in [0.8, 1.5]$, $\beta \in \{10,12,18\}$)
• No explicit per-batch normalization or weight clipping required.

The computational burden of storing and updating per-sample loss vectors is negligible compared to the cost of forward/backward passes in standard model training.

7. Implications and Theoretical Interpretations

GLOW operates by continuously steering model attention toward “hard” samples—those with persistently low inter-epoch loss improvement—which, in the context of CoT SFT, often correspond to negative or ambiguous reasoning chains. This adaptive rebalancing mitigates premature overfitting on “easy” (rapidly saturated) patterns, maintains policy diversity, and yields increased entropy at inference, associated with improved generalization and exploration capabilities (Tian et al., 8 Jan 2026). The DM framework situates GLOW as a unification of hard-focus (e.g., Focal Loss) and noise-robust (e.g., MAE, MSE) weighting schemes; tuning of weighting function parameters allows GLOW to modulate its sensitivity across the easy-hard spectrum, optimizing for robustness in diverse, heterogeneous learning settings (Wang et al., 2019).

Empirical validation across modalities—including vision benchmarks under heavy noise and language tasks with severe class imbalance—demonstrates that GLOW (via power or exponential gain) is a generic and tractable method for maximizing model robustness, OOD performance, and transfer initialization.