Class-wise inequality loss is a family of objectives that penalizes disparities between class losses to enforce balanced training signals and reduce bias.
It is applied, for example, in medical diagnosis to balance positive and negative cross-entropy losses, thereby enhancing performance on minority classes.
Empirical results demonstrate improvements in macro F1 scores and calibration error across diverse scenarios, underscoring its effectiveness in handling imbalance.
Searching arXiv for the primary paper and closely related class-wise loss-balancing work.
Class-wise inequality loss is not a single canonical formula in the literature, but a class of objectives that explicitly penalize disparities among class-level losses. In the most direct recent usage, it denotes the absolute difference between the positive-class and negative-class cross-entropy losses in a minibatch, introduced to mitigate both class imbalance and class-feature bias in medical diagnosis (Zuo et al., 9 Aug 2025). Closely related formulations synchronize class-wise losses for post-hoc calibration by minimizing their empirical standard deviation (Jung et al., 2023), while earlier deep hashing work used the label “class-wise inequality” for a class-wise likelihood with cubic and quantization constraints in binary code learning (Zhe et al., 2018). Across these usages, the common theme is that unequal class-level optimization signals are treated as a training pathology rather than as an incidental by-product of skewed data.
1. Terminological scope and problem setting
In the medical-diagnosis setting, class-wise inequality loss is introduced to address class-feature bias, defined as models’ potential reliance on features that are strongly correlated with only a subset of classes, leading to biased performance and poor generalization on other classes. The proposed objective promotes equal contributions of classification loss from positive-class and negative-class samples, and is paired with a class-wise group distributionally robust optimization objective that upweights underperforming classes (Zuo et al., 9 Aug 2025).
A separate calibration framework uses class-wise loss inequality in a different sense. There, the class-wise training losses often diverge because of different levels of intra-class and inter-class appearance variation, and the method measures inequality by the empirical standard deviation of the per-class losses, then learns class-wise scaling factors to synchronize them during post-hoc calibration (Jung et al., 2023).
An earlier and technically distinct usage appears in deep supervised hashing, where the “class-wise inequality” or “cubic-constraint” loss models each class as an isotropic Gaussian in the Hamming-code space and regularizes continuous codes so that they remain inside a slightly enlarged Hamming cube before final binarization (Zhe et al., 2018).
This suggests that the phrase should be interpreted through the surrounding optimization problem rather than as a single standardized loss.
2. Binary class-wise inequality loss in medical diagnosis
As formulated in the medical-diagnosis work, the minibatch B is partitioned into positive-class and negative-class subsets,
Its immediate effect is to penalize the model whenever one class’s loss is much higher or lower than the other’s. The stated intuition is twofold. First, if a classifier exploits class-specific or spurious features that strongly benefit one class but not the other, one side can incur very low loss while the opposite side suffers high loss. Second, in an imbalanced dataset, the majority class tends to have lower loss while the minority class has high loss. Driving Lpos≈Lneg is therefore intended to force the model toward shared discriminative signals and to increase attention to the minority group without resorting to naive oversampling (Zuo et al., 9 Aug 2025).
As presented, this formulation is explicitly binary: it is built from positive-class and negative-class losses rather than from a general C-class dispersion functional.
3. Coupling with class-weighted group DRO
The medical formulation does not use the inequality term in isolation. It is integrated with a group distributionally robust optimization term that automatically up-weights the worse-performing class. The per-class weights are defined by a softmax over the two losses, with stop-gradient: wc=sg(exp(τLpos)+exp(τLneg)exp(τLc)),c∈{pos,neg}.
The temperature parameter Bpos={(xi,yi)∈B∣yi=1},Bneg={(xi,yi)∈B∣yi=0}.0 controls how aggressively the optimization focuses on the higher-loss class: as Bpos={(xi,yi)∈B∣yi=1},Bneg={(xi,yi)∈B∣yi=0}.1, all weight goes to the class with larger Bpos={(xi,yi)∈B∣yi=1},Bneg={(xi,yi)∈B∣yi=0}.2; as Bpos={(xi,yi)∈B∣yi=1},Bneg={(xi,yi)∈B∣yi=0}.3, the objective approaches a uniform per-class average. The coefficient Bpos={(xi,yi)∈B∣yi=1},Bneg={(xi,yi)∈B∣yi=0}.4 controls the strength of the inequality penalty. In practice, both are annealed over epochs: Bpos={(xi,yi)∈B∣yi=1},Bneg={(xi,yi)∈B∣yi=0}.5 is increased from Bpos={(xi,yi)∈B∣yi=1},Bneg={(xi,yi)∈B∣yi=0}.6 toward a final Bpos={(xi,yi)∈B∣yi=1},Bneg={(xi,yi)∈B∣yi=0}.7, while Bpos={(xi,yi)∈B∣yi=1},Bneg={(xi,yi)∈B∣yi=0}.8 is decreased so that early training emphasizes stable class weighting and later training becomes more even across classes (Zuo et al., 9 Aug 2025).
which makes explicit that the update direction depends on which class currently has the larger loss.
The theoretical rationale is stated in information-theoretic terms. Equality of class-conditional entropies Lpos=−∣Bpos∣1(x,y)∈Bpos∑[ylogP^(y=1∣x)],0 implies equality of class losses when the model approximates the true posterior, so large loss differences indicate reliance on class-specific features or under-use of minority evidence. Under mild regularity and symmetry assumptions, the related calibration work further shows that a reweighting proportional to Lpos=−∣Bpos∣1(x,y)∈Bpos∑[ylogP^(y=1∣x)],1 is the steepest-descent direction for reducing class-wise loss inequality while preserving total loss magnitude (Jung et al., 2023).
4. Empirical behavior and reported gains
The medical-diagnosis evaluation spans a synthetic toy setting and several real datasets: DAIC-WOZ, MODMA, ADReSS, BreastMNIST, and RetinaMNIST. Baselines include plain cross-entropy, class-weighted ERM, per-class-average ERM, and G-DRO only. Reported metrics are Macro F1, per-class F1, and accuracy (Zuo et al., 9 Aug 2025).
On the toy example, the balanced setting reports ERM positive-class accuracy of Lpos=−∣Bpos∣1(x,y)∈Bpos∑[ylogP^(y=1∣x)],2 versus Lpos=−∣Bpos∣1(x,y)∈Bpos∑[ylogP^(y=1∣x)],3 for the class-unbiased method; in the imbalanced setting, ERM positive-class accuracy is Lpos=−∣Bpos∣1(x,y)∈Bpos∑[ylogP^(y=1∣x)],4 versus Lpos=−∣Bpos∣1(x,y)∈Bpos∑[ylogP^(y=1∣x)],5 for the proposed method. On DAIC-WOZ, described as a severe-imbalance setting, ERM collapses to the negative class; ERM with class weighting gives Lpos=−∣Bpos∣1(x,y)∈Bpos∑[ylogP^(y=1∣x)],6, G-DRO only gives Lpos=−∣Bpos∣1(x,y)∈Bpos∑[ylogP^(y=1∣x)],7, ERM plus the inequality term gives Lpos=−∣Bpos∣1(x,y)∈Bpos∑[ylogP^(y=1∣x)],8, and G-DRO plus the inequality term gives Lpos=−∣Bpos∣1(x,y)∈Bpos∑[ylogP^(y=1∣x)],9. MODMA, ADReSS, BreastMNIST, and RetinaMNIST all show consistent Macro F1 gains of Lneg=−∣Bneg∣1(x,y)∈Bneg∑[(1−y)logP^(y=0∣x)].0–Lneg=−∣Bneg∣1(x,y)∈Bneg∑[(1−y)logP^(y=0∣x)].1 points over the best ERM variant. In a subsampled RetinaMNIST case study, Macro F1 improves from Lneg=−∣Bneg∣1(x,y)∈Bneg∑[(1−y)logP^(y=0∣x)].2 to Lneg=−∣Bneg∣1(x,y)∈Bneg∑[(1−y)logP^(y=0∣x)].3 for balanced Lneg=−∣Bneg∣1(x,y)∈Bneg∑[(1−y)logP^(y=0∣x)].4-sample training and from Lneg=−∣Bneg∣1(x,y)∈Bneg∑[(1−y)logP^(y=0∣x)].5 to Lneg=−∣Bneg∣1(x,y)∈Bneg∑[(1−y)logP^(y=0∣x)].6 in the imbalanced Lneg=−∣Bneg∣1(x,y)∈Bneg∑[(1−y)logP^(y=0∣x)].7-positive setting (Zuo et al., 9 Aug 2025).
The post-hoc calibration framework reports a different empirical pattern but a closely related conclusion. Across CIFAR-10, CIFAR-100, ImageNet, and long-tailed CIFAR10-LT/CIFAR100-LT, synchronizing class-wise losses reduces Expected Calibration Error while preserving accuracy. For example, CWLS+TS reduces average ECE on CIFAR-10 from Lneg=−∣Bneg∣1(x,y)∈Bneg∑[(1−y)logP^(y=0∣x)].8 to Lneg=−∣Bneg∣1(x,y)∈Bneg∑[(1−y)logP^(y=0∣x)].9, on CIFAR-100 from Lcls-ineq=Lpos−Lneg.0 to Lcls-ineq=Lpos−Lneg.1, and on ImageNet from Lcls-ineq=Lpos−Lneg.2 to Lcls-ineq=Lpos−Lneg.3, with accuracy change Lcls-ineq=Lpos−Lneg.4. On CIFAR10-LT, average ECE drops from Lcls-ineq=Lpos−Lneg.5 to Lcls-ineq=Lpos−Lneg.6; on CIFAR100-LT, from Lcls-ineq=Lpos−Lneg.7 to Lcls-ineq=Lpos−Lneg.8, while accuracy is maintained or slightly improved by up to Lcls-ineq=Lpos−Lneg.9 (Jung et al., 2023).
Taken together, these results support a recurrent empirical claim: synchronizing class-level losses can improve either downstream classification balance or calibration reliability without the accuracy degradation often associated with naive reweighting.
5. Class-wise loss synchronization and other related objectives
The calibration framework defines per-class negative log-likelihood losses on a validation set,
Lpos≈Lneg0
collects them into Lpos≈Lneg1, and measures inequality by
Lpos≈Lneg2
The method normalizes each class loss, maps it through a sigmoid-shaped gain with parameters Lpos≈Lneg3 and Lpos≈Lneg4, and trains with
Lpos≈Lneg5
where Lpos≈Lneg6 preserves the integrated loss scale. Classes with Lpos≈Lneg7 receive extra gradient, whereas over-fitted classes with Lpos≈Lneg8 have reduced gradient (Jung et al., 2023).
Other imbalance-oriented objectives use different proxies for class-level underperformance. The Class-Wise Difficulty-Balanced loss defines class difficulty as Lpos≈Lneg9, where C0 is validation accuracy for class C1 at epoch C2, and sets class weights to C3 with a dynamic focusing exponent
C4
so that harder classes receive larger weight as training evolves (Sinha et al., 2020).
OWAdapt takes class-level cross-entropies C5, sorts them, and applies an Ordered Weighted Average,
C6
reassigning the largest weights to the currently highest class-losses at each iteration. The intended effect is again to focus optimization pressure on the worst-performing classes, whether due to imbalance or intrinsic difficulty (Maldonado et al., 2023).
From a theoretical perspective, the balanced-loss literature studies surrogate objectives such as Generalized Logit-Adjusted (GLA) and Generalized Class-Aware (GCA) losses. Both are presented as Bayes-consistent for balanced-loss targets, but their C7-consistency guarantees differ markedly: GLA requires complete hypothesis sets and yields bounds scaling at least as C8, whereas GCA is C9-consistent for any bounded or complete hypothesis set with bounds scaling as wc=sg(exp(τLpos)+exp(τLneg)exp(τLc)),c∈{pos,neg}.0 (Cortes et al., 30 Dec 2025).
These objectives are not the same loss, but they share a structural commitment: class-wise optimization signals should adapt to class-specific underperformance rather than remain fixed by raw frequency alone.
6. Earlier hashing usage and conceptual boundaries
In deep supervised hashing, the term “class-wise inequality” refers to a different construction. The method models each class as an isotropic Gaussian in Hamming-code space and minimizes the negative log-likelihood
where wc=sg(exp(τLpos)+exp(τLneg)exp(τLc)),c∈{pos,neg}.2 is the wc=sg(exp(τLpos)+exp(τLneg)exp(τLc)),c∈{pos,neg}.3-bit code and wc=sg(exp(τLpos)+exp(τLneg)exp(τLc)),c∈{pos,neg}.4 is the binary center of class wc=sg(exp(τLpos)+exp(τLneg)exp(τLc)),c∈{pos,neg}.5. Training proceeds in two stages: a “cubic” stage constrains codes and centers to remain in wc=sg(exp(τLpos)+exp(τLneg)exp(τLc)),c∈{pos,neg}.6 with ReLU penalties, and a “vertex” stage adds a quantization term wc=sg(exp(τLpos)+exp(τLneg)exp(τLc)),c∈{pos,neg}.7 with wc=sg(exp(τLpos)+exp(τLneg)exp(τLc)),c∈{pos,neg}.8 (Zhe et al., 2018).
This formulation is class-wise because it clusters samples around class centers and repels them from other class centers. It is called “class-wise inequality” in the cited summary, but it does not equalize positive- and negative-class losses or minimize a dispersion statistic over class losses. Its primary domain is semantics-preserving hashing rather than calibration or debiasing under class imbalance.
A common misconception is therefore to treat “class-wise inequality loss” as a single, universally accepted object. The literature instead contains several paper-specific definitions. Another misconception is to equate all such losses with class-frequency weighting. Related work explicitly argues that the number of available training data may not always be a good clue to determine the weighting strategy, because some minority classes may already be sufficiently represented and overweighting them can drop overall performance (Sinha et al., 2020). In that sense, the broader class-wise-inequality perspective is less about frequency per se than about unequal optimization pressure, unequal fittedness, or unequal class-specific reliability.
The most defensible general characterization is therefore narrow and operational: class-wise inequality losses are objectives that monitor some class-level discrepancy—between two class losses, among many class losses, or between class centers in an embedding space—and convert that discrepancy into a training signal. Which discrepancy is used, and why, depends entirely on the task: debiasing in medical diagnosis (Zuo et al., 9 Aug 2025), synchronization for calibration (Jung et al., 2023), adaptive class emphasis under imbalance (Sinha et al., 2020, Maldonado et al., 2023, Cortes et al., 30 Dec 2025), or semantics-preserving hashing (Zhe et al., 2018).