Adaptive Class Balancing Strategy

• Adaptive class balancing strategy is a collection of dynamic techniques that adjust data sampling, synthetic over-sampling, and loss weighting to mitigate class imbalance.
• The approach uses local data statistics and adaptive weighting mechanisms to modulate corrective actions in data, sample, or loss function space during training.
• Empirical results demonstrate enhanced minority-class recall, reduced calibration errors, and improved overall performance across applications such as medical diagnosis and federated learning.

Adaptive class balancing strategy refers to a family of data-centric and algorithmic techniques designed to dynamically mitigate class imbalance during supervised learning. Unlike static resampling or fixed weighting schemes, adaptive class balancing modulates its corrective actions in data space, sample selection, or loss function space by explicitly monitoring local data distributions, per-class difficulties, or evolving class statistics during training. This systematically reduces the dominance of majority classes and promotes robust, fair, and generalizable learning, even when labeled samples are scarce or highly skewed. Adaptive class balancing strategies span synthetic over-sampling, hybrid resampling, dynamic weighting, sample selection in online or active learning, and specialized regularization schemas, and are broadly applicable across standard classification, deep models, semantic segmentation, active/federated/continual learning, and fairness-critical domains.

1. Methodological Foundations of Adaptive Class Balancing

Adaptive class balancing strategies emerged to address severe performance degradation and fairness violations seen in imbalanced learning, where rare (minority) classes are systematically underrepresented in both data and decision boundaries. Such strategies typically adapt to the data or model state using local statistics, dynamic weighting, or feedback from the ongoing optimization process.

Synthetic Over-Sampling and Data Augmentation

Self-adaptive synthetic over-sampling methods, e.g., SASYNO (Gu et al., 2019), identify local neighborhoods among minority class samples (using data-driven distance quantifiers) and generate synthetic instances by Gaussian perturbation and linear interpolation, expanding minority support only in regions dominated by authentic data. Related frameworks such as MWMOTE and adaptive Mahalanobis-based schemes (Yousefimehr et al., 17 May 2025) calculate per-sample or subregion importance, allocating more synthetic data near the minority–majority decision boundary, thus adaptively shifting the decision surface.

Hybrid Sampling and Cleaning

Hybrid sampling frameworks, such as SMOTE-RUS-NC (Newaz et al., 2022), couple adaptive cleaning (neighborhood cleaning that removes noisy majorities) with targeted majority undersampling and locally restricted over-sampling. Parameterization is governed by class frequencies and retained sample qualities, ensuring that only essential majority information is dropped and minority boosting is limited to achievable balance.

Loss-level Adaptive Class Reweighting

Dynamic weighting strategies operate in loss space to rebalance class contributions. For instance, adaptive focal loss (Zhou et al., 20 Oct 2025) introduces class-dependent weights (often the ratio of the most to least frequent class), amplifying losses from minority classes and hard-to-classify samples simultaneously. Class Adaptive Label Smoothing (CALS) (Liu et al., 2022) assigns learnable, class-specific multipliers to balance margin violations and calibration errors, updating these multipliers online based on validation set constraint violations.

Online, Active, and Federated Learning

In streaming/environments with concept drift, algorithms such as AREBA (Malialis et al., 2020) maintain per-class queues with adaptive capacities that reflect observed class frequencies—old instances are overwritten based on a time-decay metric, preserving a near-uniform distribution regardless of class priors or concept drift. In active learning, class balance is enforced during batch selection: the acquisition process is regularized toward a uniform class distribution per query round, using an explicit balancing term in the optimization problem (Bengar et al., 2021, Das, 20 May 2024). In federated settings, global coordination of replay buffers and adaptive temperature scaling are used to balance classes across clients and incremental tasks (Qi et al., 10 Jul 2025).

Incremental and Continual Learning

Continual and incremental settings introduce catastrophic forgetting and additional class imbalance with new classes. Here, Adaptive Weight Fusion (AWF) (Sun et al., 13 Sep 2024) or adaptive prototype replay (Zhu et al., 17 Dec 2024) techniques employ trainable fusion parameters or prototype updating strategies to adaptively preserve and re-balance knowledge across class increments.

2. Mathematical Formulations and Optimization Principles

Central to adaptive class balancing are rigorously defined metrics and update rules that encourage uniformity or calibrated error distribution among classes.

Neighborhood and Distance-based Quantification

The SASYNO method (Gu et al., 2019) defines an objective quantifier $y$ based on local Euclidean distances:

$$y = \frac{1}{P_u} \sum_{|x_i - x_j| < u} |x_i - x_j|,$$

where $u$ is the mean pairwise distance. Neighbor pairs for oversampling are restricted to $|x_i - x_j| < y$.

Gaussian disturbance for synthetic point generation is controlled by the local standard deviation $\sigma$, with noise $g \sim \mathcal{N}(0, \sigma^2)$, and new samples created through interpolation

$S_k = r_k P_k + (1 - r_k)q_k,$

where $r_k$ is sampled uniformly.

Adaptive Weighting in Losses and Calibration

In adaptive focal loss (Zhou et al., 20 Oct 2025), per-class weight $\alpha_i = N_{\text{max}} / N_i$ and the modulating factor $(1 - p_t)^\gamma$ together scale the cross entropy,

$$L_{\text{adapt}} = -\sum_{i=1}^C \alpha_i (1 - p_t)^\gamma y_i \log(p_i).$$

In CALS (Liu et al., 2022), learnable classwise multipliers $\lambda_k$ update after each epoch using an augmented Lagrangian with constraint violation penalties for per-class logit margins,

$$\lambda_k^{j+1} = \frac{1}{|D_{\text{val}}|} \sum_{(x,y) \in D_{\text{val}}} P^\prime\Big(\frac{d_k}{m} - 1, \rho_k^{(j)}, \lambda_k^{(j)}\Big).$$

Distribution and Mutual Information Balancing

Class-conditional distribution balancing (Zhao et al., 24 Apr 2025) optimizes weights $\alpha_c$ on class $c$'s training samples to minimize the average 2-Wasserstein distance between per-class and marginal latent representations,

$$L_{\text{mutual}} = \frac{1}{C} \sum_{c=1}^C W_2(p(z|y=c, \alpha_c), p(z)).$$

Optimization operates under simplex constraints $\sum_i [\alpha_c]_i = 1$, $[\alpha_c]_i \ge 0$.

Buffer Management and Selection Policies

In AREBA (Malialis et al., 2020), adaptive queue size is determined by the online decayed size metric:

$$s_k^t = \theta s_k^{t-1} + I_{[y^t = k]}(1 - \theta),$$

with storage capacity adjusted to maintain a classwise balance.

3. Empirical Performance and Comparative Evaluation

Adaptive class balancing consistently improves diverse performance metrics:

• Enhanced minority-class sensitivity/recall, geometric mean (G-mean), and F-measure across classifiers, with confusion matrices showing a more balanced distribution of errors (Gu et al., 2019, Newaz et al., 2022, Liu et al., 2021).
• Reduction in miscalibration and classwise Expected Calibration Error (ECE) under long-tailed regimes (Liu et al., 2022).
• Quantitatively, methods such as SASYNO (Gu et al., 2019) and DuBE (Liu et al., 2021) outperform canonical resampling methods (SMOTE, ADASYN, RUS, standard under/oversampling) in settings with true class imbalance or complex intra-class difficulty, with substantial gains in specificity and averaged metrics.
• Hybrid sampling (SMOTE-RUS-NC (Newaz et al., 2022)) delivers up to 40 percentage points higher g-mean, especially in highly skewed datasets.
• Federated and incremental strategies achieve 2–15% Top-1 accuracy improvements over state-of-the-art baselines in heterogeneous settings (Qi et al., 10 Jul 2025), with buffer visualizations confirming preserved minority presence across tasks.
• Adaptive active learning strategies, by reweighting or inverting class frequencies at query time (Bengar et al., 2021, Das, 20 May 2024), lead to faster convergence and higher F1 under severe class imbalance, requiring fewer labels for baseline-matching performance.

4. Practical Applications and Use Cases

Adaptive class balancing is critical in domains where minority detection is paramount or where data acquisition for all classes is impractical:

• Medical diagnosis: rare disease/fault detection, severity grading (e.g., Parkinson’s stage diagnosis (Zhou et al., 20 Oct 2025)), and lesion segmentation, which benefit from dynamically upweighted rare classes in both loss and sample generation.
• Fraud and anomaly detection: adaptive balancing ensures robust sensitivity to fraud cases in otherwise overwhelming majority-class financial transaction data (Newaz et al., 2022).
• Remote sensing and environmental monitoring: agricultural pattern recognition (Agriculture-Vision (Liu et al., 18 Jun 2024)) uses adaptive per-class weights, rare class mosaic augmentation, and probability post-processing to improve semantic segmentation mIoU under long-tailed class distributions.
• Federated and distributed continual learning: cross-client and cross-task class balancing frameworks maintain global fairness and robustness in class-incremental federated settings (Qi et al., 10 Jul 2025).
• Active/online learning in robotics and industrial monitoring, where minimizing labeled data and balancing evolving class frequencies are crucial (Das, 20 May 2024).

5. Impact on Fairness, Calibration, and Generalization

Fairness and calibration are directly enhanced by adaptive class balancing:

• SASYNO and related synthetic oversampling methods rebalance the confusion matrix in favor of minority and tail classes, reducing disparate impact (Gu et al., 2019).
• CALS (Liu et al., 2022) demonstrates that classwise adaptive calibration penalties can bring ECE for rare classes down substantially (as low as 2.15% on ImageNet-LT), preventing the overconfidence typical of long-tailed learning.
• In class-conditional distribution balancing (Zhao et al., 24 Apr 2025), mutual information minimization suppresses spurious correlations, leading to better generalization in domain-shift and resource-limited regimes.
• Prototype-based replay and adaptive fusion techniques in continual learning (Zhu et al., 17 Dec 2024, Sun et al., 13 Sep 2024) prevent catastrophic forgetting due to class or representation drift, maintaining class discrimination as data and model evolve.

6. Algorithmic Extensions and Future Research Directions

Current research identifies several axes for further enhancement:

• Integration with generative models (GANs, VAEs, diffusion models (Qin et al., 2023)) can provide adaptive, data-aware augmentation that is semantically richer than linear or naive synthetic generation (Yousefimehr et al., 17 May 2025).
• End-to-end context-aware resampling that jointly adapts sample selection, windowing parameters (in time-series AL), and loss calibration during training (Das, 20 May 2024).
• Parameter auto-tuning, e.g., for the focusing parameter γ in focal loss, or hyperparameters controlling prototype adaptation and classwise weight updates, to ensure optimality in arbitrary imbalance regimes.
• Bridging hybrid-space balancing: simultaneously enforcing inter-class and intra-class balance as in DuBE (Liu et al., 2021), and cross-distribution alignment as in CCDB (Zhao et al., 24 Apr 2025).
• Extension of balancing strategies to multi-label, high-dimensional, heterogeneously distributed, or non-Euclidean domains.

7. Summary Table: Algorithmic Principles Across Techniques

Method/Framework	Principle of Adaptation	Domain/Application
SASYNO (Gu et al., 2019)	Local pairwise neighborhood/interpolation, data-driven σ	General classification
AREBA (Malialis et al., 2020)	Adaptive buffer size (per-class), time-decayed metrics	Online/stream learning
Hybrid sampling (Newaz et al., 2022)	Sequential data cleaning, controlled under/oversampling	Ensemble/classification
CALS (Liu et al., 2022)	Per-class penalty multiplier, augmented Lagrangian	Deep nets/classification, segmentation
DuBE (Liu et al., 2021)	Joint inter/intra-class balancing, error density weighting	Ensemble, UCI datasets
AWF (Sun et al., 13 Sep 2024)/Adapter (Zhu et al., 17 Dec 2024)	Adaptive (trainable) fusion, prototype compensation	Incremental segmentation
CCDB (Zhao et al., 24 Apr 2025)	Sample reweighting to minimize $\mathcal{W}_2$ divergence	Group-robust/debiased classification
SRN-BRF (Newaz et al., 2022)	Ensemble on balanced subsets after hybrid sampling	Noisy/highly imbalanced data

This comparative view contextualizes each method’s principle of adaptive balancing and its domain of effectiveness.

8. Conclusion

Adaptive class balancing strategies anchor state-of-the-art solutions for robust, fair, and generalizable learning under class-imbalanced regimes. By employing local data geometry, per-class loss weighting, online buffering, or mutual information minimization, these methods have demonstrably improved performance in practical and benchmark scenarios. As data modalities, application domains, and usage constraints evolve, adaptive strategies—especially those coupling data-centric and model-side adaptation—are central to future advances in equitable and reliable machine learning systems.