Class-Balanced Random Sampling Methods

• Class-balanced random sampling is a methodology that draws equal samples from each class to mitigate majority bias in supervised learning.
• It is applied in ensemble, active, and federated settings using techniques like balanced bootstrapping, under-sampling, and adversarial or privacy-preserving augmentation.
• Empirical studies demonstrate significant gains in minority-class metrics and overall model performance across diverse applications.

Class-balanced random sampling is a family of sampling methodologies designed to mitigate the pervasive issue of class imbalance during supervised machine learning, particularly in ensemble learning, active learning, and federated learning. The central aim is to construct training or selection pools in such a way that each class is represented with approximately equal probability or count, counteracting the statistical dominance of majority classes that otherwise degrades minority-class recall and inflates model bias. Class-balanced sampling is used in numerous variants, including balanced bootstrap construction for tree ensembles, class-weighted uncertainty in active learning proposals, stratified sampling in tabular settings, and explicit group-balancing in distributed or federated learning. This article surveys the formal definitions, canonical algorithmic treatments, recent advancements such as adversarial augmentation and privacy-preserving selection, together with theoretical guarantees and empirical impacts.

1. Formal Definitions and Core Algorithms

The canonical class-balanced random sampling procedure is most directly characterized in the context of ensemble construction and stochastic optimization. For a $C$-class supervised task with dataset $D=\{(x_i,y_i)\}_{i=1}^N$, standard random sampling or image-balanced sampling chooses each datum uniformly, resulting in class $j$ sampled with probability $n_j/N$ (where $n_j = | \{i : y_i=j\} |$), thus replicating any existing class imbalance (Liu et al., 2021).

Class-balanced random sampling instead proceeds by:

• (1) Drawing class labels uniformly: $P(y=j) = 1/C$;
• (2) For each chosen class, drawing one sample uniformly from the within-class pool: $P(x_i|y_i=j) = 1/n_j$.

Thus, $P(x_i) = 1/(C n_{y_i})$, ensuring that every class is represented equally in the sampled batch or subset.

For binary classification ($N_m \ll N_M$), as in the Balanced Random Forest (BRF) approach, each bootstrap subset contains exactly $N_m$ samples from the minority and $D=\{(x_i,y_i)\}_{i=1}^N$0 from the majority, sampled with replacement (Newaz et al., 2024). For multi-class settings, $D=\{(x_i,y_i)\}_{i=1}^N$1 samples are taken from each class. This approach is adopted in both classical algorithms (e.g., BRF, Balanced Stratified Sampling (Saleema et al., 2014)) and deep ensemble frameworks, such as DeepBalance, which constructs balanced bootstraps for each member of a deep belief network ensemble (Xenopoulos, 2017).

2. Variants Across Domains: Ensemble Learning, Survival Analysis, and Federated Systems

Balanced random sampling underlies a spectrum of algorithmic frameworks, each adapted to particular problem structures and data modalities:

• Tree Ensembles: In BRF, each decision tree is trained on a bootstrap sample where the minority and majority classes are balanced by random under-sampling of the majority. The recently introduced iBRF augments this with a hybrid pipeline of Neighborhood Cleaning (NC), random under-sampling (RUS), and SMOTE-based minority oversampling (Newaz et al., 2024). For Random Survival Forests, balanced sampling is achieved by augmenting the minority (event) class using SMOTE to match the majority (censored) class before bootstrapping (Afrin et al., 2018).
• Stratified Sampling in Tabular Data: Balanced stratified sampling, exemplified in cancer prognosis prediction, constructs subsamples by drawing the same number of samples from each class (after threshold-based exclusion of extremely rare classes), thus ensuring an exactly balanced training set (Saleema et al., 2014).
• Active Learning in Detection: For object detection with severe box-level imbalance, class-balanced random sampling is implemented by re-weighting each candidate box's uncertainty inversely by class frequency, $D=\{(x_i,y_i)\}_{i=1}^N$2, substantially increasing the selection rate for minority classes (Liao et al., 25 Aug 2025).
• Federated and Distributed Learning: In federated settings, client selection can itself be subject to class imbalance, resulting in global datasets skewed toward over-represented classes. Fed-CBS introduces an explicit quadratic class-imbalance degree (QCID) as a selection criterion, with privacy-preserving computation of client-level label histograms underlying a computationally efficient and provably imbalance-reducing client sampling procedure (Zhang et al., 2022).

A summary of core sampling procedures is provided for comparison:

Method	Sampling Criterion	Domain
Standard Balanced RF	Draw $D=\{(x_i,y_i)\}_{i=1}^N$3 samples from each class per bootstrap	Tree Ensembles
Balanced Stratified	Draw equal $D=\{(x_i,y_i)\}_{i=1}^N$4 samples per class (after pruning rare classes)	Tabular Classification
DeepBalance	Draw $D=\{(x_i,y_i)\}_{i=1}^N$5 from each class per ensemble member	Deep Ensembles
Class-Balanced AL	Weight selection by $D=\{(x_i,y_i)\}_{i=1}^N$6	Active Object Detection
Fed-CBS (QCID)	Minimize squared distance to uniform class-distribution across clients	Federated Learning

3. Theoretical Properties and Rationale

The primary theoretical motivation for class-balanced random sampling is to mitigate class prior-induced bias in learning algorithms, particularly under severe imbalance. By constructing training batches or subsets with equal class frequencies, the loss landscape presented to the learner (or ensemble member) prevents majority-class examples from dominating parameter updates and ensures that minority-class decision boundaries are adequately represented (Newaz et al., 2024, Xenopoulos, 2017).

For ensemble learners, different balanced bootstraps diversify the selection of majority-class examples, preserving more information against the information bottleneck created by simple global under-sampling (Xenopoulos, 2017). In survival analysis, balancing corrects the systematic underestimation of minority-class cumulative hazard in Random Survival Forests, as quantified by shifts in the Nelson-Aalen estimator (Afrin et al., 2018).

Hybrid samplers, such as the iBRF (NC→RUS→SMOTE), explicitly balance noise reduction (NC), bias-variance trade-off (via mild under-sampling and synthetic oversampling), and ensemble variance reduction. Theoretical analysis in federated learning demonstrates that minimizing QCID across clients strictly decreases the expected class-imbalance and provably speeds optimization convergence (Zhang et al., 2022).

4. Advanced Extensions: Hybrid, Adversarial, and Privacy-Preserving Sampling

Recent work has extended class-balanced random sampling in several significant directions:

• Hybrid Sampling (iBRF): iBRF composes multiple balancing steps within each ensemble member: Neighborhood Cleaning removes noisy or borderline majorities, RUS lightly reduces the remaining majority count, and SMOTE fills the minority to match, ensuring that each bootstrap is balanced but not over-fitted to synthetic data (Newaz et al., 2024).
• Adversarial and Feature-Augmented Sampling (Breadcrumb): A central limitation of naïve class-balance is overfitting on few-shot classes. Breadcrumb leverages adversarial feature augmentation (EMANATE), backtracking feature snapshots across epochs and aligning them to present increasingly “hard” class representations, thus enforcing both class balance and increased intra-class variance without duplication. In this procedure, the sampling within each batch remains class-balanced, but the use of historical feature states injects difficulty and prevents overfitting to the minority classes’ few instances (Liu et al., 2021).
• Privacy-Preserving Federated Sampling (Fed-CBS): When class distribution is non-IID across federated clients, performing client selection under privacy constraints is challenging. Fed-CBS uses fully homomorphic encryption (FHE) to gather only inner-product statistics over local class histograms, constructing an expected QCID for any candidate set, and then stochastically selects clients to minimize imbalance under strong privacy guarantees (Zhang et al., 2022).

5. Empirical Results and Comparative Performance

Balanced random sampling techniques consistently deliver significant gains in minority-class metrics and overall predictive validity compared to baseline or naïve approaches:

• In iBRF, the hybrid sampler achieves average MCC and F1 improvements of ∼6 percentage points over standard BRF on 44 imbalanced datasets, also outperforming SMOTE-Bagging, RUSBoost, and other baselines (Newaz et al., 2024).
• BRSF (Balanced Random Survival Forests) yields an average Brier Score reduction of 55%, and C-index improvements of ~12 points compared to unbalanced RSF in survival analysis tasks (Afrin et al., 2018).
• In DeepBalance, ensembles of balanced-bootstrapped DBNs surpass both under- and over-sampling schemes in AUC and sensitivity, particularly in highly imbalanced fraud detection datasets (Xenopoulos, 2017).
• Box-level active learning with class-balanced sampling brings 2–4 mAP gains over entropy or diversity-only baselines in object detection (Liao et al., 25 Aug 2025).
• Fed-CBS not only reduces QCID by a factor of ten compared to random selection, but under heavy imbalance, may even outperform “full participation” federated training in rounds-to-convergence and test accuracy (Zhang et al., 2022).
• In classification tasks with high class cardinality and long-tailed distributions, adversarial class-balanced augmentation with Breadcrumb improves head and tail accuracy, with substantial gains in the “few-shot” regime (+4.9% top-1 accuracy on ImageNet-LT “few”) (Liu et al., 2021).

6. Practical Guidelines, Limitations, and Extensions

Rigorous implementation of class-balanced random sampling requires several practical considerations:

• Preserve all minority-class examples in every subsample or bootstrap to avoid irrecoverable information loss (Newaz et al., 2024).
• Prefer light random under-sampling (e.g., 10–30%) and pair with noise-cleansing or synthetic augmentation to moderate the variance-inflation of severe under-sampling (Newaz et al., 2024).
• In active selection, compute empirical class frequencies from pseudo labels or model predictions, and re-weight selection scores inversely to these frequencies (Liao et al., 25 Aug 2025).
• For stratified sampling on high-cardinality labels, consider thresholding micro-classes with minuscule support to prevent over-weighting noise (Saleema et al., 2014).
• In federated learning, leverage privacy-preserving aggregation for class-support histograms when minimizing group imbalance is desired (Zhang et al., 2022).
• Adversarial augmentation addresses overfitting in few-shot or long-tailed scenarios; feature snapshot alignment can be added at little computational cost (Liu et al., 2021).

Reported limitations include the need for careful tuning of undersampling rates, reweighting strengths ($D=\{(x_i,y_i)\}_{i=1}^N$7 in detection), or selection hyperparameters. The reliability of early pseudo labels, privacy infrastructure for distributed statistics, and extension to regression or structured outputs are open areas. Generalization of balancing concepts to region-level, segmentation, or multi-label scenarios is suggested as a future trajectory (Liao et al., 25 Aug 2025).

7. References to Pivotal Works

• "iBRF: Improved Balanced Random Forest Classifier" (Newaz et al., 2024)
• "Introducing DeepBalance: Random Deep Belief Network Ensembles to Address Class Imbalance" (Xenopoulos, 2017)
• "Balanced Random Survival Forests for Extremely Unbalanced, Right Censored Data" (Afrin et al., 2018)
• "Box-Level Class-Balanced Sampling for Active Object Detection" (Liao et al., 25 Aug 2025)
• "Cancer Prognosis Prediction Using Balanced Stratified Sampling" (Saleema et al., 2014)
• "Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction" (Zhang et al., 2022)
• "Breadcrumbs: Adversarial Class-Balanced Sampling for Long-tailed Recognition" (Liu et al., 2021)

This body of literature rigorously establishes class-balanced random sampling as a foundational methodology for combating class imbalance in modern machine learning, spanning ensemble algorithms, active and federated learning, survival analysis, and long-tailed recognition. The continued evolution of hybrid and adversarial balanced samplers, together with growing attention to privacy-respecting and structured-output settings, suggests further theoretical and applied advances are forthcoming.