Class-Balanced Sampling Strategy

Updated 9 July 2026

Class-balanced sampling strategy is a method that equalizes class representation by controlling sample exposure through inverse-frequency or quota-based rules.
It is applied across diverse areas like long-tailed recognition, federated learning, active learning, and medical segmentation to enhance minority-class performance.
Various implementations such as square-root sampling, fixed quotas, and hybrid methods address exposure control while mitigating risks like overfitting.

Class-balanced sampling strategy denotes a family of data-selection procedures that explicitly control how often classes, class-conditioned instances, or class-bearing structures enter optimization, with the aim of reducing the dominance of frequent classes and improving minority-class learning. In the literature, the strategy appears in long-tailed recognition, source-free domain adaptation, federated learning, active learning, medical image classification, segmentation, 3D object detection, and time-series pre-training, but the central invariant is exposure control: the sampler changes the training distribution before the loss is evaluated, whereas loss reweighting changes the contribution of samples already present in the batch (Liu et al., 2021, Qu et al., 2022, Wong et al., 18 Apr 2025, Skylitsis et al., 19 May 2026).

1. Formal definition and statistical objective

In its most basic form, class-balanced sampling replaces uniform-over-samples training with a distribution that equalizes class frequency. For a dataset with class counts $n_c$ , instance-based sampling induces $P_{\mathrm{inst}}(c)=n_c/N$ , whereas class-balanced sampling uses either inverse-frequency rules or uniform-over-class selection followed by uniform sampling within the chosen class (Galdran et al., 2021). In decoupled long-tailed recognition, this is stated explicitly as sampling each class with probability $\frac{1}{C}$ , oversampling tail classes, with uniform sampling within the selected class (Liu et al., 2021). A broader family uses

$p(c)=\frac{n_c^q}{\sum_{k=1}^C n_k^q},$

where $q=1$ recovers instance-based sampling, $q=0$ yields uniform-over-class sampling, and $q=\frac12$ gives square-root sampling (Galdran et al., 2021).

A fixed-allocation version appears in balanced stratified sampling. Given $K$ eligible classes and a target sample size $n$ , the per-class target is

$n_c^{\mathrm{target}}=\left\lfloor \frac{n}{K}\right\rfloor,$

with classes that cannot meet this quota without oversampling removed until the allocation is feasible (Saleema et al., 2014). This formulation makes the balancing objective explicit: the sample is not merely representative of the empirical prior; it is constructed to alter the prior seen by the learner.

The same principle can be expressed in other statistical forms. In federated learning, the aim is not only balanced class counts but balanced exposure in aggregated gradients. Stratify defines a Stratified Label Schedule,

$P_{\mathrm{inst}}(c)=n_c/N$ 0

with label selection probability

$P_{\mathrm{inst}}(c)=n_c/N$ 1

so that uniform $P_{\mathrm{inst}}(c)=n_c/N$ 2 yields uniform label exposure (Wong et al., 18 Apr 2025). Fed-CBS instead measures deviation from balance via the Quadratic Class-Imbalance Degree,

$P_{\mathrm{inst}}(c)=n_c/N$ 3

and samples client subsets whose grouped data distribution is closer to uniform (Zhang et al., 2022). These formulations make clear that class-balanced sampling is not a single algorithm but a design principle for constraining the training distribution.

2. Canonical constructions

The literature implements class balancing through several recurring constructions: fixed per-class quotas, uniform-over-class batch formation, ranking-based rebalancing, stratified scheduling, and hybrid under/over-sampling. The balancing unit can be an instance, a frame, a slice, a target-domain prototype candidate, a client subset, a detection box, or an implicit cluster of unlabeled patterns.

Setting	Sampling unit	Representative rule
Long-tailed recognition	class then instance	$P_{\mathrm{inst}}(c)=n_c/N$ 4 (Liu et al., 2021)
SFDA	target category candidates	$P_{\mathrm{inst}}(c)=n_c/N$ 5 per class (Qu et al., 2022)
Federated learning	scheduled labels	$P_{\mathrm{inst}}(c)=n_c/N$ 6 (Wong et al., 18 Apr 2025)
Time-series corpus pre-training	grid cell, series, window	$P_{\mathrm{inst}}(c)=n_c/N$ 7 (Shao et al., 23 May 2025)

A quota-based design appears in source-free domain adaptation. BMD fixes an equal quota

$P_{\mathrm{inst}}(c)=n_c/N$ 8

then selects the top- $P_{\mathrm{inst}}(c)=n_c/N$ 9 most confident target instances for each class from the entire target set, rather than applying local confidence thresholds independently (Qu et al., 2022). This is a global inter-class balanced sampling strategy: every class contributes the same number of candidate instances to prototype construction, even when transfer difficulty varies across categories.

A multiset-duplication design appears in 3D object detection. CBGS constructs per-class frame sets $\frac{1}{C}$ 0, computes $\frac{1}{C}$ 1, fixes a uniform quota $\frac{1}{C}$ 2 with $\frac{1}{C}$ 3, and samples $\frac{1}{C}$ 4 frames with replacement from each $\frac{1}{C}$ 5, producing an expanded multiset dataset $\frac{1}{C}$ 6 whose duplicates are intentional (Zhu et al., 2019). In the nuScenes setting, this balanced dataset sampling is complemented by GT-AUG and a balanced grouping head (Zhu et al., 2019).

A ranking-based design appears in box-level active learning for object detection. Candidate boxes are scored by

$\frac{1}{C}$ 7

where $\frac{1}{C}$ 8 is uncertainty and $\frac{1}{C}$ 9 is the empirical pseudo-label frequency of the predicted class; the top- $p(c)=\frac{n_c^q}{\sum_{k=1}^C n_k^q},$ 0 boxes are annotated under a box budget (Liao et al., 25 Aug 2025). No hard class quotas are imposed, but the exponential factor smoothly down-weights boxes from pseudo-label-majority classes.

Hybrid samplers combine undersampling and oversampling rather than enforcing balance through a single probability law. Both iBRF and SMOTE-RUS-NC apply Neighborhood Cleaning Rule, then Random Undersampling, then SMOTE, either as a preprocessing pipeline or inside each bootstrap sample of an ensemble (Newaz et al., 2024, Newaz et al., 2022). These methods treat class balance as a constrained reconstruction problem: remove noisy or redundant majority points, reduce majority volume, then synthesize minority support until class counts match.

3. Supervised learning, long tails, and synthetic mixing

In long-tailed visual recognition, class-balanced sampling is often embedded in a two-stage pipeline. “Breadcrumbs: Adversarial Class-Balanced Sampling for Long-tailed Recognition” defines stage 1 with image-balanced random sampling for representation learning and stage 2 with class-balanced sampling for classifier learning, using uniform-over-class selection (Liu et al., 2021). The paper’s central objection is that ordinary class-balanced sampling over-fits few-shot classes because it repeatedly samples the same images; duplication alone is not adversarial in feature space (Liu et al., 2021). EMANATE and Breadcrumb replace duplicated tail features with aligned feature snapshots collected across epochs, so CBS is retained as the training distribution while the repeated draws become harder and more diverse.

“Class-Balanced Distillation for Long-Tailed Visual Recognition” uses the same basic class-balanced stage-2 prior, $p(c)=\frac{n_c^q}{\sum_{k=1}^C n_k^q},$ 1, but changes its role. Instead of freezing the representation and retraining only the classifier, CBD retrains the backbone under class-balanced sampling while distilling feature geometry from an instance-trained teacher (Iscen et al., 2021). The resulting objective combines supervised cross-entropy with a cosine-distance feature distillation term, so the class-balanced sampler no longer serves only to debias the classifier; it drives continued representation learning on under-represented classes (Iscen et al., 2021).

Balanced-MixUp introduces a different construction. One sample is drawn with instance-based sampling, another with class-balanced sampling, and the mixed example is formed with

$p(c)=\frac{n_c^q}{\sum_{k=1}^C n_k^q},$ 2

This yields an effective class prior

$p(c)=\frac{n_c^q}{\sum_{k=1}^C n_k^q},$ 3

which interpolates between the empirical long-tail prior and a uniform-over-class prior (Galdran et al., 2021). The method therefore balances exposure without pure duplication, and does so through soft targets.

In tabular imbalance, hybrid samplers express the same objective through bootstrap-level balancing. iBRF includes all minority instances in each bootstrap, fills the remainder with majority samples drawn with replacement, then applies NC, controlled RUS, and SMOTE so that the final per-bootstrap class counts are equal (Newaz et al., 2024). “A Novel Hybrid Sampling Framework for Imbalanced Learning” formalizes the same sequencing as SMOTE-RUS-NC and its ensemble version SRN-BRF, with

$p(c)=\frac{n_c^q}{\sum_{k=1}^C n_k^q},$ 4

and uses $p(c)=\frac{n_c^q}{\sum_{k=1}^C n_k^q},$ 5 to match minority to post-RUS majority (Newaz et al., 2022). Here class-balanced sampling is inseparable from noise cleaning and support synthesis.

4. Prototype, client, and box-level balancing

In source-free domain adaptation, the problem is that target pseudo labels are themselves class-biased. BMD addresses this by selecting, for each target category, an equally sized candidate pool from the entire target set, building prototypes from those balanced pools, then replacing monocentric class summaries with $p(c)=\frac{n_c^q}{\sum_{k=1}^C n_k^q},$ 6 intra-class centers and dynamic pseudo labels updated at both epoch and mini-batch scales (Qu et al., 2022). Pseudo labels are assigned by nearest multicentric prototype, and mini-batch dynamics update prototypes by EMA with $p(c)=\frac{n_c^q}{\sum_{k=1}^C n_k^q},$ 7 (Qu et al., 2022). The class-balanced sampling step is thus upstream of prototype construction and downstream of the model’s current belief state.

In federated learning, class-balanced sampling can target labels or clients. Stratify schedules labels globally through SLS, restricts participation to clients that possess the scheduled labels, and optionally weights client choice by

$p(c)=\frac{n_c^q}{\sum_{k=1}^C n_k^q},$ 8

while keeping label identities hidden by placeholders and homomorphic encryption (Wong et al., 18 Apr 2025). Fed-CBS takes a client-subset view instead: at each round it selects a subset of available clients whose grouped label distribution minimizes QCID, with theoretical guarantees that the expected QCID is lower than under random sampling (Zhang et al., 2022). Both methods treat imbalance as a property of aggregated exposure, not merely of local data silos.

In active learning for classification, the balancing constraint is imposed during acquisition rather than SGD. “Class-Balanced Active Learning for Image Classification” introduces a batch selection objective

$p(c)=\frac{n_c^q}{\sum_{k=1}^C n_k^q},$ 9

where $q=1$ 0 is the expected class composition of the selected batch and $q=1$ 1 is the target class quota for the current cycle (Bengar et al., 2021). In box-level active object detection, the same principle is transferred from image acquisition to box acquisition by class-balancing the uncertainty score itself (Liao et al., 25 Aug 2025).

These settings indicate that class-balanced sampling is not confined to minibatch formation. It can operate over pseudo-labeled target candidates, encrypted client subsets, active-learning decisions, or detector outputs, provided there is a mechanism for estimating or constraining class exposure.

5. Dense prediction, non-standard classes, and pattern strata

In medical image segmentation, class-balanced sampling has been extended from images to episodes. “Disentangling Sampling from Training Budget in Class-Imbalanced CT Body Composition Segmentation” constructs each episode by uniformly choosing $q=1$ 2 foreground classes, then drawing $q=1$ 3 support and $q=1$ 4 query slices per selected class, with 500 episodes per epoch (Skylitsis et al., 19 May 2026). Because classes are sampled uniformly across episodes, rare tissues participate as episode targets as often as frequent tissues. The paper emphasizes that this changes which classes appear in each batch, whereas CE, Dice, or class-weighted losses act only on pixels already present (Skylitsis et al., 19 May 2026).

An analogous shift appears in time-series corpus construction. In BLAST, “classes” are not human-labeled semantic categories but implicit clusters of time-series patterns induced by statistical descriptors. Each series is characterized by stationarity, trend, seasonality, volatility, scedasticity, memorability, and anomaly rate, embedded into a 2D UMAP space, partitioned into a $q=1$ 5 grid, and sampled uniformly over non-empty grid cells and then uniformly within a cell (Shao et al., 23 May 2025). The resulting rule,

$q=1$ 6

treats grid cells as classes and makes the balanced sampling target pattern coverage rather than label coverage (Shao et al., 23 May 2025). Grid mixup then interpolates between sampled cells via Dirichlet weights (Shao et al., 23 May 2025).

This broadening of the balancing unit also appears in 3D detection. CBGS samples frames containing rare object classes more frequently, while SA-CBGS keeps the balanced grouping and duplicate sampling scheme but augments the detector with point-level foreground segmentation and center estimation heads to recover localization precision lost in coarse feature maps (Nagesh et al., 2022). The sampling strategy remains class-balanced at the data-pipeline level even though the optimization target is dense detection.

A plausible implication is that class-balanced sampling is best understood as a constraint on exposure over a chosen partition of the data space. In some applications that partition is the label set; in others it is a grid cell, a tissue-conditioned episode, or a class-conditioned frame index. The operational question is not whether one is sampling “classes” in the narrow semantic sense, but whether the training process equalizes access to underrepresented regions that matter for the task.

6. Empirical behavior, limitations, and methodological cautions

Empirically, class-balanced sampling often improves minority performance and can improve overall accuracy, but the effect is strongly conditioned by setting and implementation. In SFDA, BMD improved SHOT on Office-31 from $q=1$ 7 to $q=1$ 8, on Office-Home from $q=1$ 9 to $q=0$ 0, on VisDA-C from $q=0$ 1 to $q=0$ 2, and on PointDA-10 from $q=0$ 3 to $q=0$ 4; on VisDA-C’s hardest class, truck, SHOT rose from $q=0$ 5 to $q=0$ 6, while the coefficient of variation decreased from $q=0$ 7 to $q=0$ 8 (Qu et al., 2022). In long-tailed recognition, Breadcrumb improved ImageNet-LT ResNet-10 from $q=0$ 9 to $q=\frac12$ 0 overall and from $q=\frac12$ 1 to $q=\frac12$ 2 on few-shot classes relative to Decoupling(cRT) (Liu et al., 2021). In FL, Stratify reached $q=\frac12$ 3 on CIFAR-10 $q=\frac12$ 4 at $q=\frac12$ 5, versus FedAvg $q=\frac12$ 6 at $q=\frac12$ 7, and Fed-CBS reduced QCID on CIFAR-10 with Dirichlet $q=\frac12$ 8 from $q=\frac12$ 9 under random sampling to $K$ 0 while also reducing communication rounds (Wong et al., 18 Apr 2025, Zhang et al., 2022). In low-data CT segmentation, episodic sampling achieved mean Dice $K$ 1 versus $K$ 2 and $K$ 3 for random and weighted sampling, although much of that difference was shown to be attributable to a 12-fold difference in training iterations (Skylitsis et al., 19 May 2026).

The major caution is that balancing can alter the effective training prior and thereby introduce new pathologies. Balanced stratified sampling may help decision boundaries yet hurt calibrated probabilities if deployment priors differ from the balanced training set (Saleema et al., 2014). CBS in long-tailed recognition can over-fit few-shot classes because the same tail images are repeatedly sampled (Liu et al., 2021). In BMD, severe label shift or negligible model probability for a truly present class can make the top- $K$ 4 candidate pool low quality, and if initial pseudo labels are extremely noisy, multicentric clustering can fragment clusters (Qu et al., 2022). In FL, weighted client selection slightly relaxes privacy relative to uniform client selection, and CKKS-based label discovery overhead increases with the number of classes (Wong et al., 18 Apr 2025).

A second caution is methodological. Apparent improvements from a sampler may partly reflect changed training budgets rather than changed exposure geometry. The CT segmentation study showed that random and weighted sampling received far fewer iterations per epoch than episodic sampling in the low-data regime, and that under matched iteration budgets the performance gap shrank markedly, while episodic sampling still delayed overfitting (Skylitsis et al., 19 May 2026). This suggests that class-balanced sampling should be evaluated with iteration-aware schedules rather than epoch counts alone.

A common misconception is that class-balanced sampling is interchangeable with loss reweighting. The surveyed literature does not support that equivalence. Loss-level remedies reweight pixels, anchors, or logits already present in the batch; class-balanced sampling changes which samples, labels, boxes, frames, windows, or clients contribute to the batch in the first place (Skylitsis et al., 19 May 2026, Shao et al., 23 May 2025). Conversely, sampling alone is often insufficient: many successful systems combine balancing with multicentric prototypes and dynamic SCE in SFDA, feature distillation in long-tailed recognition, soft pseudo labels in active detection, GT-AUG or auxiliary structure supervision in detection, or synthetic vicinal mixing in medical image classification (Qu et al., 2022, Iscen et al., 2021, Liao et al., 25 Aug 2025, Nagesh et al., 2022, Galdran et al., 2021).

Taken together, the literature presents class-balanced sampling strategy not as a monolithic algorithm but as a general method for reshaping training exposure under imbalance. Its strongest formulations explicitly define the balancing unit, the sampling law or quota, the interaction with augmentation or pseudo labeling, and the evaluation budget under which gains are assessed.