---
title: Class-Balanced Sampling Strategy
url: https://www.emergentmind.com/topics/class-balanced-sampling-strategy
type: topic
---

# Class-Balanced Sampling Strategy

Class-balanced sampling strategy denotes a family of data-selection procedures that explicitly control how often classes, class-conditioned instances, or class-bearing structures enter optimization, with the aim of reducing the dominance of frequent classes and improving minority-class learning. In the literature, the strategy appears in long-tailed recognition, source-free domain adaptation, federated learning, active learning, medical image classification, segmentation, 3D object detection, and time-series pre-training, but the central invariant is exposure control: the sampler changes the training distribution before the loss is evaluated, whereas loss reweighting changes the contribution of samples already present in the batch [2105.00127] [2204.02811] [2504.13462] [2605.20405].

## 1. Formal definition and statistical objective

In its most basic form, class-balanced sampling replaces uniform-over-samples training with a distribution that equalizes class frequency. For a dataset with class counts \(n_c\), instance-based sampling induces \(P_{\mathrm{inst}}(c)=n_c/N\), whereas class-balanced sampling uses either inverse-frequency rules or uniform-over-class selection followed by uniform sampling within the chosen class [2109.09850]. In decoupled long-tailed recognition, this is stated explicitly as sampling each class with probability \(\frac{1}{C}\), oversampling tail classes, with uniform sampling within the selected class [2105.00127]. A broader family uses
\[
p(c)=\frac{n_c^q}{\sum_{k=1}^C n_k^q},
\]
where \(q=1\) recovers instance-based sampling, \(q=0\) yields uniform-over-class sampling, and \(q=\frac12\) gives square-root sampling [2109.09850].

A fixed-allocation version appears in balanced stratified sampling. Given \(K\) eligible classes and a target sample size \(n\), the per-class target is
\[
n_c^{\mathrm{target}}=\left\lfloor \frac{n}{K}\right\rfloor,
\]
with classes that cannot meet this quota without oversampling removed until the allocation is feasible [1403.2950]. This formulation makes the balancing objective explicit: the sample is not merely representative of the empirical prior; it is constructed to alter the prior seen by the learner.

The same principle can be expressed in other statistical forms. In federated learning, the aim is not only balanced class counts but balanced exposure in aggregated gradients. Stratify defines a Stratified Label Schedule,
\[
\mathrm{SLS}=\mathrm{Shuffle}\!\left(\bigcup_{l\in\mathcal L} f_l\{l\}\right),
\]
with label selection probability
\[
P(l)=\frac{f_l}{\sum_{l\in\mathcal L} f_l},
\]
so that uniform \(f_l\) yields uniform label exposure [2504.13462]. Fed-CBS instead measures deviation from balance via the Quadratic Class-Imbalance Degree,
\[
QCID(\mathcal M)=\sum_{b=1}^B\left(\frac{\sum_{n\in\mathcal M} q_n\alpha_{(n,b)}}{\sum_{n\in\mathcal M} q_n}-\frac{1}{B}\right)^2,
\]
and samples client subsets whose grouped data distribution is closer to uniform [2209.15245]. These formulations make clear that class-balanced sampling is not a single algorithm but a design principle for constraining the training distribution.

## 2. Canonical constructions

The literature implements class balancing through several recurring constructions: fixed per-class quotas, uniform-over-class batch formation, ranking-based rebalancing, stratified scheduling, and hybrid under/over-sampling. The balancing unit can be an instance, a frame, a slice, a target-domain prototype candidate, a client subset, a detection box, or an implicit cluster of unlabeled patterns.

| Setting | Sampling unit | Representative rule |
|---|---|---|
| Long-tailed recognition | class then instance | \(p(c)=1/C,\; p(i\mid c)=1/n_c\) [2105.00127] |
| SFDA | target category candidates | \(M=\max\{1,\lfloor n_t/(rK)\rfloor\}\) per class [2204.02811] |
| Federated learning | scheduled labels | \(\mathrm{SLS}=\mathrm{Shuffle}(\cup_{l\in\mathcal L} f_l\{l\})\) [2504.13462] |
| Time-series corpus pre-training | grid cell, series, window | \(\mathcal P(W_{n,k,t})=1/(|\mathcal G||\mathcal G_m||\mathcal W_{n,k}|)\) [2505.17871] |

A quota-based design appears in source-free domain adaptation. BMD fixes an equal quota
\[
M=\max\left\{1,\left\lfloor \frac{n_t}{rK}\right\rfloor\right\},
\]
then selects the top-\(M\) most confident target instances for each class from the entire target set, rather than applying local confidence thresholds independently [2204.02811]. This is a global inter-class balanced sampling strategy: every class contributes the same number of candidate instances to prototype construction, even when transfer difficulty varies across categories.

A multiset-duplication design appears in 3D object detection. CBGS constructs per-class frame sets \(S_c\), computes \(T=\sum_c n_c\), fixes a uniform quota \(Q=rT\) with \(r=0.1\), and samples \(Q\) frames with replacement from each \(S_c\), producing an expanded multiset dataset \(D\) whose duplicates are intentional [1908.09492]. In the nuScenes setting, this balanced dataset sampling is complemented by GT-AUG and a balanced grouping head [1908.09492].

A ranking-based design appears in box-level active learning for object detection. Candidate boxes are scored by
\[
a(b)=u(b)\exp\!\left(-\frac{p(\hat l_b)}{\sigma}\right),
\]
where \(u(b)\) is uncertainty and \(p(\hat l_b)\) is the empirical pseudo-label frequency of the predicted class; the top-\(K\) boxes are annotated under a box budget [2508.17849]. No hard class quotas are imposed, but the exponential factor smoothly down-weights boxes from pseudo-label-majority classes.

Hybrid samplers combine undersampling and oversampling rather than enforcing balance through a single probability law. Both iBRF and SMOTE-RUS-NC apply Neighborhood Cleaning Rule, then Random Undersampling, then SMOTE, either as a preprocessing pipeline or inside each bootstrap sample of an ensemble [2403.09867] [2208.09619]. These methods treat class balance as a constrained reconstruction problem: remove noisy or redundant majority points, reduce majority volume, then synthesize minority support until class counts match.

## 3. Supervised learning, long tails, and synthetic mixing

In long-tailed visual recognition, class-balanced sampling is often embedded in a two-stage pipeline. “Breadcrumbs: Adversarial Class-Balanced Sampling for Long-tailed Recognition” defines stage 1 with image-balanced random sampling for representation learning and stage 2 with class-balanced sampling for classifier learning, using uniform-over-class selection [2105.00127]. The paper’s central objection is that ordinary class-balanced sampling over-fits few-shot classes because it repeatedly samples the same images; duplication alone is not adversarial in feature space [2105.00127]. EMANATE and Breadcrumb replace duplicated tail features with aligned feature snapshots collected across epochs, so CBS is retained as the training distribution while the repeated draws become harder and more diverse.

“Class-Balanced Distillation for Long-Tailed Visual Recognition” uses the same basic class-balanced stage-2 prior, \(p_j=1/c\), but changes its role. Instead of freezing the representation and retraining only the classifier, CBD retrains the backbone under class-balanced sampling while distilling feature geometry from an instance-trained teacher [2104.05279]. The resulting objective combines supervised cross-entropy with a cosine-distance feature distillation term, so the class-balanced sampler no longer serves only to debias the classifier; it drives continued representation learning on under-represented classes [2104.05279].

Balanced-MixUp introduces a different construction. One sample is drawn with instance-based sampling, another with class-balanced sampling, and the mixed example is formed with
\[
\lambda\sim \mathrm{Beta}(\alpha,1),\qquad
\hat x=\lambda x_I+(1-\lambda)x_C,\qquad
\hat y=\lambda y_I+(1-\lambda)y_C.
\]
This yields an effective class prior
\[
P_{\mathrm{eff}}(c)=\frac{\alpha}{\alpha+1}\frac{n_c}{N}+\frac{1}{\alpha+1}\frac{1}{C},
\]
which interpolates between the empirical long-tail prior and a uniform-over-class prior [2109.09850]. The method therefore balances exposure without pure duplication, and does so through soft targets.

In tabular imbalance, hybrid samplers express the same objective through bootstrap-level balancing. iBRF includes all minority instances in each bootstrap, fills the remainder with majority samples drawn with replacement, then applies NC, controlled RUS, and SMOTE so that the final per-bootstrap class counts are equal [2403.09867]. “A Novel Hybrid Sampling Framework for Imbalanced Learning” formalizes the same sequencing as SMOTE-RUS-NC and its ensemble version SRN-BRF, with
\[
a_{\mathrm{RUS}}=\frac{N_{\min}}{N_{\mathrm{maj}}^{(\mathrm{after\ RUS})}},\qquad
a_{\mathrm{SMOTE}}=\frac{N_{\min}^{(\mathrm{after\ SMOTE})}}{N_{\mathrm{maj}}^{(\mathrm{after\ RUS})}},
\]
and uses \(a_{\mathrm{SMOTE}}=1\) to match minority to post-RUS majority [2208.09619]. Here class-balanced sampling is inseparable from noise cleaning and support synthesis.

## 4. Prototype, client, and box-level balancing

In source-free domain adaptation, the problem is that target pseudo labels are themselves class-biased. BMD addresses this by selecting, for each target category, an equally sized candidate pool from the entire target set, building prototypes from those balanced pools, then replacing monocentric class summaries with \(S\) intra-class centers and dynamic pseudo labels updated at both epoch and mini-batch scales [2204.02811]. Pseudo labels are assigned by nearest multicentric prototype, and mini-batch dynamics update prototypes by EMA with \(\lambda=0.9999\) [2204.02811]. The class-balanced sampling step is thus upstream of prototype construction and downstream of the model’s current belief state.

In federated learning, class-balanced sampling can target labels or clients. Stratify schedules labels globally through SLS, restricts participation to clients that possess the scheduled labels, and optionally weights client choice by
\[
p_i(l)=\frac{n_i(l)}{N(l)},
\]
while keeping label identities hidden by placeholders and homomorphic encryption [2504.13462]. Fed-CBS takes a client-subset view instead: at each round it selects a subset of available clients whose grouped label distribution minimizes QCID, with theoretical guarantees that the expected QCID is lower than under random sampling [2209.15245]. Both methods treat imbalance as a property of aggregated exposure, not merely of local data silos.

In active learning for classification, the balancing constraint is imposed during acquisition rather than SGD. “Class-Balanced Active Learning for Image Classification” introduces a batch selection objective
\[
\min_z\; \boldsymbol z^{T}(P\odot \log P)\mathbf 1 + \lambda \|\Omega(c)-P^T\boldsymbol z\|_1
\quad\text{s.t.}\quad \boldsymbol z^T\mathbf 1=b,\; z_i\in\{0,1\},
\]
where \(P^Tz\) is the expected class composition of the selected batch and \(\Omega(c)\) is the target class quota for the current cycle [2110.04543]. In box-level active object detection, the same principle is transferred from image acquisition to box acquisition by class-balancing the uncertainty score itself [2508.17849].

These settings indicate that class-balanced sampling is not confined to minibatch formation. It can operate over pseudo-labeled target candidates, encrypted client subsets, active-learning decisions, or detector outputs, provided there is a mechanism for estimating or constraining class exposure.

## 5. Dense prediction, non-standard classes, and pattern strata

In medical image segmentation, class-balanced sampling has been extended from images to episodes. “Disentangling Sampling from Training Budget in Class-Imbalanced CT Body Composition Segmentation” constructs each episode by uniformly choosing \(N_C=2\) foreground classes, then drawing \(N_S=3\) support and \(N_Q=3\) query slices per selected class, with 500 episodes per epoch [2605.20405]. Because classes are sampled uniformly across episodes, rare tissues participate as episode targets as often as frequent tissues. The paper emphasizes that this changes which classes appear in each batch, whereas CE, Dice, or class-weighted losses act only on pixels already present [2605.20405].

An analogous shift appears in time-series corpus construction. In BLAST, “classes” are not human-labeled semantic categories but implicit clusters of time-series patterns induced by statistical descriptors. Each series is characterized by stationarity, trend, seasonality, volatility, scedasticity, memorability, and anomaly rate, embedded into a 2D UMAP space, partitioned into a \(100\times100\) grid, and sampled uniformly over non-empty grid cells and then uniformly within a cell [2505.17871]. The resulting rule,
\[
\mathcal P(W_{n,k,t})=\frac{1}{|\mathcal G|\cdot|\mathcal G_m|\cdot|\mathcal W_{n,k}|},
\]
treats grid cells as classes and makes the balanced sampling target pattern coverage rather than label coverage [2505.17871]. Grid mixup then interpolates between sampled cells via Dirichlet weights [2505.17871].

This broadening of the balancing unit also appears in 3D detection. CBGS samples frames containing rare object classes more frequently, while SA-CBGS keeps the balanced grouping and duplicate sampling scheme but augments the detector with point-level foreground segmentation and center estimation heads to recover localization precision lost in coarse feature maps [2205.12519]. The sampling strategy remains class-balanced at the data-pipeline level even though the optimization target is dense detection.

A plausible implication is that class-balanced sampling is best understood as a constraint on exposure over a chosen partition of the data space. In some applications that partition is the label set; in others it is a grid cell, a tissue-conditioned episode, or a class-conditioned frame index. The operational question is not whether one is sampling “classes” in the narrow semantic sense, but whether the training process equalizes access to underrepresented regions that matter for the task.

## 6. Empirical behavior, limitations, and methodological cautions

Empirically, class-balanced sampling often improves minority performance and can improve overall accuracy, but the effect is strongly conditioned by setting and implementation. In SFDA, BMD improved SHOT on Office-31 from \(88.6\%\) to \(89.4\%\), on Office-Home from \(71.5\%\) to \(72.5\%\), on VisDA-C from \(82.9\%\) to \(85.8\%\), and on PointDA-10 from \(53.1\%\) to \(57.0\%\); on VisDA-C’s hardest class, truck, SHOT rose from \(58.2\%\) to \(70.8\%\), while the coefficient of variation decreased from \(0.155\) to \(0.118\) [2204.02811]. In long-tailed recognition, Breadcrumb improved ImageNet-LT ResNet-10 from \(41.4\) to \(44.0\) overall and from \(21.5\) to \(26.4\) on few-shot classes relative to Decoupling(cRT) [2105.00127]. In FL, Stratify reached \(90.60\%\) on CIFAR-10 \(\#C=1\) at \(E8\), versus FedAvg \(23.45\%\) at \(E44\), and Fed-CBS reduced QCID on CIFAR-10 with Dirichlet \(\alpha=0.1\) from \(8.20\times 10^{-2}\) under random sampling to \(0.62\times 10^{-2}\) while also reducing communication rounds [2504.13462] [2209.15245]. In low-data CT segmentation, episodic sampling achieved mean Dice \(0.787\) versus \(0.758\) and \(0.762\) for random and weighted sampling, although much of that difference was shown to be attributable to a 12-fold difference in training iterations [2605.20405].

The major caution is that balancing can alter the effective training prior and thereby introduce new pathologies. Balanced stratified sampling may help decision boundaries yet hurt calibrated probabilities if deployment priors differ from the balanced training set [1403.2950]. CBS in long-tailed recognition can over-fit few-shot classes because the same tail images are repeatedly sampled [2105.00127]. In BMD, severe label shift or negligible model probability for a truly present class can make the top-\(M\) candidate pool low quality, and if initial pseudo labels are extremely noisy, multicentric clustering can fragment clusters [2204.02811]. In FL, weighted client selection slightly relaxes privacy relative to uniform client selection, and CKKS-based label discovery overhead increases with the number of classes [2504.13462].

A second caution is methodological. Apparent improvements from a sampler may partly reflect changed training budgets rather than changed exposure geometry. The CT segmentation study showed that random and weighted sampling received far fewer iterations per epoch than episodic sampling in the low-data regime, and that under matched iteration budgets the performance gap shrank markedly, while episodic sampling still delayed overfitting [2605.20405]. This suggests that class-balanced sampling should be evaluated with iteration-aware schedules rather than epoch counts alone.

A common misconception is that class-balanced sampling is interchangeable with loss reweighting. The surveyed literature does not support that equivalence. Loss-level remedies reweight pixels, anchors, or logits already present in the batch; class-balanced sampling changes which samples, labels, boxes, frames, windows, or clients contribute to the batch in the first place [2605.20405] [2505.17871]. Conversely, sampling alone is often insufficient: many successful systems combine balancing with multicentric prototypes and dynamic SCE in SFDA, feature distillation in long-tailed recognition, soft pseudo labels in active detection, GT-AUG or auxiliary structure supervision in detection, or synthetic vicinal mixing in medical image classification [2204.02811] [2104.05279] [2508.17849] [2205.12519] [2109.09850].

Taken together, the literature presents class-balanced sampling strategy not as a monolithic algorithm but as a general method for reshaping training exposure under imbalance. Its strongest formulations explicitly define the balancing unit, the sampling law or quota, the interaction with augmentation or pseudo labeling, and the evaluation budget under which gains are assessed.

Source: https://www.emergentmind.com/topics/class-balanced-sampling-strategy