CBRS: Class-Balanced Reservoir Sampling

• Class-Balanced Reservoir Sampling (CBRS) is a replay-buffer algorithm that balances memory by preventing majority-class dominance in continual learning.
• It uses a two-stage process with initial full memory collection followed by class-aware evictions and within-class reservoir replacement to maintain class balance.
• Empirical results on datasets like MNIST and Fashion-MNIST show that CBRS significantly improves performance over standard reservoir sampling under class imbalance.

Class-Balanced Reservoir Sampling (CBRS) is a replay-buffer population algorithm for rehearsal-based continual learning under imbalanced data streams. Its purpose is to decide which instances are stored for future replay while keeping the memory as class balanced as possible, so that minority classes are not crowded out by majority classes in a fixed-capacity buffer (Findik et al., 2022). In the continual-learning literature summarized here, CBRS is primarily associated with class-incremental settings in which streams arrive sequentially, the learner has limited replay memory, and training interleaves incoming minibatches with replayed memory minibatches. Subsequent work has treated CBRS both as a baseline for class-imbalanced rehearsal and as a building block for extensions that account for intra-class diversity, multi-label imbalance, distributed streaming constraints, and temporally shifting federated data distributions (Findik et al., 2022).

1. Problem domain and conceptual role

CBRS arises in continual learning, where a model accumulates knowledge from a sequence of learning experiences and must avoid catastrophic forgetting as additional data are seen. In the class-incremental setting emphasized by later summaries of CBRS, each stream contains examples from one class, streams arrive sequentially, and replay is used to mitigate performance degradation on previously encountered classes (Findik et al., 2022). The practical complication is that class imbalance interacts with forgetting: if the replay memory is populated by standard reservoir sampling or other class-agnostic strategies, majority classes can dominate storage and replay, which worsens retention for minority classes.

Within this setting, CBRS addresses inter-class imbalance rather than general representativeness. Its objective is not to preserve a uniform sample of the entire stream, but to keep the class histogram of memory approximately balanced. This distinction is central. Standard reservoir sampling is faithful to the arrival distribution over examples, whereas CBRS modifies admission and eviction so that underrepresented classes can grow at the expense of overrepresented ones (Findik et al., 2022).

A recurrent misconception is to treat CBRS as synonymous with generic reservoir sampling. The later literature distinguishes them sharply: reservoir sampling is example-uniform, while CBRS is class-aware and explicitly designed to prevent replay memory from being dominated by majority classes. That difference is what makes CBRS relevant in imbalanced continual-learning streams and in later federated variants under temporal drift (Findik et al., 2022).

2. Canonical algorithmic structure

A concise formal summary of CBRS is provided in "D-CBRS: Accounting For Intra-Class Diversity in Continual Learning" (Findik et al., 2022). The notation is minimal. Let $m$ denote total memory size, and let a filled memory mean that all $m$ slots are occupied. Once memory is filled, CBRS labels the largest class in memory, measured by number of stored instances, as a full class, and "once a class is labeled as full, it remains so" (Findik et al., 2022).

The algorithm has two stages.

Before memory is full, all incoming examples are stored. After memory is full, CBRS applies class-aware replacement. For an incoming sample $(x_i, y_i)$, the algorithm checks whether class $y_i$ is a full class.

If the incoming sample belongs to a full class, CBRS performs within-class reservoir-style replacement. The stated rule is

$$\Pr(\text{replace with }(x_i,y_i))=\frac{m_c}{n_c},$$

where $m_c$ is the number of instances of class $y_i$ currently in memory and $n_c$ is the total number of instances of class $y_i$ received so far. When replacement occurs, the new sample replaces one existing memory sample from the same class $y_i$ (Findik et al., 2022).

If the incoming sample does not belong to a full class, CBRS inserts it by removing randomly an instance from a full class. Operationally, this means that non-full classes are allowed to increase their memory occupancy, whereas overrepresented classes are the source of evictions. The resulting procedure is reservoir-style in spirit, but the balancing objective is imposed through the full-class logic rather than through global example-uniformity (Findik et al., 2022).

In this form, CBRS has no standalone pseudocode in the summarized source beyond the quantities $m$0, the definition of full classes, the two-stage process, and the replacement probability $m$1. The same summary notes that CBRS "follows a similar strategy as that of RS while keeping classes stored in the memory balanced" (Findik et al., 2022).

3. Assumptions, invariants, and recognized limitations

The central statistical assumption attributed to CBRS in later work is that, within a class, all examples are treated as equally informative or representative. CBRS balances memory across classes, but it does not account for intra-class diversity; it does not model substructure inside a class, and it does not distinguish among examples of the same label on the basis of cluster membership, rarity, or mode coverage (Findik et al., 2022).

That limitation becomes consequential when a class is multimodal or internally heterogeneous. In such cases, a memory can be balanced at the class level yet still underrepresent rare but informative submodes within a class. The diagnosis given in later work is explicit: CBRS preserves class counts, not within-class mode balance, so rare clusters may disappear even when the class itself remains well represented (Findik et al., 2022).

This distinction also clarifies what CBRS does preserve. It retains a two-stage memory-management logic, uses a reservoir-style acceptance probability, and protects underrepresented classes from being wiped out by majority classes. A plausible implication is that CBRS is well aligned with settings where the dominant failure mode is inter-class imbalance, but less aligned with settings where forgetting is driven by both class imbalance and strong within-class multimodality. That interpretation is consistent with subsequent experimental comparisons, though it is not presented as a formal theorem in the underlying source (Findik et al., 2022).

A second misconception is to interpret CBRS as a generic class-fairness guarantee. The summaries do not attribute to it any theorem of exact equal class occupancy, any explicit per-class quota formula, or any guarantee about rare within-class modes. Its balancing is operational and buffer-centric: minority classes are protected because full classes are preferentially thinned, not because a fixed quota is imposed analytically (Findik et al., 2022).

4. Empirical behavior in continual learning

The most detailed empirical evidence involving CBRS in the provided sources comes from comparisons against Reservoir Sampling (RS) and D-CBRS on MNIST and Fashion-MNIST under class-incremental replay with a fixed memory budget $m$2, an MLP with 2 hidden layers of 250 neurons each, ReLU activations, Adam with learning rate $m$3, cross-entropy loss, batch size $m$4, and $m$5 replay steps per incoming batch (Findik et al., 2022). Class imbalance is simulated using retention factors

$m$6

with one factor sampled without replacement for each class.

In the base case where only class imbalance is present, CBRS substantially improves over Reservoir Sampling. On MNIST, Reservoir yields $m$7, CBRS yields $m$8, and D-CBRS yields $m$9. On Fashion-MNIST, Reservoir yields $(x_i, y_i)$0, CBRS yields $(x_i, y_i)$1, and D-CBRS yields $(x_i, y_i)$2 (Findik et al., 2022). These results support the interpretation that CBRS is effective when the primary issue is inter-class imbalance.

The same study then constructs intra-class diversity by merging original classes into new classes, creating internally multimodal labels. In the omniscient setting, where the extended method receives true composing labels and therefore acts as a diversity oracle, CBRS degrades relative to D-CBRS: on MNIST, CBRS reaches $(x_i, y_i)$3 while D-CBRS reaches $(x_i, y_i)$4; on Fashion-MNIST, CBRS reaches $(x_i, y_i)$5 while D-CBRS reaches $(x_i, y_i)$6 (Findik et al., 2022). In the more realistic K-means setting, D-CBRS improves over CBRS in all reported diverse settings, including 5 merged classes and 3 merged classes.

These comparisons are important for interpreting CBRS correctly. They do not imply that CBRS is ineffective; rather, they delimit its scope. CBRS is strong on class imbalance alone, but later evidence indicates that it can underperform when memory management must also preserve within-class cluster structure. The empirical pattern therefore supports a two-part characterization: CBRS is a class-balancing replay method, not a general diversity-preserving memory optimizer (Findik et al., 2022).

5. Extensions, generalizations, and alternative balancing schemes

The most direct extension of CBRS in the provided literature is Diverse-CBRS (D-CBRS). D-CBRS "builds upon CBRS" by retaining class balancing while changing eviction so that memory also preserves diverse modes within each class. Its mechanism is to cluster instances within each class, track counts per cluster, and evict preferentially from the largest cluster inside a full class. When an incoming sample belongs to a non-largest cluster, it is always inserted by evicting from the largest cluster; when it belongs to the largest cluster, the reservoir-style probability $(x_i, y_i)$7 is applied at the cluster level rather than the class level (Findik et al., 2022). This makes D-CBRS best understood as CBRS plus cluster-aware eviction.

A different generalization appears in Partitioning Reservoir Sampling (PRS) for task-free continual learning on imbalanced multi-label streams. PRS is described as a multi-label, quota-driven, online reservoir variant that explicitly balances memory occupancy across labels using running label frequencies and specialized sample-in and sample-out rules (Kim et al., 2020). Its target partition ratio is

$(x_i, y_i)$8

with $(x_i, y_i)$9 controlling the tradeoff between equalized and frequency-proportional allocation. PRS is CBRS-like in spirit because it sets target class or label shares, promotes minority-labeled examples, and removes examples from over-occupied labels, but it is not standard single-label CBRS because one memory item can count toward multiple labels and balancing is over label incidences rather than disjoint class buckets (Kim et al., 2020).

An alternative direction is Prior-free Balanced Replay (PBR), which is explicitly framed as a prior-free alternative to explicit class-balanced replay under long-tailed continual learning. PBR uses uncertainty-guided reservoir sampling, prioritizing samples with high mutual dependence between model and sample rather than maintaining class counts or quotas (Liu et al., 2024). The paper states that this implicitly preferentially stores minority samples instead of directly adjusting label distribution. Relative to CBRS, PBR is therefore an implicit, proxy-based balancing mechanism rather than an explicit class-count correction scheme (Liu et al., 2024).

Method	Setting	Balancing mechanism
CBRS	Single-label rehearsal under class imbalance	Class-aware memory balancing via full-class eviction and within-class reservoir replacement
D-CBRS	Class-imbalanced continual learning with intra-class diversity	CBRS plus clustering within each class and eviction from the largest cluster
PRS	Task-free multi-label continual learning with long-tailed labels	Soft quota-based balancing using running frequencies and label-aware sample-in/sample-out
PBR	Long-tailed continual learning without preknown class priors	Uncertainty-guided reservoir-style replay with implicit minority prioritization

Taken together, these methods define a family rather than a single canonical doctrine. CBRS is the explicit class-balancing member of that family. D-CBRS adds within-class mode preservation, PRS generalizes balancing to multi-label incidence structure, and PBR replaces explicit label balancing with uncertainty as a proxy (Findik et al., 2022).

6. Distributed and federated realizations

Although "Parallel Weighted Random Sampling" does not discuss class balance, stratification, or per-class reservoirs explicitly, it provides an exact distributed weighted reservoir substrate that can plausibly be adapted to CBRS (Hübschle-Schneider et al., 2019). The paper develops a fully distributed mini-batch streaming model with exponential-key weighted sampling without replacement, threshold maintenance

$y_i$0

skip-distance generation

$y_i$1

and distributed selection of the global $y_i$2-th smallest key across processing elements. A plausible implication is that CBRS can be implemented in a distributed system either by maintaining one reservoir per class or by using class-adjusted effective weights in a global reservoir, although the source explicitly states that this would be an adaptation rather than a direct CBRS algorithm (Hübschle-Schneider et al., 2019).

A concrete modern CBRS instantiation appears in FlashbackCL, where CBRS is used as the client-side replay-buffer maintenance mechanism in temporally shifting federated learning (Ojewale et al., 2 Jun 2026). Each client $y_i$3 maintains a replay buffer $y_i$4 with capacity

$y_i$5

where $y_i$6 is device memory budget, $y_i$7 is participation frequency, and $y_i$8 is a system-wide budget parameter. When a new sample $y_i$9 arrives and the buffer is full, the majority class is computed as

$$\Pr(\text{replace with }(x_i,y_i))=\frac{m_c}{n_c},$$0

If $$\Pr(\text{replace with }(x_i,y_i))=\frac{m_c}{n_c},$$1, a random item of class $$\Pr(\text{replace with }(x_i,y_i))=\frac{m_c}{n_c},$$2 is evicted and $$\Pr(\text{replace with }(x_i,y_i))=\frac{m_c}{n_c},$$3 is admitted. If $$\Pr(\text{replace with }(x_i,y_i))=\frac{m_c}{n_c},$$4, within-class reservoir replacement is used with probability

$$\Pr(\text{replace with }(x_i,y_i))=\frac{m_c}{n_c},$$5

The stated invariant is that the buffer’s per-class histogram is approximately uniform over the classes the client has ever seen, regardless of arrival order (Ojewale et al., 2 Jun 2026).

In FlashbackCL, replay batches are mixed with current-data batches using

$$\Pr(\text{replace with }(x_i,y_i))=\frac{m_c}{n_c},$$6

The paper reports that FlashbackCL achieves relative improvements over Flashback of +9.0% under abrupt shift, +10.0% under gradual shift, and +6.9% under cyclic shift on CIFAR-10 with 50 clients, while reducing temporal forgetting by 42% in the abrupt case and 68% in the gradual case; a 5-variant ablation identifies CBRS replay as the critical component (Ojewale et al., 2 Jun 2026). This federated formulation is not identical to the class-incremental CBRS summary used in D-CBRS, but it preserves the same essential design principle: majority-class eviction combined with within-class reservoir replacement.

The distributed and federated literature therefore situates CBRS as more than a small continual-learning heuristic. It is a compact local policy that can be embedded in larger systems: as a client-side replay controller under temporal drift, or, plausibly, as a class-aware layer atop distributed key-based reservoir infrastructure. What remains constant across these settings is the core objective of preventing memory from collapsing toward dominant classes simply because they are frequent, recent, or locally overrepresented.