Dual-Balance Collaborative Experts (DCE)

• Dual-Balance Collaborative Experts (DCE) is a framework for imbalanced Domain-Incremental Learning that integrates frequency-aware experts and a dynamic selector to address intra-domain class imbalance and cross-domain distribution shifts.
• The system operates in two stages per domain, first training specialized experts with distinct frequency biases and then synthesizing balanced Gaussian pseudo-features to train an adaptive fusion selector.
• Designed for PTM-based, exemplar-free learning, DCE preserves old many-shot knowledge while facilitating transfer for few-shot classes through a structured balance of specialization and collaboration.

Dual-Balance Collaborative Experts (DCE) is a framework for imbalanced Domain-Incremental Learning (DIL) that addresses two coupled difficulties: intra-domain class imbalance and cross-domain class distribution shifts. In the formulation introduced in "Addressing Imbalanced Domain-Incremental Learning through Dual-Balance Collaborative Experts" (Li et al., 9 Jul 2025), DCE combines a frequency-aware expert group with a dynamic expert selector trained on balanced Gaussian-sampled pseudo-features derived from historical class statistics. The framework is explicitly designed for PTM-based, exemplar-free DIL, where the label space is fixed across domains, task boundaries are known during training, old raw data are unavailable after a domain is finished, and preservation of old knowledge must be reconciled with the possibility that later domains can improve old few-shot classes (Li et al., 9 Jul 2025).

1. Problem setting and the meaning of “dual balance”

DCE is defined in a DIL regime with a sequence of domains

$$\left\{\mathcal{D}^{1}, \mathcal{D}^{2}, \cdots, \mathcal{D}^{B}\right\},$$

where each domain is

$$\mathcal{D}^{b} = \left\{\mathcal{X}_b, \mathcal{Y}_b\right\}.$$

The input-label pairs are written as $\mathcal{X}_b = \{\mathbf{x}_i\}_{i=1}^{n_b}$ and $\mathcal{Y}_b = \{y_i\}_{i=1}^{n_b}$, with a fixed label space across all domains: $$y_i \in \mathcal{Y}, \qquad \mathcal{Y}\ \text{shared across all tasks.}$$ The paper decomposes the joint distribution as

$p_b(x,y)=p_b(y)p_b(x|y),$

and emphasizes that domain shift includes both concept drift through $p_b(x|y)$ and class distribution shift through $p_b(y)$ (Li et al., 9 Jul 2025).

Two imbalance phenomena define the target problem. The first is intra-domain class imbalance, where sample counts $n_b^c$ vary strongly across classes within a domain. The second is cross-domain class distribution shift, where the class-frequency profile changes across domains. The paper’s central claim is that these two effects create a sharper stability–plasticity tension than ordinary DIL: old many-shot classes mainly require protection from forgetting, whereas old few-shot classes may benefit from transfer from later domains (Li et al., 9 Jul 2025).

The phrase dual-balance refers to two distinct balancing objectives. The first is intra-domain frequency balance, implemented through experts with different frequency biases. The second is cross-domain knowledge balance, implemented through a selector that adaptively fuses experts so that old many-shot knowledge can be preserved while old few-shot classes can improve. This makes DCE neither a standard MoE in the generic sparse-routing sense nor a standard replay-based continual learner. It is a framework that separates specialization from fusion in an exemplar-free DIL setting (Li et al., 9 Jul 2025).

A common misconception is that DCE is simply another prompt-based continual learner. The paper instead positions it as a method in which the pre-trained Vision Transformer backbone is adapted early, then stabilized, while domain-specific expertise and cross-domain fusion are handled by separate modules. This suggests that DCE’s defining feature is not prompt tuning alone, but the joint use of frequency-aware experts and statistic-based selector training (Li et al., 9 Jul 2025).

2. System architecture and domain-wise workflow

DCE uses a pre-trained Vision Transformer encoder

$\theta(x): \mathbb{R}^D \to \mathbb{R}^d$

with frozen backbone parameters $$\mathcal{D}^{b} = \left\{\mathcal{X}_b, \mathcal{Y}_b\right\}.$$0, plus a small set of adaptable parameters $$\mathcal{D}^{b} = \left\{\mathcal{X}_b, \mathcal{Y}_b\right\}.$$1 such as prompts. The generic PTM-based DIL objective is written as

$$\mathcal{D}^{b} = \left\{\mathcal{X}_b, \mathcal{Y}_b\right\}.$$2

where $$\mathcal{D}^{b} = \left\{\mathcal{X}_b, \mathcal{Y}_b\right\}.$$3 is the classifier and $$\mathcal{D}^{b} = \left\{\mathcal{X}_b, \mathcal{Y}_b\right\}.$$4 regularizes the adaptable parameters (Li et al., 9 Jul 2025).

The framework proceeds in two stages per domain. In Stage 1, DCE trains a frequency-aware expert group on the current domain. In Stage 2, it computes historical feature statistics, synthesizes balanced pseudo-features by Gaussian sampling, and trains a dynamic expert selector on these pseudo-features (Li et al., 9 Jul 2025).

A crucial design choice is that Visual Prompt Tuning is used during the first task to learn prompts that adapt the pre-trained ViT into a DIL-aware representation space. After that, the encoder and prompts are frozen, and later tasks only add new experts and update the selector. The paper presents this as the condition that makes historical feature statistics meaningful across tasks. This suggests that DCE deliberately trades some representational plasticity for a stable feature space in which statistical replay remains coherent over time (Li et al., 9 Jul 2025).

The per-domain procedure can be summarized directly from the algorithmic description:

Stage	Operation	Data source
Stage 1	Train new domain experts with $$\mathcal{D}^{b} = \left\{\mathcal{X}_b, \mathcal{Y}_b\right\}.$$5	Real current-domain data
Stage 2	Compute $$\mathcal{D}^{b} = \left\{\mathcal{X}_b, \mathcal{Y}_b\right\}.$$6, update $$\mathcal{D}^{b} = \left\{\mathcal{X}_b, \mathcal{Y}_b\right\}.$$7, synthesize $$\mathcal{D}^{b} = \left\{\mathcal{X}_b, \mathcal{Y}_b\right\}.$$8, train selector with $$\mathcal{D}^{b} = \left\{\mathcal{X}_b, \mathcal{Y}_b\right\}.$$9	Historical statistics and pseudo-features

At inference time, the pipeline is likewise explicit: compute $\mathcal{X}_b = \{\mathbf{x}_i\}_{i=1}^{n_b}$0, feed it to all experts, obtain selector weights, and fuse expert outputs by weighted summation. The selector is therefore globally shared across all seen domains, but routing is instance-adaptive rather than task-ID based (Li et al., 9 Jul 2025).

3. Frequency-aware expert group

The expert group contains three experts per domain. For a new domain $\mathcal{X}_b = \{\mathbf{x}_i\}_{i=1}^{n_b}$1, the experts are denoted $\mathcal{X}_b = \{\mathbf{x}_i\}_{i=1}^{n_b}$2, $\mathcal{X}_b = \{\mathbf{x}_i\}_{i=1}^{n_b}$3, and $\mathcal{X}_b = \{\mathbf{x}_i\}_{i=1}^{n_b}$4, corresponding to a many-shot-biased expert, a balanced expert, and a few-shot-biased expert. All experts receive the same current-domain data, but each is optimized with a different loss encoding a different frequency prior (Li et al., 9 Jul 2025).

Each expert consists of an MLP plus a dedicated classifier. The appendix specifies a three-layer MLP with widths

$\mathcal{X}_b = \{\mathbf{x}_i\}_{i=1}^{n_b}$5

The shared encoder provides the feature $\mathcal{X}_b = \{\mathbf{x}_i\}_{i=1}^{n_b}$6, and the three experts produce logits

$\mathcal{X}_b = \{\mathbf{x}_i\}_{i=1}^{n_b}$7

The first expert uses standard cross-entropy: $\mathcal{X}_b = \{\mathbf{x}_i\}_{i=1}^{n_b}$8

The second expert uses balanced softmax: $\mathcal{X}_b = \{\mathbf{x}_i\}_{i=1}^{n_b}$9

The third expert uses the inverse distribution loss. First define

$\mathcal{Y}_b = \{y_i\}_{i=1}^{n_b}$0

Then

$\mathcal{Y}_b = \{y_i\}_{i=1}^{n_b}$1

which the paper simplifies to

$\mathcal{Y}_b = \{y_i\}_{i=1}^{n_b}$2

The overall expert-training loss is

$\mathcal{Y}_b = \{y_i\}_{i=1}^{n_b}$3

The intended effect is a spectrum of frequency biases: the cross-entropy expert is naturally biased toward frequent classes; balanced softmax reduces bias relative to the empirical class prior; the reverse expert imposes a stronger complementary bias toward classes disadvantaged under standard empirical risk minimization. The paper reports that among single experts, $\mathcal{Y}_b = \{y_i\}_{i=1}^{n_b}$4 helps many-shot most, $\mathcal{Y}_b = \{y_i\}_{i=1}^{n_b}$5 helps few-shot most, and $\mathcal{Y}_b = \{y_i\}_{i=1}^{n_b}$6 is the most balanced. This is the first of the two balances in DCE: the model does not try to make one classifier uniformly optimal across all frequency regimes, but instead creates specialized experts whose collaboration can later be learned (Li et al., 9 Jul 2025).

For evaluation and dataset characterization, the paper uses explicit frequency thresholds. On Office-Home, classes are categorized using thresholds of 20 and 60 training samples; on DomainNet, the thresholds are 20 and 100. These thresholds are used for many/medium/few-shot analysis rather than for routing samples to experts during training (Li et al., 9 Jul 2025).

4. Dynamic expert selector and balanced Gaussian pseudo-replay

After experts accumulate over domains, DCE introduces a dynamic expert selector to combine them. At task $\mathcal{Y}_b = \{y_i\}_{i=1}^{n_b}$7, the selector is an MLP

$\mathcal{Y}_b = \{y_i\}_{i=1}^{n_b}$8

because there are $\mathcal{Y}_b = \{y_i\}_{i=1}^{n_b}$9 experts after $$y_i \in \mathcal{Y}, \qquad \mathcal{Y}\ \text{shared across all tasks.}$$0 tasks. Given an input feature, the selector outputs a weight vector

$$y_i \in \mathcal{Y}, \qquad \mathcal{Y}\ \text{shared across all tasks.}$$1

which measures the importance of each expert for that sample (Li et al., 9 Jul 2025).

The selector is trained after expert training at every task. Its loss is

$$y_i \in \mathcal{Y}, \qquad \mathcal{Y}\ \text{shared across all tasks.}$$2

Thus the effective logits are formed by the weighted mixture

$$y_i \in \mathcal{Y}, \qquad \mathcal{Y}\ \text{shared across all tasks.}$$3

The paper describes this as a probabilistic routing mechanism, although it does not explicitly write a softmax normalization over the selector outputs (Li et al., 9 Jul 2025).

The selector cannot use old raw data because the framework is exemplar-free. Instead, DCE stores historical feature statistics

$$y_i \in \mathcal{Y}, \qquad \mathcal{Y}\ \text{shared across all tasks.}$$4

where $$y_i \in \mathcal{Y}, \qquad \mathcal{Y}\ \text{shared across all tasks.}$$5 and $$y_i \in \mathcal{Y}, \qquad \mathcal{Y}\ \text{shared across all tasks.}$$6 are the empirical mean and covariance for class $$y_i \in \mathcal{Y}, \qquad \mathcal{Y}\ \text{shared across all tasks.}$$7 in domain $$y_i \in \mathcal{Y}, \qquad \mathcal{Y}\ \text{shared across all tasks.}$$8, computed in the frozen feature space. The global repository is

$$y_i \in \mathcal{Y}, \qquad \mathcal{Y}\ \text{shared across all tasks.}$$9

Pseudo-features are synthesized by assuming a class-conditional Gaussian: $p_b(x,y)=p_b(y)p_b(x|y),$0 The synthetic dataset is

$p_b(x,y)=p_b(y)p_b(x|y),$1

The defining balancing rule is that the same number $p_b(x,y)=p_b(y)p_b(x|y),$2 of pseudo-features is generated for every domain-class pair. This is why the paper calls the procedure balanced Gaussian sampling: replay weight is equalized across domain-class cells regardless of the original sample counts (Li et al., 9 Jul 2025).

This second balance is central to DCE’s interpretation of imbalanced DIL. The selector is not trained on a replay distribution that reproduces historical long-tail bias; it is trained on a deliberately balanced pseudo-distribution. The paper’s argument is that this allows the selector to preserve old many-shot classes without letting them dominate, while still giving old few-shot classes enough replay mass to learn when later experts are beneficial (Li et al., 9 Jul 2025).

A further technical detail is covariance estimation. Because few-shot classes yield unstable covariance estimates, the paper uses Oracle Approximating Shrinkage (OAS): $p_b(x,y)=p_b(y)p_b(x|y),$3 with

$p_b(x,y)=p_b(y)p_b(x|y),$4

$p_b(x,y)=p_b(y)p_b(x|y),$5

and

$p_b(x,y)=p_b(y)p_b(x|y),$6

To reduce storage, the implementation averages class-specific covariances within each domain to obtain a shared domain-level covariance matrix, and OAS is only computed when a class has at least $p_b(x,y)=p_b(y)p_b(x|y),$7 samples in a domain (Li et al., 9 Jul 2025).

5. Optimization, inference behavior, and retention–transfer dynamics

The optimization schedule is sequential rather than joint. For each domain $p_b(x,y)=p_b(y)p_b(x|y),$8, DCE first optimizes the expert group on real current-domain data, then optimizes the selector on synthetic pseudo-features. If written compactly, the per-domain objective is effectively

$p_b(x,y)=p_b(y)p_b(x|y),$9

but the paper is explicit that these are minimized in two stages rather than a single pass (Li et al., 9 Jul 2025).

The first domain is special. When $p_b(x|y)$0, the prompt parameters $p_b(x|y)$1 and the first three experts are jointly trained. When $p_b(x|y)$2, $p_b(x|y)$3 and $p_b(x|y)$4 are frozen, and only the new domain’s three experts are trained. After statistics are computed and pseudo-features generated, the selector is trained over the full accumulated expert pool (Li et al., 9 Jul 2025).

This yields a specific preservation mechanism. DCE does not use real-data replay, does not use distillation, and does not rely on parameter-importance regularizers such as EWC or SI. Preservation instead comes from three structural choices: freezing the backbone and prompts after task 1, storing domain-specific knowledge in separate experts, and training a single selector on balanced pseudo-replay (Li et al., 9 Jul 2025).

The paper interprets the resulting behavior through the distinction between old many-shot and old few-shot classes. Shared-space methods can improve old few-shot classes but tend to forget old many-shot classes more severely. Domain-specific methods retain old many-shot classes better but do not improve old few-shot classes using future domains. DCE is designed to occupy the middle ground by allowing instance-dependent collaboration across all experts while preserving stable specialized experts. This suggests that the selector’s role is not merely routing, but routing under an explicit retention–transfer trade-off (Li et al., 9 Jul 2025).

The paper formalizes this analysis with Class Performance Drift (CPD): $p_b(x|y)$5 where $p_b(x|y)$6 is class $p_b(x|y)$7’s accuracy immediately after training domain $p_b(x|y)$8, and $p_b(x|y)$9 is the accuracy after the final domain. Positive CPD indicates degradation; negative CPD indicates improvement over time. On DomainNet, the paper reports that shared prompt methods show larger drift on many/medium-shot classes but smaller drift on few-shot classes, domain-specific prompt methods show smaller drift on many/medium-shot classes but worse few-shot improvement, and DCE “maintains moderate CPD across all class types” (Li et al., 9 Jul 2025). A plausible implication is that DCE’s dual-balance design is most valuable precisely when forgetting and positive backward transfer should not be treated uniformly across classes.

6. Empirical results, ablations, related context, and limitations

DCE is evaluated on Office-Home, DomainNet, CORe50, and CDDB-Hard, using five randomized task/domain orders. The main metrics are final accuracy $p_b(y)$0 and

$p_b(y)$1

the average accuracy across incremental stages. For Office-Home and DomainNet, the paper also reports $p_b(y)$2, $p_b(y)$3, and $p_b(y)$4 (Li et al., 9 Jul 2025).

The main benchmark results reported in the paper are summarized below.

Dataset	$p_b(y)$5	$p_b(y)$6	Additional breakdown
Office-Home	84.6	84.4	$p_b(y)$7 for many/med/few
DomainNet	64.3	63.5	$p_b(y)$8 for many/med/few
CORe50	80.1	84.8	Not reported by frequency group
CDDB-Hard	74.6	71.8	Not reported by frequency group

On Office-Home, DCE improves over the strongest competing $p_b(y)$9 baseline by about +1.2 and over the strongest $n_b^c$0 baseline by about +0.9. On DomainNet, the gains are about +5.3 on $n_b^c$1 and +5.6 on $n_b^c$2, which is the clearest numerical win reported in the paper. On CORe50 and CDDB-Hard, the reported margins are likewise substantial (Li et al., 9 Jul 2025).

The ablation on the number of experts is particularly important. The paper compares one expert with $n_b^c$3, two experts with $n_b^c$4, three experts with $n_b^c$5, and four experts with an additional more extreme loss

$n_b^c$6

The trend reported is that increasing expert diversity improves performance, especially on medium-shot and few-shot classes, but gains beyond three experts are marginal. This is the direct empirical basis for the three-expert design (Li et al., 9 Jul 2025).

DCE also has a clear computational profile. After the first task, the encoder is frozen, and only expert heads and the selector are trained. The appendix states that DCE uses only one encoder forward pass in both training and inference, unlike prompt-selection baselines that require two encoder forward passes. The implementation reports SGD, batch size 128, cosine decay, 20 or 30 epochs for stage 1 depending on dataset, and 10 epochs for stage 2; the main text says learning rate $n_b^c$7, while the appendix gives an initial learning rate $n_b^c$8, a discrepancy the paper itself leaves to be noted (Li et al., 9 Jul 2025).

The framework is exemplar-free in raw-data storage, but it is not constant-memory. Parameter/storage growth is linear in the number of tasks because DCE adds three experts per task and enlarges the selector output dimension accordingly. The paper quantifies the per-task stored parameters as

$n_b^c$9

with selector size

$\theta(x): \mathbb{R}^D \to \mathbb{R}^d$0

A common misunderstanding is therefore that exemplar-free means negligible memory growth; DCE is exemplar-free in raw samples, but not in parameters or statistics (Li et al., 9 Jul 2025).

In the broader collaborative-expert literature, DCE has clear conceptual relatives but a distinct problem target. "Use What You Have: Video Retrieval Using Representations From Collaborative Experts" (Liu et al., 2019) establishes collaborative experts as a way to combine multiple pretrained modality-specific streams for text–video retrieval, while "Enhancing Molecular Property Prediction via Mixture of Collaborative Experts" (Yao et al., 2023) explicitly addresses expert homogeneity and decision dominance through expert-specific projection and expert-specific loss. DCE differs in that its two balances are defined by frequency imbalance within domains and retention–transfer trade-offs across domains, rather than by modality sparsity or molecular-task collaboration. This suggests that DCE belongs to the same general expert-collaboration lineage, but its novelty lies in how collaboration is tied to long-tail continual learning rather than generic expert fusion (Liu et al., 2019, Yao et al., 2023).

The paper’s limitations are also explicit or directly implied. DCE assumes that task/domain boundaries are known during training. It depends on a stable frozen feature space after the first task. It adopts a Gaussian unimodality assumption for class-conditional PTM features. Covariance estimation for few-shot classes is fragile enough to require OAS, and parameter count grows with the number of domains. These points indicate that DCE is specifically matched to exemplar-free PTM-based DIL with strong class imbalance and cross-domain frequency shift, rather than to task-free continual learning or unconstrained replay settings (Li et al., 9 Jul 2025).