Class Imbalanced FFREEDA Benchmark

• The paper demonstrates that using frozen vision foundation models in CI-FFREEDA boosts performance (e.g., +10.5% to +16.6% MAR) over traditional aggregation methods.
• CI-FFREEDA is a realistic benchmark that integrates source and target class imbalances, label shifts, and non-IID client distributions in federated source-free adaptation.
• The method reduces communication and computational overhead by freezing the backbone and training only lightweight classifier and bottleneck layers.

Searching arXiv for the cited CI-FFREEDA paper and closely related work on FFREEDA, class imbalance in federated learning, and fairness/imbalance methods to ground the encyclopedia entry. Class Imbalanced FFREEDA (CI-FFREEDA) denotes a class-imbalanced extension of Federated Source-Free Domain Adaptation (FFREEDA) in which a server distributes a pre-trained source model to multiple clients that possess only unlabeled target-domain data, while the source data itself is unavailable during adaptation. In the formulation introduced in "Rethinking the Backbone in Class Imbalanced Federated Source Free Domain Adaptation: The Utility of Vision Foundation Models" (Kihara et al., 10 Sep 2025), CI-FFREEDA explicitly incorporates class imbalances in both source and target domains, label shift between source and target, and non-IID label distributions among target clients. The setting is presented as a more realistic and more challenging benchmark than standard FFREEDA, and its central empirical claim is that strong frozen vision foundation models (VFMs) mitigate domain gaps, class imbalance, and inter-client non-IID-ness more effectively than increasingly complex aggregation or domain-adaptation procedures (Kihara et al., 10 Sep 2025).

1. Definition and problem setting

CI-FFREEDA extends the FFREEDA setting by adding three distributional complications simultaneously: class imbalance in the labeled source domain, class imbalance in the unlabeled target domains, and label-distribution heterogeneity both across domains and across clients (Kihara et al., 10 Sep 2025). The source domain is denoted by

$D_s=\{(x_s^i,y_s^i)\mid i=1,\ldots,N_s\},$

and the unlabeled target data on client $k$ is denoted by

$$D_t^{(k)}=\{x_t^{(k),i}\mid i=1,\ldots,N_t^{(k)}\},\qquad k=1,\ldots,K.$$

The objective is to learn a single global model adapted to the target domain by aggregating information across the clients’ unlabeled datasets (Kihara et al., 10 Sep 2025).

The source-free constraint is fundamental. During adaptation, the source model is available, but the source data are not. This differentiates CI-FFREEDA from ordinary federated class-imbalance settings, where local relabeling, client reweighting, or class-count exchange may be designed with direct access to local labels or at least to class-frequency summaries. In CI-FFREEDA, the target side is unlabeled and federated, so the problem couples unsupervised domain adaptation, privacy constraints, non-IID partitioning, and class imbalance in a single formulation (Kihara et al., 10 Sep 2025).

A plausible implication is that CI-FFREEDA should be understood less as a narrow variant of federated learning and more as a compound benchmark at the intersection of source-free adaptation, label shift, and decentralized unsupervised learning. That interpretation is consistent with the paper’s emphasis on realism and on the interaction between source-target shift and inter-client skew (Kihara et al., 10 Sep 2025).

2. Distributional assumptions and benchmark construction

The CI-FFREEDA formulation adopts a shift model in which both covariate and label marginals vary:

$p(x)\neq q(x), \qquad p(y)\neq q(y),$

while retaining ideal label consistency

$p(y\mid x)=q(y\mid x),$

and allowing class-conditional feature shift

$p(x\mid y)\neq q(x\mid y).$

In the federated multi-client extension, the paper states

$$\begin{align*} p(x_s) &\neq q(x_t^{1}) \neq \ldots \neq q(x_t^{K}),\ p(y_s) &\neq q(y_t^{1}) \neq \ldots \neq q(y_t^{K}),\ p(y_s \mid x_s) &= q(y_t^{1} \mid x_t^{1}) = \ldots = q(y_t^{K} \mid x_t^{K}),\ p(x_s \mid y_s) &\neq q(x_t^{k} \mid y_t^{k}),\ q(x_t^{1} \mid y_t^{1}) &= \ldots = q(x_t^{K} \mid y_t^{K}). \end{align*}$$

These assumptions encode both inter-domain and inter-client label shift and covariate shift (Kihara et al., 10 Sep 2025).

The empirical benchmark uses Office-Home and VisDA-C as source-target families. Office-Home has 65 classes and VisDA-C has 12 classes. The number of clients is 3 for Office-Home and 50 for VisDA. Target-client class distributions are created by Dirichlet partitioning, with $\alpha<1$ producing stronger non-IID-ness; the reported experiments use $\alpha=0.5$ (Kihara et al., 10 Sep 2025).

The paper’s appendix-level split protocol is also part of the benchmark specification. Approximately 60% of a domain is used for the source side, with an 80/20 train/validation split. On the target side, a balanced test set is reserved for final global evaluation, while the remainder is split among clients to simulate non-IID and imbalanced class distributions via Dirichlet sampling; each client then uses an 80/20 train/validation split (Kihara et al., 10 Sep 2025). The balanced test set matters because conventional accuracy can obscure minority-class failures under severe skew.

The principal evaluation metric is Macro Average Recall (MAR),

$$\text{MAR}=\frac{1}{C}\sum_{c=1}^{C}\frac{\text{TP}_c}{\text{TP}_c+\text{FN}_c},$$

which the paper refers to as "accuracy" (Kihara et al., 10 Sep 2025). MAR is aligned with the benchmark’s emphasis on balanced per-class performance rather than majority-class dominance.

Aspect	Office-Home	VisDA-C
Number of classes	65	12
Number of clients	3	50
Target partitioning	Dirichlet, $k$0	Dirichlet, $k$1

3. Baselines, adaptation methods, and the shift in research emphasis

The CI-FFREEDA study evaluates several source-free domain-adaptation methods and federated aggregation strategies. The source-free domain-adaptation baselines include SHOT, NRC, and AaD, while the class-imbalance-aware methods include ISFDA and ICPR. On the federated-learning side, FedAvg is used alongside non-IID-oriented aggregation methods such as FedProx and FedETF (Kihara et al., 10 Sep 2025).

The paper’s stated motivation is notable: replication of existing methods in the CI-FFREEDA setting led the authors to rethink whether the main research bottleneck lies in aggregation and domain-adaptation design. Their conclusion is that the field may have overemphasized adaptation and aggregation mechanisms relative to backbone quality (Kihara et al., 10 Sep 2025). This is not a general claim that aggregation never matters; rather, within the reported CI-FFREEDA experiments, advanced FL aggregation methods were less effective than replacing the conventional backbone with a strong shared feature extractor.

The proposed implementation strategy has three defining elements. First, the standard CNN backbone is replaced with a frozen VFM, specifically DINOv2 ViT-S or ViT-B. Second, only the bottleneck and classifier layers are trained and communicated during federated adaptation. Third, balanced sampling is applied during source training to mitigate source-side class imbalance (Kihara et al., 10 Sep 2025). These choices reframe the adaptation problem: instead of attempting to learn increasingly robust features during federated source-free adaptation, the system begins with a stronger and more invariant representation and restricts collaborative learning to lighter task-specific layers.

This suggests a methodological inversion of much prior FFREEDA practice. Rather than compensating for weak transferable features through more elaborate adaptation losses or aggregation rules, CI-FFREEDA as formulated in (Kihara et al., 10 Sep 2025) prioritizes representation strength and parameter efficiency at the outset.

4. Frozen vision foundation models as the proposed CI-FFREEDA solution

The central proposal in the CI-FFREEDA paper is the use of a frozen VFM backbone during both source pre-training and federated adaptation, with only bottleneck and classifier layers updated and exchanged (Kihara et al., 10 Sep 2025). The paper specifically evaluates DINOv2 ViT-S and ViT-B.

Three technical rationales are given. The first is domain and label invariance: large-scale pre-training is said to endow VFMs with robust, generic features that are less sensitive to domain shift and class imbalance. The second is mitigation of non-IID effects: because the backbone is frozen and shared, all clients operate in the same feature space, reducing divergence caused by non-IID label distributions. The third is resource efficiency: freezing the backbone drastically reduces communication, memory usage, and training cost, since clients neither backpropagate through nor transmit the full backbone (Kihara et al., 10 Sep 2025).

The communication reduction is quantified in the paper’s summary of the cost table. With the frozen VFM design, only less than 1MB is communicated per round, compared with approximately 100MB when a full ResNet is communicated and fine-tuned (Kihara et al., 10 Sep 2025). The paper further states that freezing the backbone can reduce computation and communication cost by up to two orders of magnitude relative to full fine-tuning. Balanced sampling during supervised source training is reported to further help counter source-side imbalance (Kihara et al., 10 Sep 2025).

A common misconception in reading the CI-FFREEDA results would be to interpret them as showing that sophisticated source-free methods such as ICPR or ISFDA are intrinsically unnecessary. The narrower claim is that when the feature extractor is sufficiently strong and frozen, the relative advantage of specialty SFDA methods becomes much smaller, and in some cases disappears, because the harder part of cross-domain transfer has already been absorbed by the backbone (Kihara et al., 10 Sep 2025).

5. Empirical findings

The main empirical result is that VFMs consistently improve performance for all evaluated SFDA methods under class imbalance, domain gap, and non-IID label-shift scenarios (Kihara et al., 10 Sep 2025). On Office-Home, ViT-S improves average MAR by $k$2 and ViT-B by $k$3 over ResNet-50. On VisDA, the reported gain is $k$4 for ViT-based backbones over ResNet-101 (Kihara et al., 10 Sep 2025).

The paper reports several broader patterns. First, target adaptation suffers much less performance drop with VFMs, even under extreme class and client skew. Second, FedProx and FedETF are less effective than simply using a strong shared feature extractor. Third, when strong enough features are provided, specialty SFDA methods such as ICPR and ISFDA lose their advantage over simpler baselines. Fourth, larger backbones such as ViT-B yield further marginal gains in more complex settings with many classes and larger domain gaps, while those gains are less necessary when the class/domain gap is not severe, as in the paper’s discussion of VisDA (Kihara et al., 10 Sep 2025).

These findings support the paper’s broader recommendation: prioritize strong, robust feature extractors, preferably frozen VFMs, over increasingly complex adaptation or federated aggregation methods. This recommendation is paired with practical observations that hyperparameter tuning becomes simpler and that deployment becomes more feasible in low-resource or bandwidth-limited settings because only small heads are exchanged (Kihara et al., 10 Sep 2025).

A second misconception is that CI-FFREEDA is only about target-side imbalance. The reported experiments and implementation details explicitly address source imbalance as well, through imbalanced source sampling in the benchmark and balanced sampling during source training in the proposed VFM-based pipeline (Kihara et al., 10 Sep 2025). In the paper’s framing, source imbalance, target imbalance, label shift, and inter-client non-IID-ness are coexisting rather than isolated factors.

6. Position within adjacent research on federated imbalance and adaptation

CI-FFREEDA occupies a distinct position relative to adjacent federated class-imbalance literature. FedReLa addresses the coexistence of global class imbalance and cross-client heterogeneity by asymmetrically re-labeling majority-class samples into minority classes using a feature-dependent label re-allocator, without requiring knowledge of the global class distribution and without additional communication overhead (Hu et al., 24 Jun 2026). FedBB decomposes imbalance into inter-case, inter-class, and inter-client levels, using Positive Negative Balanced loss for local training and Client Balanced Reweighting for aggregation (Chung et al., 8 Jun 2026). pFedBBN addresses unsupervised test-time adaptation in federated learning under class imbalance through balanced batch normalization, BBN-statistics similarity, and class-aware personalized aggregation (Iftee et al., 22 Nov 2025).

These methods and CI-FFREEDA share a concern with class skew under decentralization, but they solve different problems. FedReLa and FedBB operate in federated learning with access to local training labels or label-derived statistics; pFedBBN studies unsupervised local adaptation at inference time under domain shift; CI-FFREEDA adds the source-free constraint, so adaptation proceeds from a pre-trained source model while the source data remain inaccessible and the target clients are unlabeled (Kihara et al., 10 Sep 2025). This makes the role of representation quality unusually central.

A plausible implication is that CI-FFREEDA functions as a stress test for whether class-imbalance remedies remain effective when label supervision is removed from the target side and source exemplars are removed from the server side. In that sense, the benchmark sharpens a broader question already visible in neighboring work: whether imbalance in federated systems should be handled primarily by local label-space corrections, by aggregation rules, by normalization and test-time adaptation, or by stronger universal feature extractors (Hu et al., 24 Jun 2026, Chung et al., 8 Jun 2026, Iftee et al., 22 Nov 2025).

Within the evidence reported so far, the CI-FFREEDA paper argues for the last of these options. Its concluding position is that the realistic path forward is not chiefly more elaborate FL algorithms or source-free losses, but improved deployment of unified, scalable feature extractors—especially frozen VFMs—under privacy-constrained, class-imbalanced, label-shifted federated conditions (Kihara et al., 10 Sep 2025).