Practical Semi-Supervised Federated Learning

• PSSFL is a framework that integrates a small, centralized labeled dataset with extensive unlabeled client data to train a global model under non-IID conditions.
• It employs key primitives such as confidence-controlled pseudo-labeling, consistency regularization, and uncertainty shaping to blend supervised and unsupervised signals.
• The approach addresses real-world challenges like label scarcity, client heterogeneity, participation imbalance, and resource constraints to advance federated learning.

Practical Semi-Supervised Federated Learning (PSSFL) denotes federated learning regimes designed for deployments in which labels are scarce, unlabeled data are abundant, and statistical heterogeneity is intrinsic. In the most emphasized formulation, the server holds a small labeled dataset while clients hold only unlabeled data, and the objective is to learn a shared global model by combining centralized supervised learning with distributed semi-supervised or self-supervised updates under non-IID client distributions (Zhang et al., 2021). More recent work extends the same practical framing to foundation model adaptation, where the server may hold limited labeled high-resolution data while clients hold only unlabeled low-resolution data under strict memory budgets (Sun et al., 22 Aug 2025). Across this literature, PSSFL is less a single algorithm than a systems-and-objectives viewpoint: it treats label placement, client heterogeneity, participation imbalance, pseudo-label reliability, communication constraints, and privacy restrictions as coequal design variables rather than secondary implementation details.

1. Canonical formulation and problem variants

The canonical labels-at-server formulation writes the server dataset as

$\mathcal{D}_s = \{(x_i, y_i)\}_{i=1}^{n},$

and the unlabeled client dataset for client $k$ as

$\mathcal{D}_k = \{u_i\}_{i=1}^{m_k}.$

A standard decomposition uses a supervised server loss

$$\mathcal{L}_s = \frac{1}{n} \sum_{(x_i, y_i) \in \mathcal{D}_s} CE\big(y_i, f_\theta(x_i)\big),$$

an unsupervised client loss

$$\mathcal{L}_u^k = \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} CE\big(\hat{y}_i, f_{\theta_k}(u_i)\big) + \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} \left\| f_{\theta_k}(u_i) - f_{\theta_k}\big(\pi(u_i)\big)\right\|^2,$$

and a global objective

$\min_\theta \mathcal{L}, \quad \mathcal{L} \overset{\text{def}{=} \sum_{k=1}^K \mathcal{L}_u^k + \mathcal{L}_s,$

where $\hat{y}_i$ is a pseudo-label and $\pi(\cdot)$ is augmentation (Zhang et al., 2021).

That labels-at-server setting is not the only regime studied under the practical banner. A second family assumes partial labels on clients, often with extreme within-client mismatch between labeled and unlabeled class distributions, and formulates client objectives that combine supervised terms on $\mathcal{D}^s_k$ with pseudo-label or consistency terms on $\mathcal{D}^u_k$ (Bai et al., 2023). A third family emphasizes personalization: clients may be fully labeled, partially labeled, or fully unlabeled, and seek helper models or private layers to compensate for local label deficiency and feature skew (Shi et al., 2022). There are also domain-specific formulations in which local streams are continuously unlabeled and labels are acquired through active learning or multiview pseudo-labeling, as in human activity recognition and speech emotion recognition (Bettini et al., 2021, Feng et al., 2022).

A persistent theme is that practicality is defined by data placement and by what cannot be assumed. The literature repeatedly rejects the classical supervised-FL premise that clients own abundant ground-truth labels, and instead models systems in which labels are expensive, often centralized, and sometimes absent on the edge altogether (Zhang et al., 2020, Long et al., 2021).

2. Core algorithmic mechanisms

Most PSSFL methods combine a small number of recurring primitives. The first is confidence-controlled pseudo-labeling, usually generated from weakly augmented views and consumed on strongly augmented views. FedMix, for example, constructs pseudo-labels by aggregating predictions over $k$0 augmentations and retaining only predictions above $k$1 (Zhang et al., 2021). Semi-FedSER generalizes this into multiview pseudo-labeling with $k$2 weakly augmented views, temperature scaling $k$3, a confidence threshold that increases from $k$4 to $k$5, and an uncertainty filter based on the standard deviation across views with $k$6 (Feng et al., 2022). This suggests that practical pseudo-labeling is rarely a single argmax rule; it is typically surrounded by filtering, scheduling, or ensembling.

The second primitive is consistency regularization. In one widely used form, clients minimize a discrepancy between predictions on two augmentations,

$k$7

optionally combined with a pseudo-label cross-entropy term and a regularizer that keeps the client model close to a server-supervised model (Zhang et al., 2021). SemiFed uses KL-based consistency with RandAugment and then treats accepted pseudo-labels as labeled samples in later rounds (Lin et al., 2021). FedCon replaces pseudo-label-centric client learning with a contractive representation objective between online and target networks, using server-side cross-entropy and client-side BYOL-style representation alignment to circumvent the fact that labeled and unlabeled data are physically decoupled (Long et al., 2021).

The third primitive is uncertainty or confidence shaping. FedSEAL uses self-ensemble predictions averaged across rounds,

$k$8

and class-wise thresholds computed on a small server validation set to decide which samples enter positive pseudo-label training (Bian et al., 2021). $k$9 introduces client-specific adaptive thresholding and class-specific thresholds derived from the current global model’s confidence on each client’s unlabeled data, explicitly to reduce confirmation bias when only tens of server labels are available (Lee et al., 2024). FedAnchor departs from confidence-thresholded classifier outputs entirely: it trains an anchor head with a label contrastive loss and pseudo-labels client samples by cosine similarity to server-side anchor embeddings (Qiu et al., 2024).

The fourth primitive is structured regularization beyond positive pseudo-labels. FedSEAL adds complementary negative learning, constructing labels of the form “not class $\mathcal{D}_k = \{u_i\}_{i=1}^{m_k}.$0” for low-confidence samples and optimizing a negative loss alongside positive pseudo-label learning (Bian et al., 2021). FedDure adds a coarse-grained regulator that tracks whether pseudo-label-driven updates improve labeled loss, and a fine-grained regulator that learns instance-specific weights for unlabeled samples through a bi-level optimization loop (Bai et al., 2023). FedTriNet uses three predictors—local, global, and a spliced model combining shallow global and deep local layers—and accepts pseudo-labels only when their averaged soft predictions exceed a dynamic threshold (Che et al., 2021).

3. Representative systems and architectural patterns

The literature is best understood through a set of reference architectures rather than through a single dominant pipeline.

Framework	Data placement	Distinctive mechanism
FedMix (Zhang et al., 2021)	server small labeled $\mathcal{D}_k = \{u_i\}_{i=1}^{m_k}.$1; clients only unlabeled $\mathcal{D}_k = \{u_i\}_{i=1}^{m_k}.$2	explicit mixing of supervised, unsupervised, and previous global models plus FedFreq
FedCon (Long et al., 2021)	labels at server; unlabeled data on clients	online/target networks with contractive client learning
FedSEAL (Bian et al., 2021)	server labeled data and validation set; clients only unlabeled data	self-ensemble learning and complementary negative learning
FedAnchor (Qiu et al., 2024)	labeled anchor set on server; unlabeled clients	double-head architecture with anchor head and label contrastive loss
$\mathcal{D}_k = \{u_i\}_{i=1}^{m_k}.$3 (Lee et al., 2024)	server has very few labels; clients only unlabeled data	client-specific adaptive thresholding, SACR, LSAA
FedMox (Sun et al., 22 Aug 2025)	server labeled high-resolution data; clients unlabeled low-resolution data	spatial sparse Mixture-of-Experts and Soft-Mixture

FedMix is one of the clearest system-level blueprints. It decomposes learning into a supervised server model $\mathcal{D}_k = \{u_i\}_{i=1}^{m_k}.$4, client unsupervised models $\mathcal{D}_k = \{u_i\}_{i=1}^{m_k}.$5, and a global model $\mathcal{D}_k = \{u_i\}_{i=1}^{m_k}.$6, and updates the global state through

$\mathcal{D}_k = \{u_i\}_{i=1}^{m_k}.$7

with $\mathcal{D}_k = \{u_i\}_{i=1}^{m_k}.$8 (Zhang et al., 2021). The added $\mathcal{D}_k = \{u_i\}_{i=1}^{m_k}.$9 term is explicitly intended to stabilize the trajectory and avoid drift toward either labeled or unlabeled data. FedFreq complements this with a participation-frequency weighting rule that downweights clients that have trained more often.

FedCon addresses a different bottleneck: the decoupling of labeled and unlabeled data. Its contractive client objective does not require labels or inter-client pseudo-label exchange, while the server continues to optimize cross-entropy and a consistency term on labeled data (Long et al., 2021). FedSEAL, in contrast, accepts the pseudo-labeling paradigm but tries to make it trustworthy through temporal ensembling and complementary labels (Bian et al., 2021). FedAnchor shifts the problem into representation geometry by turning server labels into class-structured anchors in latent space (Qiu et al., 2024).

Recent systems widen the design space further. $$\mathcal{L}_s = \frac{1}{n} \sum_{(x_i, y_i) \in \mathcal{D}_s} CE\big(y_i, f_\theta(x_i)\big),$$0 treats confirmation bias as the central failure mode and responds with adaptive thresholds, sharpness-aware consistency regularization on carefully selected high-confidence pseudo-labels, and learning-status-aware aggregation (Lee et al., 2024). FedMox reframes PSSFL for frozen-backbone foundation model adaptation and object detection, where the key issues are not only label scarcity and non-IID data but also resolution mismatch and edge memory limits (Sun et al., 22 Aug 2025).

4. Heterogeneity, mismatch, and optimization pathologies

PSSFL is shaped by several pathologies that are weak or absent in centralized SSL. One is client gradient diversity. In labels-at-server SSFL, consistency regularization on non-IID unlabeled data produces highly diverse client gradients; Batch Normalization further increases this diversity, while replacing BN with Group Normalization reduces gradient diversity and improves test accuracy (Zhang et al., 2020). Grouping-based model averaging was introduced precisely because ordinary FedAvg can destructively average client updates that point in incompatible directions, and the reported gain exceeds $$\mathcal{L}_s = \frac{1}{n} \sum_{(x_i, y_i) \in \mathcal{D}_s} CE\big(y_i, f_\theta(x_i)\big),$$1 absolute accuracy over a contemporary SSFL baseline while also surpassing four supervised FL algorithms (Zhang et al., 2020).

A second pathology is class distribution mismatch, both across clients and within a client between labeled and unlabeled partitions. Fed-SHVR formalizes this setting and adds two regularizers: a KL penalty that softens pseudo-labels toward the uniform distribution and a confidence regularizer on model outputs. Its theory proves a convergence rate of $$\mathcal{L}_s = \frac{1}{n} \sum_{(x_i, y_i) \in \mathcal{D}_s} CE\big(y_i, f_\theta(x_i)\big),$$2 for Fed-SSL under non-IID data, which the work states is the first formal convergence result for Fed-SSL problems (Wang et al., 2021). FedDure generalizes the same concern into a bi-level client objective with coarse- and fine-grained regulators, explicitly targeting internal and external imbalance when $$\mathcal{L}_s = \frac{1}{n} \sum_{(x_i, y_i) \in \mathcal{D}_s} CE\big(y_i, f_\theta(x_i)\big),$$3 and $$\mathcal{L}_s = \frac{1}{n} \sum_{(x_i, y_i) \in \mathcal{D}_s} CE\big(y_i, f_\theta(x_i)\big),$$4 are both skewed (Bai et al., 2023).

A third pathology is unfairness induced by uneven label availability and data-relatedness. Personalized semi-supervised FL addresses this by selecting helper agents whose models are most related to a client’s data, using an uncertainty-based relation score

$$\mathcal{L}_s = \frac{1}{n} \sum_{(x_i, y_i) \in \mathcal{D}_s} CE\big(y_i, f_\theta(x_i)\big),$$5

and then using the most confident helper to pseudo-label each unlabeled example (Shi et al., 2022). A plausible implication is that in practical deployments, robustness may require not only a better global optimizer but also selective collaboration structures.

A fourth pathology is confirmation bias under extreme label scarcity. $$\mathcal{L}_s = \frac{1}{n} \sum_{(x_i, y_i) \in \mathcal{D}_s} CE\big(y_i, f_\theta(x_i)\big),$$6 argues that confirmation bias is more severe in FSSL than in centralized SSL because clients perform multiple local epochs on unlabeled-only data and the server and clients optimize labeled and unlabeled objectives separately (Lee et al., 2024). FedAnchor makes a related point from another angle: pseudo-labeling based solely on high-confidence classifier predictions overfits easy examples and slows convergence, whereas a label-contrastive anchor space uses many more client samples productively (Qiu et al., 2024).

5. Empirical behavior and domain-specific instantiations

On CIFAR-10 in the labels-at-server setting, FedMix combined with FedFreq converges to $$\mathcal{L}_s = \frac{1}{n} \sum_{(x_i, y_i) \in \mathcal{D}_s} CE\big(y_i, f_\theta(x_i)\big),$$7 test accuracy under non-IID data, about $$\mathcal{L}_s = \frac{1}{n} \sum_{(x_i, y_i) \in \mathcal{D}_s} CE\big(y_i, f_\theta(x_i)\big),$$8 higher than the baseline, and the final accuracy differs by no more than $$\mathcal{L}_s = \frac{1}{n} \sum_{(x_i, y_i) \in \mathcal{D}_s} CE\big(y_i, f_\theta(x_i)\big),$$9 across Dirichlet parameters $$\mathcal{L}_u^k = \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} CE\big(\hat{y}_i, f_{\theta_k}(u_i)\big) + \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} \left\| f_{\theta_k}(u_i) - f_{\theta_k}\big(\pi(u_i)\big)\right\|^2,$$0 (Zhang et al., 2021). The same study reports best performance at $$\mathcal{L}_u^k = \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} CE\big(\hat{y}_i, f_{\theta_k}(u_i)\big) + \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} \left\| f_{\theta_k}(u_i) - f_{\theta_k}\big(\pi(u_i)\big)\right\|^2,$$1, $$\mathcal{L}_u^k = \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} CE\big(\hat{y}_i, f_{\theta_k}(u_i)\big) + \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} \left\| f_{\theta_k}(u_i) - f_{\theta_k}\big(\pi(u_i)\big)\right\|^2,$$2, $$\mathcal{L}_u^k = \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} CE\big(\hat{y}_i, f_{\theta_k}(u_i)\big) + \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} \left\| f_{\theta_k}(u_i) - f_{\theta_k}\big(\pi(u_i)\big)\right\|^2,$$3, and observes a marked degradation when the server labeled set is reduced from 800 to 700 examples (Zhang et al., 2021). This suggests that even in server-label regimes, there may be a task-dependent minimum labeled-data threshold below which supervised guidance becomes too weak.

FedCon provides a complementary benchmark in the same practical scenario. Under IID data with $$\mathcal{L}_u^k = \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} CE\big(\hat{y}_i, f_{\theta_k}(u_i)\big) + \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} \left\| f_{\theta_k}(u_i) - f_{\theta_k}\big(\pi(u_i)\big)\right\|^2,$$4, it reaches $$\mathcal{L}_u^k = \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} CE\big(\hat{y}_i, f_{\theta_k}(u_i)\big) + \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} \left\| f_{\theta_k}(u_i) - f_{\theta_k}\big(\pi(u_i)\big)\right\|^2,$$5 on CIFAR-10 and $$\mathcal{L}_u^k = \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} CE\big(\hat{y}_i, f_{\theta_k}(u_i)\big) + \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} \left\| f_{\theta_k}(u_i) - f_{\theta_k}\big(\pi(u_i)\big)\right\|^2,$$6 on SVHN; under Non-IID with $$\mathcal{L}_u^k = \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} CE\big(\hat{y}_i, f_{\theta_k}(u_i)\big) + \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} \left\| f_{\theta_k}(u_i) - f_{\theta_k}\big(\pi(u_i)\big)\right\|^2,$$7, it reaches $$\mathcal{L}_u^k = \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} CE\big(\hat{y}_i, f_{\theta_k}(u_i)\big) + \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} \left\| f_{\theta_k}(u_i) - f_{\theta_k}\big(\pi(u_i)\big)\right\|^2,$$8 on MNIST, $$\mathcal{L}_u^k = \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} CE\big(\hat{y}_i, f_{\theta_k}(u_i)\big) + \frac{1}{m_k} \sum_{u_i \in \mathcal{D}_k} \left\| f_{\theta_k}(u_i) - f_{\theta_k}\big(\pi(u_i)\big)\right\|^2,$$9 on CIFAR-10, and $\min_\theta \mathcal{L}, \quad \mathcal{L} \overset{\text{def}{=} \sum_{k=1}^K \mathcal{L}_u^k + \mathcal{L}_s,$0 on SVHN, consistently outperforming FedMatch, SSFL, and FL adaptations of FixMatch and UDA (Long et al., 2021). FedSEAL reports the same pattern from a different algorithmic route: on Fashion-MNIST under non-IID data, it improves over Server-SL from $\min_\theta \mathcal{L}, \quad \mathcal{L} \overset{\text{def}{=} \sum_{k=1}^K \mathcal{L}_u^k + \mathcal{L}_s,$1 to $\min_\theta \mathcal{L}, \quad \mathcal{L} \overset{\text{def}{=} \sum_{k=1}^K \mathcal{L}_u^k + \mathcal{L}_s,$2, and on CIFAR-10 in the FedMatch-style setting it reaches $\min_\theta \mathcal{L}, \quad \mathcal{L} \overset{\text{def}{=} \sum_{k=1}^K \mathcal{L}_u^k + \mathcal{L}_s,$3, exceeding Server-SL and substantially surpassing FedMatch and FedRGD (Bian et al., 2021). A common misconception is therefore that any use of unlabeled client data is automatically beneficial; these results instead indicate that naive SSFL can underperform a server-only supervised lower bound.

Application papers show that the design principles survive outside toy image classification. Semi-FedSER applies local multiview pseudo-labeling and uncertainty-aware selection to speech emotion recognition and improves over supervised FL at label rates $\min_\theta \mathcal{L}, \quad \mathcal{L} \overset{\text{def}{=} \sum_{k=1}^K \mathcal{L}_u^k + \mathcal{L}_s,$4 and $\min_\theta \mathcal{L}, \quad \mathcal{L} \overset{\text{def}{=} \sum_{k=1}^K \mathcal{L}_u^k + \mathcal{L}_s,$5 on IEMOCAP and MSP-Improv; for example, with APC features on IEMOCAP at $\min_\theta \mathcal{L}, \quad \mathcal{L} \overset{\text{def}{=} \sum_{k=1}^K \mathcal{L}_u^k + \mathcal{L}_s,$6, supervised FL yields $\min_\theta \mathcal{L}, \quad \mathcal{L} \overset{\text{def}{=} \sum_{k=1}^K \mathcal{L}_u^k + \mathcal{L}_s,$7 UAR whereas Semi-FedSER reaches $\min_\theta \mathcal{L}, \quad \mathcal{L} \overset{\text{def}{=} \sum_{k=1}^K \mathcal{L}_u^k + \mathcal{L}_s,$8 (Feng et al., 2022). FedHAR takes a different route for human activity recognition: it combines active learning, label propagation, FL, and personalization, and reports that the active learning query rate decreases from about $\min_\theta \mathcal{L}, \quad \mathcal{L} \overset{\text{def}{=} \sum_{k=1}^K \mathcal{L}_u^k + \mathcal{L}_s,$9 in the first shard to about $\hat{y}_i$0 by the third while maintaining F1 scores close to supervised FL baselines (Bettini et al., 2021).

The scope of PSSFL has also expanded upward to foundation models. FedMox formalizes a setting with a frozen FM backbone, high-resolution labeled server data, low-resolution unlabeled client data, and object detection heads only. In that regime, high-resolution client training would require $\hat{y}_i$1 GB, while the proposed low-resolution frozen-backbone design reduces client memory to $\hat{y}_i$2 GB; on BDD100K with $\hat{y}_i$3, FedMox reaches total mAP@50 of $\hat{y}_i$4, above FedAvg, FedProx, FedSTO, and server-only centralized baselines (Sun et al., 22 Aug 2025). This broadens PSSFL from a label-placement problem into a joint label-placement, resource-allocation, and representation-mismatch problem.

6. Limitations, misconceptions, and open directions

Several misconceptions recur in the literature. One is that pseudo-labeling can simply be bolted onto FL. Multiple papers show the opposite: without explicit control of confidence, uncertainty, or model drift, pseudo-labeling may amplify errors and even fall below server-only supervised training (Bian et al., 2021, Lee et al., 2024). A second is that non-IID robustness is mainly a question of aggregation. The evidence points to a wider dependency set: normalization layers, threshold schedules, label-distribution mismatch, participation imbalance, and representation geometry all materially affect convergence and accuracy (Zhang et al., 2020, Bai et al., 2023). A third is that privacy is solved once raw data remain local. Most algorithmic papers concentrate on learning performance and leave differential privacy, secure aggregation, poisoning resistance, model inversion, GDPR-style compliance, and client churn largely unaddressed (Zhang et al., 2021, Sun et al., 22 Aug 2025).

Open problems are correspondingly broad. Server labels may be biased or unrepresentative of some client populations, especially in medical or demographic applications; several methods implicitly assume the server sees all classes, which may fail in deployment (Zhang et al., 2021). Hyperparameter tuning remains manual in many systems, including thresholds, mixing weights, and multiple regularization coefficients (Zhang et al., 2021, Feng et al., 2022). Asynchronous FL, persistent dropouts, and large-scale client churn are only partially handled by participation-aware schemes such as FedFreq (Zhang et al., 2021). Foundation-model PSSFL introduces additional unresolved issues: partial backbone adaptation, communication-efficient MoE heads, and the interaction between semi-supervised signals and frozen pretrained representations (Sun et al., 22 Aug 2025).

Recent work also suggests that PSSFL is no longer confined to centralized server-based FL. SemiDFL studies decentralized federated learning under SSL, allowing some clients to hold a few labels, some to be fully unlabeled, and others to have both, while seeking consensus in both data and model spaces (Liu et al., 2024). This suggests that the core PSSFL questions—how to trust unlabeled data, how to calibrate inter-client influence, and how to balance global sharing with local uncertainty—will likely persist even as the communication topology changes.

In current usage, PSSFL therefore names a research program rather than a settled recipe. Its central claim is that semi-supervised federated learning becomes practically meaningful only when label scarcity, non-IID structure, participation imbalance, model instability, and resource constraints are modeled explicitly and attacked jointly. The progression from FedMix, FedCon, and FedSEAL to FedAnchor, $\hat{y}_i$5, FedDure, and FedMox indicates an increasingly system-aware field: one that treats the placement of labels, the geometry of representations, and the dynamics of aggregation as inseparable components of the same federated problem.