FedOL: One-Shot Federated Learning

• FedOL is a one-shot federated learning framework that aggregates heterogeneous client predictions on unlabeled public data to build a robust global server model.
• It employs knowledge distillation and pseudo-label refinement to effectively address non-IID data distributions and varying client architectures.
• The framework dramatically reduces communication costs by sending only prediction outputs, outperforming traditional multi-round federated methods in accuracy and efficiency.

Searching arXiv for the named method and closely related FL knowledge-distillation work to ground the article. FedOL is a one-shot federated learning framework designed to construct a larger and more comprehensive server model from heterogeneous client models without multiple communication rounds, without sharing raw data, and without requiring all clients to use the same architecture (Ye et al., 19 Aug 2025). In FedOL, each client trains locally once on private labeled data, predicts on an unlabeled public dataset, and sends only prediction outputs to the server. The server then learns a larger model through knowledge distillation and pseudo-label refinement. The method is motivated by mobile and edge settings in which clients are resource heterogeneous, data are non-IID, communication is expensive, and privacy constraints preclude raw-data exchange (Ye et al., 19 Aug 2025).

1. Conceptual scope and problem formulation

FedOL addresses a practical gap in federated learning. Classic FL, exemplified by FedAvg, assumes that clients share the same model architecture, exchange parameters over many communication rounds, and repeatedly incur communication and computation cost. FedOL departs from that regime by replacing parameter sharing with prediction sharing and by restricting client participation to a single communication round (Ye et al., 19 Aug 2025).

The setting comprises $K$ clients. Client $k$ has private labeled data

$\mathbb{D}_{l,k} = \{(x,\mathbf{y})\},$

while all clients share a public unlabeled dataset

$\mathbb{D}_u = \{x\}.$

All clients solve the same classification task with $C$ classes, but their local data are non-IID and may exhibit label skew. Clients may also use different model architectures depending on device capability. The training objective is to produce a server-side model $w_s$ that is larger or more expressive than the client models, privacy-preserving, and built in one shot (Ye et al., 19 Aug 2025).

This formulation is explicitly oriented toward heterogeneous mobile and edge networks. A plausible implication is that FedOL treats server capacity as an asset rather than a constraint: the clients act as distributed sources of task knowledge, whereas the server consolidates that knowledge into a larger global model.

2. Architectural premise: heterogeneous models and one-shot communication

FedOL is organized around three operational commitments. First, each client trains locally on its own private labeled data exactly once. Second, clients transmit prediction outputs on the public unlabeled dataset rather than full model parameters. Third, the server constructs the global model through knowledge distillation and iterative pseudo-label refinement (Ye et al., 19 Aug 2025).

This design directly addresses four intertwined challenges identified in the method description. The first is heterogeneous client architectures: traditional parameter averaging presupposes model compatibility, whereas FedOL avoids parameter sharing and therefore allows clients to use different models. The second is one-shot communication: each client trains locally, predicts on public data, sends outputs to the server, and does not participate in iterative server-client back-and-forth. The third is the use of unlabeled public data: because the public set lacks ground-truth labels, the server must rely on pseudo-labels. The fourth is non-IID client data: client predictions are biased by local distributions, so FedOL incorporates class-wise confidence, entropy-based filtering, weighted vote aggregation, and client-specific distillation weights (Ye et al., 19 Aug 2025).

A common misconception is to equate “one-shot” with the absence of iterative optimization altogether. In FedOL, one-shot refers to the client-server communication pattern, not to the server’s training dynamics. The server still alternates between pseudo-label generation and model optimization for several iterations, but the clients communicate only once (Ye et al., 19 Aug 2025).

3. Objective function and server-side learning

Each client first trains its local model $w_c^k$ by minimizing cross-entropy on its own private data: $$w_c^k = \arg \min_{w} \mathbb{E}_{(x,\mathbf{y}) \in \mathbb{D}_{l,k}} \left[\text{CE}\left(\mathbf{y}, \sigma(w(x))\right)\right],$$ where $\sigma(\cdot)$ is softmax and

$\text{CE}(\mathbf{y},p) = -\mathbf{y}^T \log p.$

FedOL then adopts a semi-supervised template at the server: $k$0 where $k$1 is a distillation loss, $k$2 is a pseudo-label loss, and $k$3 is a trade-off parameter (Ye et al., 19 Aug 2025).

Because the server does not possess the true labeled data, the labeled loss is approximated through client predictions. The distillation term is written as

$k$4

leading to the final objective

$k$5

subject to $k$6 being one-hot (Ye et al., 19 Aug 2025).

The client-specific distillation weights are not uniform. FedOL defines

$k$7

Thus lower entropy implies higher confidence, and higher confidence gives a client more influence in the server loss (Ye et al., 19 Aug 2025).

This weighting scheme is central to the method’s treatment of non-IID bias. The method description argues that biased predictions are not merely noise; they encode specialized local expertise. FedOL converts that expertise into a distillation signal by upweighting clients that are more certain on a given sample (Ye et al., 19 Aug 2025).

4. Pseudo-label refinement and confidence-filtered aggregation

FedOL’s pseudo-labeling mechanism is not a simple majority vote. It is a confidence-filtered weighted voting procedure built from class-wise confidence, entropy thresholding, negative-learning-style label vectors, and weighted aggregation (Ye et al., 19 Aug 2025).

For each client $k$8, FedOL computes a class-wise confidence vector

$k$9

This serves as a class-dependent reliability profile. FedOL then applies entropy thresholding using a confidence threshold $\mathbb{D}_{l,k} = \{(x,\mathbf{y})\},$0. For each source model $\mathbb{D}_{l,k} = \{(x,\mathbf{y})\},$1, an entropy baseline $\mathbb{D}_{l,k} = \{(x,\mathbf{y})\},$2 is defined as the $\mathbb{D}_{l,k} = \{(x,\mathbf{y})\},$3-th lowest entropy over the public set. Only samples whose entropy is below $\mathbb{D}_{l,k} = \{(x,\mathbf{y})\},$4 are treated as reliable, yielding

$\mathbb{D}_{l,k} = \{(x,\mathbf{y})\},$5

The method description characterizes this as an easy-to-hard curriculum: early iterations use only highly confident predictions, and later iterations relax the threshold to admit more samples (Ye et al., 19 Aug 2025).

For each selected model, FedOL forms a signed prediction vector

$\mathbb{D}_{l,k} = \{(x,\mathbf{y})\},$6

These vectors are aggregated as

$\mathbb{D}_{l,k} = \{(x,\mathbf{y})\},$7

and the pseudo-label is then

$\mathbb{D}_{l,k} = \{(x,\mathbf{y})\},$8

where $\mathbb{D}_{l,k} = \{(x,\mathbf{y})\},$9 converts the aggregated score into a one-hot pseudo-label. If no reliable model exists for a sample, the pseudo-label remains $\mathbb{D}_u = \{x\}.$0, and that sample is ignored in pseudo-label training (Ye et al., 19 Aug 2025).

Server training alternates between two blocks: fixing $\mathbb{D}_u = \{x\}.$1 to generate $\mathbb{D}_u = \{x\}.$2, and fixing $\mathbb{D}_u = \{x\}.$3 to update $\mathbb{D}_u = \{x\}.$4 by minimizing

$\mathbb{D}_u = \{x\}.$5

This alternation is repeated for several iterations, with $\mathbb{D}_u = \{x\}.$6 increased progressively (Ye et al., 19 Aug 2025).

The pseudo-label mechanism is significant because it does not assume that all client predictions are equally trustworthy. Instead, it operationalizes sample-wise selectivity, class-wise weighting, and progressive inclusion of harder examples. This suggests a close alignment with self-paced curriculum learning, as stated in the method description (Ye et al., 19 Aug 2025).

5. Experimental protocol and reported performance

FedOL is evaluated on CIFAR-100, comprising 60,000 images and 100 classes, with a public unlabeled set of 5,000 samples. Private data are split among clients under label skew. Two heterogeneity regimes are used: a Dirichlet split $\mathbb{D}_u = \{x\}.$7, where smaller $\mathbb{D}_u = \{x\}.$8 implies greater heterogeneity, and a pathological split with limited classes per client, specifically $\mathbb{D}_u = \{x\}.$9 and $C$0 (Ye et al., 19 Aug 2025).

By default, the experiments use $C$1 clients, 50 epochs per model training, batch size $C$2, learning rate $C$3, FedOL iteration rounds $C$4, $C$5, and a curriculum in which $C$6 starts at $C$7 and increases by $C$8 each round. In parameter-based methods, all models use ResNet20. In knowledge-based methods, clients use ResNet11 or ResNet20, and the server uses the larger ResNet56 (Ye et al., 19 Aug 2025).

The baselines are Local, FedAvg, FedProx, FedDF, FedHKT, FedET, MinE, and FedKT. Methods requiring labeled public data or synthetic labeled generation are excluded for fairness (Ye et al., 19 Aug 2025).

Reported one-round FedOL accuracies are as follows:

Partition setting	FedOL accuracy
Dirichlet $C$9	37.2%
Dirichlet $w_s$0	29.8%
Pathological $w_s$1	35.8%
Pathological $w_s$2	33.5%

FedOL is reported to outperform all baselines across all partition settings, with the best baseline often lower by more than $w_s$3 percentage points. Even when baselines are allowed up to 15 communication rounds, FedOL with only one round matches or exceeds knowledge-based methods and often rivals or surpasses parameter-based methods after many rounds (Ye et al., 19 Aug 2025).

The cost analysis reports communication per client of $w_s$4 MB for parameter-based methods such as FedAvg and $w_s$5 MB for knowledge-based methods including FedOL. The method description therefore presents FedOL as a large reduction in communication cost while keeping client computation comparable to standard supervised learning (Ye et al., 19 Aug 2025).

These results support two specific conclusions stated in the method description. First, one-shot communication does not preclude strong performance when the server can exploit unlabeled public data effectively. Second, allowing heterogeneous client architectures is compatible with competitive federation when the exchanged object is predictive knowledge rather than parameter tensors.

6. Position within federated learning and interpretive boundaries

FedOL belongs to the knowledge-based branch of federated learning. Its defining features are one-shot communication, prediction sharing rather than parameter sharing, server-side construction of a larger model, and the use of unlabeled public data for distillation and pseudo-label refinement (Ye et al., 19 Aug 2025). This differentiates it from parameter-averaging methods such as FedAvg and from multi-round personalization methods that assume repeated synchronization.

It is also distinct from methods that operate with a shared foundation-model backbone in repeated communication rounds. For example, FedOT uses a pre-trained foundation model as a frozen black-box encoder, learns a globally shared classifier on the server, and learns client-specific orthogonal feature transformations locally; it is designed to balance generalization and personalization in heterogeneous FL without fine-tuning the foundation model itself (Kong et al., 26 May 2025). By contrast, FedOL addresses one-shot federation across heterogeneous client models and transfers knowledge through client predictions on unlabeled public data rather than through shared feature-space adaptation (Ye et al., 19 Aug 2025).

The practical implications stated for FedOL are concentrated in mobile and edge networks where clients have limited compute, devices differ widely in capability, privacy matters, communication budget is tight, and one-shot deployment is preferred (Ye et al., 19 Aug 2025). A plausible implication is that FedOL is most attractive when client participation is intermittent or expensive, but a server can still sustain iterative optimization over public unlabeled data.

Two interpretive cautions follow directly from the formulation. First, FedOL depends on access to a public unlabeled dataset, so the method does not eliminate the need for shared data resources; it relocates the shared resource from labeled private data to unlabeled public inputs. Second, the server’s gains rely on the reliability of pseudo-label refinement under non-IID bias, which is why the design emphasizes entropy filtering, class-wise confidence, weighted voting, and client-specific distillation rather than naive averaging (Ye et al., 19 Aug 2025).