FedSub: Selective Federated Learning

• FedSub is a multifaceted federated learning concept encompassing personalized, subspace, and private techniques for selective model sharing.
• It enables selective sharing through class-aware model fusion, low-dimensional random projection, or privacy-preserving submodel selection, each addressing distinct system challenges.
• Empirical studies show that personalized FedSub improves F1 scores and convergence speed, while subspace and private variants balance efficiency, bias, and privacy.

FedSub is an overloaded term in federated learning literature rather than a single standardized algorithm. It denotes at least three distinct research lines and one comparative baseline: a personalized federated learning method that combines class-aware prototypes with model subnetworks for ubiquitous systems (Campana et al., 2024), an efficient subspace algorithm that uses random projection and low-dimensional dual variables under heterogeneous data (Zhang et al., 5 Sep 2025), a subspace-only baseline used in comparison with SSF (Zhu et al., 28 Apr 2026), and a private federated submodel learning protocol based on private set union and MM-SPIR (Wang et al., 2023). Across these uses, the common motif is selective sharing, but the optimization objectives, privacy assumptions, communication patterns, and theoretical guarantees differ substantially.

1. Nomenclature and variant taxonomy

The literature uses the name “FedSub” for different mechanisms that operate at different granularities of selectivity. In one line of work, the selective unit is a class-specific subnetwork extracted from client activations; in another, it is an $r$-dimensional optimization subspace; in another, it is a subset of submodels indexed privately across clients.

Variant	Core mechanism	Source
Personalized FedSub	Class-aware prototypes, clustering, and subnetwork fusion	(Campana et al., 2024)
Subspace FedSub	Random projection plus low-dimensional dual correction	(Zhang et al., 5 Sep 2025)
SSF baseline “FedSub”	Projected control variates with no residual retention	(Zhu et al., 28 Apr 2026)
Private FedSub	Private set union and private write-back over submodels	(Wang et al., 2023)

A common misconception is to treat these as successive versions of one method. The record does not support that reading. The papers target different system models: ubiquitous sensing and personalization (Campana et al., 2024), large-model optimization under heterogeneous data (Zhang et al., 5 Sep 2025), baseline comparison for heterogeneity-corrected subspace learning (Zhu et al., 28 Apr 2026), and information-theoretically private federated submodel learning with two non-colluding databases (Wang et al., 2023). This suggests that “FedSub” functions primarily as a naming pattern around selective federation rather than as a single canonical protocol.

2. Personalized FedSub for class-aware model fusion

In personalized federated learning for ubiquitous systems, FedSub was introduced to address a specific failure mode of whole-model aggregation under non-IID data: two clients may be similar for class “bike” but dissimilar for class “walk,” so averaging their full weights can worsen performance on the classes where they differ (Campana et al., 2024). The method therefore computes compact class-level prototypes to detect user similarity per label, extracts class-specific subnetworks as the set of neurons or weights most activated for that class, and fuses only those subnetworks across similar users on a per-class basis.

The client-side workflow is defined round-wise. At round $t$, the server samples $M$ clients. Each client performs local training on the current model $h(\theta_{t-1})$ for $E$ epochs, computes a prototype for each class, extracts class-wise subnetworks from selected layers, and sends prototypes and subnetworks to the server. For class $k$ on client $i$, the prototype is

$$p_k^{(i)}=\frac{1}{|D_k^{(i)}|}\sum_{x\in D_k^{(i)}} x$$

or, more generally,

$$p_k^{(i)}=\operatorname{avg}_{x\in D_k^{(i)}} f(x).$$

For subnetwork extraction, if $L$ is the selected subset of layers, then the average activation matrix for class $t$0 and layer $t$1 is

$t$2

and the class-$t$3 subnetwork is the collection $t$4. Implicitly, this defines a mask $t$5; wherever $t$6 is zero, that neuron or weight is considered inactive for class $t$7.

The server-side procedure is explicitly class-aware. It can optionally predict missing prototypes for clients that have not seen some classes, using collaborative filtering with cosine similarity. Pairwise similarity on shared classes is computed as

$t$8

For each class $t$9, the server then runs K-means on $M$0 using either Euclidean distance or $M$1–cosine-similarity, with $M$2 chosen to minimize the Davies–Bouldin score. This yields per-class clusters $M$3.

Subnetwork fusion follows the template

$M$4

where $M$5 is a reliability score for client $M$6 on class $M$7. Three fusion strategies are proposed. Cluster-AVG uses weights proportional to class frequency; Cluster-Leadership selects the client with maximal $M$8 in the cluster; Overlapping Components fuses only those weight entries where all masks are nonzero. After fusion, the server optionally normalizes weights to avoid exploding magnitudes and sends back fused parameters $M$9 to each client as its new local model.

3. Empirical behavior in ubiquitous systems

The personalized FedSub formulation was evaluated in three real-world scenarios characterized by high data heterogeneity, derived from human activity recognition and mobile health applications (Campana et al., 2024). The datasets were WISDM for HAR with 36 clients, 6 activities, and 3,776 one-second samples; WESAD for stress with 15 clients, 4 states, and 6,189 windows; and DREAMT for sleep with 100 clients, binary sleep/wake labels, and 189,211 windows. The model architecture in all experiments was a small feed-forward network with two hidden layers, 128 then 512 units, ReLU, cross-entropy loss, and SGD. Hyperparameters included $h(\theta_{t-1})$0 local epoch per round and total rounds of 300 for HAR or 1000 for Stress and Sleep.

The baselines re-implemented in the same framework were FedAvg, FedProx through FedHome’s $h(\theta_{t-1})$1 regularization, FedHome, FedCLAR, FedRep, and ProtoHAR. Metrics were per-client macro-F1 on held-out 30% test data, average test loss, 95% CI across 5 seeds, convergence speed, and personalization gap where applicable.

In the static scenario, FedSub outperformed all baselines, especially personalized ones such as FedRep and ProtoHAR, by 3–6% F1 and by 4–8% lower test loss, reaching approximately $h(\theta_{t-1})$2 F1 on HAR, Stress, and Sleep. In the dynamic scenario, where clients gain new classes mid-training, FedSub recovered accuracy within 1–2 rounds thanks to predicted prototypes and pre-fused subnetworks, while other personalized methods suffered longer dips. Averaged over the three datasets, it improved final F1 by about $h(\theta_{t-1})$3 versus FedRep and $h(\theta_{t-1})$4 versus ProtoHAR, and cut convergence rounds by 10–20%.

The ablation studies clarify where the gains arise. “Partial” subnetworks, defined as only early layers, converged faster and attained approximately 3–8% gain in F1 or loss versus “Full,” because later layers encode highly client-specific decisions that are better locally fine-tuned. Among fusion strategies, Overlapping Components outperformed Cluster-Leadership, which in turn outperformed Cluster-AVG, in both accuracy and stability, with narrower confidence intervals. The paper also identifies concrete limitations and future work: privacy of prototypes, subnetwork quality, decentralization toward P2P or opportunistic federated learning, and broader use cases such as chronic disease prediction or recommender systems. A plausible implication is that the method’s strongest regime is one where label-conditioned similarity is informative and stable enough to justify per-class aggregation.

4. Low-dimensional subspace methods that also bear the name FedSub

A different FedSub line addresses the large-model regime, where communication, memory, and computation are scarce under heterogeneous data (Zhang et al., 5 Sep 2025). Its main idea is to restrict each client’s local update to a tunable low-dimensional subspace via a random projection matrix $h(\theta_{t-1})$5, while maintaining a low-dimensional dual variable $h(\theta_{t-1})$6 to mitigate client drift. The local model is parameterized as

$h(\theta_{t-1})$7

with subspace gradient

$h(\theta_{t-1})$8

and local step

$h(\theta_{t-1})$9

The dual update is

$E$0

The stated objective is simultaneous reduction of communication, computation, and memory costs, with uplink communication $E$1 per client per round and local work that scales mostly with $E$2 rather than $E$3.

The convergence analysis assumes projection randomness, smoothness, and bounded variance of the subspace gradient. With $E$4 and $E$5, Theorem 1 gives the ergodic bound

$E$6

under a stepsize condition of order $E$7. The second term is the bias induced by random subspaces; it vanishes if $E$8 or if $E$9 deterministically. Experiments include synthetic logistic regression and CIFAR-100 with ResNet-110. On CIFAR-100, applying FedSub only to convolutional layers reduced trainable parameters to 43% of full, and FedSub-CD and FedSub-full both reached approximately 58% test accuracy while using only approximately 30% of uplink cost.

The SSF paper uses “FedSub” differently, as a baseline described as “the naïve subspace–SCAFFOLD or subspace–FedAvg with control variates,” in which model updates and drift correction live entirely in an $k$0-dimensional subspace and residual orthogonal information is discarded whenever the subspace rotates (Zhu et al., 28 Apr 2026). In that construction,

$k$1

and no residual is maintained across rounds. The SSF comparison argues that FedSub has no robust SCAFFOLD-style convergence guarantee under arbitrary heterogeneity unless one assumes uniformly bounded full-space gradients and strong correlation between consecutive random subspaces. Empirically, on a matrix-regression toy problem with $k$2, Final RelErr was $k$3 for FedSub, and on CIFAR-100 with ResNet-110 the test-accuracy ranking was Full-SCAFFOLD at approximately 53%, SSF at approximately 45%, FedAvg at approximately 38%, and FedSub at approximately 30%. The overlap in naming should therefore not be read as identity of method.

5. Private federated submodel learning under the FedSub name

Another FedSub formulation concerns federated submodel learning with information-theoretic privacy (Wang et al., 2023). Here a central server consists of two non-colluding, replicated databases, $k$4 and $k$5, and the full model is partitioned into $k$6 submodels, each of length $k$7 over a finite field $k$8. A set of $k$9 clients is selected to perform one round. Client $i$0 has a desired submodel index set $i$1, represented by an incidence vector $i$2.

The protocol has three phases. First, FSL-CRG establishes client-side common randomness, including a global scalar $i$3 and one-time pads, using RSPIR and one-time pads. Second, FSL-PSU privately computes the union of submodel indices to be updated collectively. In Step 1, each client sends masked incidence values $i$4. In Step 2, each database aggregates client contributions, adds server randomness $i$5, routes through a random relay client, and after cancellation each database recovers

$i$6

from which

$i$7

Once $i$8 is known, each database sends $i$9 to its clients, and client $$p_k^{(i)}=\frac{1}{|D_k^{(i)}|}\sum_{x\in D_k^{(i)}} x$$0 trains locally on $$p_k^{(i)}=\frac{1}{|D_k^{(i)}|}\sum_{x\in D_k^{(i)}} x$$1 and produces increments $$p_k^{(i)}=\frac{1}{|D_k^{(i)}|}\sum_{x\in D_k^{(i)}} x$$2 for each $$p_k^{(i)}=\frac{1}{|D_k^{(i)}|}\sum_{x\in D_k^{(i)}} x$$3.

Third, FSL-write performs private write-back. Clients mask updates with pads $$p_k^{(i)}=\frac{1}{|D_k^{(i)}|}\sum_{x\in D_k^{(i)}} x$$4, databases aggregate and add $$p_k^{(i)}=\frac{1}{|D_k^{(i)}|}\sum_{x\in D_k^{(i)}} x$$5, relay clients add $$p_k^{(i)}=\frac{1}{|D_k^{(i)}|}\sum_{x\in D_k^{(i)}} x$$6, and each database finally recovers $$p_k^{(i)}=\frac{1}{|D_k^{(i)}|}\sum_{x\in D_k^{(i)}} x$$7 to update

$$p_k^{(i)}=\frac{1}{|D_k^{(i)}|}\sum_{x\in D_k^{(i)}} x$$8

The paper states correctness, DB-privacy, and inter-client privacy for both FSL-PSU and FSL-Write, and Theorem 4.1 asserts robustness to any pattern of client drop-outs, late-arrivals, and single DB failure. The communication complexity of one FSL round is $$p_k^{(i)}=\frac{1}{|D_k^{(i)}|}\sum_{x\in D_k^{(i)}} x$$9 symbols for FSL-PSU, $$p_k^{(i)}=\operatorname{avg}_{x\in D_k^{(i)}} f(x).$$0 for FSL-Write, and $$p_k^{(i)}=\operatorname{avg}_{x\in D_k^{(i)}} f(x).$$1 for CRG, yielding overall cost $$p_k^{(i)}=\operatorname{avg}_{x\in D_k^{(i)}} f(x).$$2. Relative to cited prior work, this FedSub variant emphasizes exact information-theoretic privacy, uncoded storage, and many-client parallel updates.

6. Comparative interpretation and recurring themes

The main unifying feature across FedSub variants is selective federation, but the selection dimension changes. Personalized FedSub selects by class and by active components of the network (Campana et al., 2024). Subspace FedSub selects by low-dimensional projection coordinates (Zhang et al., 5 Sep 2025, Zhu et al., 28 Apr 2026). Private FedSub selects by submodel index sets and protects that selection with private set union (Wang et al., 2023). This suggests a family resemblance rather than a shared algorithmic core.

The trade-offs also differ. In the personalized setting, the central tension is balancing personalization and generalization under label-conditioned heterogeneity, with gains measured by macro-F1, test loss, convergence speed, and recovery after class drift. In the subspace setting, the central tension is between low-dimensional efficiency and the bias introduced by projection or by discarded residual information. In the private submodel setting, the central tension is between exact privacy guarantees and communication complexity under a two-database architecture.

Future directions in the cited works remain correspondingly distinct. The personalized paper proposes DP or homomorphic encryption for prototypes, pruning-inspired methods such as LRP-based relevance for subnetwork quality, decentralization toward P2P or opportunistic federated learning, and applications to chronic disease prediction or recommender systems (Campana et al., 2024). The subspace algorithm paper proposes adaptive optimizers, explicit bias correction, structured or learned subspaces, analysis of partial participation and asynchronous updates, and large LLM finetuning with DP or secure aggregation (Zhang et al., 5 Sep 2025). The private submodel paper already emphasizes robustness to client drop-outs, late-arrivals, and database failure (Wang et al., 2023).

Taken together, the literature shows that “FedSub” does not denote one settled method. It denotes several technically distinct attempts to federate only a selected part of what would otherwise be shared in full: class-conditioned model components, projected optimization directions, or privately chosen submodels.