--- title: Federated Split Learning (FedSL) url: https://www.emergentmind.com/topics/federated-split-learning-fedsl type: topic --- # Federated Split Learning (FedSL) Federated Split Learning (FedSL), also written as SFL in several papers, is a hybrid distributed-learning paradigm that combines Federated Learning (FL)’s parallel client-side training with Split Learning (SL)’s partitioning of a deep model into a client-side subnetwork and a server-side subnetwork. In its standard form, the full model is split at a cut layer, clients compute forward activations on local data, the server completes the forward and backward passes on the remaining layers, and a federated aggregation step synchronizes the client-side submodels. This design is used to reduce client computation relative to FL, avoid SL’s sequential-relay bottleneck, and keep raw data local, while introducing new trade-offs in communication, privacy leakage through smashed data, server storage, and cut-layer selection [2311.09441][2004.12088]. ## 1. Conceptual foundations and historical development FedSL emerged from the limitations of FL and SL when considered in isolation. FL requires each client to store and train the full model, which becomes impractical for large DNNs and resource-constrained devices. SL reduces client-side compute by assigning only an early network portion to the client, but classical SL is sequential and therefore incurs long end-to-end delay as the number of clients grows. SplitFed Learning was introduced specifically to amalgamate the two approaches, retaining SL’s split-model privacy and resource relief while recovering FL-style parallelism [2004.12088][2011.14818]. A common formalization writes the full model as \[ W=\bigl[\,W_C;\;W_S\bigr], \qquad |W_C|=\alpha |W|,\;\; |W_S|=(1-\alpha)|W|,\;\; \alpha\in[0,1], \] where \(W_C\) denotes the client-side submodel and \(W_S\) the server-side submodel. In another equivalent notation used for industrial settings, \[ M(x;\theta)=B(A(x;\theta_A);\theta_B), \] with \(A\) the device-side submodel and \(B\) the server-side remainder [2311.09441][2510.05713]. The literature does not treat FedSL as a single fixed protocol. Instead, it defines a family of architectures. Some papers use a main server for server-side training and a separate fed server for aggregation of client-side weights [2004.12088][2011.14818]. Others use paired edge servers with periodic aggregation of server-side weights only [2301.01824]. Still others generalize the same idea to wireless edge networks, vehicular edge computing, GANs, RNNs over sequentially partitioned data, and decentralized blockchain-managed deployments [2310.15584][2207.01750][2011.03180][2507.07637]. ## 2. Canonical architecture and training protocols A typical FedSL training round has three phases. First, each client runs a forward pass on a local mini-batch through \(W_C\), producing smashed data \(z\). Second, the server receives \(z\), continues the forward pass through \(W_S\), computes the loss, backpropagates through \(W_S\), and returns \(\nabla_z\) to the client. Third, the client completes backpropagation through \(W_C\), updates its local copy of \(W_C\), and uploads its updated client-side parameters to a federated aggregator, which applies FedAvg to \(\{W_C^{(k)}\}_{k=1}^K\) and broadcasts the new global \(W_C\) [2311.09441]. In the original SplitFed configuration, clients hold identical client-side subnetworks \(W_k^{\rm C}\), the main server holds \(W^{\rm S}\), and the fed server aggregates client-side updates by weighted averaging, \[ W^{\rm C}_{t+1}=W^{\rm C}_t-\eta \sum_{k=1}^K \frac{n_k}{n}\,\tilde g_k, \] while the main server computes losses on smashed activations and returns cut-layer gradients \(\delta A_k\) for client-side backpropagation [2004.12088]. The chapter treatment of SplitFed emphasizes that clients operate in parallel, whereas vanilla SL trains clients sequentially [2011.14818]. Not all FedSL systems aggregate the same model portion. In the hybrid architecture of [2301.01824], client–server pairs train locally on split models, but only the server-side weights are periodically averaged through “SerAVG,” while clients do not share or average their local \(W^{(1:c)}\). That design preserves the split-learning training flow yet changes the synchronization locus from client-side to server-side [2301.01824]. Several specialized protocols adapt the same template to different model classes. FedSL-GAN places the Generator \(G\) on the server and splits the Discriminator \(D\) vertically across a client’s resource-constrained devices; clients collaboratively evaluate the split discriminator on both local real data and generated samples, then the server performs FedAvg over discriminator updates and updates \(G\) using discriminator feedback [2207.01750]. For sequentially partitioned data, the RNN-based FedSL framework distributes consecutive sequence segments across clients, exchanges hidden states \(h_{\tau_i}\) and split gradients \(\delta_i\) between neighboring clients, and aggregates the segment-specific sub-networks on a server; this protocol is tailored specifically to cases where neither standard FL nor cut-layer SL can handle the temporal partitioning [2011.03180]. ## 3. Optimization of split points, bandwidth, and system resources A central technical issue in FedSL is the choice of cut layer or split point. In wireless settings, the split point simultaneously determines client compute, smashed-data size, and communication latency. One formulation considers \(K\) heterogeneous edge devices and one server, with device-specific split points \(s_k\in\{1,\dots,L\}\), bandwidth ratios \(b_k\in[0,1]\), and per-iteration latency dominated by the slowest device: \[ \mathbb E[\tau^{\rm total}] = \mathbb E\!\left[\max_{k=1\dots K}\bigl(\tau^{cp}_{s_k}+\tau^{cm}_{s_k}(b_k)\bigr)\right]. \] The corresponding optimization minimizes expected latency subject to split-point feasibility and bandwidth constraints, with an accuracy condition \[ (1-1/K)\,s_k\,\Phi^2 \le \hat \Phi^2. \] The proposed solution alternates between split-point optimization and bandwidth allocation, yielding a complexity \[ O(N_{\rm iter}\cdot K\cdot \hat \ell + N_{\rm iter}\cdot K\cdot \log(1/\epsilon)) \] and a closed-form bandwidth update \[ b_k^*= \frac{D_{s_k^*}}{W\log_2(1+\gamma_k)\,(T^*-\tau^{cp}_{s_k^*})} \] with \(T^*\) found by binary search [2310.15584]. Later heterogeneous FedSL formulations make the split point client-specific by design. HSFL allows each client \(k\) to choose its own cut-layer index \(\ell_k\), producing personalized client-side \(W_k^c\) and server-side \(W_k^s\), and jointly optimizes cut layers, main-server CPU allocation, and bandwidth. The long-timescale problem selects \(\ell_k\) by Sample Average Approximation and a Genetic Algorithm, while the short-timescale problem uses Lagrangian relaxation for CPU allocation and Branch and Bound for bandwidth allocation [2411.13907]. Vehicular edge computing adds mobility and deadline constraints to the same problem class. ASFV assigns a cut-layer \(\epsilon_n^t\) to each participating vehicle at round \(t\), selects only vehicles whose per-round delay satisfies the sojourn-time constraint, and jointly optimizes cut layers, uplink powers, CPU frequencies, and OFDMA bandwidth ratios by block coordinate descent over three subproblems [2405.18707]. A more recent wireless-edge formulation expands the decision space still further. TiLP jointly chooses bandwidth, transmit power, split-layer placement, compression level, and terminal participation under per-round deadline, memory, spectrum, and energy constraints, and evaluates candidate actions through a cross-domain digital twin comprising network, training, and task sub-twins [2604.26105]. This suggests that, in modern FedSL systems, split-layer placement is increasingly treated as one decision variable within a larger mixed continuous-discrete resource-allocation problem rather than as an isolated architectural choice. ## 4. Privacy, communication, storage, and reliability trade-offs FedSL preserves privacy in the limited sense that raw data remain local, but the main privacy concern is an honest-but-curious server attempting to reconstruct inputs from smashed data. One formal treatment measures reconstruction success by the Structural Similarity Index (SSIM) and defines \[ RS(\alpha)=\mathbb E[\mathrm{SSIM}(x,\hat x(\alpha))]. \] In the reported experiments, deeper cut layers produce more non-linear compression and lower \(RS(\alpha)\), while the client energy \[ E(\alpha)=g_e\Bigl(l_e\,D_b\,E_{\text{main}(\alpha)}+E_{\text{fed}(\alpha)}\Bigr) \] increases with \(\alpha\). The resulting privacy–energy trade-off is posed as \[ \min_{\alpha} RS(\alpha) \quad \text{s.t.}\quad E(\alpha)\le E_{\rm req}, \] with a Fashion-MNIST example yielding \(\alpha^*=0.4201\) [2311.09441]. Communication remains a major bottleneck because standard FedSL transmits smashed data and cut-layer gradients in every round. CSE-FSL addresses this by attaching an auxiliary network to the client-side cut layer so that clients can update their local models without waiting for server-side gradients, keeping only a single model at the server and uploading smashed data only every \(h\) client mini-batches. Under standard assumptions, the method guarantees \[ \frac1T\sum_{t=0}^{T-1}\mathbb E\|\nabla F(\theta^t)\|^2 = O(1/\sqrt T)+O(\Delta^2) \] for non-convex objectives [2302.05599]. A later presentation of the same line of work states client-side and server-side convergence rates of order \(O(1/\sqrt T)\) under \(L\)-smoothness, bounded variance, and bounded drift, while making explicit that server storage becomes independent of the number of clients because only a single server submodel is maintained [2507.15816]. FSL-SAGE targets the same communication bottleneck from a different angle. Each client maintains an auxiliary model \(\mathbf x_{a,i}\) that estimates the server-side gradient with respect to smashed activations, and the server aligns this auxiliary only once every \(l\) rounds. The resulting total communication is \[ O\bigl(T\,Qd_z + \tfrac{T}{l}d_a\bigr), \] with convergence \[ \min_{0\le t