Split Learning: Distributed Neural Training

• Split Learning is a distributed machine learning paradigm that partitions a neural network between client and server, exchanging intermediate activations instead of raw data.
• It optimizes the cut-layer position to balance computation, communication overhead, and privacy risks across heterogeneous devices and dynamic networks.
• Variants like U-shaped SL and Secure Split Learning extend this framework to improve inference delay, robustness, and defend against privacy leakage and backdoor attacks.

Split Learning (SL) is a distributed machine learning paradigm in which a single neural network is partitioned across multiple parties—most commonly a client or edge device and a server—so that raw data remain local while intermediate representations and corresponding gradients are exchanged during forward and backward propagation. In its standard form, the client executes the early layers, sends the cut-layer activation map or “smashed data” to the server, and receives gradients at the split during training; in inference, only the forward path is used. This distinguishes SL both from centralized training, where raw data are transferred to a server, and from federated learning, where each client trains a full local model and transmits parameter updates rather than intermediate activations (Molina et al., 12 Feb 2025, Khan et al., 2023).

1. Core protocol and mathematical structure

A canonical formulation decomposes a model $F$ into an edge- or client-side sub-model $f_{\mathrm{E}}$ and a server-side sub-model $f_{\mathrm{S}}$, with

$$F(\mathbf{x}) = f_{\mathrm{S}}(f_{\mathrm{E}}(\mathbf{x})).$$

Given raw input $\mathbf{x}$, the client computes an intermediate representation $\mathbf{z} = f_{\mathrm{E}}(\mathbf{x})$, transmits $\mathbf{z}$ to the server, and the server computes the prediction $\hat{y} = f_{\mathrm{S}}(\mathbf{z})$. During training, the server computes the loss $\mathcal{L}(\hat{y}, y)$, backpropagates through its own parameters, and returns $\partial \mathcal{L}/\partial \mathbf{z}$ so that the client can complete backpropagation through $f_{\mathrm{E}}$0 and update its local parameters by the chain rule (Molina et al., 12 Feb 2025).

This workflow is often described using the language of smashed data, activation maps, and cut layers. In wireless and edge-oriented formulations, the client-side forward pass, uplink transmission of smashed data, server-side forward and backward pass, and downlink transmission of gradients form the dominant computation–communication loop (Wu et al., 2022). In U-shaped SL, the partition is refined further: the client holds a head $f_{\mathrm{E}}$1, the server holds a backbone $f_{\mathrm{E}}$2, and the client again holds a tail $f_{\mathrm{E}}$3, so that $f_{\mathrm{E}}$4. In that arrangement, raw inputs and labels stay on the client, the server processes only the intermediate backbone segment, and the loss is computed client-side (Rieger et al., 11 Jan 2025).

A concise way to summarize the standard protocol is that SL externalizes a hidden layer interface. The split point is therefore not merely an implementation detail; it determines local computation, transmitted feature dimensionality, gradient traffic, privacy exposure, and optimization dynamics.

2. Architectural variants and partitioning regimes

SL is not limited to a single two-party, layer-wise split. The literature in the supplied corpus describes several structurally distinct variants.

Variant	Structural idea	Example
Classical layer split	Early layers on client, later layers on server	(Molina et al., 12 Feb 2025)
U-shaped SL	Head and tail on client, middle layers on server	(Khan et al., 2023)
Split RNN / LSTMSPLIT	Entire recurrent layers assigned to edge or server	(Molina et al., 12 Feb 2025)
FedSL	Split recurrent computation across clients holding sequence segments	(Abedi et al., 2020)
Hierarchical SL	User–edge–cloud partition with three model segments	(Zhang et al., 23 Jan 2025)

For recurrent models, the defining point is that the recurrence equations are not altered; rather, whole recurrent layers are assigned to different sides of the split. In the LSTMSPLIT formulation, at least two LSTM layers are used, the early LSTM layer or layers remain on the edge, the later LSTM layer or layers remain on the server, and only the output sequence or hidden representation of the last edge-side LSTM is transmitted. In the FPGA water-quality case study, this yielded two main SL configurations: Split-A, where two stacked LSTMs remain on the FPGA and FC layers move to the server, and Split-B, where one LSTM remains on the FPGA and the remaining LSTM plus FC layers move to the server (Molina et al., 12 Feb 2025).

FedSL extends SL into sequentially partitioned data. Instead of cutting along feed-forward depth between client and server, it splits a recurrent model at temporal boundaries between sequence segments stored on different clients. Hidden states move from one client to the next segment owner, gradients flow back across that boundary, and segment-specific sub-networks are then aggregated federatively on a server. This introduces a third partitioning regime beyond horizontal and vertical partitioning: sequential partitioning (Abedi et al., 2020).

For LLMs, SplitLLM introduces a three-tier hierarchical split across user, edge, and cloud, with the pre-trained model and LoRA adapters divided into three parts. The training path becomes user $f_{\mathrm{E}}$5 edge $f_{\mathrm{E}}$6 cloud in the forward pass and the reverse in the backward pass, while only LoRA adapters are updated and then aggregated at the cloud. In the reported simulations, this reduced peak memory usage by up to $f_{\mathrm{E}}$7 compared to the state-of-the-art benchmarks (Zhang et al., 23 Jan 2025).

These variants show that “split learning” names a family of partitioned training protocols rather than a single fixed algorithm. The common invariant is that intermediate states, not raw data, define the coupling interface.

3. Split-point optimization and system-level trade-offs

A recurrent theme across the supplied work is that SL performance is governed by the joint geometry of layer workloads, activation sizes, channel rates, and device heterogeneity. The split point is therefore routinely cast as an optimization variable.

In semantic segmentation with ENet, the per-device inference delay for a cut at bottleneck module $f_{\mathrm{E}}$8 is modeled as the sum of edge computation time, uplink transmission time for $f_{\mathrm{E}}$9 bits, and server computation time on the remaining workload. The paper jointly optimizes cut-layer selection, bandwidth allocation, and server resource allocation, and also distinguishes parallel and serial processing regimes. Its numerical results show that SL-based designs substantially reduce inference delay relative to “first layer” baselines in which raw images are transmitted and the whole model runs at the server (Evgenidis et al., 2024).

For EMG prosthetic control, the cut-layer problem is posed directly as convergence-rate maximization. Using per-layer computational loads $f_{\mathrm{S}}$0, activation sizes $f_{\mathrm{S}}$1, cumulative client-side parameter sizes $f_{\mathrm{S}}$2, client and server speeds $f_{\mathrm{S}}$3, and rate $f_{\mathrm{S}}$4, the paper derives an epoch-delay model

$f_{\mathrm{S}}$5

and then derives an interval condition for the optimal cut layer in terms of a communication–computation trade-off function $f_{\mathrm{S}}$6. The resulting OCLA procedure performs offline pruning and online lookup, and the reported performance evaluation shows substantially accelerated convergence in the EMG pattern-recognition task (Marinova et al., 2024).

Wireless and edge settings motivate more elaborate resource management. CPSL replaces purely sequential SL by a “first-parallel-then-sequential” design: devices are partitioned into clusters, client-side models train in parallel within each cluster, and clusters are processed sequentially. In the reported simulations, per-round latency was $f_{\mathrm{S}}$7 s for CPSL, compared with $f_{\mathrm{S}}$8 s for vanilla SL and $f_{\mathrm{S}}$9 s for FL, while maintaining similar final accuracy under non-IID data (Wu et al., 2022). Adaptive SL over energy-constrained wireless edge networks generalizes the split point from a fixed design choice to a per-device, per-episode control variable $$F(\mathbf{x}) = f_{\mathrm{S}}(f_{\mathrm{E}}(\mathbf{x})).$$0, paired with server resource allocation $$F(\mathbf{x}) = f_{\mathrm{S}}(f_{\mathrm{E}}(\mathbf{x})).$$1; the OPEN algorithm, based on Lyapunov optimization, reduces average training delay and energy consumption by $$F(\mathbf{x}) = f_{\mathrm{S}}(f_{\mathrm{E}}(\mathbf{x})).$$2 and $$F(\mathbf{x}) = f_{\mathrm{S}}(f_{\mathrm{E}}(\mathbf{x})).$$3, respectively, relative to existing SL schemes (Li et al., 2024).

Under heterogeneous devices, HSFL further allows different clients to use different cut layers, while optimizing computation and transmission resources jointly. The reported simulation results show improvements in both convergence rate and model accuracy over comparison frameworks on heterogeneous devices with non-iid data (Sun et al., 2024). A plausible implication is that a single global split point is often a systems simplification rather than an algorithmic optimum.

4. Privacy, leakage, and security

A persistent misconception in the SL literature is that keeping raw data local is equivalent to privacy preservation. The supplied papers reject that interpretation. Activation maps can preserve substantial structure from the input, and multiple works in the corpus emphasize that plaintext split activations are often invertible or at least highly informative (Khan et al., 2023, Nguyen et al., 2023).

In U-shaped 1D CNNs for ECG, the visual correspondence between raw signals and activation maps is sufficiently strong that earlier work cited in the corpus demonstrated reconstruction attacks based on activation inversion, distance correlation, and Dynamic Time Warping. This is why several papers argue that vanilla and plaintext SL do not provide strong privacy for raw client data, even when labels stay local (Khan et al., 2023). The hybrid response is to combine SL with homomorphic encryption, specifically by encrypting the client-side activation map before transmission and restricting the server-side encrypted computation to a linear layer. In the reported PTB-XL experiments, the HE-based U-shaped SL setting reduced accuracy by only $$F(\mathbf{x}) = f_{\mathrm{S}}(f_{\mathrm{E}}(\mathbf{x})).$$4 compared to training on plaintext, while preserving raw training-data privacy (Khan et al., 2023).

A second line of defense is representational degradation. Binarized Split Learning replaces client-side weights and activations with binary representations, reducing leakage from smashed data while also reducing client computation and memory. The paper reports up to $$F(\mathbf{x}) = f_{\mathrm{S}}(f_{\mathrm{E}}(\mathbf{x})).$$5 times less forward-propagation time in both training and inference phases on mobile devices, up to $$F(\mathbf{x}) = f_{\mathrm{S}}(f_{\mathrm{E}}(\mathbf{x})).$$6 times less memory and bandwidth requirements, and merely a small degradation in model accuracy; it also augments B-SL with local leak loss and differential privacy mechanisms and evaluates resilience against feature-space hijacking attack (Pham et al., 2022).

Security issues are not limited to passive privacy leakage. SafeSplit addresses client-side backdoor attacks in U-shaped SL by analyzing server-side checkpoints after each client’s training, combining a static frequency-domain comparison with a dynamic rotational-distance metric, and reverting to the latest checkpoint classified as benign. Across CIFAR-10, MNIST, Fashion-MNIST, CIFAR-100, and GTSRB, the reported backdoor accuracy drops to near zero while main-task accuracy is largely preserved (Rieger et al., 11 Jan 2025).

These results collectively support a narrow but important conclusion: SL can reduce raw-data exposure, but structural privacy is not the same as cryptographic or adversarial robustness.

5. Training order, fairness, and hybridization

Another nontrivial property of classical SL is that multi-client training is often sequential. That sequentiality affects both latency and statistical fairness. In healthcare experiments on tuberculosis classification with five clients, the model trained by classical sequential SL was biased toward clients trained later in the round. Averaged across clients and metrics, placing a client first rather than last caused an approximately $$F(\mathbf{x}) = f_{\mathrm{S}}(f_{\mathrm{E}}(\mathbf{x})).$$7 performance drop, and the effect intensified as the number of clients increased (Madaan et al., 2021).

The mechanism described in that work is straightforward: the shared server-side model is updated client by client without an averaging step, so later clients overwrite part of the optimization trajectory induced by earlier clients. SplitFedv3 removes this order sensitivity by parallelizing training and averaging server-side parameters at the end of each round; in the reported experiments, first-versus-last placement no longer changed AUPRC, F1, or Cohen’s Kappa (Madaan et al., 2021).

This order-sensitivity result clarifies why many recent SL variants move toward hybridization. FedSL uses federated aggregation for segment-specific recurrent sub-networks (Abedi et al., 2020). CPSL parallelizes within clusters before sequential inter-cluster transfer (Wu et al., 2022). HSFL combines personalized client-side splits with server-side coordination and resource optimization (Sun et al., 2024). SplitLLM similarly combines SL-style forward/backward partitioning with FedAvg-style aggregation over LoRA adapters (Zhang et al., 23 Jan 2025). This suggests that modern SL research increasingly treats pure sequential SL as only one corner of a broader design space.

6. Representative applications and empirical profiles

The supplied corpus places SL in several concrete application domains.

In FPGA-based edge IoT for river water quality monitoring, the target task is next-day dissolved oxygen forecasting from the previous 15 measurements. The teacher model contains $$F(\mathbf{x}) = f_{\mathrm{S}}(f_{\mathrm{E}}(\mathbf{x})).$$8 parameters and occupies $$F(\mathbf{x}) = f_{\mathrm{S}}(f_{\mathrm{E}}(\mathbf{x})).$$9 KB, while the compressed student model contains $\mathbf{x}$0 parameters and occupies $\mathbf{x}$1 KB. After knowledge distillation, $\mathbf{x}$2 pruning, and 8-bit quantization, the student reaches MAE $\mathbf{x}$3, MSE $\mathbf{x}$4, and $\mathbf{x}$5, compared with MAE $\mathbf{x}$6, MSE $\mathbf{x}$7, and $\mathbf{x}$8 for the teacher. On the Artix-7 FPGA, Split-A and Split-B expose the classical SL trade-off: Split-A has latency $\mathbf{x}$9 and power $\mathbf{z} = f_{\mathrm{E}}(\mathbf{x})$0 W, while Split-B increases latency to $\mathbf{z} = f_{\mathrm{E}}(\mathbf{x})$1 but lowers power to $\mathbf{z} = f_{\mathrm{E}}(\mathbf{x})$2 W and raises scalability from $\mathbf{z} = f_{\mathrm{E}}(\mathbf{x})$3 to $\mathbf{z} = f_{\mathrm{E}}(\mathbf{x})$4 processing elements (Molina et al., 12 Feb 2025).

In semantic segmentation for real-time computer vision, SL is framed as an inference-partitioning problem rather than a training-only protocol. The ENet-based formulation models the split only at bottleneck boundaries, includes the transmission of pooling indices needed by MaxUnpooling, and shows that optimal or near-optimal delay requires joint cut-layer and bandwidth allocation rather than a static “minimum data” heuristic (Evgenidis et al., 2024).

In EMG prosthetic control, SL is motivated explicitly by the mismatch between prosthetic hardware and the computational demands of deep learning. The cited experiments show that suboptimal cut-layer choice directly slows convergence, and the proposed OCLA procedure is intended to make SL viable under time-varying channel and compute conditions (Marinova et al., 2024).

In wireless LLM fine-tuning, SplitLLM shows that hierarchical SL can be combined with parameter-efficient adaptation. On MRPC, the reported IID accuracy is $\mathbf{z} = f_{\mathrm{E}}(\mathbf{x})$5 for SplitLLM, compared with $\mathbf{z} = f_{\mathrm{E}}(\mathbf{x})$6 for FL and $\mathbf{z} = f_{\mathrm{E}}(\mathbf{x})$7 for two-tier SL; on CIFAR100 with ViT-base under non-IID data, SplitLLM reaches $\mathbf{z} = f_{\mathrm{E}}(\mathbf{x})$8, compared with $\mathbf{z} = f_{\mathrm{E}}(\mathbf{x})$9 for FL and $\mathbf{z}$0 for SL, while also reducing peak memory usage up to $\mathbf{z}$1 (Zhang et al., 23 Jan 2025).

7. Limitations and open directions

Several limitations recur across the corpus. Communication conditions are often simplified; the FPGA buoy study assumes an ideal link (Molina et al., 12 Feb 2025), while the semantic-segmentation and wireless-edge papers explicitly model bandwidth and resource allocation but do not address all deployment effects such as multi-cell interference or adversarial networking (Evgenidis et al., 2024, Li et al., 2024). Privacy guarantees are often structural rather than cryptographic unless HE is added (Khan et al., 2023). Static split points remain common, even though multiple papers argue for adaptive or heterogeneous cut layers (Li et al., 2024, Sun et al., 2024). Sequential SL creates both latency bottlenecks and fairness pathologies (Madaan et al., 2021).

The trajectory suggested by the supplied work is therefore toward hybrid, adaptive, and security-aware SL: dynamic cut selection, server and radio resource co-optimization, federated averaging over split sub-models, encrypted or binarized feature transmission, and server-side defenses against poisoning and backdoor behavior. This suggests that SL is best understood not as a finished privacy-preserving primitive, but as a systems framework whose behavior depends decisively on where the model is split, what is transmitted at the cut, and how training is orchestrated across clients and servers.