Heterogeneous U-Shaped Split Learning

• The paper demonstrates that U-shaped split learning offloads heavy computation to servers while keeping raw data on clients.
• It adapts model partitioning based on client heterogeneity, optimizing for differences in compute, memory, and bandwidth.
• Practical implementations in vehicular networks, medical imaging, and IoT highlight its effectiveness in balancing privacy and efficiency.

Searching arXiv for recent and foundational papers on heterogeneous U-shaped split learning and closely related variants. I’ll look for arXiv records matching “U-shaped split learning heterogeneity”, “UFSL”, and “split federated learning”. Heterogeneous U-shaped split learning is a distributed learning topology in which each client retains an input-side subnetwork and an output-side subnetwork, while a server or edge server executes the intermediate layers. The arrangement is “U-shaped” because computation flows client $\rightarrow$ server $\rightarrow$ client in the forward pass and client $\rightarrow$ server $\rightarrow$ client in the backward pass, so that raw data and labels remain on the client while substantial computation is offloaded. In the heterogeneous setting, clients may differ in local dataset size, input and label distributions, compute capacity, memory budget, bandwidth, mobility, or availability, and the split configuration is therefore adapted across clients rather than fixed globally (Yu et al., 2024, Lin et al., 10 Jun 2025, Lin et al., 5 May 2025).

1. Architectural model and system variants

The canonical U-shaped design partitions a model into three parts. A client-side head performs early feature extraction, a server-side middle performs the heavy shared computation, and a client-side tail completes the task-specific prediction and local loss evaluation. In vehicular edge networks, this appears as part $a$ on the vehicular user, part $b$ on the edge server, and part $c$ back on the vehicular user; in medical imaging, RoS-FL places the head and tail on institutions while the computation server hosts the body; in IoT settings, EUSFL places $F_1$ and $F_3$ on devices and $F_2$ on the edge (Yu et al., 2024, Yang et al., 2022, Tang et al., 2023).

The supporting infrastructure varies across implementations. Some systems use an edge server plus a separate aggregation entity, such as a weight averaging server in U-SFL or an aggregation server in RoS-FL, so that client-side parameters are periodically averaged while the server-side middle is updated centrally. This separation is used to preserve the split topology while retaining FL-style parallelism. In U-shaped medical image networks, paired encoder–decoder skip connections are deliberately co-located on the client, so that skip features are routed locally rather than transmitted, avoiding both additional bandwidth and privacy leakage through high-fidelity encoder features (Yu et al., 2024, Yang et al., 2022).

A related distinction concerns whether the middle computation is merely offloaded or also secured cryptographically. SplitHappens extends U-shaped split learning to a two-server setting with Function Secret Sharing, so that the server-side middle is evaluated over masked inputs and function shares, while the client still performs the first and final layers locally and keeps labels private (Khan et al., 14 Jul 2025).

2. Training protocol and parameter flow

For a local sample $\rightarrow$0 with label $\rightarrow$1 on client $\rightarrow$2, the forward computation in vehicular U-SFL is written as

$\rightarrow$3

after which the client computes a local task loss $\rightarrow$4. Backpropagation then crosses both cut layers: the client updates $\rightarrow$5, the server updates $\rightarrow$6, and the client receives the boundary gradient needed to update $\rightarrow$7. In the federated version, client-side parameters are periodically averaged as

$\rightarrow$8

while the server middle remains centrally optimized (Yu et al., 2024).

The same structural principle appears in more abstract form in SplitHappens. The client computes activations $\rightarrow$9, the server computes the middle output $\rightarrow$0, and the client resumes with the final layer and loss. The boundary chain rule is explicitly

$\rightarrow$1

This formulation makes clear that U-shaped split learning changes the location of loss computation, not the end-to-end gradient semantics, provided that the boundary messages are exact (Khan et al., 14 Jul 2025).

In U-shaped medical image networks, the end-to-end mapping is written as

$\rightarrow$2

with parallel execution across clients in each communication round. The client computes $\rightarrow$3, the computation server executes $\rightarrow$4, the client completes $\rightarrow$5, and the respective head, body, and tail parameters are aggregated by weighted averaging. The Dynamic Weight Correction Strategy then corrects the round-$\rightarrow$6 aggregate relative to the previous round through

$\rightarrow$7

followed by

$\rightarrow$8

This design specifically targets instability induced by heterogeneous local data (Yang et al., 2022).

3. Heterogeneity, split adaptation, and resource control

Heterogeneity enters U-shaped split learning through both systems variables and statistical variables. In vehicular U-SFL, vehicular users differ in CPU frequency $\rightarrow$9, FLOPs per cycle $\rightarrow$0, energy budgets $\rightarrow$1, stay time $\rightarrow$2, and channel conditions determined by distance $\rightarrow$3. The joint optimization problem selects per-client split points $\rightarrow$4, uplink bandwidth $\rightarrow$5, and edge compute allocation $\rightarrow$6 to minimize a weighted combination of energy and worst-case latency:

$\rightarrow$7

subject to aggregate bandwidth, aggregate edge compute, split-ordering, stay-time, and energy-budget constraints. The inclusion of $\rightarrow$8 is explicitly intended to alleviate straggler effects across heterogeneous clients (Yu et al., 2024).

HASFL formalizes a closely related problem in split federated learning over edge systems. Its convergence bound contains a variance term that decreases with larger per-client batch sizes $\rightarrow$9 and a drift penalty that grows with the maximum retained client-side depth $a$0 when aggregation is periodic. The reported upper bound is

$a$1

On that basis, HASFL jointly optimizes batch sizes and model splitting to minimize training time to a target accuracy under memory and latency constraints (Lin et al., 10 Jun 2025).

Other systems focus on scheduling and memory feasibility. EdgeSplit optimizes a per-device partition point and bandwidth allocation by minimizing the maximum round time across synchronous clients; its alternating algorithm exploits a closed-form best partition for fixed bandwidth and a convex bandwidth subproblem for fixed partitions (Zhang et al., 2024). Makespan minimization studies further decompose each client’s work into five tasks—client forward, helper forward, client tail forward/loss/backward, helper backward, and client backward—and show that assignment and helper scheduling are already hard under cardinality or memory-capacity constraints. For the heterogeneous task setting, the paper proves that, unless $a$2, the problem cannot admit a polynomial-time approximation algorithm for any approximation factor, and therefore proposes the EquiDistributed heuristic to balance helper-side workload under capacities (Ganian et al., 6 Feb 2026).

Heterogeneity is also addressed at the parameterization level. HSplitLoRA adapts both LoRA ranks and the split point according to client memory budgets and computes a resource-normalized importance score

$a$3

then uses a noise-free aggregation rule

$a$4

so that heterogeneous adapter ranks can be merged without aggregation noise (Lin et al., 5 May 2025).

4. Privacy model, attack surface, and defenses

The central privacy property of U-shaped split learning is architectural: raw inputs and labels remain on the client because the task head and loss are client-side. Vehicular U-SFL states that raw inputs and labels are never transmitted, only intermediate features and gradients; RoS-FL extends the protection target to inputs $a$5, labels $a$6, outputs $a$7, and the full set of model parameters by ensuring that no single party sees the entire model; EUSFL similarly keeps labels and raw data on IoT clients and uses the edge only for the middle split (Yu et al., 2024, Yang et al., 2022, Tang et al., 2023).

This architecture does not eliminate leakage from intermediate representations. SDAR shows that an honest-but-curious server can infer both private features and labels in U-shaped split learning by learning decodable simulators with adversarial regularization. On CIFAR-10 at the deep split level of $a$8, SDAR reports private feature reconstruction with less than $a$9 mean squared error in U-shaped SL and a label inference accuracy of over $b$0, while the paper states that existing passive attacks fail to produce non-trivial results in that regime (Zhu et al., 2023).

Several defenses have therefore been proposed. SplitHappens combines U-shaped split learning with Function Secret Sharing, masking both smashed activations and boundary gradients as

$b$1

and evaluates server-side functions through additive shares so that, under the non-collusion assumption, a single corrupted server does not learn the original function or the true client representation (Khan et al., 14 Jul 2025). KD-UFSL instead protects the “middle” statistically through local Gaussian perturbation of raw inputs and k-anonymous microaggregation of smashed data; the paper reports that this can increase the mean squared error between actual and reconstructed images by up to $b$2 in some cases and decrease structural similarity by up to $b$3 on four benchmarking datasets while preserving global-model utility (Zaland et al., 19 Feb 2026). EUSFL focuses on label privacy by perturbing one-hot labels with Laplace noise through LabelDP before local loss computation, thereby targeting reconstruction attacks that exploit label information (Tang et al., 2023).

A common misconception is that keeping labels on the client is sufficient for privacy. The attack and defense literature indicates a narrower conclusion: U-shaped topology prevents direct label transmission, but the intermediate activations, returned server outputs, and boundary gradients remain meaningful leakage channels unless additional protections are imposed (Zhu et al., 2023, Khan et al., 14 Jul 2025, Zaland et al., 19 Feb 2026).

5. Theory and reported performance

The main convergence result directly applicable to U-shaped split learning comes from the analysis of sequential split learning on heterogeneous data. The paper states that the SSL convergence theorems carry over to U-SL if the split pipeline yields unbiased gradients with the same $b$4 bound and end-to-end $b$5-smoothness. Under those assumptions, the strongly convex rate has dominant term $b$6, and the general convex and non-convex rates have dominant term $b$7; the paper further argues that SSL can outperform FedAvg in extremely heterogeneous settings because its heterogeneity-dominated terms improve by a factor $b$8 relative to FedAvg (Li et al., 2023).

Representative empirical results span vehicular networking, medical imaging, IoT, and cryptographically protected training.

Domain and system	Configuration	Reported outcome
Vehicular edge U-SFL (Yu et al., 2024)	ResNet-18 on Caltech-101 and CIFAR-10, SAE and DRL	Comparable classification performance to SL; average latency converges to $b$9 s after $c$0 episodes; average energy stabilizes in $c$1–$c$2 J per task per device
RoS-FL for medical imaging (Yang et al., 2022)	ACDC segmentation and Mayo LDCT restoration	Segmentation: DSC $c$3, HD95 $c$4, ASD $c$5, JC $c$6; Restoration: PSNR $c$7, SSIM $c$8
EUSFL for IoT (Tang et al., 2023)	CIFAR-10 with VGG-16, Setup 2	EUSFL total time $c$9 s versus FL $F_1$0 s and USFL $F_1$1 s, with near-identical accuracy around $F_1$2 at $F_1$3 epochs
SplitHappens (Khan et al., 14 Jul 2025)	MNIST, CIFAR-10, Fashion-MNIST with FSS-secured U-shaped SL	MNIST Network 1: $F_1$4 min and $F_1$5 MB versus AriaNN $F_1$6 min and $F_1$7 MB; prior accuracy matched or slightly exceeded relative to MSnH on some settings

These results show that heterogeneous U-shaped split learning is not confined to a single optimization regime. Some systems prioritize communication reduction through semantic compression, some prioritize robustness to non-IID medical data through drift correction, some prioritize cryptographic protection of the server-side middle, and some prioritize edge-assisted acceleration. This suggests that the empirical behavior of U-shaped systems is dominated less by the abstract topology than by the particular co-design of partitioning, aggregation, and privacy mechanism.

6. Limitations, assumptions, and open directions

Across the literature, privacy claims are often architectural rather than formal. Vehicular U-SFL explicitly notes that no formal privacy metric such as differential privacy is provided; RoS-FL acknowledges that intermediate activations and gradients remain susceptible to inversion; SplitHappens assumes a semi-honest adversary that can corrupt at most one of two non-colluding servers; KD-UFSL does not privatize gradients and leaves practical sensitivity calibration to future work (Yu et al., 2024, Yang et al., 2022, Khan et al., 14 Jul 2025, Zaland et al., 19 Feb 2026).

System assumptions are also restrictive. Several frameworks assume a single edge server and a single aggregation server, synchronous rounds, or static client sets. U-SFL in vehicular networks models mobility via average rates over stay time and considers a single ES and WAS; HASFL is derived for synchronous split federated learning with periodic aggregation; makespan optimization papers assume fixed task profiles and helper capacities within each scheduling instance (Yu et al., 2024, Lin et al., 10 Jun 2025, Ganian et al., 6 Feb 2026). In large-scale deployments, these assumptions interact with client churn, variable channels, and partial participation.

Model-selection sensitivity remains a persistent issue. Later split combinations may improve accuracy but increase communication; deeper heads can improve privacy but raise client compute; semantic compression can substantially reduce transmission yet induce a minor accuracy drop; fixed-point secure training can widen the gap to plaintext performance on complex data such as CIFAR-10 (Yu et al., 2024, Zaland et al., 19 Feb 2026, Khan et al., 14 Jul 2025). HSplitLoRA adds another dimension by showing that heterogeneous split learning for LLMs is also sensitive to which weights are deemed important, how LoRA ranks are assigned, and where the model is cut (Lin et al., 5 May 2025).

The stated research directions are correspondingly broad. They include dynamic split selection, complex topologies such as multi-ES systems, tighter privacy analysis, defenses against feature and gradient inversion, adaptive semantic metrics beyond the mutual-information proxy, asynchronous aggregation, staleness-aware correction, malicious-security extensions beyond the semi-honest setting, and broader multi-client heterogeneous evaluation for cryptographically protected U-shaped protocols (Yu et al., 2024, Yang et al., 2022, Khan et al., 14 Jul 2025, Lin et al., 10 Jun 2025). A plausible implication is that heterogeneous U-shaped split learning should be understood not as a single algorithmic template but as a family of privacy-sensitive distributed systems in which partitioning, optimization, scheduling, and security must be co-designed for the target domain.