Multi-hop Split Learning (MSL)

• Multi-hop Split Learning (MSL) is a distributed training method that partitions a neural network into several sequential sub-models executed over multiple nodes.
• It leverages diverse formulations—including SFC-based, pipelined, and multi-stage protocols—to optimize communication, synchronization, and latency during both forward and backward passes.
• MSL reduces memory footprints and computation bottlenecks while enhancing privacy by integrating adaptive model splitting and privacy-preserving techniques.

Searching arXiv for papers on multi-hop split learning, pipelined split learning, and SFC-based split learning. Multi-hop Split Learning (MSL) is a form of split learning in which a global neural network is partitioned into multiple sequential sub-models and executed across a client, a server, and one or more intermediate nodes, so that forward activations and backward gradients traverse a multi-hop path rather than a single client–server cut. In the Service Function Chaining (SFC)-based formulation, split sub-models are interpreted as service functions and their composition forms a service chain representing the global model; in parallel and pipelined formulations, the same basic idea appears as a multistage training pipeline across multiple compute nodes (Hara et al., 12 Sep 2025, Tirana et al., 2024, Wei et al., 7 May 2025, Hara et al., 28 Apr 2026). Across these strands, MSL is studied as a joint systems problem involving model partitioning, placement, communication, synchronization, routing, latency, memory footprint, and, in some variants, privacy against eavesdroppers (Wei et al., 9 Jul 2025).

1. Formal model and notation

A standard formalization considers a client, a server, and $H-1$ intermediate nodes, each hosting one split sub-model. The global model $F$ with $L$ layers is written as

$F = f^1 \circ f^2 \circ \dots \circ f^L,$

and partitioned into $H$ sub-models

$F_h = f^{l_{h-1}+1} \circ \dots \circ f^{l_h},$

with $\sum_{h=1}^H |\mathrm{layers}(F_h)| = L$. If $x \in \mathbb{R}^d$ is the input, $y \in \mathbb{R}^C$ the ground-truth label, $W_h$ the parameters of sub-model $F$0, $F$1 the activation at the output of $F$2, and $F$3 the network output, then forward propagation is

$F$4

with global loss

$F$5

Backward propagation follows the chain rule:

$F$6

and

$F$7

Under this model, each hop $F$8 receives $F$9, computes $L$0 in the forward pass, then receives $L$1 in the backward pass, computes $L$2, and passes $L$3 upstream (Hara et al., 12 Sep 2025).

Related formulations use different symbols but the same sequential dependency structure. MP-SL describes a deep network split into $L$4 model-segments

$L$5

where the data owner hosts both $L$6 and $L$7, and intermediate compute nodes host $L$8; activations at cut layer $L$9 are denoted $F = f^1 \circ f^2 \circ \dots \circ f^L,$0 and gradients $F = f^1 \circ f^2 \circ \dots \circ f^L,$1 (Tirana et al., 2024). A pipelined edge-network formulation instead assumes an $F = f^1 \circ f^2 \circ \dots \circ f^L,$2-layer model split into up to $F = f^1 \circ f^2 \circ \dots \circ f^L,$3 submodels with binary cut variables $F = f^1 \circ f^2 \circ \dots \circ f^L,$4 and placement variables $F = f^1 \circ f^2 \circ \dots \circ f^L,$5, together with a micro-batch size $F = f^1 \circ f^2 \circ \dots \circ f^L,$6 (Wei et al., 7 May 2025). This suggests that “multi-hop” is not tied to a single implementation pattern: it is the broader condition that training-state transfer is distributed across more than one inter-node boundary.

2. SFC-based architecture and neural service functions

In the SFC-based architecture, each sub-model $F = f^1 \circ f^2 \circ \dots \circ f^L,$7 is encapsulated in a Neural Service Function (NSF), implemented as a Linux user-space process acting as a transparent TCP proxy over SRv6. The stated components are SRv6 encapsulation/decapsulation through an eBPF-based SFC proxy, TPROXY rules for intercepting end-to-end TCP flows, buffering logic to reassemble byte streams from packets, and libtorch sub-model execution (Hara et al., 12 Sep 2025).

The forward packet flow is defined procedurally. The client sends an original TCP SYN to the SR source node, which encapsulates it in an outer IPv6+SRH header whose segment list contains the $F = f^1 \circ f^2 \circ \dots \circ f^L,$8 NSFs and the server. At each NSF, eBPF at TC ingress matches the local SID, decapsulates the outer header, stores the $F = f^1 \circ f^2 \circ \dots \circ f^L,$9-tuple $H$0 (outer IPv6+SRH) mapping in a BPF map, and intercepts the flow via TPROXY into NSF user space. The NSF buffers TCP segments, reassembles the byte stream into a tensor $H$1, runs $H$2, retrieves the stored SRH from the BPF map keyed by the new $H$3-tuple, re-encapsulates $H$4 into SRv6 using the outer IPv6/SRH, and writes TCP segments to the next NSF. The backward pass reverses the segment list in the SRH; each NSF performs the eBPF decapsulation, TProxy interception, gradient computation, and forwarding of $H$5 upstream (Hara et al., 12 Sep 2025).

This architecture is explicitly designed to maintain compatibility with existing applications while enabling dynamic routing of smashed data. In the later optimization framework, the same SFC intuition is elevated into an augmented graph $H$6 by introducing one imaginary node $H$7 per sub-model and linking each $H$8 to its candidate host nodes $H$9; the ordered execution $F_h = f^{l_{h-1}+1} \circ \dots \circ f^{l_h},$0 is treated as the service chain itself (Hara et al., 28 Apr 2026). A plausible implication is that the SFC view turns MSL from a purely model-parallel training protocol into a network-native service orchestration problem.

3. Training protocols, communication, and pipelining

For SFC-based MSL, training is given per mini-batch $F_h = f^{l_{h-1}+1} \circ \dots \circ f^{l_h},$1. The forward pass initializes $F_h = f^{l_{h-1}+1} \circ \dots \circ f^{l_h},$2 and sequentially invokes

$F_h = f^{l_{h-1}+1} \circ \dots \circ f^{l_h},$3

At the server, the loss gradient is computed as $F_h = f^{l_{h-1}+1} \circ \dots \circ f^{l_h},$4. The backward pass then iterates from $F_h = f^{l_{h-1}+1} \circ \dots \circ f^{l_h},$5 to $F_h = f^{l_{h-1}+1} \circ \dots \circ f^{l_h},$6:

$F_h = f^{l_{h-1}+1} \circ \dots \circ f^{l_h},$7

The communication-cost model for forward activations of size $F_h = f^{l_{h-1}+1} \circ \dots \circ f^{l_h},$8 bytes is

$F_h = f^{l_{h-1}+1} \circ \dots \circ f^{l_h},$9

with backward gradients of the same order of magnitude. Per-hop computation time is denoted $\sum_{h=1}^H |\mathrm{layers}(F_h)| = L$0 and $\sum_{h=1}^H |\mathrm{layers}(F_h)| = L$1, and overall per-batch training latency is

$\sum_{h=1}^H |\mathrm{layers}(F_h)| = L$2

where $\sum_{h=1}^H |\mathrm{layers}(F_h)| = L$3 is the idle time an NSF spends waiting for upstream gradients (Hara et al., 12 Sep 2025).

A distinct line of work treats MSL as a pipeline. MP-SL uses asynchronous non-blocking send/receive threads so that while one mini-batch is computing on hop $\sum_{h=1}^H |\mathrm{layers}(F_h)| = L$4, the next batch can be transmitted from $\sum_{h=1}^H |\mathrm{layers}(F_h)| = L$5. After an initial fill-up phase of $\sum_{h=1}^H |\mathrm{layers}(F_h)| = L$6 forward steps, the pipeline runs full and each mini-batch completes in

$\sum_{h=1}^H |\mathrm{layers}(F_h)| = L$7

where

$\sum_{h=1}^H |\mathrm{layers}(F_h)| = L$8

Warm-up and unwind incur extra sums of per-hop forward and backward processing times (Tirana et al., 2024).

The edge-network pipelining framework makes the micro-batch explicit. A mini-batch of $\sum_{h=1}^H |\mathrm{layers}(F_h)| = L$9 samples is divided into micro-batches of size $x \in \mathbb{R}^d$0, with forward and backward passes overlapping across micro-batches. The total time to train one mini-batch is

$x \in \mathbb{R}^d$1

where $x \in \mathbb{R}^d$2 is the latency of the first micro-batch and $x \in \mathbb{R}^d$3 is the per-micro-batch pipeline latency, defined as the bottleneck over all compute and communication stages (Wei et al., 7 May 2025). Taken together, these formulations separate two major execution regimes in MSL: synchronous hop-by-hop training, where waiting time is prominent, and pipelined training, where bottleneck-stage latency and fill/unwind overhead become central.

4. Model splitting, placement, and chaining optimization

Once MSL is distributed across multiple hops, three coupled design decisions arise repeatedly: where to cut the model, where to place each sub-model, and how to route smashed data through the network. The SFC optimization work formulates these choices as ILPs for inference and training. With routing and placement variables

$x \in \mathbb{R}^d$4

and model-splitting variables

$x \in \mathbb{R}^d$5

the objective minimizes total latency

$x \in \mathbb{R}^d$6

subject to flow conservation, connectivity between successive subpaths, exact layer coverage, nonempty submodels, contiguity of layers, ordering across submodels, and storage and memory capacity constraints (Hara et al., 28 Apr 2026).

The same paper proposes a Block Coordinate Descent heuristic that alternates between optimizing the split and optimizing routing/placement. Model splitting is solved via a $x \in \mathbb{R}^d$7-sequence segmentation dynamic program with

$x \in \mathbb{R}^d$8

in $x \in \mathbb{R}^d$9, while placement and chaining are solved via a depth-first tour search on a modified augmented graph. The reported total complexity is approximately

$y \in \mathbb{R}^C$0

and global optimality is not guaranteed because the procedure is a non-convex BCD on an ILP (Hara et al., 28 Apr 2026).

A graph-theoretic optimization appears in the pipelined edge-network setting. There, for fixed $y \in \mathbb{R}^C$1, the model splitting and placement subproblem maps to minimizing a weighted sum of a bottleneck cost and a linear cost:

$y \in \mathbb{R}^C$2

with $y \in \mathbb{R}^C$3. A bottleneck-aware shortest-path algorithm enumerates candidate bottleneck edges, solves a shortest-path problem on the corresponding subgraph, and returns the globally optimal solution of $y \in \mathbb{R}^C$4 by Minoux’s min-max/min-sum theory. Its time complexity is

$y \in \mathbb{R}^C$5

for graph sizes $y \in \mathbb{R}^C$6 and $y \in \mathbb{R}^C$7 (Wei et al., 7 May 2025).

MP-SL adds yet another optimization layer: an ILP in which decision variables $y \in \mathbb{R}^C$8 indicate whether layer $y \in \mathbb{R}^C$9 is assigned to compute node $W_h$0, under memory and contiguity constraints, so as to minimize the maximum per-hop processing time (Tirana et al., 2024). This suggests that optimization in MSL is not monolithic. Depending on the system model, the dominant objective may be bottleneck equalization, end-to-end latency minimization, memory balancing, or network-path selection.

5. Empirical behavior and system-level trade-offs

The SFC-based architecture is evaluated on hardware consisting of an Intel i9-10980XE with 36 cores, an NVIDIA RTX A6000, and 128 GB RAM, using a Mininet emulation with NSFs at $W_h$1 and $W_h$2, an SR source at $W_h$3, a server at $W_h$4, and 1 Gbps links with jumbo frames enabled. The model is a ResNet-101-$W_h$5 width network with $W_h$6 layers split into $W_h$7 sub-models $W_h$8, trained on CIFAR-100 with SGD and learning-rate schedule $W_h$9, momentum $F$00. For inference with mini-batch $F$01, the proposed SFC-MSI records $F$02 ms/round versus $F$03 ms/round for a baseline TCP-proxy MSI, supporting the claim that SRv6 overhead is much smaller than compute time and is effective for real-time $F$04. For training with $F$05, per-round latency is approximately $F$06 ms for both SFC-MSL and the baseline; total training for $F$07 epochs is approximately $F$08 s with final test accuracy $F$09. Under simulated 1 Mbps congestion on one link, the baseline latency spikes from $F$10 ms in inference and $F$11 ms in training until congestion is removed, whereas SFC-MSL detects congestion with a $F$12 s window, installs a detour, and recovers to baseline performance within approximately $F$13 s (Hara et al., 12 Sep 2025).

MP-SL emphasizes memory footprint and heterogeneous-device robustness. On a Raspberry Pi 4, full-model FL uses approximately $F$14 of RAM for VGG-19, while MP-SL reduces data-owner memory by up to $F$15. With two compute nodes, MP-SL improves over one-hop split learning by $F$16 for ResNet and $F$17 for VGG when split points are optimizer-chosen rather than naive. Under data-owner heterogeneity with a mix of Pi 4 and Pi 3 devices, MP-SL’s epoch time varies by less than $F$18, whereas SplitNN varies by up to $F$19. Pipeline steady-state throughput improves roughly by a factor of approximately $F$20 if hops are perfectly balanced, and accuracy is unchanged because the functional DNN is identical to the one trained centrally (Tirana et al., 2024).

The pipelined edge-network study reports that pipeline parallelism cuts total latency by $F$21 to $F$22 over no-pipeline split learning with identical final accuracy. It also reports that optimal model splitting and placement plus batching differs from a suboptimal BCD by less than $F$23 in latency, while BCD runs $F$24 faster when $F$25 (Wei et al., 7 May 2025). In the SFC optimization study, ILP-optimal inference and training latencies indicate a consistent trade-off: for small batch sizes $F$26, the optimum is at $F$27, whereas for larger $F$28 or training, the optimum shifts to $F$29; beyond $F$30, added communication delays dominate and total latency rises. BCD remains within $F$31 of ILP across all $F$32 and is roughly two orders of magnitude faster than ILP in the larger network instances evaluated (Hara et al., 28 Apr 2026).

A common misconception is that adding more hops necessarily improves performance. The reported results point to a more constrained conclusion: multi-hop partitioning can reduce memory pressure and computation bottlenecks, but beyond the system-dependent “sweet spot,” added smashed-data transmission and synchronization costs dominate (Hara et al., 28 Apr 2026).

6. Privacy, limitations, and open directions

Privacy is not inherent to MSL, but it is an active design axis. In MP-SL, multihop splitting with at least two cuts is described as raising the bar against label-inference attacks since no single party sees end-to-end activations, while existing defenses such as NoPeek are said to complement MP-SL (Tirana et al., 2024). A more explicit privacy model appears in deceptive signal-assisted private multi-hop split learning, where a global model $F$33 is partitioned into $F$34 sequential sub-models $F$35, hosted on selected edge devices and the server, while subsets of nontraining devices concurrently emit jamming signals whenever genuine sub-model outputs or gradients are transmitted. The optimization objective is to minimize expected information leakage to eavesdroppers subject to delay and energy constraints, jointly over the training-device set $F$36, deceptive-jammer sets $F$37, sub-model splits $F$38, and power levels $F$39 (Wei et al., 9 Jul 2025).

Because eavesdropper positions and monitoring probabilities are unknown, that work adopts a model-free reinforcement-learning solution: Soft Actor-Critic augmented with an Intrinsic Curiosity Module and a Cross-Attention mechanism. The state is

$F$40

the action is

$F$41

and the reward is

$F$42

Simulation results report convergence up to $F$43 faster and information leakage reduced by up to $F$44 versus traditional SAC, with additional reductions of up to $F$45 versus SAC when the number of eavesdroppers is $F$46 (Wei et al., 9 Jul 2025). This suggests that privacy-preserving MSL can be framed not only through cryptographic or statistical protections but also through device-role assignment, physical-layer signaling, and adaptive model partitioning.

Several limitations recur across the literature. In the SFC-based implementation, training overhead is dominated by inter-hop waiting time as $F$47 grows, motivating asynchronous variants and stale-gradient MSL (Hara et al., 12 Sep 2025). In pipelined systems, initial warm-up and final unwind remain unresolved overheads (Tirana et al., 2024). More hops increase point-to-point transfers even when communication overlaps with compute (Wei et al., 7 May 2025). Joint optimization formulations remain computationally difficult at scale, which is why heuristic methods such as BCD are used despite the absence of global optimality guarantees (Hara et al., 28 Apr 2026).

Future directions are correspondingly consistent across papers: compression of smashed data to reduce communication cost; asynchronous or pipelined MSL to better utilize compute resources; adaptive splitting under dynamic node availability and workload shifts; incorporation of secure aggregation or differential privacy; multi-path chaining to load-balance smashed-data traffic; privacy-aware split-point selection; and online adaptation that combines decomposition-based optimization with reinforcement learning under varying batch sizes, link states, and node loads (Hara et al., 12 Sep 2025, Tirana et al., 2024, Wei et al., 7 May 2025, Hara et al., 28 Apr 2026). Collectively, these directions indicate that MSL has evolved from a simple extension of two-party split learning into a broader research area at the intersection of distributed training, network service chaining, systems optimization, and privacy engineering.