Split-LLM: Distributed Large Language Models

• Split-LLM is a distributed paradigm that partitions LLMs across client, edge, and cloud to keep raw inputs local while processing intermediate activations remotely.
• It supports flexible partitioning (e.g., Head–Tail and Head–Body–Tail splits) to optimize memory, computation, and network communication under varying resource and security demands.
• The design incorporates fine-tuning techniques, communication compression, and robust privacy defenses to efficiently balance performance with protection against data leakage.

Split-LLM denotes a family of split-learning and split-inference schemes for LLMs in which a full model is partitioned across trust or resource boundaries—most commonly client/server, user/edge/cloud, or client/edge/cloud—so that raw inputs remain local while only intermediate activations, cut-layer gradients, or compact adapter updates are exchanged. In the standard formulation, a client-side submodel $f_c(\cdot;\theta_c)$ and a server-side submodel $f_s(\cdot;\theta_s)$ satisfy

$\hat y = f_s\bigl(f_c(x;\theta_c);\theta_s\bigr),$

with forward propagation sending smashed activations to the server and backward propagation returning activation gradients to the client during fine-tuning (Liu et al., 27 Apr 2026, Gu et al., 5 Aug 2025). The label “SplitLLM” also appears in the titles of particular systems for hierarchical fine-tuning and collaborative inference, but across this literature it names a broader design space spanning privacy-aware WAN inference, split federated learning, heterogeneous edge deployment, and systematic attack-defense evaluation (Zhang et al., 23 Jan 2025, Mudvari et al., 2024, Cunningham, 18 Feb 2026).

1. Architectural forms and partitioning patterns

The basic Split-LLM pipeline is a cut-layer protocol. A client computes $z_k=f_C^{(k)}(x)$, transmits $z_k$, the server completes the forward pass to obtain $\hat y=f_S(z_k)$, computes the loss, and returns $g_z=\partial \ell/\partial z_k$ so that the client can update its own parameters. This general pattern has been codified as the end-to-end split-LLM fine-tuning pipeline in the survey literature and as the problem formulation used by VFLAIR-LLM for collaborative LLM adaptation under split learning (Liu et al., 27 Apr 2026, Gu et al., 5 Aug 2025).

Two canonical partition settings recur. In the Head–Tail (HT) split, the client holds the embedding layer plus the first $n_{\rm head}$ transformer blocks, while the server holds the remaining $n_{\rm tail}$ blocks plus the final head layer. In the Head–Body–Tail (HBT) split, the client keeps the embedding and first $n_{\rm head}$ blocks, the server executes the middle $f_s(\cdot;\theta_s)$0 blocks, and the client finishes with the last $f_s(\cdot;\theta_s)$1 blocks plus the head layer. Inference-oriented head–body–tail variants expose the same structure without cross-boundary gradients: the local head produces $f_s(\cdot;\theta_s)$2, the server-side body maps $f_s(\cdot;\theta_s)$3 to $f_s(\cdot;\theta_s)$4, and the local tail samples the next token from $f_s(\cdot;\theta_s)$5 (Gu et al., 5 Aug 2025, Gu et al., 12 Jun 2026).

Several systems specialize this template. The privacy-aware WAN inference system in “Privacy-Aware Split Inference with Speculative Decoding for LLMs over Wide-Area Networks” keeps both the token embedding and the LM head local, with the bulk of the model remote; for Mistral 7B with $f_s(\cdot;\theta_s)$6 and $f_s(\cdot;\theta_s)$7, layers $f_s(\cdot;\theta_s)$8–$f_s(\cdot;\theta_s)$9 are local, layers $\hat y = f_s\bigl(f_c(x;\theta_c);\theta_s\bigr),$0–$\hat y = f_s\bigl(f_c(x;\theta_c);\theta_s\bigr),$1 are remote, and layers $\hat y = f_s\bigl(f_c(x;\theta_c);\theta_s\bigr),$2–$\hat y = f_s\bigl(f_c(x;\theta_c);\theta_s\bigr),$3 are local again (Cunningham, 18 Feb 2026). FL-LLaMA uses three segments—client-side Blocks$\hat y = f_s\bigl(f_c(x;\theta_c);\theta_s\bigr),$4, server-side Blocks$\hat y = f_s\bigl(f_c(x;\theta_c);\theta_s\bigr),$5, and client-side Blocks$\hat y = f_s\bigl(f_c(x;\theta_c);\theta_s\bigr),$6—so that embedding, some early blocks, some late blocks, and the output head remain on the client (Zhang et al., 21 May 2025). Hierarchical SplitLLM over wireless networks deploys layer $\hat y = f_s\bigl(f_c(x;\theta_c);\theta_s\bigr),$7 on the user, layers $\hat y = f_s\bigl(f_c(x;\theta_c);\theta_s\bigr),$8 to $\hat y = f_s\bigl(f_c(x;\theta_c);\theta_s\bigr),$9 on the edge, and layers $z_k=f_C^{(k)}(x)$0 to $z_k=f_C^{(k)}(x)$1 on the cloud (Zhang et al., 23 Jan 2025). ELSA uses a related tripartite client–edge–client decomposition with a client-side “up” segment, an edge-side “middle,” and a client-side “down” segment, updating only LoRA adapters while freezing the backbone (Yang et al., 20 Jan 2026).

These variants make the split point a multi-objective control variable. The same cut determines client memory footprint, server workload, communication tensor shape, gradient path, and the attack surface exposed by smashed activations. This suggests that “Split-LLM” is less a single architecture than a parameterized family of placements whose operating point depends on hardware, network, and threat assumptions.

2. Split inference and autoregressive decoding over WANs

Split inference is structurally harder than feed-forward model serving because autoregressive decoding turns network round trips into a first-order bottleneck. The WAN-oriented system in (Cunningham, 18 Feb 2026) addresses this with an asymmetric split and with the first application of lookahead decoding to split inference over WANs. Its lookahead procedure batches multiple token predictions into one round trip by collecting $z_k=f_C^{(k)}(x)$2-gram candidates from Jacobi trajectories and verifying them locally under greedy argmax. The formal acceptance rule commits the first $z_k=f_C^{(k)}(x)$3 speculative tokens for which each local argmax matches the candidate sequence, and the paper proves a Greedy Lookahead Equivalence lemma: under greedy argmax decoding, lookahead commits exactly the same token sequence as standard sequential decoding. Empirically, all tokens matched across $z_k=f_C^{(k)}(x)$4 prompts and $z_k=f_C^{(k)}(x)$5 tokens each, with zero quality degradation. The same study reports that $z_k=f_C^{(k)}(x)$6-gram speculation accepts $z_k=f_C^{(k)}(x)$7–$z_k=f_C^{(k)}(x)$8 tokens per decoding step on average, with a peak of $z_k=f_C^{(k)}(x)$9 observed on code, and acceptance rates remain nearly identical across Mistral 7B and Mistral NeMo 12B (Cunningham, 18 Feb 2026).

The measured WAN latency decomposition on Mistral 7B over an $z_k$0 ms link was dominated by network RTT: $z_k$1 ms ($z_k$2) for RTT, $z_k$3 ms ($z_k$4) for local GPU compute, $z_k$5 ms ($z_k$6) for cloud GPU compute, and $z_k$7 ms ($z_k$8) for serialization, for a fixed overhead of approximately $z_k$9 ms. With

$\hat y=f_S(z_k)$0

the paper projects $\hat y=f_S(z_k)$1 tok/s for lookahead at $\hat y=f_S(z_k)$2 ms RTT when $\hat y=f_S(z_k)$3, compared with $\hat y=f_S(z_k)$4 tok/s at $\hat y=f_S(z_k)$5 ms RTT. Over an $\hat y=f_S(z_k)$6 ms WAN link, the implemented system achieves $\hat y=f_S(z_k)$7–$\hat y=f_S(z_k)$8 tok/s on Mistral 7B and $\hat y=f_S(z_k)$9–$g_z=\partial \ell/\partial z_k$0 tok/s on Mistral NeMo 12B; the 12B configuration uses only $g_z=\partial \ell/\partial z_k$1 GB local VRAM and still matches 7B throughput because RTT dominates. Local-only Mistral 7B inference on an RTX 3090 runs at approximately $g_z=\partial \ell/\partial z_k$2 tok/s, so split lookahead serves at $g_z=\partial \ell/\partial z_k$3–$g_z=\partial \ell/\partial z_k$4 of local speed while preserving the property that raw tokens never leave the trusted device (Cunningham, 18 Feb 2026).

Other inference work treats split placement as an optimization problem. “SplitLLM: Collaborative Inference of LLMs for Model Placement and Throughput Optimization” formulates layer assignment under a latency SLA and solves it with a pseudo-polynomial dynamic program, offloading $g_z=\partial \ell/\partial z_k$5 of server FLOPs on a base Transformer, $g_z=\partial \ell/\partial z_k$6 on BERT-Base, and $g_z=\partial \ell/\partial z_k$7 on a GPT-2-like model, while reporting roughly $g_z=\partial \ell/\partial z_k$8 reduction in server workload and $g_z=\partial \ell/\partial z_k$9 percent improvement over a greedy method (Mudvari et al., 2024). In wireless inference, the model-based reinforcement learning approach of (Chen et al., 2024) adapts the split point $n_{\rm head}$0 online using PPO and a reward surrogate; the surrogate reduces training duration from more than $n_{\rm head}$1 days to $n_{\rm head}$2 minutes with less than $n_{\rm head}$3 reward loss, and the learned policy moves the split deeper as channel noise increases.

3. Fine-tuning regimes: PEFT, heterogeneity, and hierarchy

The earliest split-LLM fine-tuning systems center on parameter-efficient fine-tuning. SplitLoRA partitions a pre-trained LLM into client and server segments, freezes backbone weights, inserts LoRA adapters, and combines split learning with split federated learning. For GPT-2 Small ($n_{\rm head}$4 M) and GPT-2 Medium ($n_{\rm head}$5 M), both split after layer $n_{\rm head}$6, SplitLoRA reports that perplexity matches centralized LoRA within $n_{\rm head}$7PPL $n_{\rm head}$8, converges about $n_{\rm head}$9 faster than FedLoRA and about $n_{\rm tail}$0 faster than centralized LoRA on GPT-2 Small, exchanges about $n_{\rm tail}$1 K floats per round versus about $n_{\rm tail}$2 K in FedLoRA, and trains only $n_{\rm tail}$3 M LoRA parameters per client on GPT-2 Small with $n_{\rm tail}$4, versus $n_{\rm tail}$5 M in FedLoRA (Lin et al., 2024).

Heterogeneous-device extensions replace static cuts and uniform LoRA ranks with adaptive allocation. HSplitLoRA ranks candidate weights with the resource-normalized gradient-weight product

$n_{\rm tail}$6

then jointly chooses split point and per-weight rank under per-client budgets, and aggregates heterogeneous low-rank updates with a “noise-free” concatenate-then-multiply rule. In the heterogeneous setting, it preserves near-homogeneous performance with PPL drop $n_{\rm tail}$7, while BLEU remains approximately $n_{\rm tail}$8 versus $n_{\rm tail}$9 for SplitLoRA (Lin et al., 5 May 2025). SplitFrozen takes a different route: the device-side model is frozen, each device performs only forward propagation through its local layers, and the server alone fine-tunes LoRA on the upper layers. On GPT-2 under highly imbalanced data, SplitFrozen reports average non-IID accuracy gains of $n_{\rm head}$0 over FedLoRA and SplitLoRA, reduces device MFLOPs from $n_{\rm head}$1 in FedLoRA and $n_{\rm head}$2 in SplitLoRA to $n_{\rm head}$3, and cuts total training time from about $n_{\rm head}$4 s to $n_{\rm head}$5 s (Ma et al., 23 Mar 2025).

Hierarchical and memory-aware split federated learning generalize these designs to multi-tier systems. “SplitLLM: Hierarchical Split Learning for LLM over Wireless Network” divides a frozen LLM and its LoRA adapters across users, edge servers, and a cloud server, updates only adapters locally, and aggregates them with FedAvg at the cloud; in MRPC and CIFAR100 experiments it reports peak memory reduction of up to $n_{\rm head}$6 versus vanilla federated learning (Zhang et al., 23 Jan 2025). “Memory-Efficient Split Federated Learning for LLM Fine-Tuning on Heterogeneous Mobile Devices” instead keeps one full LLM on the server, assigns each client a lower-layer prefix according to its capacity, and fine-tunes corresponding server-side LoRA modules sequentially. Its reported server memory drops from $n_{\rm head}$7 MB in the baseline SFL design to $n_{\rm head}$8 MB, a reduction of approximately $n_{\rm head}$9, while final accuracy and F1 remain $f_s(\cdot;\theta_s)$00 and $f_s(\cdot;\theta_s)$01 and convergence time improves from $f_s(\cdot;\theta_s)$02 s to $f_s(\cdot;\theta_s)$03 s (Chen et al., 3 Jun 2025).

Across these systems, the split point is entangled with PEFT scope. A shallow client cut reduces on-device compute, but it also changes the number and placement of LoRA modules, the dimensionality of exchanged states, and the extent to which non-IID effects are absorbed locally versus centrally. This suggests that split-LLM fine-tuning is best understood as joint optimization over partition depth, trainable parameter budget, and synchronization structure rather than as mere model slicing.

4. Communication compression, scheduling, and throughput engineering

Communication and scheduling become dominant once split training spans many devices or weak links. Split Fine-Tuning (SFT) over wireless networks jointly optimizes split point $f_s(\cdot;\theta_s)$04, sparsity $f_s(\cdot;\theta_s)$05, quantization level $f_s(\cdot;\theta_s)$06, and bandwidth allocation $f_s(\cdot;\theta_s)$07 to minimize total fine-tuning delay under accuracy and memory constraints. Its compression pipeline applies top-$f_s(\cdot;\theta_s)$08 sparsification, stochastic quantization, and lossless encoding, and its two-timescale resource management uses an augmented Lagrangian method for large-timescale configuration and sequential quadratic programming for per-round bandwidth allocation. Reported results include $f_s(\cdot;\theta_s)$09 reduction in communication overhead, up to $f_s(\cdot;\theta_s)$10 reduction in fine-tuning delay, $f_s(\cdot;\theta_s)$11 compression from $f_s(\cdot;\theta_s)$12 sparsity with $f_s(\cdot;\theta_s)$13-bit quantization, and $f_s(\cdot;\theta_s)$14 compression after lossless coding, all within $f_s(\cdot;\theta_s)$15 accuracy loss (Zhang et al., 16 Jan 2025).

Parallelism-oriented systems attack the same bottleneck from the execution side. FL-LLaMA introduces client-batch and server-hierarchical strategies for split federated training, plus attention-mask compression and collaborative KV caching for inference. On ReCoRD with $f_s(\cdot;\theta_s)$16 k samples, sequential training over $f_s(\cdot;\theta_s)$17 clients takes $f_s(\cdot;\theta_s)$18 s, client-batch mode with $f_s(\cdot;\theta_s)$19 clients reduces this to $f_s(\cdot;\theta_s)$20 s, and server-hierarchical mode with $f_s(\cdot;\theta_s)$21 clients reduces it to $f_s(\cdot;\theta_s)$22 s. During inference, transmitting only $f_s(\cdot;\theta_s)$23 instead of the full attention mask lowers per-token communication latency by $f_s(\cdot;\theta_s)$24, and the collaborative KV-cache yields up to $f_s(\cdot;\theta_s)$25 faster generation at query length $f_s(\cdot;\theta_s)$26 (Zhang et al., 21 May 2025). Splitwiser addresses a different throughput pathology: low utilization during token generation on a single GPU. It splits prompt computation and token generation into concurrent workers running under NVIDIA MPS, reports latency reduction of up to $f_s(\cdot;\theta_s)$27 in the Huggingface backend and $f_s(\cdot;\theta_s)$28 with MPS, and shows that MPSx2 in vLLM yields $f_s(\cdot;\theta_s)$29 speedup on $f_s(\cdot;\theta_s)$30 requests (Aali et al., 21 Apr 2025).

ELSA combines communication reduction with hierarchical coordination. It clusters clients using semantic fingerprints derived from public probe inputs and symmetric KL divergence, assigns them to edge servers under latency constraints, and applies Count-Sketch together with semantic subspace orthogonal perturbation. Its communication model gives a compression ratio $f_s(\cdot;\theta_s)$31, and its reported empirical outcomes include $f_s(\cdot;\theta_s)$32–$f_s(\cdot;\theta_s)$33 communication reduction, cosine similarity near zero for reconstruction attempts, token identification accuracy below $f_s(\cdot;\theta_s)$34, and an $f_s(\cdot;\theta_s)$35 convergence guarantee under its stated assumptions (Yang et al., 20 Jan 2026).

A common systems theme is that split-LLM performance is rarely compute-bound in only one place. WAN inference can be RTT-bound; wireless fine-tuning can be uplink/downlink-bound; multi-client training can be straggler-bound; and same-GPU serving can be cache- and scheduler-bound. Consequently, successful implementations treat splitting, compression, batching, and scheduling as a coupled control problem.

5. Privacy leakage, attack models, and defenses

Split-LLM is motivated by privacy, but the literature shows that privacy is not automatic. In the WAN inference setting, the threat model in (Cunningham, 18 Feb 2026) is a semi-honest cloud that knows layers $f_s(\cdot;\theta_s)$36, observes activations, and lacks the embedding and unembedding weights. Even under this asymmetric split, a $f_s(\cdot;\theta_s)$37-layer MLP inversion decoder trained on $f_s(\cdot;\theta_s)$38 samples recovers top-$f_s(\cdot;\theta_s)$39 tokens with $f_s(\cdot;\theta_s)$40 accuracy when the split is after layer $f_s(\cdot;\theta_s)$41, $f_s(\cdot;\theta_s)$42 after layer $f_s(\cdot;\theta_s)$43, $f_s(\cdot;\theta_s)$44 after layer $f_s(\cdot;\theta_s)$45, and $f_s(\cdot;\theta_s)$46 after layer $f_s(\cdot;\theta_s)$47, compared with a $f_s(\cdot;\theta_s)$48 random baseline. The same paper reports that each extra local layer costs only about $f_s(\cdot;\theta_s)$49 ms of throughput, so moving to $f_s(\cdot;\theta_s)$50 is about $f_s(\cdot;\theta_s)$51–$f_s(\cdot;\theta_s)$52 slower but yields roughly $f_s(\cdot;\theta_s)$53 reduction in recoverable tokens (Cunningham, 18 Feb 2026).

Benchmark work systematizes this privacy–utility trade-off. VFLAIR-LLM implements $f_s(\cdot;\theta_s)$54 attacks—$f_s(\cdot;\theta_s)$55 model-inversion and $f_s(\cdot;\theta_s)$56 label-inference—and $f_s(\cdot;\theta_s)$57 defenses—$f_s(\cdot;\theta_s)$58 perturbation-based and $f_s(\cdot;\theta_s)$59 learning-based. On SST-2 with an HT split, the undefended system has VMI recall of about $f_s(\cdot;\theta_s)$60 at $f_s(\cdot;\theta_s)$61 accuracy; MID with $f_s(\cdot;\theta_s)$62 lowers VMI recall to $f_s(\cdot;\theta_s)$63 while maintaining $f_s(\cdot;\theta_s)$64 accuracy, giving DCS about $f_s(\cdot;\theta_s)$65; and DP with $f_s(\cdot;\theta_s)$66 yields recall $f_s(\cdot;\theta_s)$67 at $f_s(\cdot;\theta_s)$68 accuracy, DCS about $f_s(\cdot;\theta_s)$69. Under HBT with Full-LoRA and defenses at both cuts, MID with $f_s(\cdot;\theta_s)$70 achieves the best overall C-DCS of about $f_s(\cdot;\theta_s)$71 (Gu et al., 5 Aug 2025).

Later work broadens both the attack surface and the defense toolbox. “From Prompts to Responses: Dual-Sided Data Leakage and Defense in Split LLMs” shows that leakage is not confined to private prompts: PIDI reconstructs both prompts and generated responses using dual-sided initialization and patched inversion. Its reported overall attack performance $f_s(\cdot;\theta_s)$72 is $f_s(\cdot;\theta_s)$73 on Llama3-3B/Fin, $f_s(\cdot;\theta_s)$74 on Llama3-8B/Med, and $f_s(\cdot;\theta_s)$75 on Qwen2.5-7B/Dolly, and its patched inversion remains stable for sequence length $f_s(\cdot;\theta_s)$76. The proposed ADMI defense combines adapter-based local warm-up, a VIB mutual-information regularizer, and a model-distance regularizer; on Financial QA with $f_s(\cdot;\theta_s)$77, it changes MP/AP from $f_s(\cdot;\theta_s)$78 with no defense to $f_s(\cdot;\theta_s)$79, with DCS $f_s(\cdot;\theta_s)$80 (Gu et al., 12 Jun 2026). MixGuard takes a different path, combining token-level obfuscation, representation-level mixup, a calibration model, and adaptive gradient perturbation. It reports utility close to non-split training—for example, CoLA on Llama3-1B at $f_s(\cdot;\theta_s)$81 versus $f_s(\cdot;\theta_s)$82 centralized and GSM8K at $f_s(\cdot;\theta_s)$83 versus $f_s(\cdot;\theta_s)$84—while pushing reconstruction quality to ROUGE-L F1 about $f_s(\cdot;\theta_s)$85–$f_s(\cdot;\theta_s)$86, retaining weak reconstructions under adaptive attacks, and adding about $f_s(\cdot;\theta_s)$87 s compute per instance and $f_s(\cdot;\theta_s)$88 MB communication versus $f_s(\cdot;\theta_s)$89 s and $f_s(\cdot;\theta_s)$90 MB in the baseline (Chen et al., 15 Jun 2026). FL-LLaMA adds forward-pass Gaussian noise $f_s(\cdot;\theta_s)$91, $f_s(\cdot;\theta_s)$92, and reports that with $f_s(\cdot;\theta_s)$93 even zero noise already drives inversion BLEU-4/ROUGE-2 to about $f_s(\cdot;\theta_s)$94, while $f_s(\cdot;\theta_s)$95 keeps these scores near zero with less than $f_s(\cdot;\theta_s)$96 points of EM loss (Zhang et al., 21 May 2025).

One misconception is that split learning inherently protects labels or responses once raw prompts stay local. The recent dual-sided leakage results explicitly contradict this. Another is that stronger privacy always requires severe utility collapse. The benchmarked MID, ADMI, and MixGuard results indicate that some defenses preserve main-task performance substantially better than naive perturbation alone, though rigorous information-theoretic guarantees remain open.

6. Empirical trade-offs, terminology, and open directions

The survey literature organizes split-LLM research along three axes: model-level optimization, system-level efficiency, and privacy preservation. Representative techniques include PEFT adapters such as SplitLoRA and HSplitLoRA, communication compression via sparsification and quantization, mini-batch pipelining and resource-aware split-point selection, Gaussian DP on cut activations, and secure aggregation or homomorphic encryption. The same survey lists dynamic cut-point adaptation, robust DP for LLMs, straggler-resilient split-fed protocols, and joint defenses against collusion and poisoning as open problems (Liu et al., 27 Apr 2026).

Several recurrent trade-offs are now well documented. Increasing the local depth of the split improves inversion resistance but raises local memory and latency; deeper client retention of input and output blocks, as in FL-LLaMA and the WAN inference system, can keep raw tokens and logits local but changes client resource requirements (Cunningham, 18 Feb 2026, Zhang et al., 21 May 2025). Static partition points simplify deployment, yet multiple papers identify adaptive cut-layer selection as unfinished business under heterogeneous devices and variable networks (Zhang et al., 23 Jan 2025, Chen et al., 2024). Hierarchical user–edge–cloud designs reduce peak memory and relieve cloud congestion, but they add another aggregation tier and another opportunity for mismatch between compute placement and communication topology (Zhang et al., 23 Jan 2025, Yang et al., 20 Jan 2026).

The terminology itself reflects this diversity. “SplitLLM” names both an umbrella paradigm and specific systems, including “SplitLLM: Hierarchical Split Learning for LLM over Wireless Network” (Zhang et al., 23 Jan 2025) and “SplitLLM: Collaborative Inference of LLMs for Model Placement and Throughput Optimization” (Mudvari et al., 2024). A plausible implication is that the field has moved from asking whether LLMs can be split at all to asking how splits should co-evolve with PEFT, decoding strategy, network scheduling, and privacy defense.

In that sense, Split-LLM has become a unifying abstraction for private and resource-aware LLM deployment. Its mature forms are no longer defined solely by where the model is cut, but by how the cut interacts with speculative decoding, adapter placement, batching, compression, inversion resistance, and hierarchical coordination.