Parameter-Efficient Split Learning

• Parameter-efficient split learning is a framework that minimizes active trainable or transmitted parameters through techniques such as sparse updates and optimized split-point selection.
• It leverages methods like cut-layer selection, quantization, and low-rank adapters (LoRA) to balance client workload, communication costs, and convergence performance.
• Recent evaluations show that these strategies can achieve up to 97% bandwidth reduction and higher accuracy in non-IID, low-bandwidth, and heterogeneous settings.

Searching arXiv for papers on parameter-efficient split learning and related frameworks. Parameter-efficient split learning denotes a class of split-learning, split-federated, and vertical split-learning protocols that reduce the number of trainable, transmitted, stored, or simultaneously active parameters while preserving collaborative training over partitioned models. In this literature, efficiency is pursued through sparse server-side parameter sharing, cut-layer selection, asynchronous scheduling, quantization of intermediate representations, graph and model pruning, and parameter-efficient fine-tuning (PEFT) modules such as LoRA. The problem becomes especially acute under non-IID data, heterogeneous clients, low-bandwidth links, and large foundation models, where classical split learning can reduce client computation but often increases bandwidth consumption or exhibits sub-optimal convergence (Chopra et al., 2021, Zhou et al., 2023, Lin et al., 2024).

1. Paradigms and efficiency objectives

SLPerf organizes split learning into three paradigm families: Model-Split-Only, Weight Aggregation-Based, and Intermediate Data Aggregation-Based (Zhou et al., 2023). This taxonomy is important for parameter efficiency because each family exposes a different control surface. In model-split-only systems such as Vanilla SL, U-shape SL, PSL, and AsyncSL, efficiency is governed primarily by the cut layer and by the size of the smashed data and corresponding gradients. In weight-aggregation-based systems such as SplitFed, FSL, FeSTA, HSFL, and CPSL, efficiency depends on how often parameters are exchanged and aggregated. In intermediate-data-aggregation methods such as SGLR, LocFedMix-SL, and CutMixSL, the objective is to avoid transferring complete weights by mixing or aggregating smaller intermediate objects.

Paradigm family	Representative methods	Main efficiency lever
Model-Split-Only	Vanilla SL, U-shape SL, PSL, AsyncSL	Cut layer, smashed-data size, gradient exchange
Weight Aggregation-Based	SplitFed, FSL, FeSTA, HSFL, CPSL	Aggregation frequency, model size, exchanged weights
Intermediate Data Aggregation-Based	SGLR, LocFedMix-SL, CutMixSL	Exchange of activations or cut-layer gradients

Within this taxonomy, “parameter efficiency” is not limited to raw parameter count. AdaSplit frames the problem jointly in terms of bandwidth, compute, and convergence under client heterogeneity, while SLPerf emphasizes communication cost per epoch, accuracy, AUC, cut-layer placement, and aggregation strategy as the principal empirical axes for comparison (Chopra et al., 2021, Zhou et al., 2023). This suggests that the field treats parameter efficiency as a systems-level property: a method may update fewer parameters yet still be inefficient if it induces large activation traffic, excessive synchronization, or poor convergence.

2. Core algorithmic mechanisms

AdaSplit is an explicit formulation of parameter-efficient split learning for low-resource and heterogeneous settings. Its two central mechanisms are the elimination of server-to-client gradient dependence and adaptive, sparse parameter collaboration on the server (Chopra et al., 2021). In classical split learning, communication requires both split activations and gradients at every iteration,

$$C_2^\mathrm{SL} = \sum_{i=1}^N \sum_{j=1}^R \sum_{k=1}^T (P_{is} + P_{si}) \cdot \sigma(i,j,k),$$

where $P_{is}$ and $P_{si}$ are client-to-server and server-to-client payloads. AdaSplit removes server-to-client gradient flow, setting $P_{si}=0$, and lets each client compute a local supervised contrastive loss based on NT-Xent:

$$L_\mathrm{client} = \sum_{i=1}^{|b|} \sum_{q_+ \in Q^i_+} -\log \frac{\exp(q_i \cdot q_+ / \tau)}{\sum_{j \ne i} \exp(q_i \cdot q_j / \tau)}.$$

It also introduces a local phase/global phase schedule controlled by $\kappa$, and a server-side orchestrator using UCB bandit strategies to select a top-$\eta$ subset of clients for communication.

On the server, AdaSplit associates each client $i$ with a learnable multiplicative mask $m_i$ over server parameters, so that gradients from client $i$ update only a sparse partition of the server model. The server objective is

$P_{is}$0

which enforces sparsity in parameter utilization per client (Chopra et al., 2021). The stated consequence is mitigation of destructive interference among heterogeneous clients together with tailored parameter sharing. On Mixed-NonIID, AdaSplit with $P_{is}$1 and $P_{is}$2 achieves 87.1% accuracy with 2.43GB bandwidth and 5.38 TFLOPS, compared with SplitFed at 84.7%, 84.64GB, and 3.76TFLOPS, and FedProx at 85.1%, 2.39GB, and 17.1TFLOPS; the reported C3-Score is 0.83–0.85 for AdaSplit versus 0.72–0.75 for the baselines (Chopra et al., 2021). The same study reports bandwidth reductions of 75–97% over prior split-learning methods.

SLPerf broadens these mechanisms into a benchmarking perspective. It reports that communication cost decreases significantly when clients hold two convolution layers instead of one because the choice of cut layer affects the size of the smashed data and gradients, and it identifies AsyncSL and C3-SL as examples of asynchronous update and batch-wise compression strategies for reducing transfer cost (Zhou et al., 2023). The cumulative picture is that parameter-efficient split learning operates through a combination of sparse updates, deeper or better-placed cuts, reduced synchronization, and selective aggregation rather than through a single architectural device.

3. Split-point selection and resource allocation

The split point is a first-order design variable because it determines both client workload and activation volume. In SLPerf, cut-layer placement is explicitly exposed as a configurable benchmark dimension, and deeper cuts generally reduce smashed-data size and therefore communication cost (Zhou et al., 2023). However, deeper cuts also move more layers to the client, so the gain in bandwidth comes with increased local computation.

This trade-off appears directly in SplitFed under packet loss. On a human embryo image segmentation task using Split U-Net, a deeper split point has a statistically significant advantage over a shallower split across all tested combinations of packet loss probability and affected clients (Shiranthika et al., 2024). The reported explanation is architectural: in the deep split, client-side back-end layers can recover or denoise features degraded at the split interface, and key skip connections remain entirely client-side. At the same time, the study notes the computational trade-off: shallow splits impose less client compute, whereas deep splits increase client workload but improve resilience.

Systematic optimization of split points has also been formalized. “Fast AI Model Splitting over Edge Networks” represents an arbitrary model as a DAG and reformulates optimal splitting as a minimum $P_{is}$3-$P_{is}$4 cut problem (Li et al., 23 Jun 2025). The method defines edge weights for device execution, server execution, and propagation delay, proves optimality after DAG restructuring to avoid double-counting propagation cost, and introduces a block-wise abstraction for block-structured models. The reported outcome is that the proposed algorithms determine the optimal model splitting within milliseconds and reduce training delay by 24.62%–38.95% in dynamic edge networks relative to state-of-the-art benchmarks (Li et al., 23 Jun 2025).

A related resource-aware perspective appears in ESFL, which places model splitting inside a split federated learning framework for heterogeneous end devices and jointly optimizes user-side workload and server-side computing resource allocation. Its optimization is formulated as a mixed-integer non-linear program and solved approximately by an iterative approach; the abstract reports significantly increased efficiency compared with standard federated learning, split learning, and splitfed learning (Zhu et al., 2024). Taken together, these results make split-point selection a central component of parameter efficiency rather than a static architectural choice.

4. PEFT-based split learning for LLMs

LLMs have made PEFT-based split learning a distinct subfield. SplitLoRA is presented as the first split-learning LLM fine-tuning framework and the inaugural open-source benchmark for SL LLM fine-tuning (Lin et al., 2024). It is built on split federated learning, partitions the model at a cut layer into client-side parameters $P_{is}$5 and server-side parameters $P_{is}$6, and applies LoRA so that only low-rank adapter parameters are trainable:

$P_{is}$7

The client computes forward passes on the first $P_{is}$8 Transformer layers, sends activations and labels to the server, receives activation gradients, updates client-side LoRA adapters, and periodically uploads only adapter parameters for weighted aggregation. The paper summary states that SplitLoRA reaches target accuracy 1.7–4.8× faster than FedLoRA or CenLoRA and restricts trainable client parameters to as little as $P_{is}$9 of full-model LoRA adapter parameters for GPT2-M and $P_{si}$0 for GPT2-S (Lin et al., 2024).

SflLLM extends the same PEFT logic to communication networks by jointly optimizing subchannel allocation, power control, model split point, and LoRA rank to minimize total training delay (Zhao et al., 20 Apr 2025). In this formulation, the rank $P_{si}$1 controls both trainable parameter count and convergence behavior. The framework reports comparable model accuracy while significantly reducing client-side computational requirements, and its resource-allocation plus adaptive-rank strategy reduces training latency relative to conventional approaches (Zhao et al., 20 Apr 2025). This places LoRA rank selection alongside the split point as a tunable systems parameter.

HSplitLoRA addresses heterogeneous clients more directly. It first identifies important weights using the Resource-Normalized Gradient-Weight Product

$P_{si}$2

then smooths importance over time, dynamically configures LoRA ranks for selected weights, determines the split point according to computing budgets, and performs heterogeneous adapter aggregation through a concatenation-based, noise-free construction satisfying

$P_{si}$3

The paper reports that HSplitLoRA outperforms state-of-the-art benchmarks in training accuracy and convergence speed (Lin et al., 5 May 2025). A related privacy-focused line is P$P_{si}$4EFT, which studies fine-tuning over an API in a two-party vertical setting, analyzes label leakage for LoRA, and proposes a multi-party split-learning algorithm with gradient masking, multi-adapter mixing, and privacy regularization. It is reported as competitive with existing privacy-preserving methods in multi-party and two-party setups while having higher accuracy (Zmushko et al., 2024).

5. Compression, pruning, asynchrony, and heterogeneous execution

A second major route to parameter-efficient split learning is to reduce the size of intermediate representations or the active model itself. Quantized-Tinyllava modifies a multimodal TinyLLaVA-style architecture with a client/server quantizer $P_{si}$5 that compresses transmitted embeddings into low-bit integer indices and reconstructs continuous embeddings on the server (Guo et al., 28 Nov 2025). Its training objective combines cross-entropy with a cosine-similarity commitment loss,

$P_{si}$6

and its bit-width selection is guided by Shannon’s source coding theorem through empirical entropy estimation. The summary reports that estimated entropy values around 1.8–1.84 indicate that 2 bits suffice, yielding an 8× communication reduction relative to FP16, with VQAV2 accuracy of 72.11% versus 73.04% for the original model (Guo et al., 28 Nov 2025).

DTIP addresses graph neural networks for satellite communications through differential privacy plus graph and model pruning (Sun et al., 2024). It prunes graph structure using centrality-based criteria, prunes parameters with masks $P_{si}$7 and $P_{si}$8, and then performs split learning on the reduced GNN. On Amazon2M, DTIP maintains an accuracy of 0.82 while achieving a 50% reduction in floating-point operations per second; on ArXiv, it reaches 0.85 under comparable conditions (Sun et al., 2024). In this setting, parameter efficiency is inseparable from topology-aware sparsification.

PubSub-VFL treats efficiency primarily as a scheduling and utilization problem in two-party split learning. It replaces strict batch-by-batch synchronization with a Publisher/Subscriber architecture, bounded buffers, a waiting deadline $P_{si}$9, parameter-server parallelism, and a hierarchical asynchronous mechanism whose synchronization interval shrinks over time according to

$P_{si}=0$0

The paper reports $P_{si}=0$1 acceleration, stable convergence, compatibility with differential privacy, and computational resource utilization of up to 91.07% (Liu et al., 14 Oct 2025). This suggests that a method can become “parameter-efficient” not only by reducing parameter count, but by increasing the useful work performed per unit wall-clock time.

Application-specific variants extend the same principle. In vehicular metaverses, a multi-agent split DRL framework dynamically switches between local-only inference and combined local-plus-server inference based on policy entropy, reducing the average computational parameter count by approximately 25% while maintaining similar performances and improving QoE by 29% (Chen et al., 30 Mar 2025). Split-n-Chain, by contrast, distributes consecutive layers across multiple nodes, keeping parameter and hyperparameter exposure local to assigned layers and recording only hashes and audit metadata on blockchain; its reported training loss trend remains similar to that of the same network implemented monolithically (Sahani et al., 10 Mar 2025).

6. Evaluation, privacy boundaries, and recurrent trade-offs

AdaSplit’s C3-Score is one of the clearest attempts to define a unified efficiency measure:

$P_{si}=0$2

where accuracy, bandwidth, and compute are normalized by their maxima (Chopra et al., 2021). SLPerf, by contrast, favors standardized empirical reporting of communication cost per epoch, accuracy, AUC, cut-layer choice, aggregation strategy, and data partitioning across IID and Non-IID settings (Zhou et al., 2023). The coexistence of these metrics reflects a persistent fact: there is no single notion of efficiency that dominates across bandwidth-limited, compute-limited, privacy-sensitive, and heterogeneity-dominated deployments.

Several misconceptions recur in the literature. First, parameter-efficient split learning is not automatically privacy-preserving. P$P_{si}=0$3EFT explicitly analyzes label leakage from activations and gradients in LoRA-based API fine-tuning, even when fewer than 0.1% of parameters are trainable, and introduces additional protocol machinery to address it (Zmushko et al., 2024). DTIP likewise relies on differential privacy on graph data rather than assuming that model splitting alone suffices (Sun et al., 2024). Split-n-Chain emphasizes that privacy of parameters and hyperparameters depends on how layers are distributed and what metadata is exposed to other nodes, not merely on data locality (Sahani et al., 10 Mar 2025).

Second, lower communication is not free. Deeper cuts reduce smashed-data size in SLPerf and improve packet-loss resilience in SplitFed, but they also place more computation on the client (Zhou et al., 2023, Shiranthika et al., 2024). AdaSplit reduces bandwidth by eliminating server-to-client gradients and orchestrating sparse collaboration, yet longer local phases may require more local computation (Chopra et al., 2021). Quantization and pruning decrease transfer volume and FLOPs, but the reported results also show explicit compression thresholds and privacy–utility trade-offs rather than monotonic improvement (Guo et al., 28 Nov 2025, Sun et al., 2024).

The field therefore converges on a multidimensional view of parameter-efficient split learning. Sparse masks, low-rank adapters, quantized embeddings, pruned graphs, asynchronous execution, and optimized split points are not interchangeable techniques; they address different bottlenecks and frequently need to be combined. A plausible implication is that future progress will depend less on isolated communication tricks than on joint optimization of split location, active parameter subset, synchronization schedule, privacy mechanism, and hardware-aware deployment constraints, especially for heterogeneous clients and large foundation models.