Split Computing: Inference Partitioning

• Split computing is an inference partitioning paradigm that divides a deep neural network between a resource-constrained device and a powerful server to optimize efficiency.
• It strategically selects a split point based on factors like tensor size and neuron importance, balancing latency, communication cost, and accuracy.
• SC applies supervised compression and adaptive techniques to mitigate privacy risks and robustness challenges during data transmission.

Split computing (SC) is an inference-time partitioning paradigm for deep neural networks in which a resource-constrained client or edge device executes an initial subnetwork, transmits an intermediate representation, and a more powerful edge or cloud server completes inference on the remaining subnetwork. In the formalism used across the literature, a model $M = f_L \circ f_{L-1} \circ \dots \circ f_1$ is split at layer $k$ into a client-side encoder $E(x)=f_k \circ \dots \circ f_1(x)$ and a server-side decoder or tail $D(z)=f_L \circ \dots \circ f_{k+1}(z)$, with $z=E(x)$ sent instead of the raw input $x$ (Matsubara et al., 2022). SC is motivated by the limits of full on-device inference, the communication cost and privacy exposure of full remote inference, and the possibility of trading device compute for reduced transmitted data and lower end-to-end latency (Matsubara et al., 2021). In this literature, SC is distinct from split learning and federated learning: the former is a distributed training protocol and the latter distributes model updates, whereas SC is the partitioned execution of a trained model during inference (Matsubara et al., 2 Jan 2025).

1. Formal model and architectural variants

The canonical SC system has three components: a head model on the device, a tail model on the server, and a network link between them. If the split is taken at layer $\ell$, the head computes $\mathbf{z}_\ell=\mathcal{M}_H(\mathbf{x})$, the feature tensor $\mathbf{z}_\ell$ is transmitted, and the tail computes $\hat{\mathbf{y}}=\mathcal{M}_T(\mathbf{z}_\ell)$ (Matsubara et al., 2021). A common refinement injects an explicit encoder–decoder bottleneck, so that the device produces a compressed latent $k$0, the server reconstructs $k$1, and the remaining layers consume $k$2 rather than the original intermediate tensor (Matsubara et al., 2021).

SC is typically analyzed relative to two limiting deployment modes. In local-only computing, the entire model runs on the device and no communication occurs; in remote-only computing, the device sends the raw input to the server and the full model runs remotely (Capogrosso et al., 2024). The central systems question is therefore not merely whether to offload, but where to cut the computation graph and what representation should cross the device–server boundary.

A broader systems perspective appears in split-execution computing outside deep learning. In a CPU–QPU setting, the total time was decomposed into translation, hardware initialization, accelerator execution, and post-processing, and the dominant bottleneck was the interface rather than the accelerator itself (Humble et al., 2016). This broader result suggests that, in SC for neural inference, the representation at the split boundary and the cost of crossing that boundary are first-class design variables rather than secondary implementation details.

2. Split-point selection and optimization objectives

Most SC formulations reduce to a latency–communication–accuracy trade-off. A standard decomposition writes the end-to-end latency as

$k$3

where device compute, feature transmission, and server compute all depend on the split layer $k$4 (Matsubara et al., 2021). In dynamic split computing, this becomes an explicit online decision problem: for split $k$5,

$k$6

with $k$7 the batch input size, $k$8 the compression ratio at layer $k$9, and $E(x)=f_k \circ \dots \circ f_1(x)$0 the communication data rate; the optimal split is then selected as $E(x)=f_k \circ \dots \circ f_1(x)$1 (Bakhtiarnia et al., 2022). That formulation permits per-batch adaptation to changing workload and channel conditions without retraining and, when natural bottlenecks exist, without altering the original model (Bakhtiarnia et al., 2022).

Architectural heuristics based only on tensor size are not sufficient. “I-SPLIT” introduces an interpretability-driven criterion based on Grad-CAM-style neuron importance: a split should be applied right after layers with high densities of important neurons, because those layers preserve discriminative information before compression and transmission (Cunico et al., 2022). The method aggregates per-image importance into a CUmulated Importance (CUI) curve over layers and shows that the best split point can depend on the class distribution of the deployment scenario rather than only on architecture or feature-map size (Cunico et al., 2022).

Search-based optimization extends this idea further. “NASC” jointly searches network architecture and split point under a latency penalty, optimizing a split-aware objective that combines loss under packet loss with a latency threshold term; on HW-NAS-Bench data, it reduces latency by approximately $E(x)=f_k \circ \dots \circ f_1(x)$2 from the baseline with slight accuracy degradation (Shimizu et al., 2022). This establishes that SC design can be treated not only as a systems scheduling problem but also as an architecture search problem in which the head, tail, and split location are co-optimized.

3. Bottlenecks, supervised compression, and feature transport

A recurring empirical finding is that naive splitting is often ineffective in modern architectures. For Faster R-CNN and Mask R-CNN on COCO 2017, extensive layer-wise tensor-size analysis shows that naive split computing methods would not reduce inference time, because early intermediate tensors are not smaller than the input, and branching after early backbone stages complicates transmission further (Matsubara et al., 2020). Injected bottlenecks partly recover feasibility: in those detectors, small bottlenecks reduced transmitted data by $E(x)=f_k \circ \dots \circ f_1(x)$3, but aggressively small bottlenecks substantially degraded detection performance (Matsubara et al., 2020).

The most systematic treatment of this problem is the SC2 framework, which formalizes supervised compression for SC and argues that evaluation based only on rate–distortion is incomplete (Matsubara et al., 2022). SC2 introduces a three-way trade-off over encoder size, transmitted data size, and supervised performance, and benchmarks 10 baseline methods across ImageNet classification, COCO detection, and PASCAL VOC segmentation with over 180 trained models (Matsubara et al., 2022). Within that benchmark, Entropic Student consistently occupies the Pareto frontier, and its encoder is about $E(x)=f_k \circ \dots \circ f_1(x)$4 smaller than MSHP’s encoder while yielding better ExR–D trade-offs (Matsubara et al., 2022).

Several later systems refine bottleneck design itself. “Digital-SC” treats the transmitted intermediate tensor as a semantic communication object, combining structured pruning, learned non-linear quantization with trainable quantization levels, and an adaptive split policy over a binary symmetric channel (Guo et al., 2023). Its policy network adaptively chooses the split point and transmitted feature dimension based on the input and the channel, and on CIFAR-10 the digital scheme is within about $E(x)=f_k \circ \dots \circ f_1(x)$5 accuracy of analog DJSCC at $E(x)=f_k \circ \dots \circ f_1(x)$6 dB while remaining compatible with standard digital transmission stacks (Guo et al., 2023). “HECS-B” instead grounds the bottleneck in compressed sensing: it inserts a high-efficiency compressed sensing bottleneck into a shallow layer, trains it with knowledge distillation, and reports a $E(x)=f_k \circ \dots \circ f_1(x)$7 reduction in bandwidth utilization and a $E(x)=f_k \circ \dots \circ f_1(x)$8 speed-up in computational efficiency compared to state-of-the-art methods while maintaining high accuracy (Zhong et al., 15 Apr 2025).

4. Multi-task, multi-sensor, and autoregressive extensions

SC is no longer restricted to single-task image classification. “MTL-Split” places a shared backbone $E(x)=f_k \circ \dots \circ f_1(x)$9 on the edge and multiple task-specific heads $D(z)=f_L \circ \dots \circ f_{k+1}(z)$0 on the server, with the split fixed at the output of the shared backbone (Capogrosso et al., 2024). In that architecture, the edge transmits one flattened feature tensor $D(z)=f_L \circ \dots \circ f_{k+1}(z)$1 per input regardless of the number of tasks, and the transmitted representation is remarkably compact: $D(z)=f_L \circ \dots \circ f_{k+1}(z)$2 is $D(z)=f_L \circ \dots \circ f_{k+1}(z)$3 MB for MobileNetV3 and $D(z)=f_L \circ \dots \circ f_{k+1}(z)$4 MB for EfficientNet in the reported configurations (Capogrosso et al., 2024). On the FACES dataset, sending 100 raw images of about $D(z)=f_L \circ \dots \circ f_{k+1}(z)$5 MB each over a $D(z)=f_L \circ \dots \circ f_{k+1}(z)$6 Gbit/s channel requires about $D(z)=f_L \circ \dots \circ f_{k+1}(z)$7 s, whereas sending 100 EfficientNet features of about $D(z)=f_L \circ \dots \circ f_{k+1}(z)$8 MB each requires about $D(z)=f_L \circ \dots \circ f_{k+1}(z)$9 s, corresponding to an approximate $z=E(x)$0 reduction in transmission time (Capogrosso et al., 2024). The same study also reports that all tested MTL-Split configurations were deployable on a 4 GB Jetson Nano, whereas naive local-only deployment of multiple single-task models often was not (Capogrosso et al., 2024).

“Ladon” extends supervised compression to a single compressed backbone serving multiple task heads for ImageNet classification, COCO detection, and PASCAL VOC segmentation (Matsubara et al., 2 Jan 2025). It is explicitly presented as the first multi-task-head supervised compression model for multi-task SC, and it reports end-to-end latency reductions of up to $z=E(x)$1 and mobile energy reductions of up to $z=E(x)$2 in multi-task split computing scenarios while maintaining predictive performance competitive with strong lightweight baselines (Matsubara et al., 2 Jan 2025). This shifts SC from single-output collaborative inference toward multi-head shared representations.

System	Split realization	Reported effect
MTL-Split	Shared CNN backbone on edge, multiple task heads on server	$z=E(x)$3 size $z=E(x)$4 MB; FACES transmission time reduction $z=E(x)$5 (Capogrosso et al., 2024)
Ladon	Single supervised-compression backbone with classification, detection, and segmentation heads	End-to-end latency reduction up to $z=E(x)$6; mobile energy reduction up to $z=E(x)$7 (Matsubara et al., 2 Jan 2025)
SC-MII	Voxel R-CNN split after the first 3D convolution following voxelization, with multi-LiDAR feature integration on the server	$z=E(x)$8 speed-up, $z=E(x)$9 reduction in edge processing time, at most $x$0 accuracy drop (Noguchi et al., 12 Jan 2026)
Adaptive LLM SC	Front-end/back-end transformer split with one-point split compression and two-stage intermediate compression	$x$1 inference speedup with significant communication overhead reduction (Sung et al., 6 Nov 2025)

The same pattern appears in multi-sensor perception. “SC-MII” targets roadside multi-LiDAR 3D object detection with Voxel R-CNN, splitting immediately after the first 3D convolution following voxelization and integrating multiple intermediate outputs on the server after NDT-based alignment (Noguchi et al., 12 Jan 2026). On a real-world dataset it reports a $x$2 speed-up and a $x$3 reduction in edge-device processing time, with at most a $x$4 drop in accuracy relative to centralized multi-LiDAR raw-point fusion (Noguchi et al., 12 Jan 2026).

LLMs introduce a different SC regime because decoding is autoregressive and key–value caches grow with sequence length. “Memory- and Latency-Constrained Inference of LLMs via Adaptive Split Computing” formulates the first autoregressive-aware SC framework for LLMs, with one-point split compression (OPSC), threshold splitting, and token-wise adaptive bit quantization (TAB-Q) (Sung et al., 6 Nov 2025). It jointly optimizes split point, quantization, and sequence length under memory and latency constraints and reports a $x$5 inference speedup with significant communication overhead reduction while maintaining or improving model accuracy (Sung et al., 6 Nov 2025). This establishes that SC for sequence generation is not a trivial port of image-based methods but requires explicit treatment of iterative decoding and KV-cache memory.

A separate extension combines SC with early exit and predefined sparsity. In “Enhancing Split Computing and Early Exit Applications through Predefined Sparsity,” structured predefined sparse MLPs are split in the middle for SC and equipped with an early-exit branch at the head; on MNIST, the resulting models achieve similar or better accuracy than dense baselines with reductions exceeding $x$6 in storage and computational complexity without compromising performance (Capogrosso et al., 2024). This suggests that, in some settings, the split interface, local compute budget, and exit policy can be co-designed with sparse connectivity rather than post hoc pruning.

5. Privacy, robustness, and closed-model adaptation

Privacy is one of the standard motivations for SC, because transmitting intermediate features rather than raw inputs is often presumed to be more privacy-preserving (Matsubara et al., 2021). That presumption is only partially justified. “Privacy Vulnerability of Split Computing to Data-Free Model Inversion Attacks” demonstrates that an attacker can reconstruct original inputs from intermediate features with high fidelity more than 40 layers deep into ResNet and RepVGG models, without access to real training data or pretrained generative models (Dong et al., 2021). Its Divide-and-Conquer Inversion (DCI) method shows that SC is not inherently privacy-preserving and that simple noise injection or feature dropout introduces a non-trivial utility–privacy trade-off rather than a clean solution (Dong et al., 2021).

Robustness and adaptation become especially difficult in deployments where the head and tail are inaccessible or frozen. “SALT (Split-Adaptive Lightweight Tuning)” addresses this closed-model setting by inserting a client-side residual adapter between the frozen head and the channel, leaving both head and tail unchanged and preserving the communication tensor shape (Okada et al., 16 Mar 2026). On CIFAR-10, SALT improves personalized accuracy from $x$7 to $x$8 while reducing training latency by more than $x$9 relative to retraining; it also maintains over $\ell$0 accuracy under $\ell$1 packet loss and about $\ell$2 accuracy at $\ell$3 under noise injection (Okada et al., 16 Mar 2026). In the same study, inversion-attack SSIM curves with and without SALT are nearly identical under noisy transmission, which the authors interpret as evidence that SALT does not reduce the privacy protection afforded by feature-space noise injection (Okada et al., 16 Mar 2026).

These results clarify a common misconception. SC can reduce direct exposure of raw inputs, but privacy, robustness, and personalization are separate design problems. Feature transmission may be preferable to raw-data transmission in some settings, yet feature invertibility, packet loss, noise injection, and model-access constraints all materially affect system behavior and cannot be treated as secondary concerns.

6. Evaluation criteria, misconceptions, and open directions

A mature SC methodology requires system-level evaluation rather than accuracy-only reporting. SC2 argues that bits-per-pixel can be misleading and instead evaluates absolute transmitted data size, while using encoder size as a static proxy for on-device computation (Matsubara et al., 2022). Its ExR–D metric multiplies encoder size and data rate, and its bottleneck-placement study shows that late bottlenecks may look best under conventional supervised rate–distortion curves while placing up to $\ell$4 of the model on the device, thereby defeating the purpose of SC (Matsubara et al., 2022). This is a broader warning against evaluating split methods only by compression ratio or only by top-1 accuracy.

Another persistent misconception is that split-point selection is determined solely by tensor geometry. I-SPLIT shows that architectural size drops are not enough; neuron importance and class distribution can change the best split layer (Cunico et al., 2022). Dynamic split computing shows that even when a fixed split is sensible under one pair of $\ell$5, it can be suboptimal under another, and that online adaptation across natural bottlenecks can improve latency without affecting accuracy (Bakhtiarnia et al., 2022). These results jointly imply that SC design spans architecture, data distribution, channel state, and deployment objective.

Across the surveyed literature, several open directions recur. The 2021 survey identifies realistic wireless evaluation, principled bottleneck design, and context-aware exit policies as central research challenges (Matsubara et al., 2021). Later papers add extensions to mixed task types beyond classification, segmentation, and detection (Capogrosso et al., 2024), adaptation to architectures beyond convolutional backbones (Cunico et al., 2022), broader privacy and robustness models (Okada et al., 16 Mar 2026), and tighter integration of compression with end-to-end energy and latency objectives (Matsubara et al., 2022). A plausible implication is that SC is increasingly less a single mechanism than a design space: one must jointly choose the split boundary, the feature representation crossing that boundary, the training objective used to shape it, and the online policy that reacts to device, server, and network variability.

In contemporary usage, then, split computing denotes a family of distributed inference strategies in which representation learning, compression, scheduling, and systems constraints are co-optimized around an explicit edge–server boundary. The literature has moved from static head/tail decompositions toward interpretability-guided splitting, supervised compression, multi-task and multi-sensor shared representations, closed-model adaptation, and autoregressive LLM-specific partitioning, while simultaneously showing that privacy and efficiency claims must be validated at the system level rather than assumed from the mere fact of splitting (Matsubara et al., 2022, Sung et al., 6 Nov 2025).