Split Early-Exit Neural Networks

• Split early-exit neural networks are architectures that partition computation between resource-limited devices and remote servers using adaptive exit strategies.
• They employ online decision rules and auxiliary predictors to jointly optimize accuracy, latency, and communication costs.
• Applications range from image classification to DDoS detection, achieving notable cost reductions and efficient real-time inference.

A split early-exit neural network is a deep model that combines split computing and early exiting: an initial “head” of the network is executed on a resource-constrained device, while the remaining “tail” can be executed on an edge or cloud server, and intermediate exit mechanisms decide whether inference should terminate locally or continue remotely. In the survey literature, split computing is formulated as a partition of a model into a head and a tail, while early exiting augments internal layers with auxiliary predictors so that “easy” inputs can stop before the full depth is evaluated (Matsubara et al., 2021, Scardapane et al., 2020). More recent systems make the combined decision explicit: SplitEE chooses a splitting layer and then decides whether a sample exits locally or is offloaded (Bajpai et al., 2023), while the abstract of I-SplitEE describes an online unsupervised method that determines the splitting layer and whether to infer on the edge device or offload to the cloud, considering accuracy, computational efficiency, and communication costs (Bajpai et al., 2024).

1. Concept and scope

In split computing, a model is partitioned at some layer $\ell$, with a head $\mathcal{M}_H$ executed on the device and a tail $\mathcal{M}_T$ executed on the server:

$$\mathbf{z}_{\ell} = \mathcal{M}_H(\mathbf{x}), \qquad \hat{\mathbf{y}} = \mathcal{M}_T(\mathbf{z}_{\ell}).$$

In early exiting, intermediate stages produce predictions $\mathbf{y}_i = \mathcal{B}_i(\mathbf{z}_i)$, and inference can stop when an exit is sufficiently confident (Matsubara et al., 2021). A split early-exit architecture therefore adds a second layer of adaptivity: it does not merely decide how deep to compute, but also where the remaining computation should take place.

This combination addresses a three-way systems trade-off. Full local inference avoids communication but can exceed device budgets. Full offloading preserves model depth but can be dominated by transmission delay and bandwidth volatility. Early exits reduce average computation but, by themselves, do not determine whether difficult samples should be continued locally or remotely. Split early-exit systems treat these choices jointly, so that local exits can suppress unnecessary transmission while remote completion remains available for hard inputs (Matsubara et al., 2021, Bajpai et al., 2023).

A common umbrella term in the literature is “multi-output network,” which denotes architectures that attach auxiliary predictors to intermediate representations. “Early-exit network” refers to the runtime use of those outputs for adaptive stopping, while “split computation” refers to deployment across device, edge, fog, or cloud tiers (Scardapane et al., 2020). Split early-exit neural networks lie at the intersection of these categories.

2. Architectural organization and exit criteria

The standard architectural pattern consists of a device-side prefix, one or more device-side exits, and a server-side suffix. In SplitEE, each incoming sample is processed up to a learned splitting layer $i_t$, and the confidence at that layer,

$C_i(x) = \max_{c \in \mathcal{C}} \hat{P}_i(c),$

is compared to a threshold $\alpha$. If $C_i(x) \ge \alpha$, the sample exits locally; otherwise it is offloaded and processed to the final layer $L$ (Bajpai et al., 2023).

Other systems refine this decision logic. In the device-edge co-inference framework with an Exit Predictor, the device first computes predictor scores $\mathcal{M}_H$0 for the on-device exits. The $\mathcal{M}_H$1-th exit is evaluated only if $\mathcal{M}_H$2, and early termination occurs only if the exit confidence

$\mathcal{M}_H$3

also satisfies $\mathcal{M}_H$4. This changes execution from “evaluate every exit until confident” to “evaluate only exits predicted to be useful,” thereby reducing on-device overhead from the exit heads themselves (Dong et al., 2022).

In heterogeneous split learning, Hetero-SplitEE attaches a lightweight client output layer at each client’s cut layer $\mathcal{M}_H$5. For inference, the client computes probabilities

$\mathcal{M}_H$6

measures confidence through entropy,

$\mathcal{M}_H$7

and exits locally if $\mathcal{M}_H$8; otherwise it transmits the intermediate feature $\mathcal{M}_H$9 to the server (Oda et al., 10 Dec 2025). This formulation is notable because the cut layer is not globally fixed: different clients choose different cut layers according to computational capacity.

More general early-exit CNN research clarifies the structure of exit blocks themselves. EENets use a separate confidence branch and softmax branch at each exit:

$\mathcal{M}_T$0

with inference terminating when $\mathcal{M}_T$1, where the paper uses $\mathcal{M}_T$2 (Demir et al., 2024). Although EENets are not, by themselves, split-computing systems, they provide a canonical exit-head design that later split early-exit systems either reuse or replace.

3. Optimization, calibration, and training

The training problem is not uniform across the literature. One family of methods treats early exits as jointly supervised auxiliary outputs. The survey on multi-output networks describes the standard joint objective as

$\mathcal{M}_T$3

where each exit contributes its own loss term (Scardapane et al., 2020). EENets add an explicit computation-aware component,

$\mathcal{M}_T$4

with a soft expected-cost recursion

$\mathcal{M}_T$5

so that accuracy and FLOP cost are optimized together (Demir et al., 2024).

SplitEE instead formulates split-layer selection as an online bandit problem. Let $\mathcal{M}_T$6 be the computation cost up to layer $\mathcal{M}_T$7, $\mathcal{M}_T$8 the offloading cost, and $\mathcal{M}_T$9 the cost-conversion factor. The reward for splitting at layer $$\mathbf{z}_{\ell} = \mathcal{M}_H(\mathbf{x}), \qquad \hat{\mathbf{y}} = \mathcal{M}_T(\mathbf{z}_{\ell}).$$0 is

$$\mathbf{z}_{\ell} = \mathcal{M}_H(\mathbf{x}), \qquad \hat{\mathbf{y}} = \mathcal{M}_T(\mathbf{z}_{\ell}).$$1

The optimal split layer is

$$\mathbf{z}_{\ell} = \mathcal{M}_H(\mathbf{x}), \qquad \hat{\mathbf{y}} = \mathcal{M}_T(\mathbf{z}_{\ell}).$$2

and SplitEE uses an Upper Confidence Bound rule

$$\mathbf{z}_{\ell} = \mathcal{M}_H(\mathbf{x}), \qquad \hat{\mathbf{y}} = \mathcal{M}_T(\mathbf{z}_{\ell}).$$3

to learn that policy online without labels (Bajpai et al., 2023). This is a distinct shift from supervised confidence-threshold tuning to streaming, unsupervised adaptation.

A different line of work argues that gating and intermediate predictors should not be trained separately. JEI-DNN states that decoupled gating creates a train-test mismatch, because intermediate modules are trained on the full dataset but, at inference time, only see the subset routed to them. It therefore jointly learns exit probabilities and intermediate inference modules under an expected cost-aware objective

$$\mathbf{z}_{\ell} = \mathcal{M}_H(\mathbf{x}), \qquad \hat{\mathbf{y}} = \mathcal{M}_T(\mathbf{z}_{\ell}).$$4

(Regol et al., 2023). This position contrasts with modular approaches but does not eliminate them: in pruning experiments on Shallow-Deep Networks, joint pruning generally performs better overall, while ordered pruning of the backbone and then the exit heads becomes similar at high accuracy rates, implying that pruning and early exit can sometimes be separated without loss of optimality in that regime (Görmez et al., 2022).

Compression-aware design extends the same question to model topology. The predefined sparsity approach for split computing and early exit fixes a sparse connectivity pattern before training rather than pruning after training, with junction edge counts

$$\mathbf{z}_{\ell} = \mathcal{M}_H(\mathbf{x}), \qquad \hat{\mathbf{y}} = \mathcal{M}_T(\mathbf{z}_{\ell}).$$5

This reduces storage and computation throughout training and inference while preserving the head–tail–exit deployment logic (Capogrosso et al., 2024).

4. Distributed, edge-cloud, and heterogeneous execution

Split early-exit networks are used in several deployment regimes beyond simple one-device/one-server inference. In hierarchical training, early exits are used not for adaptive inference but to separate the backward pass across edge and cloud. The edge computes a local loss from its early exit, updates its own parameters without gradients from the cloud, and transmits only an intermediate feature map. The cloud computes its own forward and backward passes independently. This removes backward-pass communication and enables overlap between edge backward computation and cloud computation (Sepehri et al., 2023).

Heterogeneous IoT training pushes the idea further. In Hetero-SplitEE, each client $$\mathbf{z}_{\ell} = \mathcal{M}_H(\mathbf{x}), \qquad \hat{\mathbf{y}} = \mathcal{M}_T(\mathbf{z}_{\ell}).$$6 chooses its own cut layer $$\mathbf{z}_{\ell} = \mathcal{M}_H(\mathbf{x}), \qquad \hat{\mathbf{y}} = \mathcal{M}_T(\mathbf{z}_{\ell}).$$7, computes

$$\mathbf{z}_{\ell} = \mathcal{M}_H(\mathbf{x}), \qquad \hat{\mathbf{y}} = \mathcal{M}_T(\mathbf{z}_{\ell}).$$8

and trains a local early-exit branch while the server trains the deeper continuation. Two cooperative strategies are proposed. Sequential training uses one shared server-side model and processes client updates sequentially. Averaging assigns each client its own server-side model and performs cross-layer aggregation

$$\mathbf{z}_{\ell} = \mathcal{M}_H(\mathbf{x}), \qquad \hat{\mathbf{y}} = \mathcal{M}_T(\mathbf{z}_{\ell}).$$9

for layers present in multiple server-side subnetworks (Oda et al., 10 Dec 2025). The defining feature is that heterogeneous cut layers are treated as first-class design variables rather than as noise around a fixed global split.

Distributed collaborative inference also appears outside client-server training. DistrEE assumes an edge cluster $\mathbf{y}_i = \mathcal{B}_i(\mathbf{z}_i)$0 with student models derived from a teacher network, and uses a feature-difference exit criterion

$\mathbf{y}_i = \mathcal{B}_i(\mathbf{z}_i)$1

Inference exits at branch $\mathbf{y}_i = \mathcal{B}_i(\mathbf{z}_i)$2 when $\mathbf{y}_i = \mathcal{B}_i(\mathbf{z}_i)$3 (Peng et al., 6 Feb 2025). This is notable because it does not rely on class-probability confidence; instead, it uses the maturity of an internal representation relative to the shallowest exit.

The same architectural pattern has also been instantiated in network security. In a P4-SDN collaborative DDoS detector, a quantized CNN and early-exit classifier run in the P4 data plane, while a GRU backend in the SDN controller handles uncertain cases. The switch exits locally if the CNN output lies outside the uncertainty interval defined by precomputed logit thresholds; otherwise it forwards a compact sequence of feature vectors to the controller (Karrakchou et al., 15 Sep 2025). Here the split early-exit design is realized as a confidence-gated switch/controller cascade rather than as a mobile/cloud application.

5. Empirical behavior and application domains

Reported results span image classification, NLP, DDoS detection, split learning, and training-time acceleration. The literature does not present a single benchmark protocol, but it consistently treats split early-exit design as an accuracy–computation–communication trade-off rather than as a pure accuracy-maximization problem.

System	Domain/setup	Reported outcome
I-SplitEE (Bajpai et al., 2024)	Caltech-256 and Cifar-10 with varied distortions	minimum 55% cost reduction with at most 5% performance degradation
SplitEE (Bajpai et al., 2023)	Five NLP datasets with ElasticBERT	significant cost reduction ($\mathbf{y}_i = \mathcal{B}_i(\mathbf{z}_i)$4) with slight accuracy drop ($\mathbf{y}_i = \mathcal{B}_i(\mathbf{z}_i)$5)
Exit Predictor (Dong et al., 2022)	CIFAR100, VGG16-BN, Raspberry Pi 3, varying bandwidth	meets the 30 ms latency target across bandwidths from 0.1 to 100 Mbit/s
Predefined sparsity with SC and EE (Capogrosso et al., 2024)	Sparse MLP split/exit models on MNIST	more than $\mathbf{y}_i = \mathcal{B}_i(\mathbf{z}_i)$6 reduction in storage and computational complexity without compromising performance
Hetero-SplitEE (Oda et al., 10 Dec 2025)	CIFAR-100, homogeneous Layer-3 setting	Sequential server accuracy 39.37% versus 25.81% for distributed training
Hierarchical training with early exits (Sepehri et al., 2023)	VGG-16 and ResNet-18, low bit rate channel	runtime reduced by 29% and 61% on CIFAR-10, and by 25% and 81% on Tiny ImageNet

These results show that the empirical advantage can arise from different sources. In SplitEE and I-SplitEE, the benefit is mainly online selection of the split and exit/offload policy under shifting conditions (Bajpai et al., 2023, Bajpai et al., 2024). In the Exit Predictor and predefined sparsity work, the benefit comes from suppressing local exit overhead or reducing the size of the head model itself (Dong et al., 2022, Capogrosso et al., 2024). In Hetero-SplitEE and hierarchical training, the gain is as much about training or system organization as about inference latency (Oda et al., 10 Dec 2025, Sepehri et al., 2023).

The application range is correspondingly broad. The survey literature emphasizes computer vision, edge inference, and 5G/fog environments (Matsubara et al., 2021, Scardapane et al., 2020). Subsequent work extends the paradigm to NLP with ElasticBERT (Bajpai et al., 2023), to heterogeneous IoT training with ResNet-18 (Oda et al., 10 Dec 2025), and to real-time DDoS mitigation in programmable networks (Karrakchou et al., 15 Sep 2025). This suggests that split early-exit logic is better understood as a systems pattern than as a modality-specific architecture.

6. Design tensions, misconceptions, and open problems

A common misconception is that early exiting is equivalent to thresholding the maximum softmax probability. That description matches some systems, but the literature is materially broader. EENets learn an explicit confidence branch rather than deriving confidence heuristically from class scores (Demir et al., 2024). Hetero-SplitEE uses entropy-based confidence (Oda et al., 10 Dec 2025). DistrEE uses feature difference instead of output confidence (Peng et al., 6 Feb 2025). EERO formulates the choice of exit head as classification with reject option and calibrates the classification probabilities across heads under a fixed budget by exponential weights, rather than by ad hoc threshold choice (Valade et al., 2024).

A second misconception is that the split point is an architecture constant. The split-computing survey notes that, without bottleneck injection, the best split in many tested models was often at the input or output, meaning that naive partitioning can collapse to full local or full edge inference (Matsubara et al., 2021). SplitEE therefore learns the split point online (Bajpai et al., 2023), while Hetero-SplitEE allows distinct cut layers across clients (Oda et al., 10 Dec 2025). This suggests that “the” optimal split is usually conditional on workload, bandwidth, device heterogeneity, and data distribution.

The literature also disagrees, in a constructive way, on how tightly the components should be optimized together. JEI-DNN argues that decoupled training of gates and intermediate modules creates train-test mismatch (Regol et al., 2023). By contrast, pruning experiments show that separated pruning of the backbone and exits can be close to joint pruning in the high-accuracy regime (Görmez et al., 2022). EERO adds a third perspective by treating head selection as a constrained statistical allocation problem with explicit risk and budget terms (Valade et al., 2024). The unresolved issue is not whether joint optimization can help, but when its additional complexity is justified.

Open problems identified in the survey literature include more realistic evaluation beyond small datasets, better bottleneck design and placement, automatic threshold selection, expansion to broader sensing domains, and stronger information-theoretic foundations for the accuracy–compression trade-off (Matsubara et al., 2021). More recent work adds online unsupervised adaptation under deployment shift: the abstract of I-SplitEE explicitly targets sequential data without ground truths and environmental distortions such as time of day, lighting, and weather (Bajpai et al., 2024). A plausible implication is that future split early-exit systems will be judged less by static average FLOPs and more by how stably they adapt to changing resource and data conditions.