Forward Split Networks Overview

• Forward split networks are a strategy that partitions deep neural networks across edge and server nodes to efficiently balance computation and communication.
• They employ saliency-based scoring and optimization algorithms to determine optimal split points that minimize latency and preserve accuracy.
• Applications in edge AI, smart manufacturing, and federated learning demonstrate significant reductions in memory footprint and inference delays.

Forward split networks are an architectural paradigm in which a deep neural network is partitioned along the forward computational path and distributed across multiple physically distinct compute nodes. Typically, early layers ("head") run on resource-constrained or edge devices, and later layers ("tail") execute on a more capable server or aggregator. This configuration is also known as split computing or split neural architectures and is used to leverage both local and remote resources for inference and training. Forward split networks reduce communication overhead by transmitting intermediate activations (“smashed data”) instead of raw inputs, enabling lower inference latency and scalable distributed learning while preserving model accuracy comparable to centralized deployment (Capogrosso et al., 2023, Li et al., 23 Jun 2025).

1. Formal Problem Statement and Mathematical Foundations

In a forward split network, a DNN model $M$ of $I$ layers is decomposed at split index $i$:

• Edge node ("head"): Executes layers $L_1,\ldots,L_i$, outputting activation $z_i$.
• Server node ("tail"): Consumes $z_i$ and executes layers $L_{i+1},\ldots,L_I$ to produce the final output.

Formally,

$$M(x) = (L_{i+1} \circ \dots \circ L_I)(\, (L_1 \circ \dots \circ L_i)(x)\,)$$

The total end-to-end inference latency for a single sample, when split at layer $i$, is modeled as:

$$T_{\text{total}}(i) = T^{\text{edge}}_{\text{comp}}(i) + T_{\text{comm}}(i) + T^{\text{server}}_{\text{comp}}(i)$$

with each term quantifying edge computation, transmission, and server computation, respectively (Capogrosso et al., 2023).

Optimally placing the split point involves a constrained optimization to maximize accuracy $I$0 while ensuring $I$1 (quality-of-service constraint) or, more generally, minimizing a weighted sum $I$2 (Capogrosso et al., 2023).

Extension to arbitrarily complex models is achieved by representing the DNN as a directed acyclic graph (DAG), with the forward splitting problem reducible to an $I$3–$I$4 minimum-cut problem. The optimal split minimizes the cost of edges crossing the cut, which encode device/server computation and communication delays (Li et al., 23 Jun 2025).

2. Candidate Split Point Selection and Saliency-Based Methods

Identifying effective split points is nontrivial. Exhaustive try-and-test approaches are inefficient for large architectures. Saliency-based scoring, as implemented in the Split-Et-Impera framework, utilizes class activation maps (Grad-CAM) to measure the information contribution of each layer. Layers corresponding to local maxima in cumulative saliency are flagged as candidate split points, as these represent structural hinge points where the network’s decision-making is most sensitive to the intermediate representation (Capogrosso et al., 2023).

The selection workflow is as follows:

1. Compute cumulative saliency for all layers.
2. Identify local maxima as candidates.
3. For each candidate, instantiate a split (with optional bottleneck autoencoder), measure accuracy, and simulate latency.
4. Prune splits violating QoS constraints.
5. Select the split maximizing the desired utility function (Capogrosso et al., 2023).

Block-wise abstraction for models with repetitive structures—such as residual or inception blocks—reduces the splitting problem's complexity. Under certain conditions, entire blocks are collapsed into supernodes in the DAG, leading to significant speedups (2–70×) in split discovery (Li et al., 23 Jun 2025).

3. Communication, Latency, and Resource Models

The communication cost between split points is a function of the activation tensor size and the available network bandwidth. For a given split at layer $I$5, the latency $I$6, where $I$7 is the size of $I$8, and the additional terms account for protocol and retransmission (Capogrosso et al., 2023). In dynamic network settings, split location can be adapted at run time based on real-time channel conditions and server load (Bakhtiarnia et al., 2022).

Key optimization objectives:

• Minimize device footprint: Offloading as much computation as possible without exceeding bandwidth or latency budgets (Tassi et al., 7 Sep 2025).
• Balance accuracy and efficiency: Selecting split points that compress raw inputs to low-dimensional features for transmission without significant accuracy degradation (Capogrosso et al., 2023).
• Support for multihop and chain topologies: Service Function Chaining architectures extend split computing to multi-hop topologies, where a sequence of sub-models is deployed over a chain of compute nodes, ensuring adaptive, efficient routing and computation as network conditions change (Hara et al., 12 Sep 2025).

Resource-constrained optimization is further formalized as an integer programming problem over layer-device mappings (subject to per-device CPU/memory constraints), with greedy heuristics offering fast, near-optimal solutions for large, heterogeneous device sets (Tassi et al., 7 Sep 2025).

4. Practical Algorithms and Network Architectures

Several system-level frameworks and architectural solutions support forward split networks:

• Split-Et-Impera (Capogrosso et al., 2023): Automates interpretability-driven split search, communication-aware simulation, and QoS-based selection, substantially reducing the manual effort for split-point design.
• Fast DAG-based model splitting (Li et al., 23 Jun 2025): Models complex network topologies as DAGs and applies max-flow/min-cut algorithms for provably optimal, millisecond-scale split discovery, supporting layered, block-structured models.
• Dynamic split computing (Bakhtiarnia et al., 2022): Identifies ‘natural bottlenecks’ in DNNs—layers where feature map size is minimized relative to input—and dynamically shifts split points based on measured link and compute metrics without retraining.
• SplitNets (Dong et al., 2022): Embeds split-awareness into Neural Architecture Search, simultaneously optimizing network structure, split location, and compression module parameters for system-constrained inference on embedded/multi-view systems.

As a hardware-agnostic abstraction, NSN-style architectures permit on-the-fly detachment of network layers at inference time, supporting a spectrum of “thin” to “wide” models with shared parameterization and efficient training regimens (Fuengfusin et al., 2019).

5. Applications and Empirical Results

Forward split networks are deployed in a broad range of distributed intelligence scenarios:

• Industry 4.0 and smart manufacturing: Edge-server splits running VGG16 for conveyor belt inspection maintain $I$9 accuracy drop while achieving up to 6.7× latency reductions compared to remote-only execution (e.g., $i$0 ms for split at layer 15 vs. $i$1 ms for fully remote) (Capogrosso et al., 2023).
• Training acceleration: In federated and split settings (e.g., group-based split federated learning), parallelism across groups and efficient split placement leads to 31.5% overall training delay reduction vs. vanilla split/federated learning, while achieving comparable or improved test accuracy (Zhang et al., 2023, Li et al., 23 Jun 2025).
• Edge/IoT systems: On resource-limited user equipment, heuristic split computing reduces memory and CPU footprints by over 33.6% and 60%, respectively, versus full local inference (Tassi et al., 7 Sep 2025), and adapts to dynamic link conditions (Bakhtiarnia et al., 2022).
• Multi-hop inference and dynamic routing: SFC-based architectures for multi-hop split inference achieve real-time inference latencies ($i$239 ms for $i$3) and automatic path reconfiguration under congestion without incurring extra overhead (Hara et al., 12 Sep 2025).
• Compression-aware splits in Transformers: Explicit forward splitting of FFN modules according to heavy-hitter neuron statistics yields up to 43.1% parameter reduction and 1.25–1.56× inference speedup with minimal accuracy loss for LLMs (Liu et al., 2024).

6. Extensions: Security, Learning, and Multi-view Fusion

SplitNN variants in vertical federated learning formalize the security–performance trade-off at the split point. Security Forward Aggregation (SFA) combines forward splitting with cryptographic masking to achieve central-model-level performance while ensuring individual feature privacy under realistic adversarial models (Cai et al., 2022).

Recent advances in decoupled split learning replace backward gradient exchange with per-partition auxiliary losses, effectively halving communication per iteration and reducing client memory by up to 58%, at negligible accuracy cost (Zihad et al., 27 Jan 2026).

Split-aware multi-view fusion, as realized in SplitNets, enables distributed camera arrays to fuse features optimally for system latency and memory—allowing simultaneous early compression and high-accuracy aggregation (Dong et al., 2022).

7. Empirical Trade-offs, Limitations, and Future Directions

While forward split networks enable substantial improvements in latency, communication and resource use, design involves nontrivial trade-offs:

• Communication bottlenecks: Finer splits increase utilization flexibility but may worsen total communication overhead (Hara et al., 12 Sep 2025).
• Static partitioning: Fixed split points may not be optimal as network conditions or system resources change, necessitating dynamic or adaptive algorithms (Bakhtiarnia et al., 2022).
• Block-structure assumption: Block abstraction accelerates split discovery but may underperform in unstructured architectures lacking repeated blocks (Li et al., 23 Jun 2025).
• Compression vs. accuracy: Aggressive compression at the split point (e.g., bottleneck autoencoders) must be balanced against information loss to avoid accuracy degradation (Capogrosso et al., 2023, Liu et al., 2024).

Future research explores adaptive or data-dependent split locations, integration of non-linear compression modules, dynamic switching of activation routing, and further synergy between split learning, federated/multi-party privacy schemas, and split-aware NAS for edge devices. The methodology continues to generalize to Transformer and LLM architectures, optical and analog compute nodes, and fine-grained IoT-surround deployment scenarios (Liu et al., 2024, Dong et al., 2022).