Papers
Topics
Authors
Recent
Search
2000 character limit reached

Adaptive DNN Partitioning and Offloading in Heterogeneous Edge-Cloud Continuum

Published 10 May 2026 in cs.DC, cs.AI, cs.LG, cs.NI, and cs.PF | (2605.09623v1)

Abstract: In recent years, the use of artificial intelligence on resource-constrained IoT devices has grown significantly. However, existing approaches to DNN partitioning and offloading across the edge-cloud continuum typically rely on static methods that ignore runtime dynamics. Furthermore, they are often evaluated in simulated environments rather than on real hardware. To address this gap, we propose a framework that dynamically splits neural network layers across the heterogeneous continuum. The framework profiles the model at startup, measures network link conditions between nodes, and periodically re-evaluates the partition to adapt to environmental changes. We created a physical testbed comprising a Raspberry Pi edge device, a laptop fog, and a high-performance desktop PC as the cloud. We evaluated the framework over three widely adopted convolutional neural networks: VGG16, AlexNet, and MobileNetV2. Our results show that the framework achieves reductions in energy and end-to-end latency of 27.09--35.82% and 6.34--22.92%, respectively, compared to a static partitioning baseline. These findings confirm the superiority of adaptive to static partitioning.

Summary

  • The paper introduces a measurement-driven scheduler that periodically reselects DNN layer splits across edge, fog, and cloud nodes using link probing, runtime profiling, deadline filtering, and an energy-weighted objective.
  • The evaluation on Raspberry Pi, laptop, and GPU hardware shows adaptive partitioning reduces latency by 6.34–22.92% and energy by 27.09–35.82% across VGG16, AlexNet, and MobileNetV2.
  • The results show why static splits can fail—especially for lightweight models such as MobileNetV2—and demonstrate a baseline safeguard that prevents reconfiguration unless the predicted objective improves by at least 3% or a deadline is violated.

Motivation and problem statement

The paper addresses a persistent gap in the literature on distributed DNN inference: most partitioning and offloading frameworks are evaluated in simulation under idealized network assumptions, and energy consumption is rarely treated as a primary optimization objective. The authors argue that static, pre-defined partition points are inherently mismatched to the edge-cloud continuum, where bandwidth fluctuates, workloads contend, and node capabilities vary by orders of magnitude. Their response is an adaptive framework that periodically re-evaluates the layer split across three tiers—edge, fog, and cloud—and reconfigures inference only when runtime measurements predict a better partition, subject to a hard latency deadline.

The work is positioned against several recent efforts. Li et al. jointly optimize fine-grained layer partitioning and magnitude-based channel pruning with an LSTM controller trained via policy gradients, but evaluate entirely in simulation in a mobile edge computing (MEC) setting. EdgeDI combines structured pruning with execution-time-aware feature-map partitioning for industrial IoT, but ignores energy. DRL-based offloading studies tend to report task completion time while omitting per-node energy accounting. The present paper's distinguishing claim is that it validates adaptive partitioning on physical hardware with hardware-measured energy metrics—a combination the authors state is largely absent from prior work.

Framework design

The framework assumes a continuum of one edge node, one fog node, and one cloud node, with a candidate split (i,j)(i,j) assigning layers 0,…,i0,\dots,i to the edge, i+1,…,ji+1,\dots,j to the fog, and j+1,…,N−1j+1,\dots,N-1 plus the classifier head to the cloud. Operation proceeds through five stages.

Offline profiling runs once at startup, producing two lookup tables: per-boundary activation sizes in bytes, and normalized per-layer compute weights derived from a single timed execution. These weights allow latency and energy estimates for arbitrary unseen splits to be extrapolated from a small number of measured probe splits.

Two-point link probing fits a linear round-trip-time model rtth(s)=ω+s/βrtt_h(s) = \omega + s/\beta per hop, where ω\omega is fixed overhead and β\beta effective throughput, estimated from averaged RTTs of two contrasting payload sizes. A malformed probe (where the larger payload does not take longer) causes retention of the previous model rather than corruption of the link estimate.

Candidate estimation predicts end-to-end latency as the sum of three per-node compute times (profiled weights scaled by fitted per-node execution rates) and two transfer times from the link models. Energy is computed per node using a fixed 12 W power model on the Raspberry Pi and empirically fitted rates on the fog and cloud nodes; prediction residuals feed back into rate refinement each cycle.

Best-candidate search exhaustively evaluates all valid (i,j)(i,j) pairs under a weighted, dimensionless objective combining predicted edge energy, total system energy, and latency, with normalization anchors drawn from startup probes. Two filters are notable: a deadline pre-filter rejecting candidates whose predicted latency exceeds Lmax⁡L_{\max}, and a guarantee that no candidate is accepted unless it beats the static baseline's score—an explicit safeguard against regression relative to the configuration being improved upon.

Adaptive scheduling operates in two phases. Initialization runs 50 inferences at the user-specified initial split to establish the threshold, then 15 inferences at each of three probe splits chosen to expose edge-heavy, balanced, and cloud-heavy distributions, before fitting rates and selecting the starting split. Steady-state operation re-fits compute rates (retaining phase-1 data to keep them grounded), re-probes links, and re-runs the search every 100 inferences, switching only if the objective improves by at least 3%. Deadline violations force a switch regardless of margin, falling back to the static baseline if no better candidate exists. Objective weights emphasize energy over latency: edge energy 0.6–0.9, total energy 0.2–0.3, latency 0.1–0.3.

Experimental setup

The testbed comprises a Raspberry Pi 4 (4-core Cortex-A76, 8 GB), a laptop with an Intel i7-10510U, and a desktop with an NVIDIA RTX 4070 Ti, interconnected over Tailscale VPN with ZeroMQ messaging; network throttling simulates constrained links. Three CNNs spanning a wide complexity range are evaluated: VGG16 (138M parameters), AlexNet (61M), and MobileNetV2 (2.2M). Inputs are random tensors of shape 1×3×224×2241 \times 3 \times 224 \times 224, deliberately isolating computational workload from dataset variability. Energy is measured independently per node—constant-power model on the Pi, Intel RAPL package counters on the laptop, NVML-integrated GPU power on the desktop—with wall-clock latency. Each configuration runs 500 inferences, repeated ten times and averaged.

Results

Single-device baselines expose roughly two orders of magnitude between edge and cloud for VGG16 and AlexNet (666.87 ms vs. 1.16 ms for VGG16, approximately 570×), but only 17× for MobileNetV2 (71.90 ms vs. 4.18 ms), consistent with that architecture's design for constrained devices. These figures exclude transfer costs and thus represent idealized lower bounds.

Static partitioning with approximately equal thirds improves VGG16 latency by 21.2% and AlexNet by 41.0% over single-device edge execution, but hurts MobileNetV2, increasing latency by 36.9% because transfer overhead outweighs its low compute cost. This is an important negative result: it demonstrates that even a reasonable static split can be counterproductive for lightweight models, motivating adaptivity rather than merely finer tuning of fixed cuts.

Under adaptive partitioning, all three models improve on both metrics relative to the static baseline:

Model Latency reduction Energy reduction
VGG16 6.34% 35.82%
AlexNet 22.92% 35.70%
MobileNetV2 14.20% 27.09%

Energy reductions consistently exceed latency reductions, which the authors attribute directly to the energy-dominant weighting of the objective function. The per-model behavior differs instructively: VGG16 yields the largest absolute energy saving (2.04 J) but the smallest latency gain, indicating the scheduler traded transfer overhead for offloading away from the energy-hungry edge node; AlexNet achieves both large energy savings and the largest latency improvement; MobileNetV2 shows the smallest energy gain, with the scheduler shifting heavier computation toward the more efficient cloud node (cloud energy rising from 0.03 J to 0.10 J).

Limitations and open questions

Several constraints bound the generality of these results. First, the evaluation uses dummy input tensors, so the measurements capture pure compute and communication cost without data-dependent behavior or real application workloads. Second, network bottlenecks are induced via Tailscale traffic throttling rather than genuinely dynamic wireless conditions, mobility, or multi-tenant contention—the very environmental variability that motivates adaptivity is only partially exercised. Third, the testbed contains exactly one node per tier with a single sequential pipeline; scaling to multiple candidate nodes per tier, parallel branches, or transformer architectures with different activation-size profiles remains unexamined. Fourth, the exhaustive candidate search is tractable only because the models have modest layer counts; complexity under deeper networks is not analyzed. Finally, the comparison against related work (e.g., Li et al.'s 27.31% latency reduction, EdgeCI's 34.72–43.52%) is qualitative, since objectives and testbeds differ; no head-to-head evaluation on shared hardware is performed. Whether the 3% switching hysteresis and 100-inference window generalize to faster-fluctuating environments is left open.

Conclusion

This paper delivers a concrete, reproducible demonstration that periodic, measurement-driven re-partitioning outperforms static splitting on real heterogeneous hardware, with energy reductions of 27–36% and latency reductions of 6–23% across three CNNs spanning two orders of magnitude in model size. Its principal contribution is methodological: hardware-measured per-node energy accounting combined with a lightweight weighted-objective scheduler that provably never underperforms its static baseline. The open questions concern scale—deeper architectures, richer topologies, and genuinely volatile networks—rather than the core mechanism, which the results establish as sound within the tested regime.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.