Design Insights into Partition Placement and Routing for DNN Inference in Multi-Hop Edge Networks

Abstract: Partitioned DNN inference is a promising approach for latency-sensitive intelligent services in edge networks, since it allows different parts of a model to be executed across end devices, edge servers, and the cloud. However, in a multi-hop edge network, partition placement and inference traffic routing are inherently coupled: raw inputs, intermediate features, and final outputs may have very different sizes, while candidate nodes also differ in computation capability. In addition, both communication and computation delays can become congestion-dependent under load. In this paper, we study joint partition placement and routing for fixed-partition DNN inference over heterogeneous multi-hop edge networks. We consider a small number of DNN partitions, each placed at exactly one node without replication, and formulate a congestion-aware mixed discrete--continuous optimization problem that captures both routing and execution costs. To solve it, we develop a practical alternating framework that couples partition placement with congestion-aware forwarding updates. Through numerical evaluation on hierarchical, regular, synthetic irregular, and real backbone-inspired topologies, we show that split flexibility is particularly important in IoT--edge--cloud settings, while congestion-aware refinement becomes increasingly beneficial as the offered load grows. We further illustrate how the preferred operating point depends on the communication--computation tradeoff.

• The paper presents an alternating algorithm that jointly optimizes DNN partition placement and routing to mitigate congestion in multi-hop networks.
• It utilizes marginal cost evaluation and greedy placement updates to effectively balance computation and communication costs.
• Numerical evaluations show significant latency and performance improvements over fixed and congestion-unaware baselines in diverse topologies.

Design Insights into Partition Placement and Routing for DNN Inference in Multi-Hop Edge Networks

Motivation and Context

Edge AI deployments increasingly utilize partitioned Deep Neural Network (DNN) inference to distribute layer-wise execution between end devices, edge nodes, and the cloud. This division mitigates resource contention on constrained nodes and reduces communication latency. However, the coupling of DNN partition placement and inference traffic routing—a critical concern in multi-hop heterogeneous edge networks—remains insufficiently explored in prior literature, which often assumes fixed pipelines or routes. The examined work directly addresses this by coupling discrete partition deployment decisions with congestion-aware multi-hop traffic forwarding, leveraging a comprehensive network model.

Figure 1: Inference with DNN partitions in a multi-hop network, highlighting the distributed execution from data source through partition hosts to output destination.

Problem Formulation and Methodology

The paper formalizes the joint DNN partition placement and routing optimization as a mixed discrete–continuous problem. The model supports arbitrary topologies (mesh, hierarchical, backbone-inspired) with heterogeneous computation/communication resources and explicitly accounts for resource congestion effects via convex, differentiable cost functions on both links and nodes. DNN partition placement is captured by binary variables under non-replication constraints: each partition is deployed uniquely at a single node. Traffic routing is described using node-based forwarding variables for each inference stage (input, activation, output), respecting structured transformations at partition points.

The overall objective is to minimize a weighted sum of network communication and computation costs across all induced flows: $$J(\boldsymbol{x}, \boldsymbol{\phi}) = \sum D_{ij}(F_{ij}) + \sum C_i(G_i)$$ where $F_{ij}$ and $G_i$ represent link and node loads respectively, as functions of placement and forwarding.

This is inherently nonconvex and NP-hard due to the combinatorial partition placement and intricate congestion dependencies. As solving this optimally is intractable at scale, the authors propose an iterative practical algorithm (ALT): alternating between fixing partition placement to optimize congestion-aware forwarding and vice versa.

Alternating Congestion-Aware Solution Framework

The proposed ALT algorithm operates on two interdependent subproblems at each iteration:

1. Forwarding Update: With partition placement fixed, it leverages marginal system cost metrics (derived from Gallager’s distributed minimum-delay routing) to adaptively re-route stage-wise DNN traffic in response to congestion.
2. Placement Update: Given current traffic forwarding, node scores for hosting each partition are computed using path-based surrogates of anticipated communication and computation cost. Placement is greedily updated to minimize objective surrogates, using single-source shortest-path procedures with congestion-adjusted link weights.

This alternating approach eschews exhaustive search or large-scale MIP solvers, relying instead on efficient node-centric message-passing and marginal-cost evaluation for scalable applicability.

Figure 3: Sample topology involving an IoT network on the edge, illustrating the spatial distribution and heterogeneity of candidate nodes.

Numerical Evaluation and Comparative Results

Extensive experiments are performed on four representative topologies: hierarchical IoT, mesh, small-world, and GEANT-like backbone. Each system has parametrized network sizes, link and node capacities, and DNN partition sizes. Four methods are compared:

• ALT: Full alternating congestion-aware placement and routing.
• OneShot: Single-pass placement and routing update.
• CongUnaware: Congestion-unaware, shortest-extended-path baseline.
• CoLocated: Only colocated partitioning (no functional split).

Strong numerical findings indicate that the ALT algorithm consistently outperforms all baselines in normalized global cost (latency/queueing functions). The performance gap widens substantially under increased load or in scenarios with heavy computation–communication heterogeneity, as shown in figures summarizing cost vs. input rate and the objective's sensitivity to the communication–computation tradeoff parameter.

Figure 2: Normalized objective $J$ across scenarios evidencing the consistent advantage of congestion-aware iterative refinement.

Figure 6: Left—Sample IoT topology with node/link heterogeneity; Center—Objective vs. input-rate scaling, exposing sensitivity to congestion; Right—Tradeoff between communication and computation costs revealing optimal operating points.

Key empirical insights include:

• ALT exploits split flexibility, especially in hierarchical IoT scenarios, by placing the compression-oriented first partition at resource-constrained nodes and heavier components on edge/cloud nodes. This yields tangible benefits in both total latency and robustness under load.
• CongUnaware and OneShot degrade faster with load as they neglect or myopically react to congestion feedback.
• The communication–computation tradeoff can be smoothly traversed; the system adapts partition placement to prioritize cost components as the tradeoff parameter varies, without incurring disproportionate loss in the other metric.

Implications and Future Directions

This research delivers a principled, numerically validated approach for DNN inference deployment in complex multi-hop edge environments. The formalization of congestion-aware, joint partition placement and routing with discrete (non-replicated) partitioning directly informs real-world networked AI system design. The results demonstrate tangible, sometimes substantial, latency reductions over conventional fixed-placement or stateless approaches, particularly under high load and strong node heterogeneity.

Practically, this method enables edge AI orchestrators to continuously adapt deployment and routing as network and workload conditions fluctuate, balancing privacy, response time, and resource utilization. From a theoretical perspective, the work offers a concrete methodological bridge between network optimization, distributed control, and DNN system design, introducing scalable alternating algorithms for challenging mixed-integer, congestion-coupled settings.

Subsequent directions could address:

• Fine-grained partitioning (more than two blocks, or dynamic replication under resource/statistical constraints).
• Adaptive online versions coping with time-varying workloads, failures, or dynamic topology.
• Privacy/security integration, e.g., partition host trust constraints.
• Distributed or hierarchical implementations potentially exploiting consensus or federated coordination to scale further.

Conclusion

This paper provides a rigorous treatment of congestion-aware DNN partition placement and routing for multi-hop edge networks, supporting heterogeneous, capacity-limited nodes and general topologies. The alternating framework—comprising iterative marginal-cost-based forwarding and placement—robustly adapts to workload, partition structure, and network congestion. Split execution flexibility and explicit congestion modeling drive substantial performance improvements, especially in realistic, heavily loaded edge environments. These methodological advancements pave the way for the next generation of adaptive and efficient edge AI system deployments.