IsoSched: Preemptive Tile Spatial Scheduler

• IsoSched is a framework for preemptive Tile Spatial Scheduling of multi-DNN workloads that reduces DRAM traffic and latency compared to traditional methods.
• It combines an ILP formulation with subgraph isomorphism using an Ullmann-based algorithm enhanced by Monte Carlo Tree Search for dynamic remapping.
• The framework integrates Layer Concatenate and Split techniques along with tile-level preemption to balance load, optimize throughput, and meet stringent real-time deadlines.

Searching arXiv for IsoSched and closely related work to ground the article in current papers. IsoSched is principally the name of a framework for preemptive Tile Spatial Scheduling (TSS) of concurrent multi-DNN workloads on edge and cloud accelerators. It was introduced as the first framework enabling preemptive multi-DNN scheduling on TSS architecture, combining an ILP formulation for compute and communication scheduling with a subgraph isomorphism formulation for dynamic remapping, an Ullmann-based algorithm enhanced by Monte Carlo Tree Search (MCTS), Layer Concatenate and Split (LCS) for load balancing, and Compressed Sparse Row (CSR) encoding for memory reduction (Zhao et al., 27 Aug 2025). In a distinct literature, the name also appears as an implication of an SDIO-based planning tool for Leksell Gamma Knife Icon radiosurgery, where “IsoSched” denotes a possible software layer around simultaneous sector duration and isocenter optimization rather than the multi-DNN scheduler itself (Cevik et al., 2018).

1. Problem setting and architectural scope

IsoSched was introduced against the contrast between Layer Temporal Scheduling (LTS) and Tile Spatial Scheduling (TSS). In LTS, intermediate activations are cached in DRAM and reloaded between layers, and the paper cites up to 27% of total energy spent on DRAM accesses in LTS. TSS instead partitions inter-layer activations into small tiles communicated through low-latency on-chip links, allowing downstream layers to begin when partial outputs arrive and thereby reducing DRAM traffic and latency (Zhao et al., 27 Aug 2025).

The framework targets concurrent execution of multiple DNNs with complex topologies, especially settings in which critical tasks must preempt others to meet stringent latency requirements. The motivating examples are explicitly real-time: obstacle detection must complete within tens of milliseconds; autonomous driving may require 6–9 ms response at high speeds; and AR/VR image update under $\sim 20$ ms is noted to avoid motion sickness. The workloads considered include graphs with thousands of nodes and $10^4$–$10^5$ edges, including DeepSeek-7B, Qwen-7B, and Llama-3-8B (Zhao et al., 27 Aug 2025).

The architectural model is a TSS accelerator with spatial compute engines, PEs, on-chip buffers, and an on-chip interconnect (NoC) with bi-directional local links and longer global links. The framework assumes bounded compute, link bandwidth, and buffer capacity. In the reported platforms, the paper states Edge: 64 MACs/engine and Cloud: 128 MACs/engine, with equalized MAC budgets across baselines (Zhao et al., 27 Aug 2025).

A central claim of IsoSched is therefore not merely that preemption is useful, but that preemption must be made compatible with TSS, so that the latency and energy benefits of on-chip tile streaming are not lost to the DRAM-heavy execution style characteristic of LTS-based preemption schemes such as PREMA, Planaria, CD-MSA, and MoCA (Zhao et al., 27 Aug 2025).

2. Formal scheduling model

IsoSched defines the tile as the fundamental scheduling unit. The engine timeslot is derived from per-tile MAC demand and the number of engine PEs, plus pipeline fill latency. For convolution and attention, the paper gives

$$T_{\mathrm{conv}} \;=\; \left\lceil \frac{W_o \cdot C_o \cdot K_h \cdot K_w \cdot C_{\mathrm{in}}}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}$$

and

$$T_{\mathrm{attn}} \;=\; \left\lceil \frac{N_k \cdot h \cdot d_k}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}.$$

The minimum tile time across compute-bearing layers defines the engine timeslot used for time discretization (Zhao et al., 27 Aug 2025).

The scheduling formulation uses a compute scheduling tensor

$$\mathcal{X} \in \{0,1\}^{D \times I \times N \times T \times P},$$

where $\mathcal{X}_{d,i,n,t,p}=1$ iff tile $(d,i,n)$ executes on PE $p$ at timeslot $t$, and a communication scheduling tensor

$10^4$0

where $10^4$1 iff DAG edge $10^4$2 uses link $10^4$3 at $10^4$4. Start and finish times are represented by $10^4$5 and $10^4$6, tile preemption by $10^4$7, and deadline satisfaction by $10^4$8 (Zhao et al., 27 Aug 2025).

The composite objective is written as

$10^4$9

with $10^5$0 as priority weights, $10^5$1 as a latency weight, $10^5$2 as a communication cost weight, and $10^5$3 as a preemption penalty weight. The paper presents this as a formulation aligned with Latency-Bound Throughput (LBT), lower latency, and lower communication cost (Zhao et al., 27 Aug 2025).

The constraint set includes:

• Tile execution exactly once:

$10^5$4

• Execution span definitions:

$10^5$5

• Precedence:

$10^5$6

• Engine capacity:

$10^5$7

• Link bandwidth:

$10^5$8

• Deadline satisfaction:

$10^5$9

Communication cost is modeled by Manhattan distance,

$$T_{\mathrm{conv}} \;=\; \left\lceil \frac{W_o \cdot C_o \cdot K_h \cdot K_w \cdot C_{\mathrm{in}}}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}$$0

The model also includes preemption-resume overhead through

$$T_{\mathrm{conv}} \;=\; \left\lceil \frac{W_o \cdot C_o \cdot K_h \cdot K_w \cdot C_{\mathrm{in}}}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}$$1

with

$$T_{\mathrm{conv}} \;=\; \left\lceil \frac{W_o \cdot C_o \cdot K_h \cdot K_w \cdot C_{\mathrm{in}}}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}$$2

This formulation makes compute placement, communication routing, deadline handling, and preemption part of a single optimization structure rather than separate heuristics (Zhao et al., 27 Aug 2025).

3. Subgraph isomorphism as the remapping core

IsoSched reduces dynamic remapping to subgraph isomorphism. The arriving DNN DAG is represented by adjacency matrix $$T_{\mathrm{conv}} \;=\; \left\lceil \frac{W_o \cdot C_o \cdot K_h \cdot K_w \cdot C_{\mathrm{in}}}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}$$3, the current preemptible DAG by $$T_{\mathrm{conv}} \;=\; \left\lceil \frac{W_o \cdot C_o \cdot K_h \cdot K_w \cdot C_{\mathrm{in}}}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}$$4, and the scheduler seeks a mapping $$T_{\mathrm{conv}} \;=\; \left\lceil \frac{W_o \cdot C_o \cdot K_h \cdot K_w \cdot C_{\mathrm{in}}}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}$$5 such that

$$T_{\mathrm{conv}} \;=\; \left\lceil \frac{W_o \cdot C_o \cdot K_h \cdot K_w \cdot C_{\mathrm{in}}}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}$$6

while resource and bandwidth constraints remain feasible (Zhao et al., 27 Aug 2025).

The paper uses an Ullmann-based algorithm because classical Ullmann matching supports pruning by degree constraints, adjacency consistency, and label constraints, though it retains worst-case exponential complexity $$T_{\mathrm{conv}} \;=\; \left\lceil \frac{W_o \cdot C_o \cdot K_h \cdot K_w \cdot C_{\mathrm{in}}}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}$$7. To improve practical runtime, IsoSched augments this with Monte Carlo Tree Search (MCTS). The reported MCTS steps are the standard sequence of selection, expansion, simulation, and backpropagation, with selection based on

$$T_{\mathrm{conv}} \;=\; \left\lceil \frac{W_o \cdot C_o \cdot K_h \cdot K_w \cdot C_{\mathrm{in}}}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}$$8

Simulation evaluates a mapping by computing $$T_{\mathrm{conv}} \;=\; \left\lceil \frac{W_o \cdot C_o \cdot K_h \cdot K_w \cdot C_{\mathrm{in}}}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}$$9 and checking whether $$T_{\mathrm{attn}} \;=\; \left\lceil \frac{N_k \cdot h \cdot d_k}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}.$$0; the reward is +1 for success, −1 otherwise (Zhao et al., 27 Aug 2025).

The paper’s pseudocode, named MCUSubgraphIsomorphism, returns the best mapping $$T_{\mathrm{attn}} \;=\; \left\lceil \frac{N_k \cdot h \cdot d_k}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}.$$1 after $$T_{\mathrm{attn}} \;=\; \left\lceil \frac{N_k \cdot h \cdot d_k}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}.$$2 search iterations. Actions are generated by pairwise swaps in $$T_{\mathrm{attn}} \;=\; \left\lceil \frac{N_k \cdot h \cdot d_k}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}.$$3. This makes IsoSched’s search neither purely exhaustive nor purely greedy: Ullmann contributes pruning and structural validity checks, while MCTS contributes guided sampling over the mapping space (Zhao et al., 27 Aug 2025).

A second ingredient is CSR encoding. IsoSched encodes $$T_{\mathrm{attn}} \;=\; \left\lceil \frac{N_k \cdot h \cdot d_k}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}.$$4, $$T_{\mathrm{attn}} \;=\; \left\lceil \frac{N_k \cdot h \cdot d_k}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}.$$5, $$T_{\mathrm{attn}} \;=\; \left\lceil \frac{N_k \cdot h \cdot d_k}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}.$$6, and $$T_{\mathrm{attn}} \;=\; \left\lceil \frac{N_k \cdot h \cdot d_k}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}.$$7 in CSR rather than dense adjacency matrices. The asymptotic memory comparison stated in the paper is

$$T_{\mathrm{attn}} \;=\; \left\lceil \frac{N_k \cdot h \cdot d_k}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}.$$8

The reported ablation shows CSR compression ratio vs dense: $$T_{\mathrm{attn}} \;=\; \left\lceil \frac{N_k \cdot h \cdot d_k}{\#\mathrm{PE}_{\mathrm{engine}}} \right\rceil + \mathrm{filling\_time}.$$9 (Simple), $$\mathcal{X} \in \{0,1\}^{D \times I \times N \times T \times P},$$0 (Middle), $$\mathcal{X} \in \{0,1\}^{D \times I \times N \times T \times P},$$1 (Complex). Likewise, MCTS acceleration yields matching-time reduction $$\mathcal{X} \in \{0,1\}^{D \times I \times N \times T \times P},$$2 (Simple), $$\mathcal{X} \in \{0,1\}^{D \times I \times N \times T \times P},$$3 (Middle), $$\mathcal{X} \in \{0,1\}^{D \times I \times N \times T \times P},$$4 (Complex) (Zhao et al., 27 Aug 2025).

This design establishes a central property of IsoSched: scheduling is not treated only as time-slot assignment, but as a graph-embedding problem over the current accelerator occupancy and interconnect state.

4. Load balancing, preemption policy, and runtime mechanics

IsoSched supplements graph matching with Layer Concatenate and Split (LCS) to reduce pipeline imbalance. The trigger is the coefficient of variation

$$\mathcal{X} \in \{0,1\}^{D \times I \times N \times T \times P},$$5

with a threshold of 15%, described as empirically chosen within a typical 10–20% definition. When imbalance is detected, concatenate merges adjacent small-workload layers into a segment mapped to one engine, while split partitions a heavy layer across engines (Zhao et al., 27 Aug 2025).

For concatenation, the paper gives the minimum buffer sizes

$$\mathcal{X} \in \{0,1\}^{D \times I \times N \times T \times P},$$6

and

$$\mathcal{X} \in \{0,1\}^{D \times I \times N \times T \times P},$$7

For splitting, H/W splitting is preferred when buffers suffice because it avoids partial-sum accumulation; otherwise IsoSched uses C splitting, which reduces buffer needs but requires accumulation (Zhao et al., 27 Aug 2025).

Preemption itself is tile-level. Before preemption, the framework offloads intermediate FM/state of the preempted task to DRAM via assigned links and overwrites weights with incoming critical-task weights via reconfiguration links. After preemption, it reloads original weights, restores routes, and resumes execution. The overhead is explicitly modeled as

$$\mathcal{X} \in \{0,1\}^{D \times I \times N \times T \times P},$$8

The paper states that IsoSched minimizes this overhead by preempting downstream engines, keeping upstream engines productive, and that among the three schemes discussed, Scheme III is preferred (Zhao et al., 27 Aug 2025).

Runtime prioritization is guided by the latency slack metric

$$\mathcal{X} \in \{0,1\}^{D \times I \times N \times T \times P},$$9

where $\mathcal{X}_{d,i,n,t,p}=1$0 is remaining execution time and $\mathcal{X}_{d,i,n,t,p}=1$1 is task priority. The paper states that larger $\mathcal{X}_{d,i,n,t,p}=1$2 indicates more slack and that tasks with smaller $\mathcal{X}_{d,i,n,t,p}=1$3 are more urgent. This policy is presented as a mechanism for avoiding starvation while preferentially preempting non-critical tasks (Zhao et al., 27 Aug 2025).

Taken together, LCS and tile-level preemption provide the runtime complement to the ILP and subgraph-isomorphism machinery: LCS stabilizes stage latency and utilization, while slack-guided preemption determines when and where the remapping mechanism should intervene.

5. Empirical results and comparative position

IsoSched is evaluated on Edge and Cloud platforms and on three workload groups: Simple (AR/VR): MobileNetV2, ResNet-50, EfficientNet; Middle (NAS): UNet, NASNet, PNASNet; and Complex (LLMs): DeepSeek-7B, Qwen-7B, Llama-3-8B. Hardware modeling uses Verilog, Synopsys DC (T-2022.03-SP5) on FreePDK45, McPAT 1.3 for NoC energy, and CACTI-P for SRAM (Zhao et al., 27 Aug 2025).

The main reported metric is Latency-Bound Throughput (LBT), defined following PREMA, Planaria, and CD-MSA as the maximum queries-per-second $\mathcal{X}_{d,i,n,t,p}=1$4 achieved under a Poisson arrival rate $\mathcal{X}_{d,i,n,t,p}=1$5 while meeting SLA. On this metric, IsoSched reports average improvements over LTS-PRM baselines of $\mathcal{X}_{d,i,n,t,p}=1$6 (PREMA-like), $\mathcal{X}_{d,i,n,t,p}=1$7 (Planaria-like), $\mathcal{X}_{d,i,n,t,p}=1$8 (CD-MSA-like), $\mathcal{X}_{d,i,n,t,p}=1$9 (MoCA-like). Against PREMA-like specifically across topology complexity, the gains are $(d,i,n)$0 (Simple), $(d,i,n)$1 (Middle), $(d,i,n)$2 (Complex) (Zhao et al., 27 Aug 2025).

For speedup, the reported average improvements versus LTS-PRM are $(d,i,n)$3 (PREMA-like), $(d,i,n)$4 (Planaria-like), $(d,i,n)$5 (CD-MSA-like), $(d,i,n)$6 (MoCA-like). For energy efficiency, the reported average improvements are $(d,i,n)$7 (PREMA-like), $(d,i,n)$8 (Planaria-like), $(d,i,n)$9 (CD-MSA-like), $p$0 (MoCA-like) (Zhao et al., 27 Aug 2025).

Against a non-preemptive TSS baseline, HASP-like, IsoSched reports higher critical task satisfaction (SLA): $p$1 (Simple), $p$2 (Middle), $p$3 (Complex). The stated explanation is that HASP’s non-preemptive policy leads to deadline misses under contention, whereas IsoSched’s preemptive mechanism admits urgent tasks in time (Zhao et al., 27 Aug 2025).

The paper’s ablations attribute gains to distinct components rather than to a single mechanism. MCTS reduces matching time; LCS improves speedup by $p$4, $p$5, $p$6 on the Cloud platform across various networks; and CSR reduces graph-memory footprint by up to $p$7 on complex workloads (Zhao et al., 27 Aug 2025).

These results position IsoSched as a TSS-preemptive alternative to LTS-based preemption frameworks and as a preemptive alternative to non-preemptive TSS frameworks.

6. Limitations, lineage, and cross-domain usage

IsoSched is not presented as a universally architecture-agnostic scheduler. The paper explicitly notes limitations and pathological cases: extremely high-degree fan-in/out nodes may exceed per-engine link ports; LCS might default to C-splitting when buffers are insufficient; hardware without reconfigurable links or with very low on-chip bandwidth reduces benefits; and objective weights $p$8 require tuning for the deployment scenario. The paper also lists future work such as integrating queueing models, extending to heterogeneous engines, using dynamic voltage-frequency scaling, and incorporating PCB-level multi-accelerator communication (Zhao et al., 27 Aug 2025).

The framework also has a clear research lineage. A later paper, "IMMSched: Interruptible Multi-DNN Scheduling via Parallel Multi-Particle Optimizing Subgraph Isomorphism", states that it directly builds on IsoSched [33], and that IsoSched first showed that TSS for multi-DNN workloads on edge accelerators reduces preemptive scheduling to a subgraph isomorphism problem over large DAGs. IMMSched retains the same formulation but replaces IsoSched’s CPU-serialized matching with a parallel, accelerator-resident algorithm that combines Multi-Particle Optimization, a probabilistic continuous-relaxation scheme, Ullmann refinement, consensus-guided exploration, and quantized scheduling (Zhao et al., 23 Mar 2026).

This suggests an important distinction within the same research thread. IsoSched established the TSS-preemptive formulation and the Ullmann+MCTS solution strategy, while IMMSched addresses the serial runtime overhead of matching by moving the search onto the accelerator itself (Zhao et al., 23 Mar 2026).

A separate ambiguity concerns the name itself. In the radiosurgery literature, the paper "Simultaneous optimization of isocenter locations and sector duration in radiosurgery" does not title its method IsoSched, but explicitly states the “IsoSched implication” of an IsoSched tool implementing SDIO and its Benders scheme. In that context, the tool would ingest candidate isocenters, dose influence matrices, and clinical dose objectives, then output scheduled shots for the Leksell Gamma Knife Icon; it is grounded in a mixed-integer model with Benders decomposition for simultaneous sector duration and isocenter optimization (Cevik et al., 2018).

Accordingly, “IsoSched” has two distinct usages in the provided literature: a named preemptive TSS multi-DNN scheduler (Zhao et al., 27 Aug 2025), and a radiosurgery tool implication built around SDIO rather than a separately named algorithmic framework (Cevik et al., 2018).