OctopuScheduler: SpiNNaker2 DNN Scheduler

• OctopuScheduler is a scheduler/runtime framework that maps individual DNN layers onto SpiNNaker2’s worker processing elements under strict 128 kB SRAM constraints.
• It employs equal-sized tile partitioning and IRQ-driven coordination to support efficient, layer-level execution and design-space exploration.
• The framework extends to full-model inference with an end-to-end pipeline from PyTorch to autonomous on-chip execution via DRAM-backed configurations.

Searching arXiv for the OctopuScheduler-related paper and closely related scheduling work. OctopuScheduler is a scheduler/runtime framework for executing deep neural network workloads on the SpiNNaker2 neuromorphic MPSoC. In the evidence available here, its original form is a single-layer DNN scheduler that maps an individual layer onto a set of worker processing elements (PEs) under tight PE-local SRAM constraints by partitioning the layer into equally sized tiles, coordinating execution with one scheduler PE, and using interrupt requests (IRQs) and signal words for control and completion notification. A later system extends OctopuScheduler into a multi-layer, model-level execution framework that supports an end-to-end path from PyTorch through ONNX, INT8 power-of-two post-training quantization, graph lowering, layer partitioning and mapping, static DRAM image generation, and autonomous on-chip inference on a single SpiNNaker2 chip (Jobst et al., 18 Jul 2025).

1. Definition and scope

OctopuScheduler, as described in the SpiNNaker2 inference framework literature, is not introduced as a general-purpose cluster scheduler or a cloud job scheduler. It is a platform-specific scheduling substrate for mapping and executing individual DNN layers on SpiNNaker2, later generalized into a broader execution backend for full-model inference. The original framework targets the central systems problem posed by SpiNNaker2: a single chip provides many PEs but each PE has only 128 kB local SRAM, so nontrivial DNN layers must be partitioned and distributed rather than executed monolithically (Jobst et al., 18 Jul 2025).

In its original form, the framework has a simple but explicit control structure. One scheduler PE coordinates execution, while multiple worker PEs execute the tiled computation. At layer start, the scheduler activates workers via IRQs; at layer completion, workers notify the scheduler with a signal word. The scheduler also performs automated design space exploration subject to PE-local memory limits. The paper is explicit that this earlier OctopuScheduler provided single-layer functionality only. It could act as a building block, but it did not autonomously traverse a whole model or provide end-to-end deployment of complex DNNs on a single edge chip (Jobst et al., 18 Jul 2025).

A useful way to characterize OctopuScheduler is as an execution and mapping backbone rather than as a purely abstract optimizer. The later framework paper states that the new contribution is not a new scheduler from scratch, but a generalization of OctopuScheduler so that complete quantized neural networks, rather than isolated layers, can run autonomously on-chip after a one-time host setup. This suggests that “OctopuScheduler” properly names the lower-level scheduler/runtime core, while the surrounding framework supplies compilation, IR lowering, DRAM organization, and model-level control flow (Jobst et al., 18 Jul 2025).

2. Hardware context and original execution model

The scheduler is tightly coupled to SpiNNaker2 architecture. A single chip contains 152 low-power PEs; each PE has an ARM Cortex-M4F, dedicated accelerators including EXP for exponential functions and MLA for machine learning operations, and 128 kB local SRAM. PEs communicate over a NoC in GALS style; inter-PE transfers use DMA; synchronization and control use IRQ; and each chip is paired with 2 GB external DRAM (Jobst et al., 18 Jul 2025).

These hardware constraints explain the original OctopuScheduler design. Large DNN layers do not fit naturally into PE-local SRAM, so the scheduler partitions a layer into equally sized tiles and maps them over worker PEs. The scheduler PE then orchestrates execution across those workers. This is not a data-center style resource scheduler; it is a fine-grained runtime scheduler for distributed execution across many microcontroller-class cores on one chip. The central dataflow pattern described for the extended framework is:

$$\text{DRAM} \rightarrow \text{worker PE local memory} \rightarrow \text{MLA/compute} \rightarrow \text{DRAM}$$

That pattern is already latent in the single-layer regime and becomes dominant at model scope because weights, inputs, outputs, and intermediates cannot all remain resident in SRAM (Jobst et al., 18 Jul 2025).

The paper also makes clear what the original framework already did. It supported tiling, mapping, and memory-constrained design-space exploration for individual layers, using existing SpiNNaker2 software support including low-level libraries and PySpiNNaker2. What it lacked was a standardized representation for heterogeneous layers across an entire model, persistent on-chip traversal from one layer to the next, and a global memory plan for parameters and activations (Jobst et al., 18 Jul 2025).

3. Extension to multi-layer, model-level execution

The key systems advance associated with OctopuScheduler is its extension from a layer-level mechanism to a complete inference framework. The later work lists three major additions: multi-layer scheduling across an entire model, standalone on-chip iteration over all layers without host interaction between layers, and an automated front-end from PyTorch, including parameter extraction and 8-bit post-training quantization (Jobst et al., 18 Jul 2025).

The end-to-end flow proceeds from a trained PyTorch model to ONNX, then to INT8 power-of-two quantization using AMD Quark, followed by lowering to an application graph, graph cleanup and fusion, topological sorting, conversion to low-level S2Layer objects, partitioning and mapping using OctopuScheduler mechanisms, assembly into an S2Model, generation of a complete DRAM image, deployment to SpiNNaker2 DRAM, IRQ-based launch of the scheduler PE, and autonomous on-chip iteration across all layers (Jobst et al., 18 Jul 2025).

A central abstraction introduced in the extended framework is a unified layer execution structure. Scheduler and worker binaries are not model-specific; instead, they interpret standardized layer configurations stored in DRAM. This converts OctopuScheduler from a per-layer execution engine into a reusable model-level runtime. The framework stores in DRAM: layer-related configuration and parameters, model-related configuration, and the inputs and outputs of all layers. The host statically plans DRAM “to avoid on-chip DRAM memory planning,” and the runtime simply dereferences addresses and executes (Jobst et al., 18 Jul 2025).

The DRAM structure is divided into four consecutive regions: Global Configuration, Time Measurements, Layer Configurations, and Data Memory. Each layer configuration begins with a fixed-size layer information header that includes the layer type, number of assigned worker PEs, and the DRAM address of the next layer’s configuration. Additional sub-blocks include scheduler-specific configuration, worker-specific configuration, DRAM address mappings for input and output tiles per PE, and layer-specific constants such as weights and biases. This DRAM-resident representation is effectively the executable IR consumed by OctopuScheduler at deployment time (Jobst et al., 18 Jul 2025).

4. Compilation path, operator support, and runtime control

The extended framework surrounding OctopuScheduler uses a multi-stage compilation path:

$$\text{PyTorch model} \rightarrow \text{ONNX} \rightarrow \text{quantized ONNX} \rightarrow \text{application graph} \rightarrow \text{topologically sorted nodes} \rightarrow \text{S2Layer instances} \rightarrow \text{S2Model DRAM image} \rightarrow \text{runtime execution}$$

The paper does not present this as a formal equation, but it explicitly describes that sequence. The application graph is a simplified IR in which nodes hold layer parameters and quantization scales, while edges represent data dependencies (Jobst et al., 18 Jul 2025).

Two graph-level transformations are especially important. First, Linear + ReLU fusion is applied when a linear node is followed immediately by ReLU, because the MLA can execute matrix multiplication plus ReLU as a single joined operation. Second, unnecessary Quantize/Dequantize nodes inserted by Quark, for example around softmax, are removed, and the quantization scaling factors are pulled into the related node. Topological sorting then produces the linear execution order used by the on-chip runtime (Jobst et al., 18 Jul 2025).

At runtime, the scheduler and workers follow a persistent execution loop. At startup, scheduler and workers read global information from DRAM; workers then enter low-power idle mode. For each layer, the scheduler reads the layer information header, learns which workers to activate and where the next layer configuration is stored, forwards layer information, and sends IRQs to selected workers. Workers fetch their own configuration data from DRAM, execute layer-specific operations, possibly iterate over multiple parts, signal the scheduler on completion, and return to sleep mode. After the final layer, the scheduler sends a special “finish” layer information to all workers, after which the host can read outputs and timing data (Jobst et al., 18 Jul 2025).

The operator classes explicitly referenced in the framework are Linear, ReLU, Softmax, and Add. The paper does not provide a full operator coverage table or detailed lowering rules for attention, normalization, or embedding layers. It does, however, state that the infrastructure is intended to support “large and complex DNNs up to transformer scale.” A plausible implication is that the scheduler/runtime abstraction is designed to be layer-generic even though the experimentally demonstrated path in the paper centers on an MLP (Jobst et al., 18 Jul 2025).

5. Performance characteristics and bottlenecks

The most concrete performance evaluation of the OctopuScheduler-based framework uses a three-layer MLP on MNIST with dimensions

$784 \rightarrow 512 \rightarrow 256 \rightarrow 16$

where the output is padded from 10 to 16 “to satisfy the minimum tile size for efficient MLA usage.” The reported accuracy results are: 98.36% for pre-trained FP32, 98.33% for INT8 Power-of-Two MSE CLE, and 98.34% measured on SpiNNaker2. The paper interprets this as negligible quantization loss and close preservation of quantized semantics under deployment (Jobst et al., 18 Jul 2025).

For batch size 1, the runtime breakdown is: Setup: 12 µs, FC1+ReLU: 323 µs with 8 workers, FC2+ReLU: 217 µs with 4 workers, FC3: 80 µs with 1 worker, Softmax: 50 µs with 1 worker, Cleanup: 9 µs, for a Total: 688 µs. The paper also reports an average per-layer scheduling overhead of about 13 µs, arising from loading layer information from DRAM, distributing it, triggering workers, and storing timing measurements (Jobst et al., 18 Jul 2025).

A central empirical result is that DRAM transfer, not MLA compute, dominates layer time. For the first layer, the total duration is 323 µs, the average time fetching weights from DRAM is 192 µs, and MLA vector-matrix computation time is 29 µs. The paper explicitly states that the accelerator is used for only 9% of the layer runtime. This is a critical point for understanding OctopuScheduler: the runtime control overhead is already relatively low, while the dominant systems bottleneck is external memory traffic rather than arithmetic throughput (Jobst et al., 18 Jul 2025).

The setup and cleanup overhead also remain limited. With 8 workers enabled, setup is 12 µs and cleanup 9 µs. If all available workers on a chip are enabled, setup is 39 µs and cleanup 93 µs; even then, the paper states that overhead is only about 16% of the runtime of this small model. This suggests that the scheduler itself is lightweight enough for model-level control, at least in the evaluated regime (Jobst et al., 18 Jul 2025).

6. Conceptual position among schedulers

OctopuScheduler belongs to a family of domain-specific schedulers that expose a narrow but consequential layer of systems control. In the SpiNNaker2 setting, that layer is PE-level orchestration, memory-constrained tiling, and autonomous layer traversal. This differs sharply from distributed quantum schedulers, cluster-wide elastic schedulers, or event-driven orchestration substrates, but useful analogies can still be drawn.

One relevant comparison is with distributed quantum network scheduling, where the scheduler is not the whole compiler but the layer that maps already-determined communication demands into resource-constrained operation schedules. In “Network Operations Scheduling for Distributed Quantum Computing,” the scheduling layer begins after partitioning and focuses on sequencing link-level entanglement generation, entanglement swapping, and move operations under communication and memory constraints (Chandra et al., 17 Nov 2025). This suggests a general systems pattern: OctopuScheduler likewise isolates a hardware-constrained execution layer beneath higher-level compilation decisions.

A second comparison comes from flexible cluster scheduling, where applications are decomposed into mandatory and optional components. “Flexible Scheduling of Distributed Analytic Applications” models each application as $(C_i, E_i)$, with core demand $C_i$ required for progress and elastic demand $E_i$ improving performance (Pace et al., 2016). A plausible analogy is that OctopuScheduler’s worker allocation per layer plays a role similar to elastic resource assignment: larger layers may use more workers, smaller layers fewer, while the scheduler PE preserves the minimal control structure. The paper on SpiNNaker2 does not formalize this analogy, but it explicitly shows layer-specific worker counts such as 8 workers for FC1+ReLU, 4 workers for FC2+ReLU, and 1 worker for FC3 and Softmax (Jobst et al., 18 Jul 2025).

It is equally important to distinguish OctopuScheduler from unrelated “Octopus” systems. The crowdsourcing controller “Octopus: A Framework for Cost-Quality-Time Optimization in Crowdsourcing” is a hierarchical POMDP controller for microtask routing, pricing, and quality control, not a neuromorphic runtime scheduler (Goel et al., 2017). Likewise, the cloud-to-edge event fabric “Octopus: Experiences with a Hybrid Event-Driven Architecture for Distributed Scientific Computing” provides an event substrate that could support a scheduler, but is not itself the SpiNNaker2 execution framework (Pan et al., 2024). The shared naming should not obscure the fact that OctopuScheduler, in the evidence considered here, is specifically the SpiNNaker2 DNN scheduler/runtime lineage (Jobst et al., 18 Jul 2025).

7. Limitations and open directions

The most explicit limitation is that the later framework paper does not provide a formal optimization objective for scheduling, a communication cost formula, a placement ILP, heuristic pseudocode for multi-layer mapping, explicit memory-capacity inequalities, or a throughput/latency analytical model. It states that S2Layer implements the partitioning and mapping mechanisms from OctopuScheduler, but does not restate their algorithmic details in mathematical form. Consequently, OctopuScheduler is best documented here as a systems and runtime framework rather than as a fully formalized scheduler with explicit objective functions (Jobst et al., 18 Jul 2025).

The framework is also strongly dependent on static host-side DRAM planning and on repeated DRAM ↔ PE ↔ DRAM movement. This enables models larger than PE-local SRAM would otherwise allow, but it also creates the dominant runtime bottleneck. The runtime analysis makes clear that future performance gains would likely require reduced DRAM traffic or greater overlap of transfer and computation rather than merely lowering scheduler overhead (Jobst et al., 18 Jul 2025).

A further limitation is operator and benchmark scope. The demonstrated path centers on an MLP and explicitly references only linear, add, softmax, and fused linear+ReLU in the scheduling context. The broader claim of supporting “large and complex DNNs up to transformer scale” should therefore be read, as the paper itself suggests, as an architectural capability claim rather than as a comprehensive transformer benchmark result (Jobst et al., 18 Jul 2025).

The overall significance of OctopuScheduler is thus precise. It provides the missing execution layer that converts SpiNNaker2 from a platform on which isolated DNN layers can be manually or semi-manually run into one capable of autonomous full-model inference after one host-side deployment step. Its enduring technical themes are static DRAM-backed execution, scheduler/worker separation, IRQ-driven layer dispatch, and reuse of layer-level tiling and mapping mechanisms at model scope. In that sense, OctopuScheduler is less a single algorithm than a scheduler-runtime architecture for memory-constrained neuromorphic inference (Jobst et al., 18 Jul 2025).