---
title: 'SubZeroCore: LLM-Guided Multi-Agent RL Scheduler'
url: https://www.emergentmind.com/topics/subzerocore
type: topic
---

# SubZeroCore: LLM-Guided Multi-Agent RL Scheduler

SubZeroCore is a zero-shot, LLM-guided hierarchical multi-agent reinforcement learning scheduler for task-to-core mapping and per-core DVFS on embedded platforms. In the formulation derived from "ZeroDVFS: Zero-Shot LLM-Guided Core and Frequency Allocation for Embedded Platforms," it targets multi-core embedded CPUs with per-core DVFS states, thermal sensors, performance counters, and OpenMP task runtime, and seeks to minimize energy and makespan while keeping temperatures below a safety threshold under low-latency runtime control. Its defining characteristic is the combination of hierarchical MARL, regression-based environment modeling, and LLM-extracted semantic features that permit zero-shot deployment for new workloads on trained platforms without workload-specific profiling samples [2601.08166].

## 1. Problem formulation and optimization targets

SubZeroCore is posed on a multi-core embedded CPU with discrete per-core DVFS sets $F_i$, where an arriving OpenMP workload $\Gamma$ is decomposed into tasks or threads whose mapping and frequency assignment determine energy consumption, thermal trajectory, and makespan. The decision variables are a task-to-core mapping vector
$$
A \in \{0,1\}^{|\Gamma|\times m}
$$
or, equivalently, core selection with CPU affinity for threads or tasks, together with a per-core frequency vector
$$
f = [f_1,\ldots,f_m], \qquad f_i \in F_i.
$$
The optimization goals are explicit: energy $E$ is minimized, makespan is minimized, and thermal safety is enforced through the runtime condition $T_i(t) \leq T_{\text{th}}$ for all cores $i$ [2601.08166].

The controller adopts canonical power and thermal relations as modeling priors and consistency checks. The dynamic power relation is
$$
P_{\text{dyn}} = \alpha C V^2 f,
$$
energy is
$$
E = \int P(t)\,dt,
$$
and the lumped RC thermal model is
$$
C_{\text{th}}\frac{dT}{dt} = P_{\text{total}} - \frac{T - T_{\text{amb}}}{R_{\text{th}}}.
$$
In deployment, however, SubZeroCore does not rely on these equations alone. It uses calibrated multivariate regression models that incorporate these physics-inspired dependencies while maintaining low inference latency. This positioning is important: the system is neither a utilization-only governor nor a purely table-driven profiler, but a learned controller intended to react to stall behavior, memory effects, and thermal evolution that utilization heuristics can miss [2601.08166].

A central motivation is the inadequacy of two baseline families. Utilization-only governors ignore stall cycles and variations in memory behavior, while exhaustive table-based profiling requires offline profiling for each workload and platform. SubZeroCore is designed to preserve runtime adaptation and to avoid workload-specific profiling at first deployment by generating synthetic experience conditioned on program semantics. This suggests a shift from per-application lookup-table construction toward platform-calibrated, workload-conditioned planning [2601.08166].

## 2. Hierarchical control architecture

The scheduler is organized as a hierarchical multi-agent RL system with two collaborative agents. The **Temperature agent** is thermal-oriented: it assigns priority scores or an ordering over cores based on current and predicted thermal states, then selects the top-$k$ cores that minimize hotspot risk and balance thermal load. The **Profiler agent** is performance- and energy-oriented: it selects how many cores to activate, denoted $k$, and chooses a compact per-core frequency plan from a small discrete template set. The blueprint gives examples of such abstractions as uniform-$f_i$ for top-$k$ prioritized cores, tapered-frequency profiles, or two-level banding for memory-bound cores versus compute-bound cores [2601.08166].

The final decision is composed from both outputs. The Temperature agent provides a prioritized list of cores; the Profiler agent chooses $k$ and a frequency plan; the runtime then maps tasks to the top-$k$ prioritized cores according to the selected plan. This decomposition is intended to avoid an $m^n$ explosion, where $n$ denotes per-core DVFS levels, and it is the basis for the reported practical decision latency of approximately $358$ ms for subsequent decisions [2601.08166].

The architecture is explicitly model-based. Two regressors form the environment model. The **Profiler model** predicts next-state performance observables such as makespan increment, IPC, stalls, energy, and power, given current state and action. The **Thermal model** predicts next-state per-core temperatures from current temperatures, power, and DVFS decisions. Lightweight FCN and Conv1D architectures are used so that prediction remains fast enough for planning. This decomposition also supports a Dyna-Q-style interaction loop in which real transitions and imaginary transitions coexist in the replay process [2601.08166].

The architectural split between thermal prioritization and profiler-driven frequency/core-count selection is the core systems idea. It separates hotspot avoidance from performance-energy trade-offs while still producing a single actuation decision over frequency vectors and CPU affinity. A plausible implication is that this partitioning makes the controller more tractable than a monolithic policy over the full joint space of mapping and per-core DVFS assignments.

## 3. State, action, reward, and learning dynamics

The state vector is defined as
$$
s_t = [T_t, f_t, x_t, \phi, \psi],
$$
with the more explicit form
$$
s_t = [T_t \in \mathbb{R}^m,\; f_t \in \mathbb{R}^m,\; x_t \in \mathbb{R}^p,\; \phi \in \mathbb{R}^{13},\; \psi \in \mathbb{R}^q].
$$
Here, $T_t$ contains per-core temperatures, $f_t$ the current DVFS states, $x_t$ aggregates performance counters such as IPC, stall cycles, instructions, cache references or misses, and context switches, $\phi$ denotes the 13 LLM-extracted semantic features, and $\psi$ contains static syntax-level features from Tree-sitter together with metadata such as problem size. Optional elements include utilization, NUMA locality metrics, and energy estimates [2601.08166].

The action is represented as
$$
a_t = [\pi_t \in \mathbb{R}^m,\; k_t \in \mathbb{N},\; \theta_t \in \Theta],
$$
where $\pi_t$ is the per-core priority vector from the Temperature agent, $k_t$ is the number of active cores, and $\theta_t$ is a compact per-core frequency template chosen from a small discrete set $\Theta$. The template instantiates a frequency vector $f'_t$ over the selected top-$k_t$ cores, yielding per-core DVFS with reduced dimensionality [2601.08166].

The Temperature agent uses a piecewise reward. For core $i$,
$$
r_{\text{temp}}^i =
\begin{cases}
-1, & \text{if } \text{temp}_i > 50^\circ\text{C} \\
50 - \text{temp}_i, & \text{otherwise,}
\end{cases}
$$
and the aggregate reward is
$$
r_{\text{temp}} = \frac{1}{m}\sum_{i=1}^{m} r_{\text{temp}}^i.
$$
The Profiler agent uses a thresholded two-term exponential reward with parameters $c_{\text{th}} = 0.3$ and $c_{\text{st}} = 0.5$, referenced to $E P_{\text{sav}}$ under the powersave governor and $\text{makespan}_{\text{Perf}}$ under the performance governor. If
$$
E_A > (1+c_{\text{th}})E P_{\text{sav}}
\quad \text{or} \quad
\text{makespan}_A > (1+c_{\text{th}})\text{makespan}_{\text{Perf}},
$$
then
$$
r_{\text{profiler}} = -1.
$$
Otherwise,
$$
r_{\text{energy}} = \exp\!\left(- c_{\text{st}} \cdot \frac{E_A - E P_{\text{sav}}}{2 c_{\text{th}}}\right) - 1,
$$
$$
r_{\text{makespan}} = \exp\!\left(- c_{\text{st}} \cdot \frac{\text{makespan}_A - \text{makespan}_{\text{Perf}}}{2 c_{\text{th}}}\right) - 1,
$$
and
$$
r_{\text{profiler}} = \frac{r_{\text{energy}} + r_{\text{makespan}}}{2}.
$$
These rewards jointly trade off energy and makespan while thermal limits are enforced through the temperature reward and action composition [2601.08166].

Learning follows a Dyna-Q-style off-policy Q-learning update,
$$
Q(s,a) \leftarrow Q(s,a) + \alpha\big[r + \gamma \max_{a'}Q(s',a') - Q(s,a)\big],
$$
implemented with D3QN agents to stabilize Q-learning through double networks and dueling value-advantage decomposition. Planning uses the learned profiler model $M_{\text{prof}}$ and derived thermal transitions to create imaginary samples. The reported effect is approximately $20\times$ faster convergence than model-free RL, attributed to the use of synthetic trajectories in the replay process [2601.08166].

## 4. Environment modeling and LLM-based zero-shot semantics

A distinctive aspect of SubZeroCore is its use of LLM-extracted semantic features to characterize OpenMP programs without executing them. These features condition the environment model so that synthetic samples can be generated for unseen workloads, enabling zero-shot deployment on trained platforms. Tree-sitter-derived syntactic features and workload metadata are ingested in parallel. The extraction method is described as a zero-shot prompt that requests strict JSON with constrained values, while Tree-sitter provides syntactic features concurrently [2601.08166].

The 13 semantic features are:

- **memory_access_pattern**: $\{\text{unit\_stride}, \text{non\_unit\_stride}, \text{random}, \text{mixed}\}$
- **spatial_locality**: $\{\text{high}, \text{medium}, \text{low}\}$
- **temporal_locality**: $\{\text{high}, \text{medium}, \text{low}\}$
- **cache_behavior_pattern**: $\{\text{streaming}, \text{random}, \text{blocked}, \text{mixed}\}$
- **numa_sensitivity**: $\{\text{high}, \text{medium}, \text{low}\}$
- **algorithmic_complexity**: $\{O(n), O(n \log n), O(n^2), O(n^3), \text{other}\}$
- **dominant_operation**: $\{\text{arithmetic}, \text{memory}, \text{logic}, \text{mixed}\}$
- **vectorization_potential**: $\{\text{high}, \text{medium}, \text{low}\}$
- **data_dependency_type**: $\{\text{none}, \text{loop\_carried}, \text{cross\_iteration}, \text{complex}\}$
- **false_sharing_risk**: $\{\text{none}, \text{low}, \text{medium}, \text{high}\}$
- **load_balance_characteristic**: $\{\text{uniform}, \text{irregular}, \text{dynamic}\}$
- **parallelization_overhead**: $\{\text{low}, \text{medium}, \text{high}\}$
- **scalability_bottleneck**: $\{\text{none}, \text{memory\_bandwidth}, \text{synchronization}, \text{load\_imbalance}\}$

These features are intended to capture algorithmic complexity, memory access behavior, parallel structure, synchronization, and scalability bottlenecks without execution. In conjunction with platform telemetry, they allow the environment model to produce imaginary experiences before a new program is run. This is the basis of the system’s zero-shot claim [2601.08166].

The regression models themselves are calibrated on IIO power telemetry, perf\_event counters, and per-core thermal sensors. The profiler next-state model is written as
$$
s'_{\text{prof}} = g_{\text{prof}}(s, a_{\text{prof}}),
$$
and the thermal model as
$$
s'_{\text{temp}} = g_{\text{temp}}(s, a).
$$
The reported characteristics are inference latency under $5$ ms and more than $6\times$ lower temperature-prediction MSE than prior heuristics. In this framework, the LLM is not the scheduler itself; it is a front-end feature extractor that conditions the regression models used for planning and synthetic data generation [2601.08166].

## 5. Runtime workflow, interfaces, and deployment mechanics

The runtime decision flow is specified in four stages. First, SubZeroCore performs one-time LLM semantic feature extraction, static parsing, and metadata collection. Second, it initializes the agents and environment models and loads cached platform calibrations. Third, at each decision point it observes $s_t$, obtains $\pi_t$ from the Temperature agent and $[k_t,\theta_t]$ from the Profiler agent, composes these into a frequency vector over the top-$k$ cores and a binding decision, executes the workload phase, collects $s'$ and rewards, updates replay memory, and, when the planning threshold is reached, runs $\zeta$ synthetic rollouts using $M_{\text{prof}}$ and derived thermal transitions to update the Q-networks. Fourth, it repeats this process for subsequent decisions [2601.08166].

The implementation relies on standard Linux interfaces. DVFS control is exposed through `cpufreq` with `scaling_max_freq`, and per-core frequency setting is enabled by Linux kernel APIs. Task-to-core binding is realized through `cpuset` and CPU affinity in the OpenMP runtime, with examples including `OMP_PLACES` and `OMP_PROC_BIND`. Telemetry comes from `perf_event` counters for instructions, cycles, IPC, and cache references or misses, from Linux in-kernel IIO power interfaces such as `in_power_input` in `sysfs`, and from per-core thermal sensors [2601.08166].

Latency is divided into first-decision and subsequent-decision regimes. For a new benchmark, the first decision takes $3.5$--$8.0$ s in total, comprising LLM API time of $3.07$--$7.64$ s, Tree-sitter time of approximately $50$ ms, and RL inference of approximately $358$ ms. Subsequent decisions take approximately $358$ ms because the semantic features are cached and no new LLM calls are required. The deployment notes further state that the Python prototype operates at this latency, whereas optimized inference in C++ with TensorRT is expected to reduce this to sub-$10$ ms, provided decisions are taken at phase boundaries or coarse-grain intervals to avoid jitter [2601.08166].

Safety and fallback behavior are explicit. Hard thermal limits are enforced through $T_{\text{th}}$, reward penalties, and action clipping; cooling-off strategies include core rotation and frequency downscaling. A watchdog can switch to a safe governor such as Linux `ondemand` on anomalies, and failures or sensor anomalies can trigger degradation to `ondemand`, `performance`, or `powersave` governors. Offline edge scenarios are also contemplated: semantic features may be pre-extracted and stored locally, and LLM inference may be done offline to avoid runtime overhead [2601.08166].

## 6. Experimental evaluation and reported performance

SubZeroCore is evaluated across NVIDIA Jetson TX2, Jetson Orin NX, RubikPi, and Intel Core i7 platforms. The Jetson TX2 configuration is listed as 6 cores with $4\times$A57 and $2\times$Denver2, Jetson Orin NX as $8\times$A78AE, RubikPi as $8\times$Kryo 585, and the Intel Core i7 platform as 4 cores, 8th generation. Benchmarks include BOTS and PolybenchC, with representative programs such as FFT, Strassen, SparseLU, GEMM, stencils, and task-parallel DAGs. Baselines are Linux `ondemand` and heuristic or table-based profiling schemes such as exhaustive lookup generation. Metrics comprise energy in mJ, makespan in s, temperature in $^\circ$C, decision latency in ms, convergence speed in episodes, and prediction accuracy in MSE, $R^2$, and MAPE [2601.08166].

Energy is measured as
$$
E \approx \sum_i P_i \Delta t_i,
$$
with IIO sensor sampling at $\Delta t = 10$ ms. Thermal data come from per-core sensors, performance data from `perf_event`, and DVFS states from `sysfs`. Within this setup, the reported quantitative outcomes are as follows [2601.08166]:

| Quantity | Reported result |
|---|---|
| Energy efficiency vs Linux `ondemand` | $7.09\times$ better |
| Makespan vs Linux `ondemand` | $4.0\times$ better |
| Convergence vs model-free RL | approximately $20\times$ faster |
| First decision latency | $3.5$--$8.0$ s |
| Subsequent decision latency | approximately $358$ ms |
| First decision vs table-based profiling | $8{,}300\times$ faster |
| Temperature-model accuracy | more than $6\times$ lower MSE than prior heuristic |
| Environment-model inference latency | less than $5$ ms |

A central comparative claim concerns profiling cost. Table-based profiling is described as requiring $8$--$12$ hours per benchmark, whereas SubZeroCore reaches a first decision in $3.5$--$8.0$ s and subsequent decisions in approximately $358$ ms. The reported $8{,}300\times$ first-decision improvement is therefore framed not merely as a latency figure but as the condition that makes runtime adaptation practical in dynamic embedded systems [2601.08166].

The benchmark outcomes also sharpen the conceptual distinction between SubZeroCore and utilization-only governors. Because the controller incorporates IPC, stalls, cache behavior, context switches, and semantic code features, it is designed to distinguish compute-bound from memory-bound regimes when selecting frequencies and active cores. This suggests that the reported performance gains arise not only from learning a better static policy but from using richer state variables than conventional governors [2601.08166].

## 7. Limitations, future directions, and nomenclatural disambiguation

Several limitations are stated directly. First, the one-time LLM extraction introduces an initial overhead of roughly $3$--$8$ s and small API fees, though caching and offline batch extraction mitigate this cost. Second, the environment models must be trained per hardware platform; zero-shot cross-platform transfer is said to work but to benefit from few-shot fine-tuning for lower MAPE. Third, thermal sensors have latency and the regressors are imperfect, so safety constraints must incorporate conservative margins. Fourth, the system is tuned for OpenMP code, and tied tasks, irregular graphs, multi-file projects, preemption, and finer-grain scheduling can reduce predictability or interfere with affinity assumptions. Future work is described as extending the method to concurrent workloads, multi-resource scheduling, GPU offloading, and uncertainty-aware policies [2601.08166].

The name **SubZeroCore** is not unique across the literature. In one unrelated use, "SubZeroCore: A Submodular Approach with Zero Training for Coreset Selection" denotes a training-free coreset selection method that unifies coverage and density in a weighted facility-location objective and uses greedy maximization under the standard $(1-1/e)$ approximation guarantee [2509.21748]. A related but distinct line of work on unlabeled data uses the name ZCore for zero-shot coreset selection from foundation-model embeddings [2411.15349]. The term also appears as an internal label in a diffusion-personalization specification associated with "SubZero," where it denotes the combination of orthogonalized temporal aggregation, DSOC, and projector modules for subject-style-action composition without fine-tuning [2502.19673].

Other occurrences are metaphorical or mapped rather than nominally canonical. One text uses SubZeroCore to describe the millikelvin core of the CUORE cryostat, emphasizing a contiguous volume at about $10$ mK in the context of rare-event physics [1410.1560]. Another maps the term onto an Ozone-like core-level execution resource for zero timing leakage in modern microarchitectures [1703.07706]. A further usage treats it as a design target for a sub-kelvin electronic cooling module based on cascade NIS refrigeration [1605.00830]. These occurrences are conceptually unrelated to the embedded-systems scheduler derived from ZeroDVFS, but they establish that the label has acquired a broader, domain-dependent usage.

In the embedded-systems sense, SubZeroCore denotes a controller that operationalizes zero-shot, LLM-guided core and frequency allocation through a hierarchical MARL scheduler, calibrated environment models, and semantic feature extraction. Its distinguishing claims are the elimination of workload-specific profiling at deployment, the decomposition of action selection into temperature-aware prioritization and profiler-driven frequency planning, and the combination of low-latency runtime decisions with reported gains of $7.09\times$ in energy efficiency and $4.0\times$ in makespan relative to Linux `ondemand` [2601.08166].

Source: https://www.emergentmind.com/topics/subzerocore