Cache-Aware Schedule Alignment

• Cache-aware schedule alignment is a systems principle that coordinates execution order with physical and statistical cache behavior to optimize data reuse.
• Techniques include aligning thread placement with home caches, collapsing tasks in DAG scheduling, and predicting miss ratios to match tasks with cache capacities.
• Applications range from manycore processors and embedded systems to real-time and generative model scheduling, yielding notable performance and resource efficiency improvements.

Cache-aware schedule alignment denotes the coordination of execution order, task placement, cache partitioning, or time discretization with cache state and cache topology so that reuse is preserved and avoidable misses, interference, or redundant recomputation are reduced. In the literature, the idea appears in several forms: aligning thread placement with home caches on NUCA manycores, collapsing same-object DAG nodes to exploit instruction-cache sharing, reordering LLM training around reusable prefix $K/V$, and adapting diffusion schedules to fixed cache policies (Tousimojarad et al., 2014, Tessler et al., 2020, Li et al., 31 May 2026).

1. Conceptual scope

The term is not used uniformly. Some work uses it explicitly, while other work presents the same structure without that label. In manycore programming, the central rule is to place threads where their data are homed and to home data where their threads run; one paper states this directly as the principle that “tasks are scheduled where their data lives, and data homed where their tasks run” (Tousimojarad et al., 2014). In real-time DAG scheduling, the same idea appears as the collapse of nodes that invoke the same executable object so that their threads run together under BUNDLE scheduling and share the instruction cache (Tessler et al., 2020). In grounded RAG serving, the alignment target is not a core or a page but the token prefix seen by a prefix cache: evidence must be ordered so that “set overlap” becomes reusable “prefix overlap” (Tan et al., 18 Jun 2026).

Across these domains, the object being aligned changes, but the structural problem is similar. One class of methods aligns tasks with physical cache topology, such as home tiles, L1 caches, or LLC partitions. Another class aligns schedules with statistical cache behavior, for example by predicting miss ratios from stack-distance distributions and then mapping tasks to the nodes whose caches they fit best. A third class aligns execution phases with reusable cache interfaces, such as prefix $K/V$ and accumulated $gK/gV$ in LLM RL training, or cache-valid denoising steps in diffusion inference (0902.4822, Li et al., 31 May 2026, Lei et al., 11 Jun 2026).

This suggests that cache-aware schedule alignment is best understood as a systems principle rather than a single algorithm. The common requirement is that the schedule must respect the granularity at which reuse is actually available: page home tiles in NUCA, object-level instruction footprints in DAG-OT, fixed-size token blocks in vLLM APC, or timestep-local redundancy in diffusion caches.

2. Topology-aware placement on manycore and embedded processors

A canonical hardware-locality formulation appears on the Tilera TILEPro64, a 64-tile manycore arranged in an $8\times 8$ mesh with a Dynamic Distributed Cache and a “home tile” associated with each physical memory address (Tousimojarad et al., 2014). Because “the access latency to the home core depends on the physical on-die location of the requesting core,” uniform virtual address space hides a non-uniform cache architecture. The paper compares three homing modes—local homing, remote homing, and hash for home—and argues that for “memory-bound parallel array computations with mostly sequential access,” hash-for-home is too fine-grained. Its “localised programming” technique combines three building blocks: local homing (none), static thread mapping via sched_setaffinity(), and home-cache localisation via per-thread copying. Under this formulation, the best-performing case is “Localised, Static Mapper, local homing (none),” and the measured speedup improvements are attributed primarily to “aligning schedule and data with cache topology,” not to memory striping (Tousimojarad et al., 2014).

The same theme appears in embedded MPSoC OS scheduling, but expressed through inter-process data reuse rather than home tiles. In “Locality-Aware Process Scheduling for Embedded MPSoCs,” the scheduler uses a process graph, an extended process graph with inter-task dependences, and a sharing matrix $M[p][q]$ derived from statically computed shared data sets $SS_{p,q} = DS_p \cap DS_q$ (0710.4652). Two complementary rules define the schedule: processes that do not share data are scheduled on different cores if possible, while processes that cannot execute at the same time but share data are mapped to the same core so that they share cache contents. For a core whose current process is $i$, the next process $j$ is selected such that

$|SS_{ij}| \ge |SS_{ik}|$

for any candidate $k$. This is a direct schedule-alignment criterion: temporal adjacency on a core is chosen to maximize reuse, while concurrent placement across cores is chosen to avoid unnecessary duplication and conflict (0710.4652).

These two lines of work differ in mechanism—home-cache placement versus reuse-aware OS scheduling—but converge on the same operational point. Cache awareness is not only a matter of which data are accessed; it is also a matter of where the thread runs when it accesses them, and which task ran immediately before on that same cache.

3. Real-time formulations, cache partitioning, and schedulability

In hard real-time multicore scheduling, cache-aware schedule alignment is often posed as an explicit optimization problem. One formulation uses an interval-based hyper-period schedule with execution fractions $K/V$0, binary job-to-core variables $K/V$1, per-core L1 capacity constraints

$K/V$2

and a quadratic objective that maximizes communication affinity among jobs co-located on the same L1 cache in the same interval (Zaourar et al., 2013). The objective rewards pairs with large $K/V$3, the number of communication flows between jobs $K/V$4 and $K/V$5, and therefore makes temporal and spatial co-location part of the static schedule itself (Zaourar et al., 2013).

A different real-time formulation appears in federated scheduling of sporadic DAG tasks. The DAG-OT model extends DAG nodes to $K/V$6, where $K/V$7 is the executable object, $K/V$8 is a concave BUNDLE-style WCETO function, and $K/V$9 is the number of threads (Tessler et al., 2020). The core allocation bound remains

$gK/gV$0

but both workload $gK/gV$1 and critical-path length $gK/gV$2 now include inter-thread cache benefits. Collapsing nodes with the same object reduces workload through the concavity of $gK/gV$3, and the synthetic evaluation reports “about 20% reduction in the number of cores dedicated to high-utilization tasks,” while the platform study reports “25%–50% reduction in cores per task” (Tessler et al., 2020). Here, alignment means forcing multiple requests for the same object to execute as a single cache-sharing unit.

Mixed-criticality scheduling introduces another form of alignment: dynamic redistribution of shared cache across modes. In that model, task WCETs are functions $gK/gV$4 and $gK/gV$5 of the number of LLC pages locked for the task, with separate allocations $gK/gV$6 and $gK/gV$7 and the monotonic condition $gK/gV$8 for high-criticality tasks (Awan et al., 2017). When the system switches to high-criticality mode, all low-criticality tasks are discarded and their cache portions are reassigned to high-criticality tasks. The schedulability analysis therefore uses $gK/gV$9 for carry-over jobs and $8\times 8$0 for full high-mode jobs. This is schedule alignment at mode-transition granularity: the schedule, the criticality semantics, and the cache partition must agree on which jobs will execute with which cache budget (Awan et al., 2017).

The most explicit co-optimization of cache and schedule is the three-layer framework of breadth-first search over cache partitions, first-fit task allocation, and per-core schedulability analysis (Sun et al., 2023). The framework improves NP-FP schedulability “by an average of 15.2% with a maximum improvement of 233.6% (when tasks are highly cache-sensitive) and a minimum of 1.6% (when cache sensitivity is low)” (Sun et al., 2023). It also reports that “clustering similar-period (or mutually compatible) tasks often leads to higher schedulability (on average 7.6%) than clustering by cache sensitivity.” This directly qualifies a common simplification: exploiting cache sensitivity is not always the right alignment objective. Under non-preemptive scheduling, period compatibility and blocking can dominate.

4. Predictive and adaptive forms of alignment

Cache-aware alignment need not require explicit placement primitives. In heterogeneous clusters, it can be based on compact statistical prediction. “Lightweight Task Analysis for Cache-Aware Scheduling on Heterogeneous Clusters” models stack distance as a random variable $8\times 8$1, fits its distribution from a very small sample of memory accesses, and predicts miss ratio in constant time as

$8\times 8$2

The evaluation reports that around $8\times 8$3 samples were adequate for soplex data misses with “~99%” prediction accuracy, and around $8\times 8$4 samples were adequate for dealII instruction misses with “99.6%” accuracy (0902.4822). The scheduler itself is not fixed in the paper; the contribution is the task characterization that makes cross-platform cache-aware placement feasible.

Adaptive runtime control can also use cache metrics directly. THEAS, implemented in gem5 for a 4-core ARM Cortex-A72, reads PMCs including IPC, dcache_overallMisses, and overallAccesses, and changes per-core resource levels LOW, MEDIUM, and HIGH (Muhammad et al., 10 Oct 2025). Its CPU power model includes cache misses explicitly,

$8\times 8$5

while L2 dynamic power is modeled as proportional to overallAccesses. The decision rule downscales cores with low IPC and high cache miss rate and upscales cores with high IPC and high fetch rate. This is not explicit task migration, but it is still a form of schedule alignment: the effective computational capability of each core is aligned with the cache behavior of the workload currently executing there (Muhammad et al., 10 Oct 2025).

A broader implication of these predictive and adaptive approaches is that cache-aware alignment can be mediated by abstractions rather than by explicit memory-placement APIs. This is especially important when the scheduler does not control home tiles, replacement state, or exact eviction order.

5. Prefix caches, serving pipelines, and schedule-level reuse in LLM systems

In LLM RL training, cache-aware schedule alignment can be expressed as a reordering of the training step around a reusable cache interface. “Schedule-Level Shared-Prefix Reuse for LLM RL Training” decomposes a GRPO/PPO update into three phases: prefix forward once, suffix computation as ordinary microbatches while reading prefix $8\times 8$6 and accumulating prefix-side $8\times 8$7, and prefix backward once on the accumulated gradient cache (Li et al., 31 May 2026). The schedule is proved equivalent to baseline training “over real arithmetic” and shown to align numerically within finite-precision tolerance. On dense Llama3-8B, Qwen3-8B, and MoE Qwen3-MoE-30B-A3B, the schedule reaches “up to 4.395x speedup (2.930x under a conservative compile-on comparison),” reduces “Phase-B peak HBM by up to 59.1%,” and extends the Llama3-8B capacity frontier “from 17,920 to 29,696 total tokens” (Li et al., 31 May 2026). The aligned object here is neither a task nor a core but a prefix computation boundary.

Serving systems expose a different version of the same principle. In grounded RAG, CacheWeaver observes that adjacent queries can retrieve overlapping evidence in different orders, so “document overlap is common in RAG; prefix overlap is rare” (Tan et al., 18 Jun 2026). It keeps a prefix tree over recently served evidence sequences and greedily reorders the retrieved set to maximize reusable prefix depth without changing either the evidence set or the serving engine. Across three vLLM configurations, this lowers median TTFT by “about 20-33 percent” relative to retrieval-order prefix caching, and the greedy policy reaches “97.5 percent of the median TTFT gain from oracle ordering” (Tan et al., 18 Jun 2026). The crucial misconception it corrects is that set overlap alone is sufficient for prefix-cache reuse.

Cluster-scale prefix caching raises a further issue: a scheduler can become over-coupled to cache placement. TokenLake addresses this by introducing a “unified segment-level prefix cache pool” with a declarative cache interface exposing query tensors, prefix caches, and cache-aware operations (Wu et al., 24 Aug 2025). Powered by segment-level management and “a heavy-hitter-aware load balancing algorithm,” it allows the scheduler to “schedule requests elastically by using existing techniques without considering prefix cache management.” On real-world workloads, throughput improves by “up to 2.6\times and 2.0\times,” and hit rate by “2.0\times and 2.1\times,” relative to cache-aware routing and cache-centric PD-disaggregation baselines (Wu et al., 24 Aug 2025). This suggests that one important form of schedule alignment is the separation of concerns between cache control and compute scheduling.

6. Time-discretization and cache-conditioned alignment in generative models

Recent work on diffusion and video generation extends cache-aware alignment to the timestep schedule itself. BudCache fixes the compute budget

$8\times 8$8

in advance, searches for a step-level cache mask $8\times 8$9, and then performs “cache-aware schedule alignment” by optimizing the time discretization $M[p][q]$0 for the selected mask $M[p][q]$1 (Lei et al., 11 Jun 2026). The cached Euler update uses the last active prediction $M[p][q]$2,

$M[p][q]$3

and the paper notes that the step size $M[p][q]$4 multiplies the cache error term. Under very tight budgets, schedule alignment is therefore used to reduce “cache-induced trajectory mismatch.” In the low-NFE ablation, BudCache-Opt improves over BudCache at 5, 6, and 7 NFE, while leaving latency essentially unchanged (Lei et al., 11 Jun 2026).

SeaCache solves an analogous problem by changing the cache metric rather than the schedule optimizer. It derives a Spectral-Evolution-Aware filter

$M[p][q]$5

from a Wiener-style MMSE denoiser under the forward process $M[p][q]$6, normalizes it, and computes redundancy in a “spectrally aligned representation” instead of raw feature space (Chung et al., 22 Feb 2026). The dynamic cache schedule still uses an accumulated-distance threshold, but the distances are computed after SEA filtering. Empirically, the method yields better PSNR-refresh and latency-quality trade-offs than TeaCache, TaylorSeer, and related baselines, and the paper interprets this as respecting the fact that “low-frequency structure appears early and high-frequency detail is refined later” (Chung et al., 22 Feb 2026).

Salt generalizes the alignment problem to autoregressive video generation with a KV cache whose quality depends on previous denoising schedules (Ge et al., 3 Apr 2026). Its Self-Consistent Distribution Matching Distillation adds a semigroup-style regularizer on denoising composition,

$M[p][q]$7

and its cache-aware training samples mixed step counts $M[p][q]$8, treating the resulting cache states as a “quality-parameterized condition.” Low-step rollouts are aligned to higher-step references with a cache-conditioned feature alignment loss. The paper reports consistent improvements on autoregressive backbones such as Self Forcing, LongLive, and Causal Forcing, including at “2 NFE” and “4 NFE,” while remaining “compatible with diverse KV-cache memory mechanisms” (Ge et al., 3 Apr 2026). In this formulation, schedule alignment spans timesteps, NFE budgets, and cache states simultaneously.

7. Assumptions, recurring misconceptions, and open issues

The literature repeatedly narrows the cache model in order to make alignment tractable. Some works consider instruction-cache sharing only and ignore data caches (Tessler et al., 2020). Others model only private L1 data caches and a shared L2 through working set size and communication affinities (Zaourar et al., 2013). Mixed-criticality analyses assume page-based LLC partitioning with Colored Lockdown and identical cores (Awan et al., 2017). Statistical miss-ratio prediction explicitly states that it predicts misses, not “memory latency or bandwidth,” out-of-order execution, or prefetching (0902.4822). These are not contradictions; they are modeling boundaries.

Several misconceptions recur across domains. One is that maximizing locality is always the dominant objective. The co-optimization results on multicore real-time systems show otherwise: when tasks are not strongly cache-sensitive, clustering by period compatibility can outperform clustering by cache sensitivity by “on average 7.6%” (Sun et al., 2023). Another is that cache-aware scheduling must rely on architecture-specific APIs. The TILEPro64 work deliberately avoids Tilera-specific user libraries and uses standard Linux affinity APIs instead (Tousimojarad et al., 2014). A third is that overlap in content is sufficient for reuse. CacheWeaver demonstrates that, in grounded RAG, evidence overlap matters only when it is converted into reusable prefix overlap (Tan et al., 18 Jun 2026).

A plausible implication is that future progress will come less from isolated cache heuristics than from joint models of task structure, cache state, and scheduler action. Several papers already point in that direction: runtime systems that automate task–data affinity, compilers that insert localisation patterns, more advanced collapse heuristics in DAG-OT, integration of cache pools with elastic schedulers, and richer cache-conditioned training for generative models (Tousimojarad et al., 2014, Tessler et al., 2020, Wu et al., 24 Aug 2025, Ge et al., 3 Apr 2026). Across these settings, cache-aware schedule alignment remains a coordination problem: a schedule is effective only when the cache abstraction it assumes is the cache abstraction the system actually provides.