Lookahead-Aware Scheduler

• Lookahead-aware scheduling is a strategy that uses partial future execution information to improve current dispatch and resource allocation decisions.
• Formal models demonstrate its effectiveness through competitive ratios, index policies, and delay game frameworks that counter uncertainty and contention.
• Empirical systems like SageSched, Astro, and SQUASH show significant latency improvements and enhanced system performance under variable workloads.

A lookahead-aware scheduler is a scheduler that uses explicit partial information about future execution when making present dispatch, prioritization, batching, or resource-allocation decisions. The term is not standardized across the literature, but the underlying idea is recurrent: current decisions are improved by incorporating some forecast, preview, or structurally informative proxy of future cost, future contention, future job phase, or future feasibility. In current work, this future information appears in several forms, including exact preview of upcoming jobs, output-length distributions for LLM inference, compiler-derived program phases, predicted latency under candidate batchings, deadline-period progress trajectories, and expected future holding costs (Gan et al., 9 Mar 2026, Novaes et al., 2019, Huang et al., 21 Apr 2025, Usui et al., 2015, Gurushankar et al., 13 Jan 2026).

1. Formal foundations of lookahead

The most direct formalization appears in semi-online scheduling on identical machines. In the $k$-lookahead model, when job $J_i$ arrives, the scheduler knows $p_i$ and also $p_{i+1}, p_{i+2}, \ldots, p_{i+k}$, but must still schedule $J_i$ non-preemptively and irrevocably without knowledge of $p_{i+k+1}, p_{i+k+2}, \ldots, p_n$. For makespan minimization on two identical machines, the paper proves a lower bound of $\frac{4}{3}$ for deterministic algorithms and gives an optimal $1$-lookahead algorithm with a matching upper bound of $\frac{4}{3}$. For three identical machines, it proves a lower bound of $\frac{15}{11}$ and gives a $J_i$0-competitive improved semi-online algorithm (Dwibedy et al., 2023).

A different theoretical lineage comes from delay games, where one player may delay her moves in order to obtain lookahead on the opponent’s moves. For WMSO$J_i$1U delay games, bounded lookahead is not always sufficient: the paper presents a game that can be won with unbounded lookahead, but not with bounded lookahead. It then shows that, for this class, the exact evolution of the lookahead is irrelevant: the winner is always the same, as long as the initial lookahead is large enough and the lookahead tends to infinity (Zimmermann, 2015).

These formal models clarify that lookahead is not a single mechanism. It may be a fixed finite preview window, an unboundedly growing preview process, or a structured partial-information model. They also clarify that the value of lookahead depends on the objective class: some objectives improve under small preview, while others require genuinely unbounded preview.

2. Probabilistic lookahead in LLM inference scheduling

In shared-backend LLM inference, lookahead has recently been instantiated as probabilistic anticipation of future service burden. SageSched is described in its paper as an efficient scheduler for LLM inference under demand uncertainty and hybridity, and its policy is described as uncertainty-aware. In substance, it is lookahead-aware because, before placing or reprioritizing a request, it predicts an output-length distribution, maps that distribution into a future service-cost distribution that reflects both compute and memory effects, and then prioritizes using the Gittins index rather than a point estimate. The problem setting is concurrent LLM requests on GPU servers, and the primary metric is Time-to-Last-Token (TTLT), motivated by complete-answer latency. The two stated obstacles are demand uncertainty, since output length is unknown at arrival and varies even for the same prompt, and hybridity, since LLM inference is both compute-intensive and memory-intensive because the KV cache grows with sequence length and occupies GPU memory throughout decoding (Gan et al., 9 Mar 2026).

The prediction pipeline is deliberately lightweight and training-free. SageSched uses a semantic-aware history-based predictor: it stores recently served requests in a FIFO history window of size 10,000, computes prompt embeddings, retrieves sufficiently similar historical prompts with FAISS IndexFlat, and uses the empirical output-length distribution of those matched requests. The default cosine-similarity threshold is $J_i$2, and if the system is in a warm-up phase it augments the retrieval pool with public-dataset requests. The reported average per-request prediction latency is below $J_i$3 ms, broken down as $J_i$4 ms for semantic embedding extraction and $J_i$5 ms for vector retrieval, versus about $J_i$6 ms for an LLM-based predictor.

Its hybrid cost model begins from the observation that both memory-bound and compute-bound regimes induce cumulative cost proportional to accumulated sequence length across decode steps. Let $J_i$7 denote input prompt length and $J_i$8 denote output token length. The memory-bound cost is

$J_i$9

and the compute-bound cost is

$p_i$0

Because the two expressions share the same order structure up to scaling, SageSched adopts the unified practical service-cost model

$p_i$1

Scheduling is then performed on cost distributions rather than means. The priority rule is the Gittins index: $p_i$2 The paper notes that the typesetting is malformed, but the intended meaning is the usual infimum over service budget $p_i$3 of expected consumed service before stopping divided by probability of completion within that budget. Requests with smaller Gittins indices are prioritized. Priorities are not recomputed after every token; instead, SageSched refreshes the remaining-cost distribution and recomputes the index only at service bucket boundaries. In the design discussion it defaults to 10 buckets, while experiments use a bucket size of 200. Implemented on top of vLLM, SageSched reports the best TTLT across the tested configurations, including a 28.7% average-TTLT improvement over the second-best baseline, TRAIL, at 8 requests per second on Qwen3-32B.

3. Systems mechanisms for anticipatory scheduling

Lookahead in systems work often appears as a structured proxy for near-future behavior rather than as direct future token prediction. In Astro, a compiler/runtime scheduler for ARM big.LITTLE systems, the scheduler’s state is explicitly

$p_i$4

where $p_i$5 is the hardware configuration, $p_i$6 is the compiler-derived program phase, and $p_i$7 is the hardware phase from runtime counters. The compiler partitions code into coarse phases such as $p_i$8, $p_i$9, $p_{i+1}, p_{i+2}, \ldots, p_{i+k}$0, and $p_{i+1}, p_{i+2}, \ldots, p_{i+k}$1, and the runtime uses this phase identity together with dynamic hardware state in a Q-learning-based decision process. The anticipatory element is that the scheduler can react at function boundaries and blocking calls before counters alone have fully exposed the new execution regime (Novaes et al., 2019).

In multi-SLO LLM serving, lookahead appears as batch-level future-cost evaluation. The SLO-aware scheduler models prefill and decode latency as functions of batch size and sequence lengths, uses estimated output length or output-length distributions, and searches over request orderings and batch assignments using simulated annealing. Its objective is

$p_{i+1}, p_{i+2}, \ldots, p_{i+k}$2

where $p_{i+1}, p_{i+2}, \ldots, p_{i+k}$3 is the number of requests meeting their SLOs and $p_{i+1}, p_{i+2}, \ldots, p_{i+k}$4. This yields a predictive, future-cost-aware batch scheduler rather than a purely deadline-aware or FCFS policy. The paper reports improvements of up to $p_{i+1}, p_{i+2}, \ldots, p_{i+k}$5 in SLO attainment and up to $p_{i+1}, p_{i+2}, \ldots, p_{i+k}$6 reduction in average latency relative to vLLM and LMDeploy on the cited datasets (Huang et al., 21 Apr 2025).

SmoothAgent relocates lookahead into agent-serving runtimes. It observes that representative context-engineering transformations are segment-decomposable: $p_{i+1}, p_{i+2}, \ldots, p_{i+k}$7 so transformed prefixes and their KV caches can be prepared asynchronously before the main agent reaches the transformation point. The serving backend then schedules these lookahead transformations as best-effort requests beside latency-critical foreground requests, using TTFT slack in PD disaggregation and a TBT-constrained hybrid-batch policy in PD co-location. This is a scheduler that is lookahead-aware not because it predicts user requests, but because it schedules asynchronous future-needed work early enough to avoid blocking later (Pan et al., 30 Jun 2026).

SQUASH provides another distinctive mechanism: continuous feasibility tracking over the remainder of a deadline period. For long-deadline-period accelerators, it defines

$p_{i+1}, p_{i+2}, \ldots, p_{i+k}$8

and

$p_{i+1}, p_{i+2}, \ldots, p_{i+k}$9

An accelerator becomes urgent when it is not on track, rather than only when it is very near deadline. For short-deadline accelerators, SQUASH instead uses worst-case urgent-period estimates such as

$J_i$0

This is lookahead over the finite deadline horizon, expressed as remaining-work versus remaining-time feasibility (Usui et al., 2015).

4. Index policies, preview rules, and stochastic optimality

A major branch of lookahead-aware scheduling is index-based. In the Time-Varying Holding Cost problem, the policy called LookAhead proves that when class-1 holding costs increase with age, the right comparison is not current cost rate but future expected cost rate. In the two-class preemptive M/M/1 setting studied in that paper, the optimal non-decreasing index policy is

$J_i$1

Equivalently, the threshold $J_i$2 satisfies

$J_i$3

The paper’s central claim is that an age-sensitive scheduler should “look at their cost some $J_i$4 time into the future,” where $J_i$5 is the optimal lookahead amount, rather than applying a myopic generalized $J_i$6 rule (Gurushankar et al., 13 Jan 2026).

In online packet scheduling with bounded delay, lookahead is formalized as exact one-step arrival preview. For 2-bounded instances with 1-lookahead, the paper gives a deterministic online algorithm with competitive ratio

$J_i$7

and proves a nearly tight lower bound of

$J_i$8

The scheduler computes a short optimal plan over the visible horizon and, under a calibrated condition, may transmit the packet planned for the next slot instead of the packet planned for the current slot. The key comparison is between immediate execution and the best visible two-step continuation; the paper contrasts this with the no-lookahead setting, where the optimal competitive ratio for 2-bounded instances is already $J_i$9 (Böhm et al., 2016).

These results illustrate two distinct roles for lookahead. In stochastic queueing, lookahead is expectation over future cost conditioned on a model. In competitive online scheduling, lookahead is exact preview over a short adversarial horizon. Both depart from purely reactive scheduling, but they do so through different mathematical objects: index policies in the former, local plan-based decisions in the latter.

5. Empirical behavior and operational trade-offs

The empirical record across systems papers is consistent on one point: lookahead is most valuable when uncertainty or contention makes current-state heuristics unreliable. In SageSched, gains are larger under heavier contention, which the paper presents as exactly the regime where better probabilistic lookahead should help most. Its ablations show that both Gittins-based ordering and runtime refresh at bucket boundaries improve TTLT, and that the Gittins-based policy degrades less than mean-based ranking when synthetic noise is injected into cost distributions (Gan et al., 9 Mar 2026).

The SLO-aware LLM scheduler exhibits a similar dependency on prediction quality. Its sensitivity study shows that better output-length prediction systematically improves the objective $p_{i+k+1}, p_{i+k+2}, \ldots, p_n$0, and one highlighted setting reports a 65% increase in $p_{i+k+1}, p_{i+k+2}, \ldots, p_n$1 over the paper’s original SLO-aware scheduler and an 84% increase over baseline when output-length error is under 2.5%. At the same time, its lookahead remains limited to the currently visible request set and does not model future arrivals (Huang et al., 21 Apr 2025).

SmoothAgent shows that lookahead can improve TTFT even when the “future” being prepared is not future user work but future context state. The paper reports transform-point TTFT reductions up to $p_{i+k+1}, p_{i+k+2}, \ldots, p_n$2, average tail-TTFT reduction of $p_{i+k+1}, p_{i+k+2}, \ldots, p_n$3 on Qwen3-8B and $p_{i+k+1}, p_{i+k+2}, \ldots, p_n$4 on Qwen3-32B in PD co-location, and $p_{i+k+1}, p_{i+k+2}, \ldots, p_n$5 average transform-point TTFT reduction in a PD-disaggregated configuration. Its scheduler-specific evaluation also shows why interference control matters: a token-budget-only latency model reaches maximum relative error of $p_{i+k+1}, p_{i+k+2}, \ldots, p_n$6, while the context-aware model keeps maximum relative error to $p_{i+k+1}, p_{i+k+2}, \ldots, p_n$7 (Pan et al., 30 Jun 2026).

In heterogeneous hardware scheduling, anticipatory mechanisms produce mixed but concrete gains. Astro is faster than GTS in six of eight benchmarks and more energy efficient in five of eight, but the paper also stresses that there is no universal winner between Astro-static and Astro-hybrid: hybrid adapts better when static phase-to-configuration mappings are imperfect, as in ParticleFilter (Novaes et al., 2019). SQUASH similarly shows that progress-aware priority is more effective than late urgency: relative to FRFCFS-DynOpt, it achieves 100% deadline-met ratio and 10.1% better system performance, while the full design also yields a 26% average reduction in maximum slowdown with only 1.7% performance loss versus the non-probabilistic variant (Usui et al., 2015).

The trade-offs are therefore not uniform. Some lookahead policies optimize completion latency rather than first-token latency; some improve throughput at modest energy cost; some depend heavily on accurate future estimates; and some rely on coarse but low-overhead proxies such as phase labels or progress curves.

6. Boundaries, misconceptions, and current directions

A common misconception is that lookahead-aware scheduling requires deterministic clairvoyance. Much of the literature states the opposite. SageSched’s lookahead is probabilistic rather than exact; Astro uses compiler-visible program structure as a prior rather than full future execution traces; SQUASH uses on-track/off-track progress; and SmoothAgent exploits asynchronous preparation of deterministic future-needed transformations rather than future-arrival prediction. This suggests that the defining property is not exact foresight, but the use of explicit future-relevant information that is richer than instantaneous load or queue length.

Another misconception is that all such schedulers reason over long horizons. Some are deliberately local. JASDA, for example, is a window-aware, negotiation-based extension of Scheduler-Driven Job Atomization for MIG-enabled GPUs. It announces a single future execution window

$p_{i+k+1}, p_{i+k+2}, \ldots, p_n$8

jobs generate feasible non-preemptive variants for that announced window, and the scheduler clears them by weighted interval scheduling. The paper is explicit that JASDA is locally optimal per scheduling iteration, but not globally optimal against an offline clairvoyant scheduler; it is best understood as a myopic, rolling, single-window lookahead-aware scheduler (Konopa et al., 16 Oct 2025).

Lookahead can also move from execution cost to response quality. In multi-model LLM routing, Lookahead Routing predicts latent representations of candidate models’ future responses and uses those predicted futures to dispatch queries. Its routing policy maximizes expected downstream score

$p_{i+k+1}, p_{i+k+2}, \ldots, p_n$9

while the lookahead signal is a latent response representation $\frac{4}{3}$0. The paper reports an average performance gain of 7.7% over the state of the art, but it also notes that it does not explicitly optimize cost trade-offs across large and small models (Huang et al., 22 Oct 2025).

Across these works, several open directions recur. Current systems often remain short-horizon, rely on coarse predictors, or decouple dispatch from future arrivals and future queue evolution. Some papers provide exact optimality within restricted policy classes, while others provide empirical systems designs with no global guarantees. A plausible implication is that the field is converging on a broad principle rather than a single algorithmic template: the scheduler should encode as much future-relevant structure as can be obtained cheaply enough, then use that structure in a decision rule whose overhead does not erase the benefit of anticipation.