ReflecSched: Hierarchical LLM Scheduling

• ReflecSched is a hierarchical framework for dynamic flexible job-shop scheduling that decouples strategic reflection from direct execution using simulated futures.
• It employs multi-level heuristic rollouts to compare best and worst trajectories, distilling a concise Strategic Experience to guide final decisions.
• This approach mitigates long-context overload, heuristic underutilization, and myopic decisions, achieving statistically significant improvements in makespan optimization.

Searching arXiv for the specified ReflecSched paper and closely related scheduling context. ReflecSched is a hierarchical LLM-based framework for Dynamic Flexible Job-Shop Scheduling (DFJSP) that does not use the LLM as a direct, one-shot scheduler. Instead, it separates scheduling into strategic reflection over heuristic-driven simulated futures and final execution guided by a distilled natural-language summary called “Strategic Experience” (Cao et al., 3 Aug 2025). In the formulation studied, DFJSP combines machine flexibility, where each operation can be processed on a subset of candidate machines, with online dynamism induced by events such as new job arrivals and machine breakdowns, and the objective is to minimize the makespan under precedence and machine resource constraints (Cao et al., 3 Aug 2025).

1. Problem domain and scheduling objective

ReflecSched is situated in Dynamic Flexible Job-Shop Scheduling, which the source characterizes as an NP-hard problem challenged by real-time event adaptation and complex machine routing (Cao et al., 3 Aug 2025). The setting involves operations from multiple jobs scheduled on a set of machines under real-time stochastic events. Flexibility means that each operation can be assigned to a subset of candidate machines, while dynamism means that online disruptions alter the feasible and desirable scheduling decisions over time (Cao et al., 3 Aug 2025).

The optimization target is the makespan, namely the maximum completion time among all jobs (Cao et al., 3 Aug 2025). The validity conditions emphasized in the source are standard for shop scheduling but decisive for interpreting ReflecSched’s action space: operations of the same job must be processed in order, and a machine can process at most one operation at a time (Cao et al., 3 Aug 2025). This positions the framework within online combinatorial scheduling rather than static sequence optimization.

A useful context is that the paper contrasts ReflecSched with both traditional dispatching rules and direct LLM control. Traditional dispatching rules are described as efficient but rigid, whereas deep learning approaches are described as opaque and requiring intricate feature engineering. ReflecSched is proposed in response to the claim that LLMs promise adaptive reasoning without this engineering overhead, yet direct application remains suboptimal (Cao et al., 3 Aug 2025). This suggests that the contribution is not merely another policy class for DFJSP, but a restructuring of how an LLM participates in online scheduling.

2. Failure modes of direct LLM scheduling

The paper studies a baseline called LLM-Direct, in which the current state of the shop floor is encoded into a prompt and the LLM directly selects the next action (Cao et al., 3 Aug 2025). ReflecSched is motivated by three failure modes identified for this baseline.

The first is the long-context paradox. The prompt contains large amounts of static and dynamic information, including job structures, machine data, candidate actions, and machine statuses. The source reports that omitting the static information block, despite it being over 70% of the prompt, causes negligible performance change, and interprets this as evidence that the model is underutilizing the information rather than reasoning effectively over it (Cao et al., 3 Aug 2025). The diagnosis is therefore not only that prompts are long, but that verbose state descriptions fail to produce corresponding gains in decision quality.

The second is the underutilization of expert heuristics. The paper argues that LLMs are poor at reliably applying procedural scheduling knowledge such as Priority Dispatching Rules (PDRs). To isolate this issue, it introduces PDR-Bench, where each instance is constructed so that one particular heuristic is known by design to be optimal. Even when the prompt explicitly instructs the LLM to use the optimal heuristic, the model often fails to reproduce the heuristic’s performance and instead falls back on pretrained generalized behavior (Cao et al., 3 Aug 2025). In this account, the weakness is not declarative comprehension but procedural policy enactment.

The third is myopic decision-making. The source attributes this to autoregressive next-token generation, which biases decisions toward local improvements rather than long-horizon schedule quality. To quantify this, it uses the Greedy Decision Ratio (GDR) on GEN-Bench and reports consistently high GDR for direct LLMs, indicating a strong preference for greedy actions (Cao et al., 3 Aug 2025). The paper terms this “myopic greed.”

Taken together, these three failure modes define the design space that ReflecSched addresses: prompt compression for the long-context paradox, operational rather than textual use of heuristics for heuristic underutilization, and explicit lookahead for myopic greed (Cao et al., 3 Aug 2025).

3. Hierarchical architecture and “Strategic Experience”

ReflecSched addresses those failures by decoupling strategy from execution. Its pipeline has two sequential modules: a Hierarchical Reflection Module and an Experience-Guided Decision-Making Module (Cao et al., 3 Aug 2025). The workflow given in the source is: receive a decision point, run multi-level heuristic simulations into the future, compare best and worst simulated trajectories, ask the LLM to distill them into a short natural-language summary called Strategic Experience $\mathcal{E}$, inject $\mathcal{E}$ into the final prompt, and then ask the LLM to select the next action in light of that strategic guidance (Cao et al., 3 Aug 2025).

The central artifact is Strategic Experience, defined as a concise text summary synthesizing why the best trajectory is better, what scheduling pattern should be preferred, what pitfalls should be avoided, and which actions or priorities matter strategically (Cao et al., 3 Aug 2025). The source emphasizes that the LLM is not merely summarizing data but extracting the distinguishing strategic principle between the best and worst outcomes. This makes the LLM a policy synthesizer rather than a direct scheduler.

The reflection stage is invoked selectively, only when a dynamic event occurs, such as a job arrival or machine failure (Cao et al., 3 Aug 2025). That selective triggering is part of the framework’s computational design. A plausible implication is that ReflecSched treats environmental discontinuities as the natural moments for re-estimating strategic priorities, rather than recomputing reflection at every local dispatch decision.

This division of labor is the framework’s defining structural feature. The LLM is assigned the tasks most aligned with its purported strengths in the paper: synthesizing and reasoning over structured natural-language evidence, rather than directly processing a long, overloaded shop-floor prompt (Cao et al., 3 Aug 2025).

4. Reflection module, rollout design, and policy view

The Hierarchical Reflection Module uses a recursive Simulate–Reflect–Refine loop inspired by prior work on reflection and tree-based reasoning (Cao et al., 3 Aug 2025). The simulations are deterministic over the lookahead horizon, assuming no new stochastic events during that window (Cao et al., 3 Aug 2025). Multi-level lookahead proceeds from a maximum level $l_{\max}$ down to $l=0$.

A randomized base policy $\pi_{\text{base}}$ is defined by sampling a heuristic or PDR from a predefined pool at each decision point (Cao et al., 3 Aug 2025). At higher levels $l>0$, the framework performs sparse, long-range exploration by applying $\pi_{\text{base}}$ throughout the rollout. At the base level $l=0$, it systematically evaluates all immediate actions $a \in A(S_t)$ by first executing $a$ and then continuing with $\mathcal{E}$0 (Cao et al., 3 Aug 2025). This means the system evaluates immediate actions in the context of heuristic-driven downstream consequences rather than in isolation.

For a rollout trajectory $\mathcal{E}$1, the hypothetical partial makespan cost is defined as

$\mathcal{E}$2

The best and worst trajectories are then selected by

$\mathcal{E}$3

$\mathcal{E}$4

Strategic Experience is produced by the LLM as

$\mathcal{E}$5

(Cao et al., 3 Aug 2025).

The contrastive best-versus-worst design is central to the method. The source presents it as a way to expose causal differences in scheduling behavior, rather than inundating the LLM with all rollouts (Cao et al., 3 Aug 2025). This suggests that ReflecSched’s hierarchy is not only computational but epistemic: it filters evidence before asking the model for strategy.

The paper further interprets the framework through Approximate Policy Iteration and rollout theory. It states two assumptions: that the rollout cost $\mathcal{E}$6 approximates the true cost-to-go under the base heuristic $\mathcal{E}$7, and that the LLM performs faithful reflection by correctly capturing that the best trajectory is superior to the worst one (Cao et al., 3 Aug 2025). Under these assumptions, the induced policy $\mathcal{E}$8 is greedily improved with respect to the strategic guidance, with the stated Monotonic Improvement Guarantee

$\mathcal{E}$9

The paper presents this as a conditional guarantee grounded in rollout-based dynamic programming ideas (Cao et al., 3 Aug 2025).

5. Decision module and mitigation of the three pitfalls

Once $l_{\max}$0 has been produced, the Experience-Guided Decision-Making Module uses only the immediate dynamic state and the strategic experience summary in the final prompt (Cao et al., 3 Aug 2025). The final action is written as

$l_{\max}$1

This prompt design is explicitly the mechanism by which ReflecSched addresses the long-context paradox: the large raw state is not passed through unchanged, but compressed into a strategic message (Cao et al., 3 Aug 2025).

The same architecture addresses heuristic underutilization by changing how heuristics enter the system. In ReflecSched, heuristics are not merely described to the LLM; they are executed in the rollout generator. The LLM then compares heuristic-driven futures rather than attempting to instantiate a heuristic procedurally from text instructions (Cao et al., 3 Aug 2025). The paper treats this as an operational embedding of expert knowledge.

Myopic greed is addressed by forcing reflection on simulated future outcomes before action selection. The source reports that ReflecSched reduces the Greedy Decision Ratio consistently across models, and interprets this as evidence that the strategic summary steers the LLM toward non-greedy actions (Cao et al., 3 Aug 2025). This framing is consistent with the system’s two-stage structure: lookahead occurs in simulation, while the LLM transforms the results into a strategic prior for the immediate decision.

The following table summarizes the relation between the identified failure modes and the corresponding components of ReflecSched.

Failure mode	Source diagnosis	ReflecSched mechanism
Long-context paradox	Static information is over 70% of the prompt yet omission causes negligible performance change	Compress future-relevant information into Strategic Experience $l_{\max}$2
Underutilization of expert heuristics	LLM often fails to reproduce even the optimal instructed heuristic on PDR-Bench	Use heuristic/PDR policies operationally in rollouts
Myopic decision-making	Direct LLMs exhibit consistently high Greedy Decision Ratio on GEN-Bench	Reflect on simulated future outcomes before final action selection

A plausible implication is that ReflecSched should be understood less as a pure prompting technique than as a hybrid decision architecture in which search, heuristic prior knowledge, and natural-language synthesis are explicitly separated.

6. Experimental protocol, ablations, and reported performance

The paper evaluates ReflecSched on two benchmark suites. GEN-Bench is a general-purpose benchmark with 20 Normal-scale instances and 20 Small-scale instances. PDR-Bench is a diagnostic benchmark for testing heuristic utilization, with 112 Normal-scale instances and 12 Small-scale instances (Cao et al., 3 Aug 2025). The LLMs evaluated are GPT-4o, DeepSeek-V3, Qwen3-8B, Qwen3-14B, and Qwen3-32B, all used in zero-shot mode with Chain-of-Thought prompting (Cao et al., 3 Aug 2025).

The primary performance metric is Relative Percent Deviation (RPD), calculated against the best makespan found for each instance across all methods and runs, with lower values being better. The paper also reports Win Rate and uses the Wilcoxon signed-rank test for significance (Cao et al., 3 Aug 2025). For the diagnostic analyses, it uses 5 independent generations, majority vote, and temperature $l_{\max}$3. For final comparisons, it uses temperature $l_{\max}$4 and averages over 3 independent runs (Cao et al., 3 Aug 2025).

The main reported comparison is against LLM-Direct. ReflecSched is reported to statistically significantly outperform the baseline with a 71.35% Win Rate and a 2.755% Relative Percentage Deviation reduction, with $l_{\max}$5 under the Wilcoxon signed-rank test (Cao et al., 3 Aug 2025). The framework is also reported to outperform all evaluated individual PDR heuristics overall and to perform on par with the best heuristic tailored to each instance across all problem cases. For that comparison, the paper reports average RPD values of 0.0620% on Normal-scale instances with $l_{\max}$6 and 0.9331% on Small-scale instances with $l_{\max}$7 (Cao et al., 3 Aug 2025).

The paper further reports that token-consumption analysis in Appendix H shows that on larger Normal-scale problems, ReflecSched uses fewer tokens than the baseline because the final prompt is compressed via $l_{\max}$8 (Cao et al., 3 Aug 2025). On PDR-Bench, it substantially reduces RPD relative to LLM-Direct across all evaluated models, and it reduces GDR consistently across models (Cao et al., 3 Aug 2025).

The ablation study compares full ReflecSched, with $l_{\max}$9 and $l=0$0, against an ablated single-level version with $l=0$1, evaluating the latter across $l=0$2 (Cao et al., 3 Aug 2025). The paper reports that full ReflecSched is consistently better than the single-level model; increasing rollouts in the single-level model does not close the gap; and the flat model can plateau or even degrade with more rollouts (Cao et al., 3 Aug 2025). The stated interpretation is that the gain is not merely from doing more search, but from the hierarchical structure that helps identify and summarize strategically meaningful contrasts.

7. Interpretation, positioning, and relation to scheduling methodology

ReflecSched is presented as a hybrid of heuristic rollout, tree-like lookahead, and natural-language policy distillation (Cao et al., 3 Aug 2025). In that sense, it departs from both classic dispatching-rule scheduling and end-to-end learned schedulers. Traditional dispatching rules appear in the framework as rollout primitives rather than final decision policies, while the LLM is repositioned from direct action selector to strategic analyst and guided executor.

The framework’s emphasis on non-myopic action selection situates it in a broader scheduling tradition that attempts to reconcile immediate dispatch efficiency with future congestion or blocking effects. The source explicitly argues that direct LLM scheduling fails because it tends toward local greedy choices, whereas ReflecSched injects future consequences through simulated makespan comparisons (Cao et al., 3 Aug 2025). This suggests a methodological affinity with rollout-based approximate dynamic programming, which the paper itself invokes in its Approximate Policy Iteration interpretation (Cao et al., 3 Aug 2025).

The framework also implies a specific view of expert knowledge in scheduling. Rather than replacing heuristics, it uses them as structured priors embedded in simulation. This is different from directly asking an LLM to “follow” a heuristic and also different from selecting a single best heuristic once and for all. The reported result that ReflecSched outperforms all individual heuristics overall while performing on par with the best heuristic tailored to each instance indicates that the framework is intended to aggregate heuristic strengths without being fixed to one dispatching rule (Cao et al., 3 Aug 2025).

A possible misconception is that ReflecSched is simply a prompt-engineering method for LLM scheduling. The source does not support that reduction. Its core mechanism is a hierarchical decision architecture with event-triggered reflection, multi-level heuristic-driven rollouts, best-worst trajectory comparison, and an experience-conditioned policy $l=0$3 (Cao et al., 3 Aug 2025). Another possible misconception is that improvement comes only from larger search budgets. The ablation results are presented specifically to reject that interpretation, emphasizing hierarchical reflection rather than raw rollout count (Cao et al., 3 Aug 2025).

In summary, ReflecSched defines a particular model of LLM use in DFJSP: the LLM does not replace scheduling structure, but operates atop it. It synthesizes contrastive evidence from heuristic simulations into Strategic Experience, and that summary guides the final action under dynamic events. Within the reported experiments, this design is associated with statistically significant gains over direct LLM scheduling, mitigation of the long-context paradox, reduced heuristic underutilization, and lower myopic greed, while remaining competitive with the best instance-specific heuristic across the tested problem cases (Cao et al., 3 Aug 2025).