DyRo-MCTS: Robust Dynamic MCTS for Job Shop Scheduling

• The paper introduces DyRo-MCTS by integrating a normalized robustness measure derived from machine idleness into the decision-making of standard MCTS.
• It combines offline policy guidance with online search to improve scheduling decisions in dynamic job shop environments with unpredictable job arrivals.
• Empirical results show that DyRo-MCTS consistently outperforms vanilla MCTS, especially under Poisson job arrivals and high machine utilization.

Searching arXiv for the specified paper to verify metadata and support the article. {"query":"arXiv (Chen et al., 26 Sep 2025) DyRo-MCTS: A Robust Monte Carlo Tree Search Approach to Dynamic Job Shop Scheduling","max_results":5} I found the paper metadata for "DyRo-MCTS: A Robust Monte Carlo Tree Search Approach to Dynamic Job Shop Scheduling" on arXiv with id (Chen et al., 26 Sep 2025). Dynamic Robust MCTS (DyRo-MCTS) is a Monte Carlo Tree Search method for dynamic job shop scheduling that augments policy-guided online planning with an explicit estimate of action robustness under future disturbances. It is introduced for settings in which new jobs arrive during execution, so decisions made from incomplete information can be invalidated by subsequent events. The method retains the standard MCTS workflow of selection, expansion, simulation, and backpropagation, but modifies the exploitation term so that action choice depends jointly on expected schedule quality and a robustness signal derived from machine-idleness patterns. In the reported study, DyRo-MCTS is positioned as an online improvement layer over offline-learned scheduling policies and is evaluated on dynamic job shop scheduling instances with Poisson job arrivals, mean tardiness objectives, and limited planning budgets (Chen et al., 26 Sep 2025).

1. Problem setting and objective

DyRo-MCTS is formulated for Dynamic Job Shop Scheduling (DJSS) cast as a Markov decision process under event-driven simulation. A state $s$ corresponds to the moment when a machine becomes idle and a buffer of $n$ candidate jobs is available for dispatch. The state is summarized by features such as machine loads, remaining operations, and due dates; these features may be manually crafted or produced by graph-based encodings. An action $a \in A(s)$ selects one of the $n$ candidate jobs for processing on the idle machine, so the action-space cardinality varies with the buffer size (Chen et al., 26 Sep 2025).

The transition model used for online planning is deliberately simplified. After choosing an action, the selected job’s operation executes and time advances; in the underlying environment, new jobs may arrive according to a Poisson process, but in the lookahead tree stochastic future arrivals are ignored and planning is performed only over currently existing jobs. This yields deterministic transitions inside the search tree despite the dynamic environment outside the tree.

The reward is defined through tardiness. For a terminal or rollout schedule,

$$r = -\mathcal{T}, \qquad \mathcal{T} = \frac{1}{|\mathcal{J}|}\sum_{i\in\mathcal{J}} w_i \max\{c_i-d_i,0\},$$

where $w_i$ is the job weight, $c_i$ is the completion time, $d_i$ is the due date, and $\mathcal{J}$ is the set of all jobs in the horizon. The optimization objective is therefore to minimize mean weighted tardiness. Performance under disturbances is measured by the average tardiness over a long horizon, exemplified by evaluation on the next 5,000 jobs after warm-up.

2. Policy-guided MCTS foundation

DyRo-MCTS is built on a standard policy-guided MCTS framework often referred to as PUCT. The baseline procedure follows the conventional four-phase structure: selection from the root to a leaf according to a tree policy; expansion of a non-terminal, not fully expanded leaf; simulation or rollout to a terminal state to obtain a reward, optionally replacing rollout with a learned value function; and backpropagation of the reward to update visit counts and action values (Chen et al., 26 Sep 2025).

In the baseline formulation, the mean action value is

$Q(s,a)=\frac{w(s,a)}{n(s,a)},$

with $n$0 the visit count and $n$1 the accumulated return. Classic PUCT selection chooses

$n$2

where $n$3 is a prior probability supplied by an offline policy, $n$4, and $n$5 is the exploration constant.

This baseline is important because DyRo-MCTS does not replace policy-guided MCTS; it generalizes it. The same offline policy can provide both the prior probabilities used in the PUCT exploration term and the default rollout policy used during simulation. The reported implementation allows random policies, manual heuristics, deep reinforcement learning policies, and genetic programming policies to play this guiding role.

3. Robustness as a scheduling signal

The defining addition in DyRo-MCTS is an action-level robustness estimate $n$6 intended to capture how well a decision will tolerate unforeseen job arrivals. The robustness of a complete schedule is defined through machine idleness over time, with earlier idleness penalized more heavily than later idleness: $n$7 where $n$8 is the makespan, $n$9 if machine $a \in A(s)$0 is idle at time $a \in A(s)$1 and $a \in A(s)$2 otherwise, and $a \in A(s)$3 is a tunable parameter controlling the severity of the early-idleness penalty (Chen et al., 26 Sep 2025).

For each action $a \in A(s)$4 at state $a \in A(s)$5, DyRo-MCTS performs $a \in A(s)$6 Monte Carlo rollouts, still ignoring future arrivals, and collects robustness values $a \in A(s)$7. Let $a \in A(s)$8 and $a \in A(s)$9 denote the maximum and minimum robustness values observed in the tree so far. The normalized robustness estimate is then

$n$0

so that $n$1.

The paper’s interpretation is that a schedule with less harmful early idleness is more adaptable to later job arrivals. This suggests that robustness is not modeled as uncertainty over arrival realizations directly; instead, it is approximated by a structural property of the current schedule that is expected to remain valuable when disturbances occur.

4. DyRo-UCT and search-time updates

The principal algorithmic change appears in the selection rule. DyRo-MCTS defines a combined exploitation score

$n$2

and then selects actions according to

$n$3

When $n$4, the method reduces to standard PUCT; when $n$5, robustness contributes directly to exploitation (Chen et al., 26 Sep 2025).

Expansion remains unchanged relative to standard MCTS: nodes are added when first visited, and robustness does not alter the rule for creating children. The distinction emerges after expansion, because each new edge accumulates both value and robustness statistics.

During simulation, rollouts use the offline policy or a random policy to complete a schedule. For each rollout, the method records tardiness $n$6 and computes a normalized value

$n$7

so that $n$8, and also computes the machine-idleness-based robustness value $n$9. In backpropagation, every visited edge $$r = -\mathcal{T}, \qquad \mathcal{T} = \frac{1}{|\mathcal{J}|}\sum_{i\in\mathcal{J}} w_i \max\{c_i-d_i,0\},$$0 along the selected path is updated by incrementing $$r = -\mathcal{T}, \qquad \mathcal{T} = \frac{1}{|\mathcal{J}|}\sum_{i\in\mathcal{J}} w_i \max\{c_i-d_i,0\},$$1, accumulating $$r = -\mathcal{T}, \qquad \mathcal{T} = \frac{1}{|\mathcal{J}|}\sum_{i\in\mathcal{J}} w_i \max\{c_i-d_i,0\},$$2 and $$r = -\mathcal{T}, \qquad \mathcal{T} = \frac{1}{|\mathcal{J}|}\sum_{i\in\mathcal{J}} w_i \max\{c_i-d_i,0\},$$3, and recomputing

$$r = -\mathcal{T}, \qquad \mathcal{T} = \frac{1}{|\mathcal{J}|}\sum_{i\in\mathcal{J}} w_i \max\{c_i-d_i,0\},$$4

The final action at the root is chosen by visit count. At a high level, the method preserves the operational simplicity of MCTS while changing the semantics of exploitation from pure reward seeking to a convex combination of reward and robustness.

5. Integration with offline policies

DyRo-MCTS is explicitly designed to operate on top of offline-learned dispatching policies. These policies serve two functions: they provide the prior probabilities $$r = -\mathcal{T}, \qquad \mathcal{T} = \frac{1}{|\mathcal{J}|}\sum_{i\in\mathcal{J}} w_i \max\{c_i-d_i,0\},$$5 used in the search tree’s exploration term, and they act as default rollout policies during simulation (Chen et al., 26 Sep 2025).

The framework is policy-agnostic in the sense that any policy outputting a distribution over candidate actions can be inserted. The study reports experiments with four categories of guiding policy: random, manually designed rules, deep reinforcement learning, and genetic programming. Among these, genetic-programming-based priors yielded the best guidance in practice.

This integration strategy is central to the method’s intended use. DyRo-MCTS does not assume that an offline policy is optimal; rather, it treats that policy as an informative prior whose imperfections can be corrected by search at online decision time. The reported results position robustness-aware planning as a way to improve the long-run behavior of such policies without replacing them.

6. Experimental configuration, empirical findings, and interpretation

The benchmark configuration uses 10 machines. Each arriving job has $$r = -\mathcal{T}, \qquad \mathcal{T} = \frac{1}{|\mathcal{J}|}\sum_{i\in\mathcal{J}} w_i \max\{c_i-d_i,0\},$$6 operations, and each processing time is drawn as $$r = -\mathcal{T}, \qquad \mathcal{T} = \frac{1}{|\mathcal{J}|}\sum_{i\in\mathcal{J}} w_i \max\{c_i-d_i,0\},$$7. The arrival rate $$r = -\mathcal{T}, \qquad \mathcal{T} = \frac{1}{|\mathcal{J}|}\sum_{i\in\mathcal{J}} w_i \max\{c_i-d_i,0\},$$8 is chosen so that utilization $$r = -\mathcal{T}, \qquad \mathcal{T} = \frac{1}{|\mathcal{J}|}\sum_{i\in\mathcal{J}} w_i \max\{c_i-d_i,0\},$$9, with $w_i$0. Evaluation uses a warm-up of 1,000 jobs and then measures performance over the next 5,000 jobs. Two objectives are considered: $w_i$1, where $w_i$2, and $w_i$3, where $w_i$4 with proportions $w_i$5. The MCTS budget is $w_i$6 iterations, with additional tests up to 1,000 iterations. Key tuned parameters are $w_i$7, $w_i$8, and $w_i$9 (Chen et al., 26 Sep 2025).

Aspect	Reported configuration or finding
Shop setting	10 machines
Job generation	$c_i$0, $c_i$1
Utilization	$c_i$2
Evaluation horizon	warm-up 1,000 jobs, then next 5,000 jobs
Objectives	$c_i$3, $c_i$4
Search budget	$c_i$5, also up to 1,000
Tuned parameters	$c_i$6, $c_i$7, $c_i$8
Online time	$c_i$9 at 100 iterations; $d_i$0 at 1,000

The main empirical findings are reported in relative terms. Vanilla MCTS with $d_i$1 already improves the offline policy by 6–50%. DyRo-MCTS adds a further 5–15% improvement over vanilla MCTS. The best absolute performance is obtained when the search is guided by high-quality genetic programming policies. The online time per decision is approximately $d_i$2 for 100 iterations and scales linearly to approximately $d_i$3 for 1,000 iterations.

Under continuous disturbances over 5,000 arrivals, vanilla MCTS exhibits rapid early gains but then plateaus, with later backlog reducing robustness. DyRo-MCTS improves more slowly initially, but maintains sustained gains and overtakes vanilla MCTS after approximately 600 disruptions. The authors attribute this behavior to the method’s capacity to make robust scheduling decisions that produce long-term, sustainable performance improvements under disturbances (Chen et al., 26 Sep 2025).

The paper’s discussion links these results to the early-idleness penalty. By penalizing early machine idleness, DyRo-UCT steers the schedule toward “front-loaded” resource usage that leaves less idle capacity to be wasted. A plausible implication is that the method favors schedule structures that preserve adaptability even when future arrivals are not explicitly simulated in the tree.

7. Limitations and prospective extensions

The reported limitations are specific and methodological. The robustness measure considers only the distribution of machine idleness over time; it does not account for due-date slack or machine–job compatibility at later stages. Likewise, future arrival characteristics such as operation counts and processing times are not modeled explicitly in the planning process (Chen et al., 26 Sep 2025).

Several extensions are proposed. One direction is to learn a richer robustness metric, for example one based on predicted arrival profiles or job-level slack buffers. Another is to integrate a learned transition model capable of sampling plausible future job arrivals during simulation. A third is to co-train offline policies and robustness estimators end-to-end.

These extensions indicate the present scope of DyRo-MCTS. The method is not a full stochastic planner over future arrivals; it is a lightweight robustness-aware variant of policy-guided MCTS whose robustness estimate is intentionally simple. Within that scope, the paper concludes that augmenting MCTS with machine-idleness-based robustness yields significant and sustained gains in highly dynamic job shop environments while adding negligible online planning cost (Chen et al., 26 Sep 2025).