OptalCP: Enhanced CP Solver for Scheduling

• OptalCP is a modern constraint programming solver specialized in scheduling, incorporating enhanced Failure-Directed Search and reinforcement-learning techniques.
• It employs advanced branching strategies, including multi-armed bandit algorithms and choice rollback, to minimize search tree size and strengthen propagation.
• Benchmark results on JSSP and RCPSP benchmarks demonstrate significant speed improvements, with up to 3.5× faster performance than state-of-the-art solvers.

OptalCP is both a new CP solver implementation focused on scheduling and the vehicle for a new, enhanced Failure-Directed Search (FDS) strategy. It is presented as a modern constraint programming solver for scheduling problems, supporting major scheduling constructs such as precedences, unary/no-overlap and cumulative resources, arithmetic expressions, optional interval variables, alternative constraints, and execution costs. Within that solver, the authors implemented the original FDS from Vilím et al. and then improved it using reinforcement-learning ideas, especially multi-armed bandit (MAB) algorithms; the resulting system is evaluated primarily on the Job Shop Scheduling Problem (JSSP) and the Resource-Constrained Project Scheduling Problem (RCPSP) (Heinz et al., 27 Aug 2025).

1. Definition, scope, and scheduling context

OptalCP is not presented as a brand-new CP paradigm. Its main contribution is a substantial redesign of the search guidance inside FDS, implemented in a solver whose intended role is complete CP search for hard scheduling problems. In that role, complete search functions as a “Plan B” that complements a primal heuristic “Plan A” such as Large Neighborhood Search (LNS): LNS quickly finds incumbents, while FDS tries to prove that no better solution exists. This framing is central to the system’s identity, because the paper targets not only feasible scheduling but also proving optimality, infeasibility, or stronger lower bounds.

The solver is described as scheduling-oriented rather than domain-general. The main empirical study focuses on JSSP and RCPSP, and the search engine is specialized around interval-variable branching and propagation behavior that is natural for scheduling models. A common misconception is therefore addressed directly by the paper’s positioning: OptalCP is best understood as a modern CP solver for scheduling with an enhanced FDS core, rather than as a general-purpose reinforcement-learning solver detached from CP search.

2. Failure-Directed Search as the core search engine

Failure-Directed Search differs from solution-oriented search heuristics because it tries to find infeasibility quickly and use those failures to shrink the explored search space. It builds a binary search tree from branching “choices,” where each choice splits the domain of a variable into two interval branches. For a choice $c$, FDS maintains ratings for the left and right branch, and the total choice rating is

$rating(c) := rating^l(c) + rating^r(c).$

Smaller ratings are better. The branch-local propagation outcome is encoded by

$$localRating := \begin{cases} 0 &\text{if infeasibility is detected}, \ 1 + R &\text{otherwise}, \end{cases}$$

where $R = 0.5^n$ and $n$ is the number of variable domains reduced by propagation. The rating update is

$$rating := (1-\alpha)\cdot rating + \alpha \cdot localRating.$$

The paper’s theoretical reinterpretation starts from a simplified FDS variant, sFDS, where the reduction factor is omitted: $$localRating := \begin{cases} 0 &\text{if branch is infeasible},\ 1 &\text{otherwise}, \end{cases} \qquad rating := (1-\alpha)\cdot rating + \alpha \cdot localRating.$$ Using

$\text{localReward} = - localRating,$

the choice-level reward becomes

$$\text{reward} := \begin{cases} 0 &\text{if both branches are infeasible},\ -1 &\text{if one branch is infeasible},\ -2 &\text{if both branches are feasible}. \end{cases}$$

Under this transformation, the rating update is equivalent to

$$Q(c) := Q(c) + \alpha \cdot \bigl(\text{reward} - Q(c)\bigr).$$

This is the paper’s key bridge from FDS to MAB. The stated interpretation is that every newly opened non-failing search node increases the penalty sum, and since internal nodes are proportional to total binary tree size, minimizing cumulative penalty is equivalent to minimizing search tree size (Heinz et al., 27 Aug 2025).

3. Reinforcement-learning formulation and branching policies

OptalCP studies four standard MAB algorithms as branching-choice selection strategies inside FDS: $rating(c) := rating^l(c) + rating^r(c).$0-greedy, Boltzmann exploration, UCB-1, and Thompson sampling. For Boltzmann exploration, the probability of choosing action $rating(c) := rating^l(c) + rating^r(c).$1 is

$rating(c) := rating^l(c) + rating^r(c).$2

with fixed $rating(c) := rating^l(c) + rating^r(c).$3. For UCB-1, the selected action is

$rating(c) := rating^l(c) + rating^r(c).$4

where $rating(c) := rating^l(c) + rating^r(c).$5 is the total number of actions taken and $rating(c) := rating^l(c) + rating^r(c).$6 is the number of times action $rating(c) := rating^l(c) + rating^r(c).$7 was selected. Thompson sampling is implemented with a Gaussian distribution for each action’s expected value, although the exact posterior update equations are not given.

A central empirical conclusion is that pure versions of Boltzmann, UCB-1, and Thompson sampling are inferior to hybrid strategies that combine the RL strategy only for exploratory moves with a default greedy exploitative move. These are denoted $rating(c) := rating^l(c) + rating^r(c).$8-greedy, $rating(c) := rating^l(c) + rating^r(c).$9-greedy, $$localRating := \begin{cases} 0 &\text{if infeasibility is detected}, \ 1 + R &\text{otherwise}, \end{cases}$$0-greedy, and $$localRating := \begin{cases} 0 &\text{if infeasibility is detected}, \ 1 + R &\text{otherwise}, \end{cases}$$1-greedy. The best strategy is reported as $$localRating := \begin{cases} 0 &\text{if infeasibility is detected}, \ 1 + R &\text{otherwise}, \end{cases}$$2-greedy with rollback, written $$localRating := \begin{cases} 0 &\text{if infeasibility is detected}, \ 1 + R &\text{otherwise}, \end{cases}$$3-greedy$$localRating := \begin{cases} 0 &\text{if infeasibility is detected}, \ 1 + R &\text{otherwise}, \end{cases}$$4. Relative to FDS without exploration, the best strategy gives 1.5× faster on JSSP and 2.2× faster on RCPSP.

The most distinctive refinement is choice rollback. An exploratory choice is evaluated in the current state, its rating is updated, and the solver may then rollback instead of permanently branching on it. If both branches fail, or even if one branch fails, the choice can still be useful because it does not enlarge the tree and may strengthen propagation. At a $$localRating := \begin{cases} 0 &\text{if infeasibility is detected}, \ 1 + R &\text{otherwise}, \end{cases}$$5 exploration rate, about $$localRating := \begin{cases} 0 &\text{if infeasibility is detected}, \ 1 + R &\text{otherwise}, \end{cases}$$6 of exploratory choices are rolled back, and only $$localRating := \begin{cases} 0 &\text{if infeasibility is detected}, \ 1 + R &\text{otherwise}, \end{cases}$$7 remain in the tree.

The paper also redesigns FDS parameterization. It advocates a hybrid learning-rate rule based on branch counters capped at $$localRating := \begin{cases} 0 &\text{if infeasibility is detected}, \ 1 + R &\text{otherwise}, \end{cases}$$8: $$localRating := \begin{cases} 0 &\text{if infeasibility is detected}, \ 1 + R &\text{otherwise}, \end{cases}$$9 With $R = 0.5^n$0, $R = 0.5^n$1 ranges from $R = 0.5^n$2 down to about $R = 0.5^n$3. The initial value is also made optimistic by reducing $R = 0.5^n$4 from $R = 0.5^n$5 to $R = 0.5^n$6, and the tuned exploration rate is

$R = 0.5^n$7

Among the most influential tuned parameters are LengthStepRatio (LSR), UniformChoiceStep (UCS), RatingAverageComparison (RAC), and RatingAverageLength (RAL); the final configuration omits $R = 0.5^n$8, uses uniform intervals, and sets $R = 0.5^n$9 (Heinz et al., 27 Aug 2025).

4. Solver architecture and workflow

The enhanced FDS is embedded in a complete CP workflow. A high-level execution cycle is: generate initial choices from interval-variable domains; select the next choice according to a learned ranking strategy; skip choices already decided by current propagation; rotate branches so the lower-rated one is explored first; propagate after adding the branch constraint; update branch and choice ratings from the observed local outcome; backtrack or restart when necessary; and, if all current choices are exhausted but the problem is not decided, search heuristically for a feasible assignment or generate additional finer-grained choices.

The paper gives the main implementation hooks as GenerateInitialChoices, SelectChoice, IsDecided, RotateBranches, Propagate, UpdateBranchRating, FindFeasibleAssignment, GenerateAdditionalChoices, and GenerateNoGoodConstraints. Restarts use a geometric strategy: if FailCount = FailLimit, the solver restarts, retains ratings, records nogoods, returns all path choices to the pool, clears the branch stack, and updates the fail limit by

$n$0

Choice generation is explicit. The step size is

$n$1

or

$n$2

Pivot points are

$n$3

and the generated choices are

$n$4

This branching scheme is one of the solver’s most problem-dependent components: bad LSR values can degrade performance by an order of magnitude, and the best values differ between JSSP and RCPSP.

5. Benchmark results and scheduling performance

The paper evaluates OptalCP on the two most fundamental scheduling problems, JSSP and RCPSP. It states that the resulting enhanced FDS, using the best extended MAB algorithm and configuration, performs 1.7 times faster on the JSSP and 2.1 times faster on the RCPSP benchmarks compared to the original implementation in a new solver called OptalCP, while also being 3.5 times faster on the JSSP and 2.1 times faster on the RCPSP benchmarks than the current state-of-the-art FDS algorithm in IBM CP Optimizer 22.1.

A second line of evidence concerns lower bounds. Using only a 900-second time limit per instance, the enhanced FDS improved the existing state-of-the-art lower bounds of 78 of 84 JSSP and 226 of 393 RCPSP standard open benchmark instances while also completely closing a few of them. In the summary table reproduced in the detailed account, JSSP contributes 74 improved instances and 4 newly closed ones, while RCPSP contributes 223 improved instances and 3 newly closed ones.

The experiments were run with OptalCP 0.8.5 on an Intel Xeon 4110 @ 2.1 GHz, single core. The main evaluation uses 161 JSSP instances and 200 RCPSP instances, split into development and evaluation subsets, and reported times and explored-branch totals are 7-run averages over different random seeds. The paper emphasizes that the benchmark regime is tailored to the realistic use case of FDS: many instances are converted into infeasibility-under-bound instances, so the objective is to certify that no better schedule exists rather than simply to find any feasible schedule (Heinz et al., 27 Aug 2025).

6. Interpretation, limitations, and significance

OptalCP’s significance lies in making complete CP search for scheduling explicitly into a search-tree-size minimization problem guided by learned branching decisions. The paper argues that this reframing explains why FDS can benefit from MAB-style exploration and exploitation control, and why branch ratings should be treated as non-stationary value estimates rather than fixed heuristic scores. At the same time, the paper is careful about the limits of the analogy: the available set of choices changes with the search state, and the reward distribution is non-stationary because the quality of a branch depends on previous branching decisions and current propagation context. No formal regret bound is claimed.

The method is also not presented as universally tuned. The strongest caveat is that parameter settings are problem-dependent, especially LengthStepRatio. The empirical study is restricted to JSSP and RCPSP, and the paper does not experimentally verify transfer to broader CP domains. It further notes that the evaluation emphasizes the “prove infeasible under incumbent bound” regime, which is the intended role of FDS inside OptalCP but not the only possible CP-search scenario.

Within those limits, OptalCP is best viewed as a scheduling-oriented CP solver whose FDS component is redesigned using multi-armed bandit reinforcement-learning ideas. Its distinctive elements are the reinterpretation of FDS ratings as value estimates, the use of hybrid MAB branching strategies, the introduction of choice rollback, and the integration of these mechanisms into a complete-search workflow aimed at proving optimality and improving lower bounds rather than merely finding feasible schedules (Heinz et al., 27 Aug 2025).