- The paper presents a neuro-symbolic pipeline that converts natural-language constraints and preferences into weighted partial MaxSAT code, uses the exact RC2 solver for optimization, and independently verifies feasibility and optimality.
- The paper shows that MaxSAT with intermediate planning substantially outperforms direct answering, chain-of-thought, and program-of-thought methods, reaching up to 87% acceptance on set cover versus 0% for direct methods on several MIS and scheduling settings.
- The paper demonstrates that problem structure, preference complexity, and model capability strongly affect reliability, while noting that solver guarantees apply only to the LLM-generated encoding and do not ensure faithful interpretation of user intent.
This paper presents a hybrid neuro-symbolic pipeline in which LLMs translate natural-language optimisation problems—comprising hard constraints and user-defined preferences—into weighted partial MaxSAT encodings via Python code generation, delegating the actual optimisation to the exact RC2 solver through PySAT (2605.29687). The central claim is that LLMs should not be trusted to perform combinatorial optimisation internally, but are effective at translating user intent into executable solver code, provided that solutions are independently verified for feasibility and optimality against a canonical reference encoding.
Motivation and problem setting
The motivating example is a single-machine scheduling task with six jobs, precedence constraints, and weighted deadline preferences. When prompted directly or with chain-of-thought (CoT), GPT returns an infeasible schedule violating a precedence constraint, despite producing plausible reasoning traces. Encoding the same instance as a weighted partial MaxSAT formula and solving it yields a verifiably optimal schedule. The authors use this to argue that failures stem from the nature of optimisation—the interaction of precedence, resource, and weighted-preference constraints induces a search space requiring exact reasoning—rather than from misunderstanding of the natural-language specification.
Formally, the setting is weighted partial MaxSAT: hard clauses must be satisfied while the total weight of falsified soft clauses is minimised, with soft clauses encoding preference priorities. A key methodological point is that verification is decoupled from encoding: since different encodings of the same problem may yield different optimal models, candidate solutions are checked against a canonical MaxSAT encoding by testing (i) feasibility under all hard constraints and (ii) equality of objective value with the canonical optimum, thereby accepting alternative encodings and multiple optima.
Pipeline
The pipeline comprises four stages. In intermediate planning, the LLM produces a structured symbolic description of variables, hard constraints, preferences, and objective before writing code—a step inspired by prior work showing planning improves encoding quality. In code generation, the LLM emits standalone Python using the PySAT API, guided by a fixed RC2 usage template to reduce API misuse. In MaxSAT solving, RC2 computes a feasible, optimal assignment with respect to the generated encoding; at this point correctness no longer depends on the LLM. Finally, verification checks the parsed solution against the canonical encoding used solely for evaluation. On execution errors or format violations, structured feedback triggers refinement for up to five iterations; crucially, canonical feasibility/optimality results are never fed back to the model, so refinement does not leak the reference semantics.
An important caveat stated plainly by the authors: the canonical encoding exists only for evaluation. In deployment, the solver guarantees optimality only relative to whatever the LLM encoded, so the approach makes solutions solver-verifiable but does not guarantee that the encoding captures the user's intended semantics.
Experimental design
The benchmark contains 300 instances: three families (Maximum Independent Set, Scheduling, Set Cover), 25 instances each, four preference variants per instance (none, p1, p2, p3). Four LLMs are evaluated—Gemini 3.1 Pro, GPT-5.5, Llama 3.3 (70B), and Qwen3 (30B)—at temperature zero, against direct-answer, CoT, and program-of-thought (PoT) baselines, plus two MaxSAT configurations (with and without intermediate planning). Acceptance requires both feasibility and optimality under the canonical encoding.
Results
The headline finding is stark: direct-answer prompting achieves 0% acceptance on MIS and scheduling for all four models, and CoT reaches at most 4% (MIS) and 1% (scheduling). PoT improves marginally but remains far below the MaxSAT pipeline. Aggregated acceptance rates:
| Family |
Model |
Direct |
CoT |
PoT |
MaxSAT |
MaxSAT+plan |
| MIS |
Gemini |
0 |
4 |
11 |
36 |
56 |
| MIS |
GPT |
0 |
2 |
13 |
32 |
51 |
| Scheduling |
Gemini |
0 |
1 |
14 |
44 |
59 |
| Scheduling |
GPT |
0 |
0 |
9 |
41 |
56 |
| Set cover |
Gemini |
34 |
36 |
46 |
79 |
87 |
| Set cover |
GPT |
29 |
33 |
45 |
76 |
82 |
Several patterns deserve emphasis. First, problem structure dominates difficulty: set cover is consistently easiest (baselines even achieve non-zero rates), whereas MIS and scheduling nearly defeat all prompting-only methods, indicating that coverage-style constraints are easier to encode than exclusivity and precedence interactions. Second, planning helps strong models but can hurt weak ones: it lifts Gemini and GPT uniformly across families, but for Llama and Qwen it sometimes degrades performance severely—for example, Qwen's scheduling acceptance drops from 22% (no plan) to 4% (with plan)—suggesting error propagation when a model cannot faithfully translate its own plan into executable code. Third, preference complexity matters non-monotonically: Qwen reaches 68% on MIS under p1 but 0% under p2 and p3, showing that specific preference structures can be qualitatively harder rather than difficulty scaling smoothly. Fourth, cross-model plan transfer is brittle: Gemini improves on some MIS preference variants using GPT- or Llama-generated plans (e.g., 84% on p1 with Llama plans versus 44% self-generated), but transferred plans frequently produce near-zero acceptance for weaker models, and transfer works best for modular problems like set cover.
Relation to prior work
The paper positions itself against SATLM, Logic-LM, ConstraintLLM, and SMT-based planning frameworks (LLMFP and related work). Its distinguishing claims are threefold: it targets preference-aware optimisation rather than feasibility checking or single-objective planning; it verifies optimality independently via weighted soft constraints rather than trusting the LLM's reported objective; and it uses SAT/MaxSAT as the natural formalism for predominantly Boolean structure. The authors state this is the first work combining LLM-based intent extraction with MaxSAT solvers for rigorous preference-aware optimisation, though this novelty claim rests on the scope of the surveyed literature.
Limitations and open questions
The paper concedes several limitations. The evaluation covers only three Boolean optimisation families; problems with continuous variables, numeric constraints, uncertainty, or multi-agent structure may require formalisms beyond MaxSAT. Verification depends on canonical encodings that would not exist in deployment, leaving open whether LLM-generated encodings faithfully capture user intent—an assumption the pipeline itself cannot check. Performance remains strongly model-dependent: weaker open-source models benefit far less, so solver-backed reasoning reduces but does not eliminate the need for accurate semantic parsing. Finally, the benchmark uses controlled, structured descriptions; handling ambiguous or inconsistent real-user specifications would require interactive clarification mechanisms not evaluated here. A concrete question left open is how to validate encoding fidelity without a hidden reference specification, perhaps via interactive user-in-the-loop refinement, which the authors identify as future work.
Conclusion
The paper demonstrates empirically that externalising constrained optimisation from LLMs to exact MaxSAT solvers—via LLM-generated PySAT code with independent feasibility and optimality verification—yields substantially higher acceptance rates than internal reasoning, CoT, or PoT prompting, with peak rates exceeding 80% on set cover. The most reliable configuration is characterised not by more reasoning text but by executable, solver-verifiable encodings. The residual risks—encoding fidelity, model dependence, and benchmark scope—are clearly delineated and frame the boundary conditions within which the approach's guarantees hold.