SATQuest: Verifier for LLM Logical Reasoning

• SATQuest is a systematic verifier for logical reasoning in LLMs, generating diverse CNF instances to create controlled, satisfiability-based challenges.
• It combines randomized SAT-based problem generation with orthogonal controls over instance scale, problem type, and question format to isolate key performance factors.
• The framework employs objective verification via PySAT and standardized solver metrics to guide fine-tuning and expose format-specific vulnerabilities in LLM reasoning.

Searching arXiv for the SATQuest paper and closely related sources. {"^^^^^^^^3query3 OR abs:\3^^^^^^^^"SATQuest\"","max_results":5,"sort_by":"submittedDate","sort_order":"descending"} arXivSearch(^^^^^^^^3query3 verifier logical reasoning evaluation reinforcement fine-tuning of LLMs", max_results=^^^^3ti:\3query3^^^^, sort_by="submittedDate", sort_order="descending") SATQuest is a systematic verifier for evaluating and enhancing logical reasoning in LLMs by generating diverse, satisfiability-based reasoning problems directly from Conjunctive Normal Form (CNF) instances. Introduced as a verifier-driven framework in ^^^^3 OR abs:\3query3 OR abs:\3^^^^5, it combines randomized SAT-based problem generation, objective answer verification via PySAT, and three orthogonal control axes—instance scale, problem type, and question format—to support fine-grained analysis and reinforcement fine-tuning. Its design is intended to mitigate memorization issues, separate scale effects from semantic and presentation effects, and expose failure modes that are often obscured in heterogeneous reasoning benchmarks (&&&^^^^3query3^^^^&&&).

^^^^3ti:\3^^^^. Motivation and scope

SATQuest addresses a specific methodological gap in reasoning evaluation. Current reasoning benchmarks are described as lacking scientific variable control and multi-dimensional analysis: some provide broad content, such as MMLU and GPQA, but are weak on structure, difficulty calibration, and objective verification, while others control only a single dimension and provide a narrow set of problem types and formats. SATQuest is built around the claim that logical reasoning requires systematic manipulation of instance scale, task semantics, and presentation format; without orthogonal control, it is not possible to diagnose whether a model fails because of scale, type, or format, nor to train it with verifiable rewards (&&&^^^^3query3^^^^&&&).

The framework therefore emphasizes controlled instance synthesis rather than corpus aggregation. CNF generation is parameterized by the number of variables PRESERVED_PLACEHOLDER_^^^^3query3^^^^, the number of clauses PRESERVED_PLACEHOLDER_^^^^3ti:\3^^^^, clause width distribution, literal polarity, and solver-derived hardness. SATQuest also separates satisfiable and unsatisfiable cases and constructs matched SAT/UNSAT pairs to eliminate superficial cues. This suggests a research orientation closer to experimental design than to static benchmark compilation: the underlying proposition is that logical competence should be probed under controlled perturbations of representation and optimization structure, not only via aggregate benchmark scores.

A recurring misconception in reasoning evaluation is that strong performance on familiar mathematical notation implies broad logical competence. SATQuest explicitly tests that assumption by rendering the same CNF instance in multiple encodings, including mathematical notation, machine-readable DIMACS, and two narrative variants. Its reported results show that accuracy often drops sharply outside mathematical notation, indicating that format robustness is a substantive variable rather than a superficial presentation issue (&&&^^^^3query3^^^^&&&).

^^^^3 OR abs:\3^^^^. Formal foundations and verifier semantics

SATQuest is grounded in standard propositional satisfiability. Given variables PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3^^^^, literals are elements of $\{x_k,\neg x_k\}$, and a CNF formula $F$ is written as

$F = \bigwedge_{j=1}^m C_j,$

where each clause $C_j$ is a disjunction of literals. The SAT decision problem asks whether there exists an assignment $\alpha \in \{0,1\}^n$ such that $F(\alpha)=\mathrm{True}$. SATQuest distinguishes satisfiable formulas from unsatisfiable ones and builds task families around that distinction (&&&^^^^3query3^^^^&&&).

Several classical SAT concepts are part of the framework’s formal backbone. Unit propagation sets the remaining literal in a unit clause and can iteratively produce implications or detect conflict. Resolution derives $(A \lor B)$ from PRESERVED_PLACEHOLDER_^^^^3ti:\3query3^^^^ and PRESERVED_PLACEHOLDER_^^^^3ti:\3ti:\3^^^^, and is treated as sound and complete for propositional entailment. For unsatisfiable instances, SATQuest also uses minimality notions central to diagnostic reasoning: an UNSAT core is a subset of clauses that is itself unsatisfiable; a minimal unsatisfiable subset (MUS) is an UNSAT core with no unsatisfiable proper subset; and a minimal correction subset (MCS) is a minimal set of clauses whose removal makes the remaining formula satisfiable.

MaxSAT is included as an optimization task, defined by

PRESERVED_PLACEHOLDER_^^^^3ti:\3 OR abs:\3^^^^

This extends SATQuest beyond pure satisfiability into clause-satisfaction optimization, while MUS and MCS require minimality reasoning over clause subsets. The resulting task set spans classification, constructive search, combinatorial optimization, and minimality certification. That breadth is important because the framework reports that models competent on search may still falter on minimality constraints, so the notion of “logical reasoning” is deliberately decomposed rather than treated as a single monolithic ability (&&&^^^^3query3^^^^&&&).

The verifier is central. SATQuest uses binary correctness for core evaluation,

PRESERVED_PLACEHOLDER_^^^^3ti:\33^^^^

and a reward function for fine-tuning,

PRESERVED_PLACEHOLDER_^^^^3ti:\3^^^^4

where PRESERVED_PLACEHOLDER_^^^^3ti:\3^^^^5 counts >, </think>, <answer>, and </answer> tags, and PRESERVED_PLACEHOLDER_^^^^3ti:\3^^^^6 measures the matched span ratio of the pattern <think>…</think><answer>…</answer>. This architecture ties evaluation directly to machine-checkable correctness rather than to heuristic judgment (&&&^^^^3query3^^^^&&&).

3. Controlled generation and orthogonal design

The problem-generation pipeline starts from randomized CNF synthesis. SATQuest parameterizes formulas by PRESERVED_PLACEHOLDER_^^^^3ti:\3^^^^7 and PRESERVED_PLACEHOLDER_^^^^3ti:\3^^^^8, with clause width PRESERVED_PLACEHOLDER_^^^^3ti:\3^^^^9 drawn from a mixed distribution: PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3query3^^^^ bounded by PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3ti:\3^^^^, and literal polarity assigned randomly. In the GenCNFPair procedure, clauses are repeatedly sampled until a solver reports UNSAT; a satisfiable twin is then created by flipping polarities until a solver reports SAT. The SAT and UNSAT members of the pair share structure such as PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3 OR abs:\3^^^^, PRESERVED_PLACEHOLDER_^^^^3 OR abs:\33^^^^, and literal counts, so label balance is not confounded with superficial artifacts (&&&^^^^3query3^^^^&&&).

SATQuest’s distinctive feature is its enforcement of orthogonality across three axes. The first axis is instance scale and difficulty, represented structurally by PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3^^^^4, PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3^^^^5, and total literals, and operationally by hardness proxies from solver runs: decisions, conflicts, and propagations. The second axis is problem type. The third is question format. Each CNF instance is rendered independently into all four formats, and each instance participates in each applicable task, so variation along one axis is not confounded by changes in the others.

The five task types and their output semantics are as follows:

Task	Required output	Verification
SAT decision (SATDP)	^^^^3ti:\3^^^^ bit	Correct classification of both SAT and UNSAT pair members
SAT search	PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3^^^^6-bit assignment	Check PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3^^^^7
MaxSAT	PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3^^^^8-bit assignment	Compare satisfied-clause count to RC^^^^3 OR abs:\3^^^^ optimum
MCS	PRESERVED_PLACEHOLDER_^^^^3 OR abs:\3^^^^9-bit mask	Remaining CNF must be SAT and removal set must be minimal
MUS	$\{x_k,\neg x_k\}$^^^^3query3^^^^-bit mask	Selected subset must be UNSAT and inclusion set must be minimal

The four question formats are mathematical notation, machine/DIMACS, a narrative OR semantics variant, and a narrative AND semantics variant. Mathematical notation renders formulas directly with $\{x_k,\neg x_k\}$^^^^3ti:\3^^^^, $\{x_k,\neg x_k\}$^^^^3 OR abs:\3^^^^, and $\{x_k,\neg x_k\}$3. DIMACS uses a p cnf n m header followed by integer-coded clause lines ending in ^^^^3query3^^^^. The narrative OR format expresses disjunctions in friendly text. The narrative AND format presents equivalent constraints through negated semantics and requires tracking De Morgan mappings. SATQuest emphasizes that the underlying variables and polarities remain identical across these encodings; only the surface form changes (&&&^^^^3query3^^^^&&&).

Objective verification is performed through PySAT wrappers to Glucose 4.^^^^3ti:\3^^^^ for SAT and model checking, RC^^^^3 OR abs:\3^^^^ for MaxSAT, LBX for MCS extraction, and MUSX for MUS extraction. Answers are parsed from the final line or <answer> span using regular expressions, with expected output lengths of ^^^^3ti:\3^^^^ bit, $\{x_k,\neg x_k\}$4 bits, or $\{x_k,\neg x_k\}$5 bits depending on the task. Deterministic seeds and fixed solver configurations are used to ensure reproducible verification and hardness metrics (&&&^^^^3query3^^^^&&&).

4. Datasets, evaluation protocol, and worked semantics

SATQuest includes separate datasets for evaluation and reinforcement fine-tuning. The evaluation dataset contains ^^^^3ti:\3max_results3query3^^^^ satisfiable/unsatisfiable CNF pairs with $\{x_k,\neg x_k\}$6 and $\{x_k,\neg x_k\}$7, using matched structural pairs to eliminate superficial cues. Because each pair is instantiated across five task types and four formats, each pair yields ^^^^3 OR abs:\3query3^^^^ evaluations. Accuracy is the principal reported metric, defined as binary correctness (&&&^^^^3query3^^^^&&&).

The training dataset, SATQuest-RFT-3k, contains 3,^^^^3query3query3query3^^^^ CNF pairs with $\{x_k,\neg x_k\}$8, clause-to-variable ratios from $\{x_k,\neg x_k\}$9 to $F$^^^^3query3^^^^, and ^^^^3 OR abs:\3^^^^5 pairs per $F$^^^^3ti:\3^^^^ configuration. The smaller scale is intended for on-policy reinforcement fine-tuning, while the evaluation set reaches larger $F$^^^^3 OR abs:\3^^^^ to test scale generalization. This suggests a clean separation between training-time tractability and evaluation-time extrapolation.

The framework includes worked examples to make output semantics explicit. For

$F$3

SATQuest identifies the instance as satisfiable, gives $F$4 as one valid satisfying assignment in the variable order $F$5, and states that the MaxSAT optimum is 3, also attained by $F$6. The same CNF is then rendered in mathematical notation, in DIMACS, in a narrative OR form, and in a narrative AND form that requires mapping back from the negation of clauses to the original disjunctions (&&&^^^^3query3^^^^&&&).

For unsatisfiable reasoning, the framework uses

$F$7

The MUS is $F$8, since the first two clauses alone are UNSAT and any proper subset is SAT. The MCS can be either $F$9 or $F = \bigwedge_{j=1}^m C_j,$^^^^3query3^^^^, since removing either contradictory unit clause makes the remaining formula satisfiable. In bit-mask form with $F = \bigwedge_{j=1}^m C_j,$^^^^3ti:\3^^^^, MUS is ^^^^3ti:\3ti:\3query3^^^^, while MCS can be ^^^^3ti:\3query3query3^^^^ or ^^^^3query3ti:\3query3^^^^. These examples clarify that SATQuest does not treat verification as a single-output problem: multiple answers may be valid for search and MCS, while MUS and MaxSAT additionally require optimality or minimality checks (&&&^^^^3query3^^^^&&&).

5. Reinforcement fine-tuning pipeline

SATQuest is designed not only for evaluation but also for verifier-driven reinforcement fine-tuning. The reported training setup uses on-policy GRPO (Group Relative Policy Optimization) via TRL, with prompts instructing chain-of-thought in <think>… and final outputs in <answer>…</answer>. The baseline model is Qwen^^^^3 OR abs:\3^^^^.5-7B-Instruct, described as a small vanilla model, and the GRPO objective is noted as being used per DeepSeekMath (&&&^^^^3query3^^^^&&&).

Training uses the SATQuest-RFT-3k data and targets specific task-format pairs. Three models are trained on designated targets such as math-format decision, math-format search, and narrative-format decision, and then evaluated for generalization across larger $F = \bigwedge_{j=1}^m C_j,$^^^^3 OR abs:\3^^^^ and other formats or task types. The primary reward is verifier correctness, while format rewards softly encourage structural compliance. Invalid formatting or failure to produce binary strings yields $F = \bigwedge_{j=1}^m C_j,$3; partial credit is not used for core tasks in the reported experiments, although the framework notes that MaxSAT could be extended to fractional rewards through satisfied-clause fractions (&&&^^^^3query3^^^^&&&).

The hyperparameters are specified precisely: num_generations=^^^^3ti:\3^^^^6, batch size $F = \bigwedge_{j=1}^m C_j,$4, temperature $F = \bigwedge_{j=1}^m C_j,$5, learning rate $F = \bigwedge_{j=1}^m C_j,$6, max_grad_norm=^^^^3query3^^^^.3, beta=^^^^3query3^^^^.^^^^3query3ti:\3^^^^, max_steps=5^^^^3query3query3^^^^, and scale_rewards=True. Length budgets are max_prompt_length=^^^^3 OR abs:\3query3^^^^48, max_completion_length=8^^^^3ti:\3SATQuest verifier logical reasoning evaluation reinforcement fine-tuning of LLMs3 OR abs:\3^^^^ with mask_truncated_completions=True, and evaluation length budget up to 3^^^^3 OR abs:\3^^^^768 tokens for prompt and completion together. Compute consists of $F = \bigwedge_{j=1}^m C_j,$7A^^^^3ti:\3query3query3^^^^ 8^^^^3query3^^^^GB, split into 4 GPUs for training and 4 GPUs for vLLM inference. Reported training times are approximately 3^^^^3query3^^^^ hours for math decision, ^^^^3 OR abs:\3^^^^6 hours for math search, and 9 hours for narrative decision (&&&^^^^3query3^^^^&&&).

The reported training behavior is asymmetric. Training on math-format tasks increases rewards and response lengths quickly, whereas narrative-format training is less effective for the 7B baseline. Generalization to larger $F = \bigwedge_{j=1}^m C_j,$8 on the same task and format is positive. Cross-task transfer is also asymmetric: fine-tuning on a more complex math-format task such as MaxSAT improves a simpler related task such as search, whereas fine-tuning only on the simpler task does not improve the more complex one. By contrast, cross-format generalization remains weak; models fine-tuned on math often do not transfer to machine or narrative encodings, and verbose but flawed reasoning is reported as a characteristic failure mode (&&&^^^^3query3^^^^&&&).

6. Empirical findings, limitations, and significance

SATQuest reports extensive evaluation across reasoning-enhanced and vanilla models. The reasoning-enhanced group includes o3-mini, DeepSeek-R^^^^3ti:\3^^^^, QwQ-3^^^^3 OR abs:\3^^^^B, and DeepSeek-R^^^^3ti:\3^^^^-Distill-Qwen-7B/3^^^^3 OR abs:\3^^^^B; the vanilla group includes GPT-4.^^^^3ti:\3^^^^, DeepSeek-V3-^^^^3query33 OR abs:\3^^^^4, and Qwen^^^^3 OR abs:\3^^^^.5-7B/3^^^^3 OR abs:\3^^^^B-Instruct. Overall average accuracies are reported as: o3-mini ^^^^3query3^^^^.56, DeepSeek-R^^^^3ti:\3^^^^ ^^^^3query3^^^^.4^^^^3 OR abs:\3^^^^, QwQ-3^^^^3 OR abs:\3^^^^B ^^^^3query3^^^^.4^^^^3query3 DeepSeek-R^^^^3ti:\3^^^^-Distill-Qwen-3^^^^3 OR abs:\3^^^^B ^^^^3query3^^^^.39, GPT-4.^^^^3ti:\3^^^^ ^^^^3query3^^^^.38, DeepSeek-V3-^^^^3query33 OR abs:\3^^^^4 ^^^^3query3^^^^.36, and Qwen^^^^3 OR abs:\3^^^^.5-7B-Instruct below ^^^^3query3^^^^.^^^^3ti:\3^^^^ (&&&^^^^3query3^^^^&&&).

Three empirical regularities dominate the results. First, accuracy declines as solver decisions increase, and hallucinations increase at high complexity; examples include fabricated solver calls, and some models shorten responses rather than engage fully. Second, there is a task-type hierarchy: SAT decision and SAT search are easier, MaxSAT is intermediate, and MCS and MUS are notably harder, with sharp accuracy drops. Third, there is strong format sensitivity: mathematical notation yields the best performance, DIMACS follows, and the narrative formats produce the lowest accuracy. The framework further notes that reasoning models show marked drops outside math, while vanilla models, though weaker overall, are more format-balanced and produce shorter, more structured reasoning chains (&&&^^^^3query3^^^^&&&).

These results support SATQuest’s core diagnostic claim. The same CNF instance, when presented in a different format, can induce large performance changes, implying that many models do not reliably translate unfamiliar surface forms into stable internal symbolic representations. SAT-style reasoning also imposes global consistency requirements under disjunctions and minimality constraints, which the paper identifies as difficult to maintain across long reasoning chains without symbolic scaffolding. This suggests that strong performance on conventional mathematical word-problem distributions should not be conflated with broad logical robustness.

The framework’s limitations are also explicit. It is confined to CNF and therefore does not cover richer logics such as SMT or first-order logic, proof obligations, or multi-step compositional tasks beyond propositional SAT. Evaluation reaches $F = \bigwedge_{j=1}^m C_j,$9, but scaling to much larger instances would stress LLM context limits and solver interfacing for on-policy training. Smaller models also struggle to produce long binary strings reliably. Proposed extensions include moving from SAT to SMT, quantified Boolean formulas, and first-order logic; integrating proof checking through resolution or DRAT; adding multi-format curricula and translation scaffolds; and expanding the task set to entailment, redundancy detection, backbone or forced literals, and approximate $C_j$^^^^3query3^^^^SAT (&&&^^^^3query3^^^^&&&).

In research terms, SATQuest occupies a specific niche: it is a verifier-centered experimental framework for controlled reasoning analysis and reward-grounded training. Its significance lies less in proposing a new solver than in establishing an orthogonally controlled testbed where scale sensitivity, task sensitivity, and format sensitivity can be disentangled. That methodological emphasis, together with objective verification through PySAT and the availability of code and datasets, makes it a practical basis for studying whether LLM reasoning improvements reflect genuine logical generalization or narrower adaptation to familiar representational regimes (&&&^^^^3query3^^^^&&&).