Solver-Based Formal Reasoning

• Solver-based formal reasoning is a neuro-symbolic approach that combines language models with solvers (e.g., SAT, SMT) to transform informal problems into precise logical specifications.
• It addresses the challenge of formalization by mapping natural language and multimodal inputs into structured, verifiable representations, enabling robust solutions in tasks ranging from mathematics to legal reasoning.
• Architectural patterns such as adaptive routing, protocol-oriented model editing, and step-wise mathematical verification enhance both reliability and performance in complex problem-solving scenarios.

Solver-based formal reasoning is a family of neuro-symbolic methods that combines the generative ability of LLMs with the proof and decision procedures of logical solvers in order to obtain correct, robust answers on natural-language, multimodal, and formal reasoning tasks. In this setting, the “solver” may be a SAT or SMT engine, a constraint programming system, a theorem prover, or a proof assistant kernel, while the LLM handles formalization, decomposition, repair, and explanation; across recent work, the central technical difficulty is usually not sound inference inside the solver, but translating the source problem into a formal specification that actually captures the intended constraints (Raza et al., 28 Jan 2025, Szeider, 2024, Xu et al., 8 Oct 2025).

1. Formal foundations and solver semantics

At a foundational level, solver-based formal reasoning inherits two complementary abstractions. In abstract modular systems, an abstract module over a vocabulary is a directed graph whose nodes are consistent sets of literals plus a fail node; soundness is defined semantically, terminal complete consistent nodes are model nodes, and a sound abstract modular system induces a finite, acyclic transition system whose terminal non-fail states are models (Lierler et al., 2013). In formalized SAT solving, DPLL and CDCL are represented as state-transition systems over trails, clauses, learned clauses, conflict states, and restart policies, and total correctness—soundness, termination, and completeness—has been machine-verified in Isabelle/HOL (Maric et al., 2011).

Instantiation-based first-order reasoning adds a second foundational strand. By Herbrand-style reduction, first-order problems can be attacked by selecting finitely many ground instances from the Herbrand universe and passing the resulting propositional abstraction to a ground solver; the central difficulty is choosing the right instances from an often infinite candidate space (Piepenbrock et al., 2022). The GNN2RNN architecture addresses this by learning clause instantiations while remaining invariant to symbol names, a property motivated by the prevalence of Skolem symbols in clausified problems (Piepenbrock et al., 2022).

Quantified SMT reasoning exposes a related issue at solver level. A formal operational semantics for E-matching models how triggers enable quantifier instantiations, making it possible to prove termination of complex axiomatisations and to reason precisely about brittleness, incompleteness, and matching loops (Ge et al., 2024). Taken together, these lines of work frame solver-based formal reasoning not merely as “LLM plus solver,” but as a layered discipline involving model-finding semantics, transition systems, instantiation control, and proof-carrying execution.

2. Formalization as the principal bottleneck

Across contemporary systems, formalization fidelity is the dominant bottleneck. Semantic Self-Verification (SSV) makes this explicit by modeling a natural-language input $n \in N$, an abstract formalization $A=f(n)=\{\phi_1,\dots,\phi_m\}$ in logic $L$, and a set of concrete instantiations $I=\{i_1,\dots,i_k\}$ whose encodings $C(i)$ are checked by a solver (Raza et al., 28 Jan 2025). A positive instantiation requires $\mathrm{Solver}(A \land C(i))=\mathrm{SAT}$, a negative instantiation requires $\mathrm{Solver}(A \land C(i))=\mathrm{UNSAT}$, and verification succeeds only when every instantiation is consistent and well-formedness checks pass, yielding isVerified=true (Raza et al., 28 Jan 2025). The resulting generation–verification–repair loop treats concrete examples as semantic diagnostics for the abstract program.

Step-wise mathematical verification addresses the same issue from the side of human-readable derivations. MATH-VF uses a Formalizer to translate a natural-language solution into a first-order–like formal context in SimpleMath, organized as Fitch-style statements and a Solution Graph, and a Critic checks local judgments of the form $T_i \vdash Q_i$ by reducing them to satisfiability or symbolic simplification with Z3 and SymPy (Zhou et al., 27 May 2025). Here the challenge is not only whether a final answer is derivable, but whether each intermediate claim is justified under explicit side conditions such as typing, nonzero denominators, or branch structure.

In multimodal geometry, SD-GPS pushes formalization even further into the training objective. A formalizer $F_\theta$ maps text and diagram inputs directly to a typed geometry specification $\hat{\Lambda}$, and Solvability-Guided Reinforcement Learning optimizes a sequence-level reward

$A=f(n)=\{\phi_1,\dots,\phi_m\}$0

so that parser success, executable proof states, and final answer derivability become direct training signals (Li et al., 26 Jun 2026). This makes solver compatibility part of the learned representation itself, rather than a post hoc validation step.

3. Architectural patterns for coupling models and solvers

One major architectural divide is between static solver integration and dynamic composition. Adaptive LLM-Symbolic Reasoning decomposes an input into subquestions $A=f(n)=\{\phi_1,\dots,\phi_m\}$1 and reasoning types $A=f(n)=\{\phi_1,\dots,\phi_m\}$2, routes each $A=f(n)=\{\phi_1,\dots,\phi_m\}$3 to a solver family such as LP, FOL, CSP, or SMT, and executes

$A=f(n)=\{\phi_1,\dots,\phi_m\}$4

This replaces fixed solver selection with a routing stage that can handle mixed and multi-question prompts through a workflow graph over solver nodes and shared memory (Xu et al., 8 Oct 2025).

A protocol-oriented variant appears in MCP-Solver. Instead of focusing on solver routing, it exposes model inspection, atomic edits, and solving as tool calls such as get_model, add_item, replace_item, and solve_model, while preserving the invariant that the persisted model is always syntactically and semantically valid (Szeider, 2024). In the implementation described in the paper, this edit–validate–solve loop is realized with MiniZinc, compiled to FlatZinc and run with Chuffed by default, and is presented as a standardized bridge from language-model interaction to symbolic backends (Szeider, 2024).

In interactive theorem proving, composition often means decomposing proof obligations rather than selecting among heterogeneous solvers. Mechanic uses Lean’s sorry placeholder to isolate failing proof fragments, extracts each unresolved subgoal into a minimal closed context $A=f(n)=\{\phi_1,\dots,\phi_m\}$5, and reintegrates independently proved lemmas into the surrounding script; the paper states a soundness proposition showing that if each extracted subgoal has a term $A=f(n)=\{\phi_1,\dots,\phi_m\}$6 with $A=f(n)=\{\phi_1,\dots,\phi_m\}$7, then replacing the corresponding sorry nodes yields a Lean-accepted proof (Qiu et al., 25 Mar 2026). A related but strategically different pattern appears in decoupled Reasoner–Prover pipelines for Olympiad mathematics, where a general-purpose Reasoner proposes Lean theorem declarations ending in := by sorry, and an independent Prover attempts up to $A=f(n)=\{\phi_1,\dots,\phi_m\}$8 proof attempts per lemma before verified lemmas are added to the target theorem’s context (Liang et al., 7 Jul 2025).

4. Verification regimes, certificates, and dependable outputs

Verification regimes differ in both granularity and meaning. In SSV, “near-certain reasoning” denotes empirically near-perfect precision on the verified subset: with GPT-4, verified answers reached 100% precision at 21.7% coverage on AR-LSAT, 25.0% on FOLIO, 43.7% on LogDeduction, 66.0% on PrOntoQA (5-hop), and 75.2% on ProofWriter (open-world, 5-hop), while overall accuracy reached 71.3%, 80.9%, 89.7%, 100.0%, and 98.0% respectively (Raza et al., 28 Jan 2025). The system is explicit that these are not classical formal guarantees, because the natural-language specification remains informal.

A stricter notion of internal validity appears in report auditing and legal adjudication. In clinical reasoning, free-text findings are autoformalized into propositional evidence $A=f(n)=\{\phi_1,\dots,\phi_m\}$9, a clinician-audited knowledge base $L$0 is encoded in Z3, and a diagnosis $L$1 is certified precisely when

$L$2

which allows each impression claim to be labeled as entailed, hallucinated, omitted, or correctly excluded (Singh et al., 27 Feb 2026). In legal reasoning, L4M compiles statutes and case facts into quantifier-free SMT constraints over finite enumerations and linear arithmetic, then uses satisfiability checks and minimal unsat cores to drive up to three rounds of targeted self-critique before a Judge-LLM verbalizes the resulting model and sentence (Chen et al., 26 Nov 2025).

Recent work also shows that verified inference is not identical to verified delivery. The narration-gap analysis models an LLM–solver loop with formalization $L$3, decision $L$4, and narration $L$5, proves that certificate gating makes the solver verdict sound, and then shows empirically that an adversary can still invert the narrated conclusion unless the delivered conclusion is enforced directly as $L$6 from the verified verdict (Huang et al., 17 Jun 2026). In that sense, solver-based formal reasoning has expanded from checking proofs and models to checking the last natural-language step that communicates them.

5. Domains of application and empirical performance

Mathematics and theorem proving are the most prominent proving grounds for these methods. MA-LoT reports a 61.07% accuracy rate on the Lean4 version of MiniF2F-Test, compared with 33.61% for DeepSeek-V3, 50.70% for InternLM-Step-Prover, and 55.33% for Godel-Prover (Wang et al., 5 Mar 2025). Mechanic reports 11 of 12 Putnam 2025 problems solved under budget, with average time-to-proof of approximately 114 minutes and average proof length of approximately 656 lines, while the decoupled Reasoner–Prover framework solves 5 post-2000 IMO non-geometry problems for which no prior open-source prover had reported success (Qiu et al., 25 Mar 2026, Liang et al., 7 Jul 2025).

Geometry and symbolic math show a different balance between perception, autoformalization, and solver search. Inter-GPS translated text and diagrams into a unified formal language of 91 predicates and achieved 57.5% overall accuracy on Geometry3K, rising to 78.3% with ground-truth formal language (Lu et al., 2021). SD-GPS closes the loop between multimodal perception and symbolic execution more tightly, reaching 86.4% completion and 90.4% choice accuracy on Geometry3K, and 79.8% completion and 84.5% choice accuracy on PGPS9K (Li et al., 26 Jun 2026). For process auditing rather than end-task solving, MATH-VF achieved 95.7% discriminative accuracy with DeepSeek-v3 on solution-correctness judgments and improved refinement over self-refinement baselines (Zhou et al., 27 May 2025).

Solver-backed evaluation has also become a domain in its own right. LogicSkills isolates symbolization, countermodel construction, and validity assessment in the two-variable fragment of first-order logic and reports high validity performance together with substantially lower symbolization and countermodel performance across leading models (Rabern et al., 6 Feb 2026). LLMEval-Logic applies Z3 verification and expert rubrics to Chinese situational reasoning, releasing a 246-item Base subset with 1,400 rubric atoms and a 190-item Hard subset with 938 sub-questions; the best model reaches only 37.5% Hard Item Accuracy, and the highest joint Z3+Rubric formalization score in the fixed-symbol setting reaches only 60.16% (Zhang et al., 19 May 2026). These benchmarks reposition solver-based reasoning from a method of solving tasks to a method of auditing the formal competence of the models themselves.

6. Limitations, controversies, and research directions

The dominant limitations recur with notable regularity. SSV identifies insufficient instantiation coverage, mutual consistency of a wrong formalization and a wrong instantiation, and missing or superfluous constraints as core failure modes, while quantified SMT introduces timeouts and quantifier-instantiation pitfalls (Raza et al., 28 Jan 2025). The formal model for E-matching-based axiomatisations frames the same problem at solver level as brittleness, incompleteness, and divergence, showing that trigger design can determine whether a quantifier discipline is predictable or stuck in a matching loop (Ge et al., 2024).

Most reported guarantees are therefore conditional. Clinical report verification is explicitly conditioned on the truthfulness of the Findings section and the fidelity of the autoformalization function $L$7 and the knowledge base $L$8 (Singh et al., 27 Feb 2026). Legal adjudication guarantees are conditioned on extracted facts and statute encodings, and solver-backed sentence validity is separated from substantive rationality (Chen et al., 26 Nov 2025). Benchmark work reaches the same conclusion from the opposite side: once task semantics, formal language, and solver protocol are fixed, large residual error on symbolization or hardened closed-world reasoning remains measurable rather than hidden by lexical overlap or free-form explanation (Rabern et al., 6 Feb 2026, Zhang et al., 19 May 2026).

Reported future directions are correspondingly concrete: more diverse and adversarial instantiations, stronger structural parsers, richer logics, and probabilistic aggregation for instantiation-based verification (Raza et al., 28 Jan 2025); extension of solver-driven autoformalization to domains beyond plane geometry that provide typed formal languages and execution oracles (Li et al., 26 Jun 2026); tighter adaptive routing across additional solver families beyond LP, FOL, CSP, and SMT (Xu et al., 8 Oct 2025); and verified delivery mechanisms that enforce conclusions directly from certified solver verdicts rather than trusting unconstrained narration (Huang et al., 17 Jun 2026). This suggests a transition from isolated neuro-symbolic prototypes toward end-to-end verified pipelines in which formalization, decision, and delivery are all treated as first-class objects of verification.