AutoModSAT: LLM-Driven SAT Solver Optimization

• AutoModSAT is an LLM-driven methodology that automatically discovers and integrates heuristic code in modular CDCL solvers.
• It leverages code synthesis and iterative prompt optimization to enhance solver performance using runtime feedback and statistical metrics.
• The approach shifts from traditional parameter tuning to program-level modifications, enabling adaptive and efficient SAT solving strategies.

Searching arXiv for recent and foundational papers on AutoModSAT and related LLM-driven SAT solver optimization. AutoModSAT denotes an LLM-driven methodology for automatically discovering, modifying, and integrating heuristic code in SAT solvers, especially within Conflict-Driven Clause Learning (CDCL) architectures. In the literature, the term is used most directly for a tool that optimizes a complex modularized CDCL solver through code-level heuristic synthesis (Sun et al., 30 Jul 2025), while closely related frameworks such as AutoSAT instantiate essentially the same workflow on simpler solver baselines (Sun et al., 2024). In this usage, AutoModSAT is not a new satisfiability formalism; it is an automated solver-engineering paradigm in which heuristic functions are exposed as editable modules, LLMs propose source-code variants, and an evaluation loop accepts modifications according to runtime-oriented objectives such as solved-instance counts and PAR-2 (Sun et al., 30 Jul 2025). This suggests a shift from parameter tuning over fixed search spaces toward program synthesis over solver internals.

1. Conceptual scope and problem setting

AutoModSAT arises from the observation that CDCL performance is dominated by heuristics governing branching, polarity, restart, clause deletion, and activity dynamics. CDCL solvers repeatedly pick a decision variable and phase, run Boolean Constraint Propagation using two-watched-literal propagation, analyze conflicts to derive a learned clause, backjump non-chronologically, and periodically restart. The efficiency of this loop depends heavily on heuristic choices such as VSIDS or EVSIDS variable scoring, phase saving, LBD-based clause retention, and restart triggers based on conflict counts or moving averages of LBD (Sun et al., 2024).

In conventional solver development, these mechanisms are designed and tuned manually, or adjusted by automatic configuration systems that search over scalar parameters. AutoModSAT is framed as a response to the limits of such approaches. AutoSAT states that existing automated algorithm design approaches focus on hyperparameter tuning and operator selection, whereas its framework can generate new efficient heuristics (Sun et al., 2024). The later AutoModSAT work makes the same distinction more sharply, arguing that traditional configuration tools such as ParamILS, SMAC, and irace tune numerical or Boolean parameters of a fixed solver, while AutoModSAT discovers code-level heuristics with conditional logic, new thresholds, stochastic policies, and multi-signal triggers (Sun et al., 30 Jul 2025).

The central object of optimization is therefore not merely a parameter vector but a set of solver functions. In the most explicit formulation, seven heuristic slots are exposed in a modularized solver: restart_function, restart_condition, reduce_condition, rephase_function, rephase_condition, varBumpActivity, and claBumpActivity (Sun et al., 30 Jul 2025). In the earlier AutoSAT formulation, the search space similarly covers branching score systems, restart policies, clause deletion and retention, polarity schemes, and activity dynamics (Sun et al., 2024). A plausible implication is that “AutoModSAT” functions as an umbrella term for automated heuristic code discovery in SAT solvers, with AutoSAT representing an earlier instantiation on a more restricted modular baseline.

2. Architectural pattern: modular solver, LLM roles, and search loop

The architectural core of AutoModSAT is a modular solver plus an iterative feedback loop. AutoSAT describes a three-role prompting structure: an Advisor analyzes solver code and suggests heuristic directions, a Coder generates code variants, and an Evaluator selects the best code based on runtime feedback and crafts the next prompt (Sun et al., 2024). The related SolSearch framework uses a Selector, Searcher, and Evaluator decomposition with similar semantics: the selector chooses a solver patch, the searcher uses an LLM to synthesize new code for that patch, and the evaluator compiles and benchmarks the modified solver (Sheng et al., 20 Feb 2025). The later AutoModSAT system preserves this general pattern but emphasizes modularization and automatic prompt optimization (Sun et al., 30 Jul 2025).

A defining prerequisite is “LLM-friendly” solver design. The AutoModSAT paper proposes systematic guidelines for developing a modularized solver to meet LLM compatibility, emphasizing code simplification, information share and bug reduction (Sun et al., 30 Jul 2025). The detailed prescriptions are concrete: large intertwined procedures are broken into small focused functions; local-scope variables that are useful to heuristic design are lifted into class member variables; and preventive measures such as adding needed includes and type-safe wrappers are introduced to reduce common LLM-generated errors (Sun et al., 30 Jul 2025). AutoSAT reports a comparable mechanism through prompts that include the solver’s “key code” context, role guidance, localized replacement regions marked by // start … // end, and constraints requiring executable differences from the original (Sun et al., 2024).

The search loop itself is black-box optimization over code proposals. AutoSAT explicitly interprets its loop as either a greedy hill climber or a $(1+1)$-EA / $(1+\lambda)$-EA depending on the number of proposals generated per iteration (Sun et al., 2024). The paper’s algorithm uses $m$ parallel code variants each iteration and selects the best, corresponding to $(1+\lambda)$-EA with $\lambda=m$; when $m=1$, the procedure reduces to a hill-climber or $(1+1)$-EA (Sun et al., 2024). The later AutoModSAT system formalizes a two-stage version of this idea: a presearch phase prunes low-impact functions using quick tests on 50% of the instances, then a focused EA mutates only the retained functions under an evaluation budget of 50 heuristic requests (Sun et al., 30 Jul 2025).

Compilation and fault tolerance are integral to the architecture rather than ancillary engineering details. AutoSAT constrains generated code to specific functions or guards, performs static checks to ensure the code compiles, rejects identical code, and feeds syntax or runtime errors back to the Coder for correction (Sun et al., 2024). SolSearch similarly requires that prompts fix the function name and language, and disallow undefined functions, variables, or external libraries (Sheng et al., 20 Feb 2025). AutoModSAT extends this into a broader reliability program: proactive bug prevention, code repair, and modular interfaces are treated as first-class design conditions for successful heuristic discovery (Sun et al., 30 Jul 2025).

3. Search space and heuristic families

The search space of AutoModSAT is deliberately centered on CDCL heuristics that have large empirical leverage and are local enough to be edited safely. Across the relevant papers, the main target families are branching activity updates, restart logic, rephasing, clause activity updates, and reduction policies (Sun et al., 2024, Sun et al., 30 Jul 2025).

For variable activity, the canonical background is VSIDS or EVSIDS. AutoSAT lists additive bumps, controlled decay, EVSIDS-style exponential decay, bump/decay rate rules, tie-breaking in decision heaps, var_inc update policy, overflow normalization, and EVSIDS $\lambda$-decay parameterization as editable dimensions (Sun et al., 2024). SolSearch provides a concrete patch example aimed at the inc_activity function, with explicit behavioral requirements: increase activity by a fixed amount, check for overflow, rescale if needed, and manage variable priorities within a priority queue (Sheng et al., 20 Feb 2025). AutoModSAT reports a discovered varBumpActivity heuristic that scales the increment by decision level, rescales at threshold 1e100 using 1e-100, enforces a minimum activity floor 1e-100, and updates the heap lazily (Sun et al., 30 Jul 2025).

For restarts, the baseline landscape includes Luby-based schedules, Glucose-style moving averages of LBD, thresholds based on conflict counts, queue sizes, and ratios of fast and slow LBD sums (Sun et al., 2024). AutoModSAT reports discovered adaptive restart behavior based on exponential moving averages fast_avg and slow_avg, using the ratio $r=\text{fast\_avg}/\text{slow\_avg}$ to choose full, partial, or minimal restart depth, and occasionally coupling restarts with clause-database reduction and heap rebuilding (Sun et al., 30 Jul 2025). It also reports a restart_condition that combines average LBD, conflict rate, restart_first, and progressEstimate, reducing the threshold aggressively when progress is small (Sun et al., 30 Jul 2025).

Rephasing and polarity policies are another major family. AutoSAT identifies phase saving, deterministic phase, randomized phase flips, adaptive rephasing, rephase limits, thresholds, and phase inversion cadence as pluggable modules (Sun et al., 2024). In one worked example, it describes a generated rephase heuristic that computes average LBD from slow_lbd_sum / max(conflicts,1), adjusts thresholds upward or downward according to that average, adapts rephase_limit, penalizes variables not recently involved in useful conflicts by halving activity, and resets var_inc = 1.0 (Sun et al., 2024). AutoModSAT generalizes this pattern with a rephase_condition based on normalized progress and conflict density, and a rephase_function that mixes several phase-selection policies in fixed proportions, including local best, global inversion, randomized choices for low-activity variables, and user phases (Sun et al., 30 Jul 2025).

Clause management is treated analogously. AutoSAT’s search space includes clause deletion criteria based on LBD thresholds, clause length filters, clause activity-based pruning, and periodic reduction intervals (Sun et al., 2024). AutoModSAT reports both claBumpActivity and reduce_condition variants that add minimum activity floors, conflict-sensitive decay rules, and composite triggers involving memory pressure, learnt-to-original ratio, and conflict-driven early reduction (Sun et al., 30 Jul 2025). This suggests that AutoModSAT’s effective scope extends across the full control surface of a CDCL solver except for deeper structural internals such as watched-literal propagation, which the papers describe as left largely unchanged (Sun et al., 30 Jul 2025).

4. Objectives, metrics, and evaluation protocols

The principal optimization objective in AutoModSAT-style systems is solver performance measured on benchmark sets under timeouts. AutoSAT states that training primarily optimizes total runtime, while testing reports SAT-competition metrics: number of instances solved within timeout and PAR-2 (Sun et al., 2024). In its general formulation,

$f(H) = \sum_{i=1}^{N} I_i(H),$

where $(1+\lambda)$0 may denote solved status, penalized runtime, or a negated cost so that higher is better during search (Sun et al., 2024). SolSearch likewise optimizes two complementary objectives: maximize solved instances within a timeout and minimize PAR-2, where unsolved instances incur twice the cutoff (Sheng et al., 20 Feb 2025). AutoModSAT uses PAR-2 as the primary fitness, lower being better, and also reports solved counts and a speedup measure (Sun et al., 30 Jul 2025):

$(1+\lambda)$1

and

$(1+\lambda)$2

The main reported experimental settings differ across papers but share the same overall structure.

Framework	Baseline / targets	Search/eval setting	Main reported benchmarks
AutoSAT	EasySAT; designed plug-and-play for CDCL solvers	48-instance training set; 1,200 testing instances; training timeouts 500 s and 1000 s; testing 1500 s	SAT Competitions 2018/2022/2023, CNP, SCPC (Sun et al., 2024)
SolSearch	CaDiCaL, Kissat-sc2024, Z3 SAT module	Curriculum starting at 100 s and increasing	SAT Competition 2023; Knights Tour 2024 task (Sheng et al., 20 Feb 2025)
AutoModSAT	ModSAT baseline, compared with Kissat and CaDiCaL and SMAC3-tuned variants	Budget $(1+\lambda)$3; training timeout half the median solving time; testing 5000 s	11 datasets including SC 2023/2024 families, Picat-generated sets, and industrial EDA (Sun et al., 30 Jul 2025)

AutoSAT reports that a realization of the framework outperforms MiniSat on 9 out of 12 datasets and surpasses Kissat on 4 datasets in the abstract, although its detailed tables in the provided material compare directly against EasySAT and do not report direct MiniSat/Kissat head-to-heads (Sun et al., 2024). On SAT Competition 2022 with $(1+\lambda)$4 s, EasySAT solves 75 instances with PAR-2 = 2541.7, whereas AutoSAT_bump2_rephase_restart solves 96 with PAR-2 = 2393.11 (Sun et al., 2024). On CNP, the baseline solves 26 instances with PAR-2 = 926.23, while AutoSAT_restart solves 34 with PAR-2 = 504.71; on SCPC, the baseline solves 0 with PAR-2 = 3000, while combined variants solve up to 11 with PAR-2 as low as approximately 663.73 (Sun et al., 2024).

SolSearch reports improvements on SAT Competition 2023 benchmarks for CaDiCaL and Kissat-sc2024, and an especially explicit result on the Knights Tour benchmark with Z3: baseline Z3 solves 17 instances; with SolSearch, solved ratio at $(1+\lambda)$5 s improves from 7.5% to 10.0%, at $(1+\lambda)$6 s from 35.0% to 42.5%, at $(1+\lambda)$7 s from 42.5% to 52.5%, and PAR-2 decreases from 660.09 to 583.52, which the paper summarizes as approximately 11% improvement (Sheng et al., 20 Feb 2025).

The most explicit “AutoModSAT” paper reports 11 datasets, a budget of 50 heuristic proposals, and testing at 5000 s (Sun et al., 30 Jul 2025). Its headline numbers are 50% performance improvement over the ModSAT baseline, 30% superiority against state-of-the-art solvers on average, and 20% average speedup over parameter-tuned alternatives of the state-of-the-art solvers (Sun et al., 30 Jul 2025). Reported examples include cryptography-ascon, where AutoModSAT achieves PAR-2 140.72 with 20 solved versus 155.85 for Kissat-para; register-allocation, where it achieves 1177.8 with 18 solved versus 6321.95 with 8 solved for Kissat-para; hamiltonian, where it reaches 133.13 with 40 solved versus 296.75 with 39 solved; and EDA, where it obtains 376.01 with 49 solved versus 416.42 with 49 solved for Kissat-para (Sun et al., 30 Jul 2025).

5. Automatic prompt optimization and diversity control

A distinctive methodological contribution of the later AutoModSAT work is unsupervised automatic prompt optimization. Whereas AutoSAT and SolSearch use structured prompts with role descriptions, behavioral constraints, and code context (Sun et al., 2024, Sheng et al., 20 Feb 2025), AutoModSAT explicitly treats the prompt as an object of optimization (Sun et al., 30 Jul 2025).

The prompt is decomposed into Role, Goal, and Tips, to which “key code” context is appended. At each iteration, one of these sections is refined, the LLM generates code samples, and the refined prompt is accepted if both output diversity and compilation success improve (Sun et al., 30 Jul 2025). Diversity is measured by computing CodeT5+ embeddings of generated code, clustering them with K-Means++, and evaluating Shannon entropy:

$(1+\lambda)$8

Compilation and execution success rate is used as a second acceptance criterion (Sun et al., 30 Jul 2025).

This mechanism is intended to counter two failure modes that are recurrent in program-synthesis loops: “code ossification,” in which the model repeats minor mutations of an already-accepted pattern, and syntactic degeneration, in which diversity is gained only by producing invalid code. AutoSAT’s analysis already suggests that multiple proposals help avoid local optima and code ossification (Sun et al., 2024). The AutoModSAT formulation makes that intuition operational through entropy over embedding clusters plus compile success (Sun et al., 30 Jul 2025). A plausible implication is that prompt search is being used as a surrogate for maintaining mutation diversity in evolutionary program search.

Prompt content remains highly constrained. AutoSAT requires proposals to be localized, syntactically compatible, and different from the original, and supplies runtime feedback in evaluator prompts (Sun et al., 2024). SolSearch fixes the function name and language and forbids undefined dependencies (Sheng et al., 20 Feb 2025). AutoModSAT likewise standardizes instructions, disallows new globals, and appends the “key code” needed to keep edits consistent with the caller context and class state (Sun et al., 30 Jul 2025). The convergence of these design choices indicates that prompt engineering in this domain is less about natural-language nuance than about interface discipline and search diversity.

6. Relation to automated configuration, solver synthesis, and adjacent SAT/SMT frameworks

AutoModSAT belongs to a broader family of automated reasoning systems that modify solver behavior, but its position within that family is specific. It differs from classical automatic algorithm configuration because the latter usually searches over predeclared parameters or operator selections. Both AutoSAT and AutoModSAT explicitly contrast themselves with this tradition, emphasizing that they can synthesize novel heuristic code rather than merely tune scalar knobs (Sun et al., 2024, Sun et al., 30 Jul 2025).

It also differs from SAT modulo theories or DPLL(T)-style SMT frameworks. Monotonic-theory SMT extends SAT solving by coupling CDCL with theory propagation and explanation over theory atoms, using under- and over-approximations for partial assignments and constructive witnesses such as paths, cuts, flows, or trees (Bayless et al., 2014). General SMT, in the DPLL(T) account, splits Boolean abstraction from theory reasoning and learns theory lemmas through consistency checks and propagation (Monniaux, 2016). These frameworks enlarge the semantic scope of satisfiability; AutoModSAT instead optimizes the implementation of a solver, typically remaining within propositional CDCL [(Bayless et al., 2014); (Monniaux, 2016)]. The shared elements are architectural rather than semantic: modular interfaces, conflict explanations, and incremental search.

The name “AutoModSAT” can create ambiguity because “ModSAT” sometimes denotes unrelated uses of “modulo” in SAT or SAT-based compilation. For example, pseudo-Boolean constraints can be translated into SAT through decomposition into PB-Mod constraints with exactness guarantees when the least common multiple of chosen moduli exceeds the coefficient sum (Aavani, 2011). That line of work uses “modular” in the arithmetic sense of modular congruence. By contrast, AutoModSAT uses “mod” or “modification” in the sense of automated solver modification (Sun et al., 30 Jul 2025). The distinction is substantive: one concerns encoding constraints into SAT; the other concerns synthesizing heuristic code for SAT solvers.

The relation to SolSearch is especially close. SolSearch is described as an LLM-driven, plug-and-play framework that automatically discovers, modifies, and generates SAT-solving code, using curriculum-based trial-and-error and integrating with CaDiCaL, Kissat-sc2024, and Z3 without bespoke APIs (Sheng et al., 20 Feb 2025). This suggests that SolSearch and AutoModSAT are not competing conceptual categories but neighboring points in the same design space. The main differences in the provided materials are emphasis: SolSearch foregrounds curriculum learning and solver-agnostic patch selection, while AutoModSAT foregrounds modular solver design, automatic prompt optimization, and presearch plus EA (Sheng et al., 20 Feb 2025, Sun et al., 30 Jul 2025).

7. Limitations, robustness, and future directions

The papers identify several limitations that constrain current AutoModSAT systems. The first is computational cost. Evaluating candidate heuristics requires repeated compilation and execution on nontrivial benchmark suites, often under long timeouts; AutoSAT uses multiple servers with Intel Xeon Gold 6230 CPUs and an AMD EPYC 7763 server with 8× NVIDIA H100 GPUs for LLM inference (Sun et al., 2024). AutoModSAT similarly notes that solver evaluations are expensive and mitigates this by using shorter training timeouts and a budget of 50 heuristic requests (Sun et al., 30 Jul 2025).

The second is overfitting and dataset sensitivity. AutoSAT observes that heuristics learned on its 48-instance training set often improved across three competition years, but some variants that excel on 2022 regress on 2018 or 2023, illustrating dataset sensitivity (Sun et al., 2024). AutoModSAT likewise notes risk of dataset-specific optimization, even though its reported performance spans competition, crafted, and industrial datasets (Sun et al., 30 Jul 2025). This suggests that the empirical gains should not be read as evidence of universally superior heuristics; rather, they show that LLM-guided code search can traverse useful regions of heuristic space under realistic distributional structure.

A third limitation is LLM dependence. AutoSAT compares GPT-4, Qwen-72B, and DeepSeek-Coder and reports that GPT-4 generally yields stronger proposals, while Qwen and DeepSeek produce mixed results (Sun et al., 2024). AutoModSAT states that LLM quality and long-context ability matter, especially because complex solvers exceed typical token limits; its answer is modularization, not end-to-end whole-solver synthesis (Sun et al., 30 Jul 2025). SolSearch uses deepseek-coder in the reported experiments, but the paper also calls for deeper analysis of how model size, training data, and specialization affect performance (Sheng et al., 20 Feb 2025).

Maintainability and reproducibility remain open concerns. AutoSAT notes that generated code can be harder to maintain and that versioning and strict logging are required for reproducibility (Sun et al., 2024). AutoModSAT does not provide a public repository in the material provided, and similarly lacks reported seed details (Sun et al., 30 Jul 2025). SolSearch also does not provide hardware specifics or a public code repository in the provided content (Sheng et al., 20 Feb 2025). For a methodology centered on code synthesis, this absence is nontrivial because the behavior of the system depends not only on benchmarks and budgets but also on prompts, models, and repair logic.

Future directions in the papers cluster around search-space enlargement, stronger feedback, and cross-domain transfer. AutoSAT proposes richer search spaces, multi-objective optimization over runtime, memory, and clause-database size, cross-domain generalization via instance features, and hybridization with reinforcement learning or bandit selection (Sun et al., 2024). SolSearch explicitly names Max-SAT and QBF as potential target domains (Sheng et al., 20 Feb 2025). AutoModSAT suggests transfer to other complex solvers such as MIP, SMT, and CSP, conditioned on careful modularization of heuristics and exposure of key solver state (Sun et al., 30 Jul 2025). This suggests that the lasting significance of AutoModSAT may lie less in any one discovered heuristic than in establishing a reusable methodology for LLM-mediated program search in optimization-intensive systems.