---
title: 'AutoModSAT: LLM-Driven SAT Solver Optimization'
url: https://www.emergentmind.com/topics/automodsat
type: topic
---

# AutoModSAT: LLM-Driven SAT Solver Optimization

Searching arXiv for recent and foundational papers on AutoModSAT and related LLM-driven SAT solver optimization.
AutoModSAT denotes an LLM-driven methodology for automatically discovering, modifying, and integrating heuristic code in SAT solvers, especially within Conflict-Driven Clause Learning (CDCL) architectures. In the literature, the term is used most directly for a tool that optimizes a complex modularized CDCL solver through code-level heuristic synthesis [2507.22876], while closely related frameworks such as AutoSAT instantiate essentially the same workflow on simpler solver baselines [2402.10705]. In this usage, AutoModSAT is not a new satisfiability formalism; it is an automated solver-engineering paradigm in which heuristic functions are exposed as editable modules, large language models propose source-code variants, and an evaluation loop accepts modifications according to runtime-oriented objectives such as solved-instance counts and PAR-2 [2507.22876]. This suggests a shift from parameter tuning over fixed search spaces toward program synthesis over solver internals.

## 1. Conceptual scope and problem setting

AutoModSAT arises from the observation that CDCL performance is dominated by heuristics governing branching, polarity, restart, clause deletion, and activity dynamics. CDCL solvers repeatedly pick a decision variable and phase, run Boolean Constraint Propagation using two-watched-literal propagation, analyze conflicts to derive a learned clause, backjump non-chronologically, and periodically restart. The efficiency of this loop depends heavily on heuristic choices such as VSIDS or EVSIDS variable scoring, phase saving, LBD-based clause retention, and restart triggers based on conflict counts or moving averages of LBD [2402.10705].

In conventional solver development, these mechanisms are designed and tuned manually, or adjusted by automatic configuration systems that search over scalar parameters. AutoModSAT is framed as a response to the limits of such approaches. AutoSAT states that existing automated algorithm design approaches focus on hyperparameter tuning and operator selection, whereas its framework can generate new efficient heuristics [2402.10705]. The later AutoModSAT work makes the same distinction more sharply, arguing that traditional configuration tools such as ParamILS, SMAC, and irace tune numerical or Boolean parameters of a fixed solver, while AutoModSAT discovers code-level heuristics with conditional logic, new thresholds, stochastic policies, and multi-signal triggers [2507.22876].

The central object of optimization is therefore not merely a parameter vector but a set of solver functions. In the most explicit formulation, seven heuristic slots are exposed in a modularized solver: `restart_function`, `restart_condition`, `reduce_condition`, `rephase_function`, `rephase_condition`, `varBumpActivity`, and `claBumpActivity` [2507.22876]. In the earlier AutoSAT formulation, the search space similarly covers branching score systems, restart policies, clause deletion and retention, polarity schemes, and activity dynamics [2402.10705]. A plausible implication is that “AutoModSAT” functions as an umbrella term for automated heuristic code discovery in SAT solvers, with AutoSAT representing an earlier instantiation on a more restricted modular baseline.

## 2. Architectural pattern: modular solver, LLM roles, and search loop

The architectural core of AutoModSAT is a modular solver plus an iterative feedback loop. AutoSAT describes a three-role prompting structure: an Advisor analyzes solver code and suggests heuristic directions, a Coder generates code variants, and an Evaluator selects the best code based on runtime feedback and crafts the next prompt [2402.10705]. The related SolSearch framework uses a Selector, Searcher, and Evaluator decomposition with similar semantics: the selector chooses a solver patch, the searcher uses an LLM to synthesize new code for that patch, and the evaluator compiles and benchmarks the modified solver [2502.14328]. The later AutoModSAT system preserves this general pattern but emphasizes modularization and automatic prompt optimization [2507.22876].

A defining prerequisite is “LLM-friendly” solver design. The AutoModSAT paper proposes systematic guidelines for developing a modularized solver to meet LLM compatibility, emphasizing code simplification, information share and bug reduction [2507.22876]. The detailed prescriptions are concrete: large intertwined procedures are broken into small focused functions; local-scope variables that are useful to heuristic design are lifted into class member variables; and preventive measures such as adding needed includes and type-safe wrappers are introduced to reduce common LLM-generated errors [2507.22876]. AutoSAT reports a comparable mechanism through prompts that include the solver’s “key code” context, role guidance, localized replacement regions marked by `// start … // end`, and constraints requiring executable differences from the original [2402.10705].

The search loop itself is black-box optimization over code proposals. AutoSAT explicitly interprets its loop as either a greedy hill climber or a $(1+1)$-EA / $(1+\lambda)$-EA depending on the number of proposals generated per iteration [2402.10705]. The paper’s algorithm uses $m$ parallel code variants each iteration and selects the best, corresponding to $(1+\lambda)$-EA with $\lambda=m$; when $m=1$, the procedure reduces to a hill-climber or $(1+1)$-EA [2402.10705]. The later AutoModSAT system formalizes a two-stage version of this idea: a presearch phase prunes low-impact functions using quick tests on 50% of the instances, then a focused EA mutates only the retained functions under an evaluation budget of 50 heuristic requests [2507.22876].

Compilation and fault tolerance are integral to the architecture rather than ancillary engineering details. AutoSAT constrains generated code to specific functions or guards, performs static checks to ensure the code compiles, rejects identical code, and feeds syntax or runtime errors back to the Coder for correction [2402.10705]. SolSearch similarly requires that prompts fix the function name and language, and disallow undefined functions, variables, or external libraries [2502.14328]. AutoModSAT extends this into a broader reliability program: proactive bug prevention, code repair, and modular interfaces are treated as first-class design conditions for successful heuristic discovery [2507.22876].

## 3. Search space and heuristic families

The search space of AutoModSAT is deliberately centered on CDCL heuristics that have large empirical leverage and are local enough to be edited safely. Across the relevant papers, the main target families are branching activity updates, restart logic, rephasing, clause activity updates, and reduction policies [2402.10705; 2507.22876].

For variable activity, the canonical background is VSIDS or EVSIDS. AutoSAT lists additive bumps, controlled decay, EVSIDS-style exponential decay, bump/decay rate rules, tie-breaking in decision heaps, `var_inc` update policy, overflow normalization, and EVSIDS $\lambda$-decay parameterization as editable dimensions [2402.10705]. SolSearch provides a concrete patch example aimed at the `inc_activity` function, with explicit behavioral requirements: increase activity by a fixed amount, check for overflow, rescale if needed, and manage variable priorities within a priority queue [2502.14328]. AutoModSAT reports a discovered `varBumpActivity` heuristic that scales the increment by decision level, rescales at threshold `1e100` using `1e-100`, enforces a minimum activity floor `1e-100`, and updates the heap lazily [2507.22876].

For restarts, the baseline landscape includes Luby-based schedules, Glucose-style moving averages of LBD, thresholds based on conflict counts, queue sizes, and ratios of fast and slow LBD sums [2402.10705]. AutoModSAT reports discovered adaptive restart behavior based on exponential moving averages `fast_avg` and `slow_avg`, using the ratio $r=\text{fast\_avg}/\text{slow\_avg}$ to choose full, partial, or minimal restart depth, and occasionally coupling restarts with clause-database reduction and heap rebuilding [2507.22876]. It also reports a `restart_condition` that combines average LBD, conflict rate, `restart_first`, and `progressEstimate`, reducing the threshold aggressively when progress is small [2507.22876].

Rephasing and polarity policies are another major family. AutoSAT identifies phase saving, deterministic phase, randomized phase flips, adaptive rephasing, rephase limits, thresholds, and phase inversion cadence as pluggable modules [2402.10705]. In one worked example, it describes a generated rephase heuristic that computes average LBD from `slow_lbd_sum / max(conflicts,1)`, adjusts thresholds upward or downward according to that average, adapts `rephase_limit`, penalizes variables not recently involved in useful conflicts by halving activity, and resets `var_inc = 1.0` [2402.10705]. AutoModSAT generalizes this pattern with a `rephase_condition` based on normalized progress and conflict density, and a `rephase_function` that mixes several phase-selection policies in fixed proportions, including local best, global inversion, randomized choices for low-activity variables, and user phases [2507.22876].

Clause management is treated analogously. AutoSAT’s search space includes clause deletion criteria based on LBD thresholds, clause length filters, clause activity-based pruning, and periodic reduction intervals [2402.10705]. AutoModSAT reports both `claBumpActivity` and `reduce_condition` variants that add minimum activity floors, conflict-sensitive decay rules, and composite triggers involving memory pressure, learnt-to-original ratio, and conflict-driven early reduction [2507.22876]. This suggests that AutoModSAT’s effective scope extends across the full control surface of a CDCL solver except for deeper structural internals such as watched-literal propagation, which the papers describe as left largely unchanged [2507.22876].

## 4. Objectives, metrics, and evaluation protocols

The principal optimization objective in AutoModSAT-style systems is solver performance measured on benchmark sets under timeouts. AutoSAT states that training primarily optimizes total runtime, while testing reports SAT-competition metrics: number of instances solved within timeout and PAR-2 [2402.10705]. In its general formulation,
$$
f(H) = \sum_{i=1}^{N} I_i(H),
$$
where $I_i(H)$ may denote solved status, penalized runtime, or a negated cost so that higher is better during search [2402.10705]. SolSearch likewise optimizes two complementary objectives: maximize solved instances within a timeout and minimize PAR-2, where unsolved instances incur twice the cutoff [2502.14328]. AutoModSAT uses PAR-2 as the primary fitness, lower being better, and also reports solved counts and a speedup measure [2507.22876]:
$$
\text{PAR-2} = \frac{1}{n}\sum_{i=1}^{n}\tau_i,\quad
\tau_i =
\begin{cases}
t_i, & t_i \le \mathcal{T} \\
2\mathcal{T}, & t_i > \mathcal{T}\text{ or failure}
\end{cases}
$$
and
$$
\text{speedup} = \frac{v_a - v_b}{\max(v_a, v_b)}.
$$

The main reported experimental settings differ across papers but share the same overall structure.

| Framework | Baseline / targets | Search/eval setting | Main reported benchmarks |
|---|---|---|---|
| AutoSAT | EasySAT; designed plug-and-play for CDCL solvers | 48-instance training set; 1,200 testing instances; training timeouts 500 s and 1000 s; testing 1500 s | SAT Competitions 2018/2022/2023, CNP, SCPC [2402.10705] |
| SolSearch | CaDiCaL, Kissat-sc2024, Z3 SAT module | Curriculum starting at 100 s and increasing | SAT Competition 2023; Knights Tour 2024 task [2502.14328] |
| AutoModSAT | ModSAT baseline, compared with Kissat and CaDiCaL and SMAC3-tuned variants | Budget $B=50$; training timeout half the median solving time; testing 5000 s | 11 datasets including SC 2023/2024 families, Picat-generated sets, and industrial EDA [2507.22876] |

AutoSAT reports that a realization of the framework outperforms MiniSat on 9 out of 12 datasets and surpasses Kissat on 4 datasets in the abstract, although its detailed tables in the provided material compare directly against EasySAT and do not report direct MiniSat/Kissat head-to-heads [2402.10705]. On SAT Competition 2022 with $\tau=1000$ s, EasySAT solves 75 instances with PAR-2 = 2541.7, whereas `AutoSAT_bump2_rephase_restart` solves 96 with PAR-2 = 2393.11 [2402.10705]. On CNP, the baseline solves 26 instances with PAR-2 = 926.23, while `AutoSAT_restart` solves 34 with PAR-2 = 504.71; on SCPC, the baseline solves 0 with PAR-2 = 3000, while combined variants solve up to 11 with PAR-2 as low as approximately 663.73 [2402.10705].

SolSearch reports improvements on SAT Competition 2023 benchmarks for CaDiCaL and Kissat-sc2024, and an especially explicit result on the Knights Tour benchmark with Z3: baseline Z3 solves 17 instances; with SolSearch, solved ratio at $\le 100$ s improves from 7.5% to 10.0%, at $\le 300$ s from 35.0% to 42.5%, at $\le 500$ s from 42.5% to 52.5%, and PAR-2 decreases from 660.09 to 583.52, which the paper summarizes as approximately 11% improvement [2502.14328].

The most explicit “AutoModSAT” paper reports 11 datasets, a budget of 50 heuristic proposals, and testing at 5000 s [2507.22876]. Its headline numbers are 50% performance improvement over the ModSAT baseline, 30% superiority against state-of-the-art solvers on average, and 20% average speedup over parameter-tuned alternatives of the state-of-the-art solvers [2507.22876]. Reported examples include `cryptography-ascon`, where AutoModSAT achieves PAR-2 140.72 with 20 solved versus 155.85 for `Kissat-para`; `register-allocation`, where it achieves 1177.8 with 18 solved versus 6321.95 with 8 solved for `Kissat-para`; `hamiltonian`, where it reaches 133.13 with 40 solved versus 296.75 with 39 solved; and `EDA`, where it obtains 376.01 with 49 solved versus 416.42 with 49 solved for `Kissat-para` [2507.22876].

## 5. Automatic prompt optimization and diversity control

A distinctive methodological contribution of the later AutoModSAT work is unsupervised automatic prompt optimization. Whereas AutoSAT and SolSearch use structured prompts with role descriptions, behavioral constraints, and code context [2402.10705; 2502.14328], AutoModSAT explicitly treats the prompt as an object of optimization [2507.22876].

The prompt is decomposed into `Role`, `Goal`, and `Tips`, to which “key code” context is appended. At each iteration, one of these sections is refined, the LLM generates code samples, and the refined prompt is accepted if both output diversity and compilation success improve [2507.22876]. Diversity is measured by computing CodeT5+ embeddings of generated code, clustering them with K-Means++, and evaluating Shannon entropy:
$$
H(X) = -\sum_{i=1}^{K} p_i \log(p_i), \quad p_i=\frac{|C_i|}{N}.
$$
Compilation and execution success rate is used as a second acceptance criterion [2507.22876].

This mechanism is intended to counter two failure modes that are recurrent in program-synthesis loops: “code ossification,” in which the model repeats minor mutations of an already-accepted pattern, and syntactic degeneration, in which diversity is gained only by producing invalid code. AutoSAT’s analysis already suggests that multiple proposals help avoid local optima and code ossification [2402.10705]. The AutoModSAT formulation makes that intuition operational through entropy over embedding clusters plus compile success [2507.22876]. A plausible implication is that prompt search is being used as a surrogate for maintaining mutation diversity in evolutionary program search.

Prompt content remains highly constrained. AutoSAT requires proposals to be localized, syntactically compatible, and different from the original, and supplies runtime feedback in evaluator prompts [2402.10705]. SolSearch fixes the function name and language and forbids undefined dependencies [2502.14328]. AutoModSAT likewise standardizes instructions, disallows new globals, and appends the “key code” needed to keep edits consistent with the caller context and class state [2507.22876]. The convergence of these design choices indicates that prompt engineering in this domain is less about natural-language nuance than about interface discipline and search diversity.

## 6. Relation to automated configuration, solver synthesis, and adjacent SAT/SMT frameworks

AutoModSAT belongs to a broader family of automated reasoning systems that modify solver behavior, but its position within that family is specific. It differs from classical automatic algorithm configuration because the latter usually searches over predeclared parameters or operator selections. Both AutoSAT and AutoModSAT explicitly contrast themselves with this tradition, emphasizing that they can synthesize novel heuristic code rather than merely tune scalar knobs [2402.10705; 2507.22876].

It also differs from SAT modulo theories or DPLL(T)-style SMT frameworks. Monotonic-theory SMT extends SAT solving by coupling CDCL with theory propagation and explanation over theory atoms, using under- and over-approximations for partial assignments and constructive witnesses such as paths, cuts, flows, or trees [1406.0043]. General SMT, in the DPLL(T) account, splits Boolean abstraction from theory reasoning and learns theory lemmas through consistency checks and propagation [1606.04786]. These frameworks enlarge the semantic scope of satisfiability; AutoModSAT instead optimizes the implementation of a solver, typically remaining within propositional CDCL [1406.0043; 1606.04786]. The shared elements are architectural rather than semantic: modular interfaces, conflict explanations, and incremental search.

The name “AutoModSAT” can create ambiguity because “ModSAT” sometimes denotes unrelated uses of “modulo” in SAT or SAT-based compilation. For example, pseudo-Boolean constraints can be translated into SAT through decomposition into PB-Mod constraints with exactness guarantees when the least common multiple of chosen moduli exceeds the coefficient sum [1104.1479]. That line of work uses “modular” in the arithmetic sense of modular congruence. By contrast, AutoModSAT uses “mod” or “modification” in the sense of automated solver modification [2507.22876]. The distinction is substantive: one concerns encoding constraints into SAT; the other concerns synthesizing heuristic code for SAT solvers.

The relation to SolSearch is especially close. SolSearch is described as an LLM-driven, plug-and-play framework that automatically discovers, modifies, and generates SAT-solving code, using curriculum-based trial-and-error and integrating with CaDiCaL, Kissat-sc2024, and Z3 without bespoke APIs [2502.14328]. This suggests that SolSearch and AutoModSAT are not competing conceptual categories but neighboring points in the same design space. The main differences in the provided materials are emphasis: SolSearch foregrounds curriculum learning and solver-agnostic patch selection, while AutoModSAT foregrounds modular solver design, automatic prompt optimization, and presearch plus EA [2502.14328; 2507.22876].

## 7. Limitations, robustness, and future directions

The papers identify several limitations that constrain current AutoModSAT systems. The first is computational cost. Evaluating candidate heuristics requires repeated compilation and execution on nontrivial benchmark suites, often under long timeouts; AutoSAT uses multiple servers with Intel Xeon Gold 6230 CPUs and an AMD EPYC 7763 server with 8× NVIDIA H100 GPUs for LLM inference [2402.10705]. AutoModSAT similarly notes that solver evaluations are expensive and mitigates this by using shorter training timeouts and a budget of 50 heuristic requests [2507.22876].

The second is overfitting and dataset sensitivity. AutoSAT observes that heuristics learned on its 48-instance training set often improved across three competition years, but some variants that excel on 2022 regress on 2018 or 2023, illustrating dataset sensitivity [2402.10705]. AutoModSAT likewise notes risk of dataset-specific optimization, even though its reported performance spans competition, crafted, and industrial datasets [2507.22876]. This suggests that the empirical gains should not be read as evidence of universally superior heuristics; rather, they show that LLM-guided code search can traverse useful regions of heuristic space under realistic distributional structure.

A third limitation is LLM dependence. AutoSAT compares GPT-4, Qwen-72B, and DeepSeek-Coder and reports that GPT-4 generally yields stronger proposals, while Qwen and DeepSeek produce mixed results [2402.10705]. AutoModSAT states that LLM quality and long-context ability matter, especially because complex solvers exceed typical token limits; its answer is modularization, not end-to-end whole-solver synthesis [2507.22876]. SolSearch uses `deepseek-coder` in the reported experiments, but the paper also calls for deeper analysis of how model size, training data, and specialization affect performance [2502.14328].

Maintainability and reproducibility remain open concerns. AutoSAT notes that generated code can be harder to maintain and that versioning and strict logging are required for reproducibility [2402.10705]. AutoModSAT does not provide a public repository in the material provided, and similarly lacks reported seed details [2507.22876]. SolSearch also does not provide hardware specifics or a public code repository in the provided content [2502.14328]. For a methodology centered on code synthesis, this absence is nontrivial because the behavior of the system depends not only on benchmarks and budgets but also on prompts, models, and repair logic.

Future directions in the papers cluster around search-space enlargement, stronger feedback, and cross-domain transfer. AutoSAT proposes richer search spaces, multi-objective optimization over runtime, memory, and clause-database size, cross-domain generalization via instance features, and hybridization with reinforcement learning or bandit selection [2402.10705]. SolSearch explicitly names Max-SAT and QBF as potential target domains [2502.14328]. AutoModSAT suggests transfer to other complex solvers such as MIP, SMT, and CSP, conditioned on careful modularization of heuristics and exposure of key solver state [2507.22876]. This suggests that the lasting significance of AutoModSAT may lie less in any one discovered heuristic than in establishing a reusable methodology for LLM-mediated program search in optimization-intensive systems.

Source: https://www.emergentmind.com/topics/automodsat