Using deep learning to construct stochastic local search SAT solvers with performance bounds (2309.11452v2)
Abstract: The Boolean Satisfiability problem (SAT) is the most prototypical NP-complete problem and of great practical relevance. One important class of solvers for this problem are stochastic local search (SLS) algorithms that iteratively and randomly update a candidate assignment. Recent breakthrough results in theoretical computer science have established sufficient conditions under which SLS solvers are guaranteed to efficiently solve a SAT instance, provided they have access to suitable "oracles" that provide samples from an instance-specific distribution, exploiting an instance's local structure. Motivated by these results and the well established ability of neural networks to learn common structure in large datasets, in this work, we train oracles using Graph Neural Networks and evaluate them on two SLS solvers on random SAT instances of varying difficulty. We find that access to GNN-based oracles significantly boosts the performance of both solvers, allowing them, on average, to solve 17% more difficult instances (as measured by the ratio between clauses and variables), and to do so in 35% fewer steps, with improvements in the median number of steps of up to a factor of 8. As such, this work bridges formal results from theoretical computer science and practically motivated research on deep learning for constraint satisfaction problems and establishes the promise of purpose-trained SAT solvers with performance guarantees.
- Stephen A. Cook. The complexity of theorem-proving procedures. In Proceedings of the Third Annual ACM Symposium on Theory of Computing, STOC ’71, pages 151–158, New York, NY, USA, 1971. Association for Computing Machinery.
- L. A. Levin. Problems of information transmission. 1973.
- Machine learning methods in solving the boolean satisfiability problem, 2022.
- Graph neural networks and boolean satisfiability, 2017.
- Learning a sat solver from single-bit supervision, 2019.
- Goal-aware neural SAT solver. In 2022 International Joint Conference on Neural Networks (IJCNN). IEEE, jul 2022.
- Guiding high-performance sat solvers with unsat-core predictions, 2019.
- Neural heuristics for sat solving, 2020.
- Jesse Michael Han. Enhancing sat solvers with glue variable predictions, 2020.
- Learning local search heuristics for boolean satisfiability. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
- NLocalSAT: Boosting local search with solution prediction. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization, jul 2020.
- Robin A. Moser. A constructive proof of the lovasz local lemma, 2008.
- A constructive proof of the general lovasz local lemma, 2009.
- Algorithmic and enumerative aspects of the moser-tardos distribution, 2017.
- Beyond the lovasz local lemma: Point to set correlations and their algorithmic applications, 2020.
- Interaction networks for learning about objects, relations and physics, 2016.
- Solving mixed integer programs using neural networks, 2020.
- P. Erdős and L. Lovász. Problems and results on 3-chromatic hypergraphs and some related questions. Infinite and finite sets, 2:609–627, 1975.
- An algorithmic proof of the lovasz local lemma via resampling oracles, 2015.
- T. Schöning. A probabilistic algorithm for k-sat and constraint satisfaction problems. In 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039), pages 410–414, 1999.
- C.H. Papadimitriou. On selecting a satisfying truth assignment. In [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science, pages 163–169, 1991.
- Local search strategies for satisfiability testing. In D. S. Johnson and M. A. Trick, editors, Cliques, Coloring, and Satisfiability: the Second DIMACS Implementation Challenge in Discrete Mathemathics and Theoretical Computer Science, volume 26, pages 521–532. American Mathematical Society, 1996.
- Choosing probability distributions for stochastic local search and the role of make versus break. volume 7317, pages 16–29, 06 2012.
- Relational inductive biases, deep learning, and graph networks, 2018.
- Layer normalization, 2016.
- Abien Fred Agarap. Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375, 2018.
- Robin A. Moser. Exact Algorithms for Constraint Satisfaction Problems. PhD thesis, 2012.
- The Moser-Tardos Resample algorithm: Where is the limit? (an experimental inquiry), pages 159–171.
- Critical behavior in the satisfiability of random boolean expressions. Science, 264(5163):1297–1301, 1994.
- MassimoLauria/cnfgen: CNFgen registered with Zenodo, November 2019.
- On the glucose sat solver. International Journal on Artificial Intelligence Tools, 27:1840001, 02 2018.
- PySAT: A Python toolkit for prototyping with SAT oracles. In SAT, pages 428–437, 2018.
- Satenstein: Automatically building local search sat solvers from components. volume 232, pages 517–524, 01 2009.
- JAX: composable transformations of Python+NumPy programs, 2018.
- Jraph: A library for graph neural networks in jax., 2020.
- Maximilian Kramer (1 paper)
- Paul Boes (10 papers)