2000 character limit reached
Global Benchmark Database (2405.10045v2)
Published 16 May 2024 in cs.DB, cs.AI, and cs.LO
Abstract: This paper presents Global Benchmark Database (GBD), a comprehensive suite of tools for provisioning and sustainably maintaining benchmark instances and their metadata. The availability of benchmark metadata is essential for many tasks in empirical research, e.g., for the data-driven compilation of benchmarks, the domain-specific analysis of runtime experiments, or the instance-specific selection of solvers. In this paper, we introduce the data model of GBD as well as its interfaces and provide examples of how to interact with them. We also demonstrate the integration of custom data sources and explain how to extend GBD with additional problem domains, instance formats and feature extractors.
- Fahiem Bacchus. Maxsat formulas library, 2006. URL: http://www.cs.toronto.edu/maxsat-lib/maxsat-instances/.
- A comprehensive study of k-portfolios of recent SAT solvers. In Intl. Conf. on Theory and Applications of Satisfiability Testing, SAT, pages 1–18, 2022. doi:10.4230/LIPIcs.SAT.2022.2.
- Experiment design and administration for computer clusters for sat-solvers (EDACC). J. Satisf. Boolean Model. Comput., pages 77–82, 2010. doi:10.3233/SAT190078.
- The Satisfiability Modulo Theories Library (SMT-LIB), 2016. URL: www.smt-lib.org.
- Aslib: A benchmark library for algorithm selection. Artif. Intell., 237:41–58, 2016. doi:https://doi.org/10.1016/j.artint.2016.04.003.
- Seeking practical CDCL insights from theoretical SAT benchmarks. In Intl. Joint Conf. on Artificial Intelligence, IJCAI, pages 1300–1308, 2018. doi:10.24963/ijcai.2018/181.
- Roy Thomas Fielding. REST: Architectural Styles and the Design of Network-based Software Architectures. PhD thesis, University of California, Irvine, 2000. URL: http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm.
- SAT Competition 2020. Artificial Intelligence, 2021. doi:10.1016/j.artint.2021.103572.
- Quantified boolean formulas satisfiability library (QBFLIB), 2005. URL: www.qbflib.org.
- MIPLIB 2017: Data-Driven Compilation of the 6th Mixed-Integer Programming Library. Mathematical Programming Computation, pages 443–490, 2021. doi:10.1007/s12532-020-00194-3.
- Effective auxiliary variables via structured reencoding. In Intl. Conf. on Theory and Applications of Satisfiability Testing, SAT, pages 1–19, 2023. doi:10.4230/LIPICS.SAT.2023.11.
- SATLIB: An online resource for research on SAT. SAT 2000, pages 283–292, 2000. URL: https://api.semanticscholar.org/CorpusID:17486963.
- Markus Iser. Benchmark Compilation for SAT Competition 2023. In Proceedings of SAT Competition 2023: Solver, Benchmark and Proof Checker Descriptions, 2023.
- GBD Tools, November 2023. URL: https://github.com/Udopia/gbd, doi:10.5281/zenodo.10213944.
- A problem meta-data library for research in SAT. In Proceedings of Pragmatics of SAT, POS, pages 144–152, 2018. doi:10.29007/gdbb.
- Wes McKinney et al. Data structures for statistical computing in Python. In Proceedings of the 9th Python in Science Conference, pages 51–56, 2010. doi:10.25080/majora-92bf1922-00a.
- Better evaluations by analyzing benchmark structure. In Pragmatics of SAT workshop, POS, 2016. URL: http://www.pragmaticsofsat.org/2016/reg/POS-16_paper_4.pdf.
- Christos H. Papadimitriou. Computational complexity. 1994.
- Dominik Pascal Schreiber. Scalable SAT Solving and its Application. PhD thesis, Karlsruhe Institute of Technology, KIT, 2023. doi:10.5445/IR/1000165224.
- Satex: A web-based framework for SAT experimentation. Electron. Notes Discret. Math., pages 129–149, 2001. doi:https://doi.org/10.1016/S1571-0653(04)00318-X.
- Satzilla: portfolio-based algorithm selection for sat. J. Artif. Int. Res., pages 565–606, 2008. doi:10.5555/1622673.1622687.