Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

LLMTune: Accelerate Database Knob Tuning with Large Language Models (2404.11581v1)

Published 17 Apr 2024 in cs.AI and cs.DB

Abstract: Database knob tuning is a critical challenge in the database community, aiming to optimize knob values to enhance database performance for specific workloads. DBMS often feature hundreds of tunable knobs, posing a significant challenge for DBAs to recommend optimal configurations. Consequently, many machine learning-based tuning methods have been developed to automate this process. Despite the introduction of various optimizers, practical applications have unveiled a new problem: they typically require numerous workload runs to achieve satisfactory performance, a process that is both time-consuming and resource-intensive. This inefficiency largely stems from the optimal configuration often being substantially different from the default setting, necessitating multiple iterations during tuning. Recognizing this, we argue that an effective starting point could significantly reduce redundant exploration in less efficient areas, thereby potentially speeding up the tuning process for the optimizers. Based on this assumption, we introduce LLMTune, a LLM-based configuration generator designed to produce an initial, high-quality configuration for new workloads. These generated configurations can then serve as starting points for various base optimizers, accelerating their tuning processes. To obtain training data for LLMTune's supervised fine-tuning, we have devised a new automatic data generation framework capable of efficiently creating a large number of <workload, configuration> pairs. We have conducted thorough experiments to evaluate LLMTune's effectiveness with different workloads, such as TPC-H and JOB. In comparison to leading methods, LLMTune demonstrates a quicker ability to identify superior configurations. For instance, with the challenging TPC-H workload, our LLMTune achieves a significant 15.6x speed-up ratio in finding the best-performing configurations.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes. arXiv preprint arXiv:2304.09433 (2023).
  2. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
  3. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712 (2023).
  4. Cgptuner: a contextual gaussian process bandit approach for the automatic tuning of it configurations under varying workload conditions. Proceedings of the VLDB Endowment 14, 8 (2021), 1401–1413.
  5. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology (2023).
  6. PaLM: Scaling Language Modeling with Pathways. CoRR abs/2204.02311 (2022). arXiv:2204.02311
  7. Hebo: Heteroscedastic evolutionary bayesian optimisation. arXiv preprint arXiv:2012.03826 (2020), 7.
  8. Tuning database configuration parameters with ituned. Proceedings of the VLDB Endowment 2, 1 (2009), 1246–1257.
  9. Efficient and robust automated machine learning. Advances in neural information processing systems 28 (2015).
  10. WATuning: a workload-aware tuning system with attention-based deep reinforcement learning. Journal of Computer Science and Technology 36, 4 (2021), 741–761.
  11. Benjamin Hilprecht and Carsten Binnig. 2021. One model to rule them all: towards zero-shot learning for databases. arXiv preprint arXiv:2105.00642 (2021).
  12. Sequential model-based optimization for general algorithm configuration. In Learning and Intelligent Optimization: 5th International Conference, LION 5, Rome, Italy, January 17-21, 2011. Selected Papers 5. Springer, 507–523.
  13. Mistral 7B. arXiv preprint arXiv:2310.06825 (2023).
  14. Mistral 7B. arXiv:2310.06825 [cs.CL]
  15. LlamaTune: sample-efficient DBMS configuration tuning. arXiv preprint arXiv:2203.05128 (2022).
  16. Mayuresh Kunjir and Shivnath Babu. 2020. Black or white? how to develop an autotuner for memory-based analytics. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. 1667–1683.
  17. Meghdad Kurmanji and Peter Triantafillou. 2023. Detect, Distill and Update: Learned DB Systems Facing Out of Distribution Data. Proceedings of the ACM on Management of Data 1, 1 (2023), 1–27.
  18. GPTuner: A Manual-Reading Database Tuning System via GPT-Guided Bayesian Optimization. arXiv preprint arXiv:2311.03157 (2023).
  19. How good are query optimizers, really? Proceedings of the VLDB Endowment 9, 3 (2015), 204–215.
  20. Optimizing hyperparameters of support vector machines by genetic algorithms.. In IC-AI, Vol. 74. 82.
  21. Qtune: A query-aware database tuning system with deep reinforcement learning. Proceedings of the VLDB Endowment 12, 12 (2019), 2118–2130.
  22. Resdsql: Decoupling schema linking and skeleton parsing for text-to-sql. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 13067–13075.
  23. Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls. Advances in Neural Information Processing Systems 36 (2024).
  24. SMAC3: A versatile Bayesian optimization package for hyperparameter optimization. The Journal of Machine Learning Research 23, 1 (2022), 2475–2483.
  25. Chatkbqa: A generate-then-retrieve framework for knowledge base question answering with fine-tuned large language models. arXiv preprint arXiv:2310.08975 (2023).
  26. OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL]
  27. The synthetic data vault. In 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA). IEEE, 399–410.
  28. Zero-infinity: Breaking the gpu memory wall for extreme scale deep learning. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 1–14.
  29. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3505–3506.
  30. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950 (2023).
  31. Murray Shanahan. 2024. Talking about large language models. Commun. ACM 67, 2 (2024), 68–79.
  32. Deterministic policy gradient algorithms. In International conference on machine learning. Pmlr, 387–395.
  33. ibtune: Individualized buffer tuning for large-scale cloud databases. Proceedings of the VLDB Endowment 12, 10 (2019), 1221–1234.
  34. Llama 2: Open foundation and fine-tuned chat models, 2023. URL https://arxiv. org/abs/2307.09288 (2023).
  35. Transaction Processing Performance Council (TPC). Year of the specification version. TPC-H Benchmark Specification. Online. Available: http://www.tpc.org/tpch/.
  36. Immanuel Trummer. 2022a. CodexDB: Synthesizing code for query processing from natural language instructions using GPT-3 Codex. Proceedings of the VLDB Endowment 15, 11 (2022), 2921–2928.
  37. Immanuel Trummer. 2022b. DB-BERT: a Database Tuning Tool that” Reads the Manual”. In Proceedings of the 2022 International Conference on Management of Data. 190–203.
  38. Immanuel Trummer. 2023a. Can Large Language Models Predict Data Correlations from Column Names? Proceedings of the VLDB Endowment 16, 13 (2023), 4310–4323.
  39. Immanuel Trummer. 2023b. Demonstrating GPT-DB: Generating Query-Specific and Customizable Code for SQL Processing with GPT-4. Proceedings of the VLDB Endowment 16, 12 (2023), 4098–4101.
  40. Immanuel Trummer. 2023c. From bert to gpt-3 codex: harnessing the potential of very large language models for data management. arXiv preprint arXiv:2306.09339 (2023).
  41. Automatic database management system tuning through large-scale machine learning. In Proceedings of the 2017 ACM international conference on management of data. 1009–1024.
  42. An inquiry into machine learning-based automatic configuration tuning services on real-world database management systems. Proceedings of the VLDB Endowment 14, 7 (2021), 1241–1253.
  43. Attention is all you need. Advances in neural information processing systems 30 (2017).
  44. UDO: universal database optimization using reinforcement learning. arXiv preprint arXiv:2104.01744 (2021).
  45. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019).
  46. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. arXiv preprint arXiv:1809.08887 (2018).
  47. KIEval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Models. arXiv preprint arXiv:2402.15043 (2024).
  48. An end-to-end automatic cloud database tuning system using deep reinforcement learning. In Proceedings of the 2019 International Conference on Management of Data. 415–432.
  49. Facilitating database tuning with hyper-parameter optimization: a comprehensive experimental evaluation. Proceedings of the VLDB Endowment 15, 9 (2022), 1808–1821.
  50. Restune: Resource oriented tuning boosted by meta-learning for cloud databases. In Proceedings of the 2021 international conference on management of data. 2102–2114.
  51. Towards dynamic and safe configuration tuning for cloud databases. In Proceedings of the 2022 International Conference on Management of Data. 631–645.
  52. An Efficient Transfer Learning Based Configuration Adviser for Database Tuning. Proceedings of the VLDB Endowment 17, 3 (2023), 539–552.
  53. Automatic Database Knob Tuning: A Survey. IEEE Transactions on Knowledge and Data Engineering (2023).
  54. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems 36 (2024).
  55. Database meets artificial intelligence: A survey. IEEE Transactions on Knowledge and Data Engineering 34, 3 (2020), 1096–1116.
  56. Llm as dba. arXiv preprint arXiv:2308.05481 (2023).
  57. DB-GPT: Large Language Model Meets Database. Data Science and Engineering (2024), 1–10.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now