Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

KnobCF: Uncertainty-aware Knob Tuning (2407.02803v1)

Published 3 Jul 2024 in cs.DB

Abstract: The knob tuning aims to optimize database performance by searching for the most effective knob configuration under a certain workload. Existing works suffer two significant problems. On the one hand, there exist multiple similar even useless evaluations of knob tuning even with the diverse searching methods because of the different sensitivities of knobs on a certain workload. On the other hand, the single evaluation of knob configurations may bring overestimation or underestimation because of the query uncertainty performance. To solve the above problems, we propose a decoupled query uncertainty-aware knob classifier, called KnobCF, to enhance the knob tuning. Our method has three significant contributions: (1) We propose a novel concept of the uncertainty-aware knob configuration estimation to enhance the knob tuning process. (2) We provide an effective few-shot uncertainty knob estimator without extra time consumption in training data collection, which has a high time efficiency in practical tuning tasks. (3) Our method provides a general framework that could be easily deployed in any knob tuning task because we make no changes to the knob tuners and the database management system. Our experiments on four open-source benchmarks demonstrate that our method effectively reduces useless evaluations and improves the tuning results. Especially in TPCC, our method achieves competitive tuning results with only 60% to 70% time consumption compared to the full workload evaluations.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. HUNTER: an online cloud database hybrid tuning system for personalized requirements. In Proceedings of the 2022 International Conference on Management of Data. 646–659.
  2. ASTROMER-A transformer-based embedding for the representation of light curves. Astronomy & Astrophysics 670 (2023), A54.
  3. Mastering uncertainty in performance estimations of configurable software systems. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering. 684–696.
  4. Neural message passing for quantum chemistry. In International conference on machine learning. PMLR, 1263–1272.
  5. Learning transferable features in deep convolutional neural networks for diagnosing unseen machine conditions. ISA transactions 93 (2019), 341–353.
  6. Approximate Query Answering in Complex Gaussian Mixture Models. In 2019 IEEE International Conference on Big Knowledge (ICBK). IEEE, 81–86.
  7. Jon C Helton and Freddie Joe Davis. 2003. Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems. Reliability Engineering & System Safety 81, 1 (2003), 23–69.
  8. Benjamin Hilprecht and Carsten Binnig. 2022. Zero-shot cost models for out-of-the-box learned cost prediction. Proceedings of the VLDB Endowment 15, 11 (2022), 2361–2374.
  9. Mohammad Hossin and Md Nasir Sulaiman. 2015. A review on evaluation metrics for data classification evaluations. International journal of data mining & knowledge management process 5, 2 (2015), 1.
  10. Too many knobs to tune? towards faster database tuning by pre-selecting important knobs. In 12th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 20).
  11. LlamaTune: Sample-efficient DBMS configuration tuning. arXiv preprint arXiv:2203.05128 (2022).
  12. Nautilus: A Benchmarking Platform for DBMS Knob Tuning. In Proceedings of the Eighth Workshop on Data Management for End-to-End Machine Learning. 72–76.
  13. Mayuresh Kunjir and Shivnath Babu. 2020. Black or white? how to develop an autotuner for memory-based analytics. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. 1667–1683.
  14. GPTuner: A Manual-Reading Database Tuning System via GPT-Guided Bayesian Optimization. Proc. VLDB Endow. 17, 8 (may 2024), 1939–1952.
  15. Qtune: A query-aware database tuning system with deep reinforcement learning. Proceedings of the VLDB Endowment 12, 12 (2019), 2118–2130.
  16. Robust Estimation of Resource Consumption for SQL Queries using Statistical Techniques. Proceedings of the VLDB Endowment 5, 11 (2012).
  17. SMAC3: A versatile Bayesian optimization package for hyperparameter optimization. Journal of Machine Learning Research 23, 54 (2022), 1–9.
  18. A strategy on selecting performance metrics for classifier evaluation. International Journal of Mobile Computing and Multimedia Communications (IJMCMC) 6, 4 (2014), 20–35.
  19. Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems 30 (2017).
  20. Bao: Making Learned Query Optimization Practical. In Proceedings of the 2021 International Conference on Management of Data (Virtual Event China, 2021-06-09). ACM, 1275–1288.
  21. Ryan Marcus and Olga Papaemmanouil. 2019. Plan-Structured Deep Neural Network Models for Query Performance Prediction. Proc. VLDB Endow. 12, 11 (2019), 1733–1746. arXiv:1902.00132 [cs]
  22. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32 (2019).
  23. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015).
  24. Douglas A Reynolds et al. 2009. Gaussian mixture models. Encyclopedia of biometrics 741, 659-663 (2009).
  25. Conditional generative model based predicate-aware query approximation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 8259–8266.
  26. Michael Stein. 1987. Large sample properties of simulations using Latin hypercube sampling. Technometrics 29, 2 (1987), 143–151.
  27. Using probabilistic reasoning to automate software tuning. ACM SIGMETRICS Performance Evaluation Review 32, 1 (2004), 404–405.
  28. Ji Sun and Guoliang Li. 2019. An End-to-End Learning-Based Cost Estimator. Proc. VLDB Endow. 13, 3 (2019), 307–319.
  29. ibtune: Individualized buffer tuning for large-scale cloud databases. Proceedings of the VLDB Endowment 12, 10 (2019), 1221–1234.
  30. Vamsidhar Thummala and Shivnath Babu. 2010. iTuned: a tool for configuring and visualizing database parameters. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of data. 1231–1234.
  31. Immanuel Trummer. 2022. DB-BERT: a Database Tuning Tool that” Reads the Manual”. In Proceedings of the 2022 international conference on management of data. 190–203.
  32. Automatic database management system tuning through large-scale machine learning. In Proceedings of the 2017 ACM international conference on management of data. 1009–1024.
  33. An inquiry into machine learning-based automatic configuration tuning services on real-world database management systems. Proceedings of the VLDB Endowment 14, 7 (2021), 1241–1253.
  34. Ž Vujović et al. 2021. Classification model evaluation metrics. International Journal of Advanced Computer Science and Applications 12, 6 (2021), 599–606.
  35. UDO: universal database optimization using reinforcement learning. arXiv preprint arXiv:2104.01744 (2021).
  36. Industry practice of configuration auto-tuning for cloud applications and services. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1555–1565.
  37. Uncertainty aware query execution time prediction. Proceedings of the VLDB Endowment 7, 14 (2014), 1857–1868.
  38. Cost-based or learning-based? A hybrid query optimizer for query plan selection. Proceedings of the VLDB Endowment 15, 13 (2022), 3924–3936.
  39. Overfitting and underfitting analysis for deep learning based end-to-end communication systems. In 2019 11th international conference on wireless communications and signal processing (WCSP). IEEE, 1–6.
  40. An end-to-end automatic cloud database tuning system using deep reinforcement learning. In Proceedings of the 2019 international conference on management of data. 415–432.
  41. Openfe: Automated feature generation with expert-level performance. In International Conference on Machine Learning. PMLR, 41880–41901.
  42. Facilitating database tuning with hyper-parameter optimization: a comprehensive experimental evaluation. Proceedings of the VLDB Endowment 15, 9 (2022), 1808–1821.
  43. Restune: Resource oriented tuning boosted by meta-learning for cloud databases. In Proceedings of the 2021 international conference on management of data. 2102–2114.
  44. Towards dynamic and safe configuration tuning for cloud databases. In Proceedings of the 2022 International Conference on Management of Data. 631–645.
  45. Zhilu Zhang and Mert Sabuncu. 2018. Generalized cross entropy loss for training deep neural networks with noisy labels. Advances in neural information processing systems 31 (2018).
  46. Automatic Database Knob Tuning: A Survey. IEEE Transactions on Knowledge and Data Engineering 35, 12 (2023), 12470–12490.
  47. QueryFormer: A Tree Transformer Model for Query Plan Representation. Proc. VLDB Endow. 15, 8 (2022), 1658–1670.
  48. Bestconfig: tapping the performance potential of systems via automatic configuration tuning. In Proceedings of the 2017 symposium on cloud computing. 338–350.
  49. BestConfig: tapping the performance potential of systems via automatic configuration tuning. In Proceedings of the 2017 Symposium on Cloud Computing (Santa Clara, California) (SoCC ’17). Association for Computing Machinery, New York, NY, USA, 338–350.

Summary

We haven't generated a summary for this paper yet.