PLRTune: Importance Pre-Sampling and LLM-Guided Reinforcement Learning for Automatic Database Tuning

Abstract: Configuration tuning is critical to database performance, yet automatic database tuning remains challenging due to high-dimensional knob spaces, substantial online tuning cost, unreliable textual hints derived from LLMs or community documents, and the difficulty of exploiting the remaining optimization room after initialization. Hence, we propose PLRTune, a staged database tuning system that leverages workload-specific domain knowledge to identify a reduced search space and further optimize within this promising region. First, we develop an importance pre-sampling and reranking strategy to identify the dominant knob subset in a workload-specific manner and derive a compact state representation. Second, we design an execution-guided hint refinement technique to improve the initialization quality of documentation-guided tuning. Finally, we propose a post-tuning refinement stage that leverages Twin Delayed Deep Deterministic Policy Gradient (TD3) to explore the dominant knob subset and further exploit the remaining optimization room. We evaluate PLRTune on MySQL and PostgreSQL across diverse benchmark workloads. Compared with state-of-the-art approaches, PLRTune achieves the best final result on all tested workloads, improving over the corresponding best-performing alternative by 9.50% on average. Moreover, PLRTune reaches the strongest baseline's best performance level 9.03 times faster on average across workloads, demonstrating its practical runtime efficiency without sacrificing final tuning quality.

• The paper introduces a staged pipeline that combines importance pre-sampling, LLM-guided hint refinement, and TD3-based RL to tackle DBMS tuning inefficiencies.
• It leverages a reduced search space with empirical feedback to achieve up to 9.5% performance improvements and 9× faster convergence than previous baselines.
• The approach demonstrates robust cross-hardware generalization and sets a framework for future automated, RL-driven system optimization.

Staged Importance Pre-Sampling and LLM-Guided RL for DBMS Knob Tuning: An Expert Essay on PLRTune

Introduction and Motivation

Automatic tuning of database management system (DBMS) configurations is a long-standing challenge due to the high dimensionality, structural interdependence, and workload sensitivity of exposed knobs. Manual tuning by DBAs is labor-intensive, while contemporary automatic tuners face persistent challenges: sample inefficiency in high-dimensional spaces, limited effectiveness of warm-start strategies, unreliable LLM or community-derived tuning hints, and suboptimal post-initialization exploitation. The "PLRTune: Importance Pre-Sampling and LLM-Guided Reinforcement Learning for Automatic Database Tuning" (2606.14312) addresses these issues via a three-phase, staged tuning architecture that unifies reduced-space prior construction, execution-guided hint refinement, and RL-based final exploitation.

PLRTune System Architecture

PLRTune separates the tuning process into discrete, pipeline stages, each addressing distinct limitations of previous approaches (e.g., cold-start inefficiency, poor context adaptation of textual hints, and incomplete local exploration after warm start).

Figure 2: Overall architecture of PLRTune, emphasizing staged pipelines and shared pool reuse between importance pre-sampling, execution-guided hint refinement, and RL-based post-tuning.

Phase 0: Importance Pre-Sampling and Knob Reranking

PLRTune introduces a representative sample collection phase to identify workload-sensitive dominant knobs and compress state representations. Latin hypercube sampling (LHS) generates a broad, uniform set of configurations. A random forest regressor, trained on observed performance, assigns importance scores to individual knobs to form a workload-specific Top-K set. Concurrently, internal DBMS metrics are compressed via principal component analysis (PCA) to distill a compact, informative subset for RL state representations. This reduces tuning to a minimal, workload-orthogonal parameter and state subspace, side-stepping the curse of dimensionality and mitigating inefficient exploration.

Figure 1: Stability of Top-K important knob identification under increasing pre-sampling, demonstrating consistent convergence after a moderate budget.

Figure 5: Cumulative explained variance curves confirm that a small number of principal components capture nearly all metric variance across MySQL and PostgreSQL.

Phase 1: Execution-Guided Hint Refinement

Raw LLM-derived or documentation-based hints are contextually filtered and adapted using performance feedback. PLRTune collects external textual recommendations, normalizes resource-relative recommendations, and prunes negative- or redundant-impact hints via measured marginal contribution. A BERT-based LLM converts heterogeneous hints to structured parameters, but only empirically validated (by observed improvement) hints are retained in a persistent hint store. Values are further optimized around the suggested regime via a lightweight actor-critic policy, ensuring robust warm-starts focused on empirically verified, workload-relevant parameterizations.

Figure 6: Workflow of Phase 1 combines hint extraction, normalization, empirical prior fusion, and online validation with execution-guided hint pruning.

Phase 2: State-Aware TD3-Based Post-Tuning Refinement

While hint-guided initialization confers early rapid gains, textual sources alone leave substantial remaining optimization untapped. PLRTune further exploits the compact action/state subspace with a Twin Delayed Deep Deterministic Policy Gradient (TD3) agent. Unlike vanilla DDPG or black-box optimizers, TD3 operates only on the reduced Top-K knob set and the PCA-compressed state, leveraging twin critics and delayed actor updates to enhance stability under severely limited online budget. Thus, RL drives continued, workload-informed exploitation, surpassing the plateau often reached by purely documentation-guided tuners.

Figure 3: Structure and feedback loop of TD3-based post-tuning, where only the most influential knobs are updated in the RL loop.

Numerical Results and Empirical Analysis

Rapid Convergence and Final Quality

Empirical studies on MySQL and PostgreSQL with industry-standard benchmark workloads (Sysbench, TPC-H, TPC-C, YCSB) demonstrate:

• PLRTune improves over the strongest prior baseline by 9.50% on average in final objective.
• PLRTune reaches the same best baseline performance 9.03× faster on average.
• For transactional/mixed workloads (TPS/p95), improvement averages ~10.7%; for MySQL/PG Sysbench workloads, specific gains exceed 14%.

Figure 4: Motivating experiment on MySQL shows DB-BERT's performance saturates quickly, while PLRTune's RL-based post-tuning achieves significant additional optimization.

Figure 8: PLRTune exhibits notable early-stage gains after hint refinement, followed by a secondary improvement phase unique among evaluated baselines.

Component and Sensitivity Ablations

• Top-K size choice (default: 50): Large enough for expressivity, small enough for efficiency; performance saturates near Top-50 Figure 9.
• Disabling either reduced-space analysis, hint refinement, or TD3-based post-tuning degrades both convergence time and final performance Figure 10.

Cross-Hardware Generalization

PLRTune's knob relevance prior, derived during sampling on one hardware platform, demonstrates robust efficacy for the same workload on lower- and higher-resource DBMS instances. The system reuses only the dominant knob set, not concrete values, and adapts locally—preserving gains across heterogeneous targets.

Figure 7: PLRTune maintains strong tuning quality and convergence on hardware with varied resource profiles, validating the generality of learned priors.

Theoretical and Practical Implications

PLRTune's staged pipeline substantiates several principles for next-generation automatic DBMS tuning:

• Prioritizing context-sensitive subspaces: Direct, per-workload identification and reduction of parameter space is superior to global or uniform optimization, especially when search budgets are tight.
• Necessity of empirical feedback over textual priors: Blind application of LLM- or documentation-derived recommendations yields poor robustness. Execution-guided filtering is necessary to eliminate negative or irrelevant actions.
• Two-phase learning for sample efficiency: Fast convergence is achieved by bootstrapping with validated hints, immediately followed by targeted RL exploitation; neither stage alone is sufficient.

These principles can frame future DBMS, middleware, and even LLM agent tuning systems facing similar high-dimensional, context-dependent policy search problems.

Conclusion

PLRTune establishes a unified, staged approach for automatic DBMS configuration tuning. By combining importance pre-sampling with RL state-space reduction, execution-guided hint refinement, and state-aware RL-based exploitation, it overcomes the core bottlenecks in sample efficiency and final solution quality. Empirically, PLRTune dominates existing RL and LLM-guided baselines both in convergence time and final objective, across both workload and hardware variation. The modular pipeline, workload-sensitive subspace reduction, and closed-loop hint adaptation frameworks introduced in PLRTune are likely to inform future automated system tuning, LLM-augmented optimization, and efficient RL deployments for large-scale, real-world systems.