A Case for Agentic Tuning: From Documentation to Action in PostgreSQL

Abstract: Documentation has long guided computer system tuning by distilling expert knowledge into per-parameter recommendations. Yet such guides capture only what experts conclude, discarding how they reason. This fundamental gap manifests in three concrete deficiencies: documentation grows stale as software evolves, fails under heterogeneous workloads, and ignores inter-parameter dependencies. We propose shifting from static documentation to dynamic action for system tuning. We introduce PerfEvolve, which translates expert tuning methodologies into executable skills that equip LLM-based agents to perform version-consistency verification, workload-specific profiling, and multi-parameter joint optimization. Evaluated on PostgreSQL under TPC-C and TPC-H benchmarks, PerfEvolve outperforms state-of-the-art documentation-driven tuning baselines by up to 35.2%. The tool is available at https://github.com/ISCAS-OSLab/PerfEvolve.

• The paper introduces PerfEvolve, a two-phase method that profiles 116 PostgreSQL parameters, discovers interactions with factorial ANOVA, and compiles findings into 23 executable skills covering 160 parameter profiles.
• The paper shows that documentation-based tuning can harm performance: PostgreSQL guidelines and PGTune degraded 7 of 22 TPC-H queries by over 10%, while workload-specific configurations improved results by up to 35.2% over defaults.
• The paper demonstrates that procedural knowledge outperforms declarative recommendations, raising trial validity from 68% to 100% and recovering up to 58.9% of cross-hardware performance loss, though recalibration remains necessary for major hardware or workload changes.

Motivation: documentation as a flawed tuning oracle

PerfEvolve addresses a structural weakness in automated system tuning: the reliance on static documentation as the knowledge source for LLM-based tuners. Systems such as GPTuner ingest official manuals and community guides, use LLMs to extract parameter semantics and value constraints, and inject this into Bayesian optimization. The authors argue that this paradigm rests on an unexamined assumption—that documentation is sufficiently correct—and that when it is not, tuning quality is bounded by documentation quality.

Through a survey of Linux, PostgreSQL, MySQL, and Apache Kafka documentation, the paper identifies three recurring deficiencies:

• Staleness: recommendations calibrated to obsolete hardware or software assumptions, e.g., PostgreSQL's random_page_cost = 4.0 (HDD-era) versus ~1.1 optimal on SSDs.
• Context insensitivity: generic defaults with no conditioning on workload type, storage medium, or read/write ratio; e.g., the canonical shared_buffers = 25% of RAM rule.
• Correlation absence: parameters documented in isolation despite pervasive interactions; the shared_buffers × work_mem interaction alone explains 39% of performance variance (ANOVA $\eta^2$).

The central insight is that these share one root cause: documentation records results (what experts concluded) rather than process (how they concluded it). A result is an environment-specific snapshot that goes stale; a process is a reproducible methodology that re-derives the answer for any target environment.

Empirical characterization on PostgreSQL

The paper grounds its claims in a 150-VM study of PostgreSQL v16 (8 vCPU, 8 GB DRAM, SSD) under TPC-C-r, TPC-C-w, and TPC-H workloads, yielding three findings:

Established best practices can be actively harmful. Both PG-Official rules and PGTune degrade 7 of 22 TPC-H queries by more than 10%, inflating sort/aggregation latency by 12–27% due to a memory allocation mismatch (shared_buffers = 2 GB shrinking page cache while work_mem = 4 MB forces disk spills). Similarly, the recommended checkpoint_completion_target = 0.9 loses by 15–24% against workload-specific optima on SSDs.

No universally optimal configuration exists. A configuration improving TPC-C-r throughput by +35.2% degrades TPC-C-w by −4.3% and TPC-H by up to −18.6%; the gap between workload-specific optima reaches 48.6%.

Correlations are the norm, not the exception. Across all 55 pairs of 11 high-sensitivity parameters, 60% exceed a 15% interaction threshold, with the strongest pair (commit_delay × cpu_operator_cost) reaching 96%. Notably, 8 of the top-10 interactions span different PostgreSQL subsystems—precisely the relationships documentation omits.

These findings carry a direct implication: any tuner consuming raw documentation inherits stale values, ignores context, and assumes independence, capping achievable performance regardless of search algorithm quality.

PerfEvolve design

PerfEvolve operationalizes procedural knowledge through a two-phase workflow producing executable skills for LLM agents:

Dimensionality reduction. A fully parallel single-parameter sweep over PostgreSQL's 116 performance-relevant parameters (~6,297 runs, completed in one day on the cluster) computes per-parameter sensitivity via coefficient of variation. The distribution exhibits a pronounced long tail: only 15 of 116 parameters exceed the 5% threshold, reducing the downstream search space by an order of magnitude. The sweep also yields empirical safe ranges—which eliminate the unsafe configurations responsible for GPTuner's observed 32% crash rate—and response-curve shape classifications that inform search strategy.

Correlation topology discovery. A two-stage factorial ANOVA screens all $\binom{k}{2}$ pairs: a liberal $2\times2$ coarse screen (intentionally maximizing recall, since $n=1$ designs overestimate interaction strength by roughly 2×), followed by a $4\times4$ design with Benjamini-Hochberg FDR correction retaining pairs with $\eta^2 > 0.15$ and corrected $p < 0.05$. The resulting graph decomposes into low-dimensional connected components—the largest contains only 3–4 parameters—reducing joint optimization from an intractable $4^{15}$ global space to small sub-problems such as 192 runs for a 3-parameter component.

Procedural document generation. Outputs are compiled into a DAG of skills (preconditions, procedures, decision criteria, postconditions, reference data). For PostgreSQL v16, the document comprises 23 skills and 160 parameter profiles. The agent operates either in full procedural execution mode or in knowledge-injection mode, exporting constraints to existing frameworks like GPTuner's SMAC optimizer without algorithm changes.

Evaluation results

Evaluated against GPTuner and E2ETune on two testbeds, PerfEvolve delivers several strong results:

Setting	Result
Matched environment (VM cluster)	Up to +35.2% over default PostgreSQL in 30 trials; trial validity raised from 68% to 100%
Cross-hardware transfer (96× core gap, 128× memory gap)	GPTuner-Original collapses to −54% to −56%; with PerfEvolve knowledge, degradation recovered by up to 58.9%

The most instructive experiment isolates knowledge form while controlling information quantity. Injected into E2ETune, even correct declarative values degrade performance relative to no knowledge at all (−0.4% to −1.9%), because the LLM anchors on injected numbers and reduces exploration; incorrect values are catastrophic on TPC-H (−8.3%). In contrast, procedural structure alone—with no calibration data—outperforms every declarative variant on TPC-H (+5.3%), and full PerfEvolve knowledge achieves the best results across all workloads (+8.4%, +12.7%, +35.2%). This directly validates the paper's thesis that the form of knowledge matters more than specific recommendations.

The ablation confirms complementarity: sensitivity ranking dominates read-heavy OLTP (+9.5% on TPC-C-r alone), correlation knowledge dominates write-heavy OLTP (+13.2% on TPC-C-w), and full knowledge is most robust. The single most direct evidence for topology discovery is the shared_buffers × checkpoint_completion_target case: independent tuning of each parameter yields −13.4% combined, whereas joint 2D optimization yields +35.2%—a 48.6% gap attributable solely to whether correlation was modeled.

Cost analysis reveals discrete stage-level returns: Stage 1 (dimensionality reduction, 57% of budget) yields +5%; Stage 2 (correlation screening, 32%) brings cumulative gain to +12.4%; Stage 3 (joint optimization, only 11%) adds +22.8%, enabled by topology-aware decomposition.

Limitations and open questions

The paper is candid about several constraints. First, reference data is hardware-specific: on the 192-core server, PerfEvolve knowledge fails to improve write-heavy OLTP (−10.6% for GPTuner+PerfEvolve), because memory pressure characteristics differ fundamentally from the 8 GB profiling VMs. The methodology transfers; the empirical ranges do not, and substantial hardware divergence requires recalibration. Second, the approach assumes representative offline workloads; workload drift activating previously irrelevant subsystems can leave the skill set incomplete, necessitating monitoring and periodic re-profiling. Third, the analysis covers pairwise and low-order interactions only, potentially missing higher-order effects invisible to pairwise screening. Fourth, skill structure reduces but does not eliminate agent reliability failures—misread measurements, premature termination, misapplied decision rules remain possible, so PerfEvolve is a structured control layer rather than a guarantee of autonomous correctness. Finally, the ~11,000-run offline cost suits mature, stable software stacks better than rapidly evolving ones, and extension beyond DBMS may require stronger rollback and sandboxing given harsher failure modes.

Conclusion

This paper makes a well-supported case that the binding constraint on documentation-driven LLM tuning is not the search algorithm but the form of the knowledge consumed. By systematically characterizing staleness, context insensitivity, and correlation absence, and by demonstrating that procedural knowledge outperforms—even correct—declarative knowledge under controlled injection, the authors substantiate their proposal to encode tuning methodology as executable, verifiable skills. The cross-framework gains (improving both a BO-based and a fine-tuned generative tuner without algorithm modification) indicate the contribution lies in the knowledge representation itself. Open questions remain around higher-order interaction discovery, hardware-generalizable reference data, and robustness under workload drift.