DivSkill-SQL: Residual Ensemble Optimization

• DivSkill-SQL is a framework that employs residual skill optimization to enhance Text-to-SQL ensemble diversity.
• The method leverages high-level system prompts rather than model fine-tuning to generate complementary SQL candidates.
• Empirical evaluations across SQL dialects show significant accuracy gains and reduced error rates in candidate selection.

DivSkill-SQL is a framework for constructing Text-to-SQL ensembles by optimizing each additional agentic “skill” on the failures left by the current ensemble, rather than relying on stochastic decoding or prompt variation to induce diversity. In this formulation, a skill is a high-level system prompt that shapes reasoning and tool-use behavior, not a change to model parameters. The method is presented as a residual skill optimization procedure that targets candidate-set coverage under a Pass@K objective, and it is implemented without model fine-tuning. Empirically, it is reported to improve selected accuracy on Spider2-Lite across multiple SQL dialects and to transfer across dialects and task formulations, including BIRD-Critic (Zhu et al., 20 May 2026).

1. Problem setting and motivation

DivSkill-SQL addresses a specific bottleneck in Text-to-SQL ensembling: the ceiling imposed by Pass@K, the probability that at least one of $K$ generated candidates is correct. In a standard ensemble pipeline, multiple SQL candidates are generated and a selector chooses one. Even a perfect selector cannot recover a correct query that never appears in the candidate pool. On this view, the central problem is not only candidate ranking but candidate-set coverage.

The framework is motivated by the observation that existing diversity mechanisms—stochastic decoding, prompt variants, or multiple generators—often produce correlated failures. The reported failure correlations include repeated schema mistakes, join errors, and dialect mistakes. In agentic systems, the paper argues that this effect is amplified because randomness introduced early in a long tool-using trajectory tends to propagate into noisy variants of the same failure mode, rather than into genuinely complementary solutions. The practical implication is that Pass@K tends to plateau as additional samples cease to add new capabilities.

DivSkill-SQL therefore reframes ensemble construction as a residual coverage problem. Instead of asking each new agent to be globally stronger over the whole training distribution, it asks the next skill to recover examples that all previously selected skills still fail on. This distinguishes complementary skill acquisition from mere surface-form variation and locates ensemble design within a coverage-maximization perspective (Zhu et al., 20 May 2026).

2. Formal objective and residual optimization

The formalism centers on the success probability of an individual skill and the induced population Pass@K of a skill bank. For a skill $s$, let $p_s(x)\in[0,1]$ denote the probability that one execution of the skill on input $x$ returns a correct SQL query. For a skill bank $A=\{s_1,\ldots,s_K\}$, the paper defines

$$\operatorname{Pass@K}(A)=\mathbb{E}_{x\sim P}\!\left[1-\prod_{j=1}^{K}(1-p_{s_j}(x))\right].$$

This objective makes explicit that ensemble quality depends on the joint coverage of the skill bank. If several skills succeed on the same subset of examples, their marginal utility is limited. Conversely, a skill that succeeds primarily where the existing ensemble fails has high marginal value.

The residual set after selecting $s_1,\ldots,s_{j-1}$ is defined as the training examples on which all previous skills fail. The next skill is then chosen to maximize empirical success on that residual set. The paper’s theoretical claim is that this residual optimization directly targets the marginal contribution of a new skill to Pass@K. The marginal gain of adding skill $s$ to an existing bank $A$ is written as

$$\Delta(s\mid A)=\mathbb{E}_{x\sim P}\!\left[p_s(x)\prod_{s'\in A}(1-p_{s'}(x))\right].$$

Under the formulation in the appendix, greedy residual selection yields the standard monotone submodular approximation guarantee,

$s$0

This guarantee formalizes the paper’s notion of “marginal contribution”: a skill is valuable insofar as it solves residual failure mass not already covered by the current bank. A plausible implication is that the method shifts the objective of prompt optimization from single-agent improvement to incremental ensemble coverage, which is a materially different target than optimizing one prompt for average-case performance (Zhu et al., 20 May 2026).

3. Skill representation, optimization loop, and inference pipeline

In DivSkill-SQL, a skill is a high-level system prompt that determines an agent’s reasoning and tool-use policy, for example whether it should explore the schema first, decompose the question, draft quickly and repair, stress-test joins, or retrieve templates. The paper explicitly states that skills are prompt-level instructions rather than model parameters. The initial seed pool is manually curated and includes: default exploration-and-test, direct_coder, decompose, explore_heavy, conservative, adversarial_checker, template_first, and fast_error_repair.

The optimization procedure is batch-sequential. It starts from the seed pool, samples a subset of training examples, runs the current skill bank, removes examples that are solved, and optimizes the next skill on the remaining failures. In the reported experiments, the reflective prompt optimizer is GEPA. Accepted prompt refinements are committed back into the seed pool. The result is a bank of skill-conditioned agents specialized to different regions of the residual error distribution. The paper emphasizes that the ensemble diversity comes from learned skills rather than from high-temperature sampling, and it reports low-temperature evaluation with a large token budget.

At inference time, the $s$1 learned skill-conditioned agents run in parallel on a test question, each producing a candidate SQL query. The system then deduplicates candidates that execute to the same result. Selection is performed by pairwise LLM-based candidate comparison, with order swapping to reduce position bias, and the final winner is chosen by win count. This preserves the ensemble character of the method: the output is selected from a bank of complementary candidates rather than emitted by a single monolithic generator.

The framework is therefore modular in two senses. First, it decouples complementary behavior from model fine-tuning. Second, it separates candidate generation from candidate selection. The paper later identifies the second separation as a limitation, because improvements in candidate coverage do not automatically eliminate selection errors (Zhu et al., 20 May 2026).

4. Empirical evaluation on Spider2-Lite and BIRD-Critic

The main evaluation is conducted on Spider2-Lite across three dialect subsets: 135 SQLite examples, 207 Snowflake examples, and 209 BigQuery examples. The framework is tested with two base models, Opus-4.6 and GPT-5.4. Skills are optimized on roughly 200 Snowflake-style training examples for Spider2-Lite transfer, and BIRD-mini-dev is used for the BIRD-Critic transfer experiment.

For Opus-4.6, DivSkill-SQL achieves selected accuracy of 64.44 on SQLite, 64.25 on Snowflake, and 64.88 on BigQuery. The strongest ensemble baseline in the reported table has selected accuracy 63.70 on SQLite, 53.14 on Snowflake, and 56.59 on BigQuery. The paper highlights improvements of +11.11 points on Snowflake and +8.29 points on BigQuery over the strongest ensemble baseline. For GPT-5.4, DivSkill-SQL reports selected accuracy of 71.85 on SQLite, 61.84 on Snowflake, and 63.41 on BigQuery, and it is reported to outperform the strongest baseline in each case while improving both Pass@8 and selected accuracy.

On BIRD-Critic, described as a PostgreSQL debugging benchmark derived from BIRD in which the model repairs a buggy SQL query given a natural-language issue description and database context, the method transfers without retraining and improves selected accuracy by +2.64 points, with corresponding gains in both Pass@1 and Pass@8.

Setting	Reported result	Note
Opus-4.6, Snowflake	64.25 selected accuracy	+11.11 over strongest ensemble baseline
Opus-4.6, BigQuery	64.88 selected accuracy	+8.29 over strongest ensemble baseline
GPT-5.4, SQLite	71.85 selected accuracy	Best reported selected accuracy for that model/dialect
BIRD-Critic transfer	+2.64 selected accuracy	No retraining

The paper also states that to match DivSkill-SQL’s Pass@8 coverage, baselines often need several additional passes. This suggests that the reported gains are not solely a consequence of producing more candidates, but of producing candidates with higher complementarity under the same small-$s$2 budget (Zhu et al., 20 May 2026).

5. Transferability, trajectory diversity, and error diagnostics

A central empirical claim of DivSkill-SQL is that skills optimized on one dialect can transfer without retraining across Snowflake, BigQuery, and SQLite. The reported interpretation is that the learned prompts encode higher-level behaviors—schema exploration, decomposition, grain alignment, and error checking—rather than merely memorizing dialect-specific syntax. The same pattern extends to task transfer: skills trained on standard Text-to-SQL are reported to improve BIRD-Critic, which is a debugging formulation rather than from-scratch generation.

The paper’s trajectory analysis is intended to show that the diversity is behavioral rather than superficial. Repeated runs of the same base skill cluster tightly in trajectory similarity, indicating that they tend to produce near-variants of the same reasoning path. In contrast, learned skills spread trajectories across a much wider range. Specific skills such as direct_coder, template_first, exploration-heavy, and decomposition are associated with distinct schema-inspection, drafting, and repair patterns. This is presented as evidence that DivSkill-SQL is not just sampling different surface forms of SQL.

Error diagnostics on Snowflake reinforce this claim. The number of pools containing an invalid-reference candidate drops from 10 to 7, and among solvable pools from 6 to 2. Missing-function hallucinations fall from 6 to 2. The paper summarizes these results as up to 3x fewer hallucinated schema references and function calls. Structural mismatch errors also drop, including wrong distinct usage, wrong window-function usage, and wrong union structure. The stated conclusion is that the gains arise from more reliable complementary skills rather than from injecting random variance into the decoding process.

A neighboring line of work illustrates the distinction. DIVER focuses on robustness through dynamic interactive value linking and evidence reasoning, improving Text-to-SQL performance by automatically generating evidence rather than by optimizing candidate-set complementarity (Nan et al., 12 Feb 2026). DivSkill-SQL instead targets the generation side of ensembling: its primary object is not evidence quality but residual skill coverage. This suggests that the two approaches occupy adjacent but non-identical positions in the Text-to-SQL design space.

6. Ablations, limitations, and significance

The ablations support the residual optimization claim. Repeated sampling from the base skill improves Pass@K only slowly, which the paper interprets as evidence of correlated failures. Optimizing a single base skill improves individual quality but still leaves many residual failures uncovered. A manually diverse seed pool helps somewhat, but the full residual optimization procedure performs best across almost all $s$3, especially at small and moderate $s$4, where each new candidate must cover a fresh failure mode to remain useful.

The framework’s principal acknowledged limitation is that it primarily improves candidate generation, not final selection. The paper reports a noticeable gap between Pass@8 and selected accuracy in some settings, meaning that the correct SQL is often present in the candidate pool but not chosen. The current selector depends on an LLM judge for pairwise comparisons, and this stage scales quadratically with the number of surviving candidates. The authors therefore suggest combining residual skill optimization with stronger verifiers or reward models to improve selection.

Another important clarification concerns what DivSkill-SQL does not do. It does not fine-tune the underlying LLM, and it does not equate diversity with prompt paraphrase. Its notion of diversity is operational: different skills induce different agent trajectories and therefore different regions of candidate-space coverage. This makes the framework a principled approach to Text-to-SQL ensembling under a Pass@K objective rather than an ad hoc collection of prompts.

Within the broader Text-to-SQL literature, DivSkill-SQL is significant because it treats ensemble construction as a greedy residual coverage problem with an explicit submodular-style guarantee and then demonstrates that the resulting skills transfer across dialects and even across task formulations. A plausible implication is that it shifts attention from “how many samples to draw” to “how to make each additional sample solve a new class of failures,” which is a sharper formulation of ensemble utility than conventional diversity heuristics (Zhu et al., 20 May 2026).