PaVeRL-SQL: Partial-Match RL in Text-to-SQL

• The paper introduces a dual-pipeline framework that integrates in-context verbal self-evaluation and chain-of-thought reinforcement learning to enhance Text-to-SQL performance.
• It employs partial-match rewards to provide dense, non-negative feedback, significantly boosting execution accuracy on benchmarks like Spider2.0-SQLite and BIRD.
• The framework demonstrates robust gains in mixed-dialect and low-resource settings, addressing challenges in complex queries and domain-specific business logic.

PaVeRL-SQL is a Text-to-SQL framework that combines Partial-Match Rewards and Verbal Reinforcement Learning to drive self-improvement in reasoning LLMs (RLMs) for Text-to-SQL. It is designed for settings in which current Text-to-SQL methods still suffer from low execution accuracy on industry-scale databases and complex questions involving domain-specific business logic. The framework adopts two pipelines: a newly designed in-context learning framework with group self-evaluation using capable open- and closed-source LLMs as backbones, and a chain-of-thought (CoT) RL pipeline with a small backbone model, OmniSQL-7B, trained with a specially designed reward function and two-stage RL. Reported results indicate state-of-the-art performance on Spider, Spider 2.0, and BIRD, with particular gains on the industrial-level Spider2.0-SQLite benchmark and in mixed-dialect training settings (Hao et al., 8 Sep 2025).

1. Problem setting and design goals

PaVeRL-SQL is motivated by several limitations of existing Text-to-SQL systems in realistic deployment conditions. The framework targets complexity, because real-world queries are longer, more compositional, and loaded with domain-specific logic such as multi-way joins, CTEs, and window functions. It also targets uncertainty, since user questions may be ambiguous or under-specified and the available schema or context at inference may be partial or noisy. Additional design pressures include data sparsity for many domains and less popular SQL dialects, generalization failures when models overfit to specific datasets, resource constraints in on-premises or auditable deployments, and reward sparsity in RL regimes that rely on binary execution signals (Hao et al., 8 Sep 2025).

Within this problem setting, PaVeRL-SQL is intended to provide dense, graded feedback during training through partial-match rewards while also supporting two operational regimes. One regime is resource-rich and relies on in-context, self-evaluative loops with large models; the other is resource-constrained and uses efficient on-prem RL with a small backbone. This suggests that the framework is not only an algorithmic proposal but also a deployment-oriented architecture spanning hosted LLM use and smaller auditable models.

The emphasis on reward sparsity places PaVeRL-SQL in the broader line of RL-enhanced Text-to-SQL research that argues execution-only rewards are too coarse for complex semantic parsing. Closely related work, such as "Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQL," likewise frames Text-to-SQL as a multi-step, reasoning-intensive task and proposes multiple partial rewards with GRPO to encourage intermediate reasoning and better generalization (Pourreza et al., 29 Mar 2025).

2. Two-pipeline architecture

PaVeRL-SQL is organized around two main pipelines with distinct computational assumptions and optimization mechanisms.

Pipeline	Backbone	Main mechanism
Verbal RL	capable open- and closed-source LLMs	in-context learning with group self-evaluation
CoT RL	OmniSQL-7B	specially designed reward function and two-stage RL

The Verbal RL pipeline is described as a "Generate-and-Judge" framework. For each question, it samples up to $K=10$ valid executable SQLs from a backbone LLM, with at most 200 generation attempts and with duplicates and non-executable results filtered. Each candidate is executed to check validity. The same LLM is then prompted, using a scoring prompt, to independently score all $K$ candidate SQLs for their likelihood of fully and precisely answering the question given schema and context. Each candidate’s final score is the mean of 20 scoring passes, and the system returns the candidate with the highest average score, breaking ties randomly (Hao et al., 8 Sep 2025).

This verbal pipeline uses the backbone LLM as both generator and judge of SQL quality. No gradient updates are performed; the procedure is an in-context decision process rather than parameter tuning. The paper characterizes this as simulating a group-preference RL signal, as in GRPO, purely through in-context voting. A plausible implication is that PaVeRL-SQL treats RL not solely as a training-time paradigm but also as a decision-theoretic template that can be approximated through structured self-evaluation.

The CoT RL pipeline is intended for efficient, auditable, on-premises Text-to-SQL generation. It starts with Supervised Fine-Tuning (SFT) Warmup from a well-trained, SFT-primed code model, OmniSQL-7B, derived from Qwen2.5-coder with SynSQL-2.5M data. It then applies Group-Relative Policy Optimization (GRPO): for each prompt consisting of question, schema, and context, the model generates $G$ SQL candidates, executes them, and computes rewards. Training uses a two-stage schedule with restart from the best-performing checkpoint from Stage 1, adaptive learning-rate reset depending on plateauing or fluctuation in accuracy, and cosine annealing, with total epochs bounded by 20 (Hao et al., 8 Sep 2025).

3. Partial-Match Rewards and verbal reinforcement learning

The central technical contribution of PaVeRL-SQL is the use of Partial-Match Rewards as reward shaping for Text-to-SQL. The framework begins from the observation that binary rewards based on execution success are sparse and do not provide useful learning signal when outputs are nearly correct. To address this, it defines two metrics.

The first is Binary Execution Accuracy $(\text{EX}_b)$, which is looser than exact correctness and is considered correct if all target columns are included and at most $\tau$ extra columns are present, with $\tau=5$ by default. The second is Fractional Execution Accuracy $(\text{EX}_f)$, defined for each sample as the proportion of correct columns in the result table, with values in $[0,1]$; each column in the gold table counts as 1 if its values match and 0 otherwise, after row-order and formatting normalization (Hao et al., 8 Sep 2025).

The reward function is specified as

$$R = \begin{cases} 10 \cdot \text{EX}_f, & \text{if SQL executes and returns results;}\ 0.5, & \text{if SQL executes but is incorrect;}\ 0, & \text{if SQL fails to execute.} \end{cases}$$

Two properties are emphasized. First, the reward is dense relative to binary execution accuracy because it provides nonzero signal for partial correctness. Second, the design uses no negative reward, because mixing signs destabilizes RL updates; the paper reports that removing negative rewards improved RL stability (Hao et al., 8 Sep 2025).

Verbal Reinforcement Learning in PaVeRL-SQL refers to the self-evaluating generate-and-score loop in which the LLM both proposes candidate SQL and assesses their quality in context with group-level comparison. The paper states that this simulates RL preference signals without model weight updates and can outperform vanilla zero-shot prompting. This suggests a distinction between reward as an explicit optimization target in CoT RL and reward as an implicit ranking signal in the verbal pipeline.

The use of partial rewards also aligns PaVeRL-SQL with adjacent RL-based Text-to-SQL methods. Reasoning-SQL, for example, uses execution accuracy, AI feedback, syntax check, schema linking, n-gram similarity, and format reward to mitigate reward sparsity, and argues that such signals encourage intermediate reasoning and generalization (Pourreza et al., 29 Mar 2025). PaVeRL-SQL differs in the specific definition of partial-match rewards, especially its table-column-based execution criteria and its combination with verbal self-evaluation.

4. Optimization procedure and training dynamics

The CoT RL pipeline in PaVeRL-SQL uses a GRPO-based objective with PPO-style advantage updates, groupwise relative comparison, and a KL penalty to a reference policy. The paper gives the objective as

$\begin{split} \mathcal{J}_{\text{GRPO}(\theta)} := &\ \mathbb{E}\left[ \frac{1}{G} \sum_{i=1}^G \min\left(r_i A_i, \mathrm{clip}(r_i, 1-\epsilon, 1+\epsilon) A_i\right) \right] \ & - \beta\,\mathrm{D_{KL}(\pi_\theta \,\|\, \pi_\text{ref})} \end{split}$

where

$K$0

is the likelihood ratio for output $K$1, and $K$2 is the groupwise advantage (Hao et al., 8 Sep 2025).

Training proceeds in two stages. Stage 1 uses a standard RL schedule with learning-rate warmup, linear ramp-up, and cosine annealing. Stage 2 restarts from the best-performing checkpoint from Stage 1 rather than necessarily the last checkpoint, resets the learning rate at a higher value or half the previous value depending on whether accuracy has plateaued or fluctuated, and applies cosine annealing again. The reported rationale is that best accuracy is not always at the last epoch, so checkpoint restart can accelerate convergence and prevent overfitting.

The paper also reports ablation observations about inference-time aggregation. Majority voting improves execution accuracy and saturates around group size $K$3; larger group sizes increase cost without comparable performance gains. Learning-curve analysis on Spider, Spider2.0-SQLite, and BIRD is reported to show fast, stable convergence under partial-match rewards (Hao et al., 8 Sep 2025).

A common misconception in RL-based Text-to-SQL is that execution-only supervision is sufficient if enough rollouts are sampled. PaVeRL-SQL explicitly rejects that assumption by adopting non-negative, partial-match rewards. Closely related work reaches a similar conclusion from a different reward design: Reasoning-SQL argues that reward signals must be frequent and granular enough to reinforce partial schema matching, syntax validity, and structural alignment, and it likewise uses GRPO to exploit those shaped rewards (Pourreza et al., 29 Mar 2025).

5. Empirical results on Spider, Spider 2.0, BIRD, and mixed dialects

PaVeRL-SQL is evaluated on Spider, Spider 2.0, Spider2.0-SQLite, BIRD, and SynSQL10K. The main empirical claims focus on execution accuracy gains over existing systems and on robustness under realistic industrial constraints (Hao et al., 8 Sep 2025).

For the Verbal RL pipeline, the paper reports the following results on Spider2.0-SQLite:

Method	Backbone	EX (Spider2.0-SQLite)
PaVeRL (verbal RL)	GPT-5 mini	37.0%
CHESS	GPT-5 mini	29.6%
Agent	GPT-5 mini	24.4%
0-shot	GPT-5 mini	33.1%

The reported gain is 7.4\% higher than SOTA on Spider2.0-SQLite for the verbal-RL pipeline. On BIRD dev subsamples, PaVeRL-Verbal RL is described as competitive or better than baseline SOTA, and the summary table reports 55.8–60.5 EX for PaVeRL (verbal RL), compared with 61.2 for CHESS, 29.9 for Agent, and 55.1 for 0-shot. The paper characterizes the improvements as consistent over zero-shot and prompt-only methods (Hao et al., 8 Sep 2025).

For the CoT RL pipeline, the paper reports:

Model & Training	EX (Spider2.0-SQLite)
PaVeRL (CoT RL, SynSQL10K)	19.3% (Maj. voting)
OmniSQL-7B	17.0%
Arctic-Text2SQL-R1	15.6%

This corresponds to +1.4\% over OmniSQL-7B on Spider2.0-SQLite. On the standard Spider dev set, the paper states that PaVeRL matches or exceeds leading methods, with 81.7 execution accuracy for PaVeRL (CoT RL, SynSQL10K) in the summary table and 81.6 for OmniSQL-7B, while also noting that it achieves over 86% EX with majority voting. On BIRD dev, the summary table gives 60.6 for PaVeRL (CoT RL, SynSQL10K), 63.9 for OmniSQL-7B, and 66.6 for SQL-R1-7B, while the narrative states that PaVeRL matches SOTA and in some cases slightly exceeds recent RL-based systems (Hao et al., 8 Sep 2025).

The mixed-dialect training result is among the most distinctive findings. When mixing high-resource SQLite and low-resource in-house MySQL data, the paper reports that on in-house MySQL, $K$4 improves from 20.1% for OmniSQL-7B to ~75% for PaVeRL-Mixed, described as an approximately threefold gain, while performance on Spider2.0-SQLite is maintained at EX_b~14%. The broader claim is that RL training with mixed SQL dialects yields strong, threefold gains, particularly for dialects with limited training data (Hao et al., 8 Sep 2025).

6. Position within the research landscape

PaVeRL-SQL sits at the intersection of Text-to-SQL, RL for structured generation, and practical industrial evaluation. Its nearest methodological neighbor in the supplied literature is Reasoning-SQL, which also employs GRPO and task-tailored partial rewards for reasoning-enhanced Text-to-SQL. Both frameworks are motivated by reward sparsity and both argue that RL-only or RL-centered training can surpass standard supervised fine-tuning under suitable reward design (Pourreza et al., 29 Mar 2025).

The two works differ in how they instantiate dense supervision. Reasoning-SQL uses a composite reward function with schema-linking, AI feedback, n-gram similarity, syntax check, execution accuracy, and format reward, summed as

$K$5

with weights chosen so that no incorrect query receives a higher reward than a correct one. PaVeRL-SQL instead centers the reward on partial execution matching at the result-table level and combines this with a separate verbal-RL pipeline that performs in-context group self-evaluation without parameter updates. This suggests two complementary interpretations of RL-enhanced Text-to-SQL: one centered on reward decomposition across SQL subskills, and another centered on partial semantic matching plus self-evaluative ranking.

PaVeRL-SQL’s emphasis on mixed SQL dialects also points toward broader SQL generalization problems beyond standard single-dialect Text-to-SQL evaluation. The PARROT benchmark for cross-system SQL translation argues that most existing SQL benchmarks are poorly suited for SQL-to-SQL evaluation because they focus on limited systems and do not capture many dialect-specific syntax rules, functions, and data types (Zhou et al., 27 Sep 2025). Although PARROT targets SQL-to-SQL rather than Text-to-SQL, its findings support the relevance of PaVeRL-SQL’s dialect-mixing result: dialect coverage and dialect transfer are practical bottlenecks rather than peripheral concerns.

A further misconception addressed indirectly by this landscape is that gains on Spider-style benchmarks necessarily transfer to industrial settings. PaVeRL-SQL explicitly distinguishes industrial-level Spider2.0-SQLite from simpler academic settings and reports its strongest headline improvement there. PARROT similarly shows that LLMs achieve lower than 38.53% accuracy on average in realistic cross-system SQL translation, underscoring that industrial SQL competence remains substantially harder than canonical benchmark performance may suggest (Zhou et al., 27 Sep 2025).

7. Practical implications, limitations, and resources

PaVeRL-SQL is presented as a framework for reliable, SOTA Text-to-SQL under realistic industrial constraints. The verbal pipeline is suitable when fine-tuning is infeasible or when hosted LLM inference is available, because it requires no gradient updates and relies on candidate generation plus group self-evaluation. The CoT RL pipeline is suitable when compute is available for RL training and when an auditable, on-premises model is preferred. This suggests that the framework’s main practical significance lies in offering two distinct operating points rather than a single monolithic model (Hao et al., 8 Sep 2025).

The paper also reports several implementation-level observations. Enriching the schema with minimal data profiles, including PK/FK, min/max values, and column frequencies, improves performance for filter and join operations. The use of non-negative rewards is reported to stabilize RL. Majority voting improves accuracy up to approximately 32 candidates, after which returns diminish. These details collectively indicate that PaVeRL-SQL is not limited to reward shaping alone; prompt and schema representation remain important components of performance.

The principal public resource is the code release at https://github.com/PaVeRL-SQL/PaVeRL-SQL. The datasets used include the public benchmarks Spider, Spider2.0, BIRD, and SynSQL-2.5M, as well as an in-house dataset for mixed-dialect evaluation (Hao et al., 8 Sep 2025).

A plausible limitation, based on the reported design, is that the verbal-RL pipeline depends on repeated candidate generation, execution, and repeated scoring passes, while the CoT RL pipeline depends on warm-start quality and RL stability. The paper does not present these as failures, but the architecture itself implies a trade-off between inference-time self-evaluation cost and training-time RL cost. Within the reported results, however, the framework is positioned as a practical method for handling complex queries, domain-specific logic, low-resource dialects, and resource-constrained deployment regimes simultaneously.