FINER-SQL: RL for Text-to-SQL

• FINER-SQL is a reinforcement-learning framework for Text-to-SQL that transforms sparse binary supervision into dense, interpretable execution rewards.
• It combines supervised distillation of reasoning traces with critic-free Group Relative Policy Optimization to train small models effectively.
• The method uses fine-grained rewards across formatting, execution, memory, and atomic SQL structure to enhance performance and deployment efficiency.

Searching arXiv for the cited FINER-SQL paper and closely related Text-to-SQL work. FINER-SQL is a reinforcement-learning framework for Text-to-SQL that targets small LLMs rather than large API-based systems. In its 2026 formulation, it is defined as a scalable and reusable method that replaces sparse binary supervision with fine-grained execution feedback, coupling supervised distillation of reasoning traces with critic-free Group Relative Policy Optimization (GRPO). Its central claim is that dense and interpretable rewards can make 0.5B–3B models competitive on Spider and BIRD while preserving the deployment advantages of small models, including low latency, modest VRAM requirements, and on-premise operation (Hoang et al., 5 May 2026). The name also sits within a broader line of work that treats Text-to-SQL as a problem requiring explicit fine-grained structure, whether at the token/span level, the reasoning-trace level, or the tool-interaction level (Wang et al., 2022, Dai et al., 6 May 2026).

1. Problem setting and conceptual basis

FINER-SQL formulates Text-to-SQL as the generation of an executable SQL query $\mathcal{Y}$ from a natural-language question $q$, schema $S$, and optionally knowledge $K$:

$\mathcal{Y} = f(q, S, K \mid \boldsymbol{\theta})$

The framework is motivated by a specific deployment tension. LLMs can achieve strong Text-to-SQL performance, but the reported costs, latency, and privacy profile are often unsuitable for production settings. The FINER-SQL paper explicitly cites privacy exposure from sending schemas and queries to external APIs, high token costs for complex pipelines, and the hardware burden of large models. It positions small LLMs as an alternative that can run on a single 12–24 GB GPU, but notes that such models exhibit weak reasoning, poor instruction following, and unstable reinforcement learning when only sparse execution rewards are used (Hoang et al., 5 May 2026).

The framework’s diagnosis of the RL failure mode is specific. Conventional Text-to-SQL RL commonly uses a sparse binary execution reward, $R_{\text{exec}}^{\text{sparse}} \in \{0,1\}$, so almost all early rollouts from a weak model receive zero reward. The paper identifies three resulting pathologies: vanishing learning signal, absence of partial credit for structurally close SQL, and unstable optimization driven by rare successes rather than consistent incremental improvement. FINER-SQL is therefore designed to transform discrete correctness into continuous learning through dense reward shaping, without introducing a heavy process reward model or human-annotated step supervision (Hoang et al., 5 May 2026).

A common source of confusion is that the phrase “fine-grained” had already been used in earlier Text-to-SQL research, but with a different emphasis. “Improving Text-to-SQL Semantic Parsing with Fine-grained Query Understanding” defined fine-grained query understanding at the token/span level via a modular NER–NEL–NSP architecture, with role-specific labels such as SELECT_COLUMN, WHERE_COLUMN, and GROUPBY_COLUMN (Wang et al., 2022). FINER-SQL instead relocates fine granularity from lexical-semantic parsing to reward construction over reasoning traces and SQL structure. This shift does not replace earlier fine-grained schema understanding; it changes the primary optimization locus.

2. Two-stage framework

FINER-SQL is organized as a two-stage pipeline: distillation, followed by reinforcement learning (Hoang et al., 5 May 2026).

In the first stage, a Reasoning Bank is built from large teacher models, including DeepSeek-R1, GPT-4o, and Qwen-2.5-72B. For each question–schema pair, teachers are prompted to produce outputs of the form $$\langle q, \text{schema} \rangle \rightarrow \langle \texttt{<think>reasoning</think> SQL} \rangle$$. These multi-teacher traces are used to supervised fine-tune a small backbone so that it learns both executable SQL and a structured reasoning format. The reported SFT loss is

$$\mathcal{L}_{\text{completion}} = -\sum_{j=C+1}^{\tau} \log P_{\theta}(y_j \mid y_{<j}, \chi),$$

where $C$ is the prompt length and $\chi$ is the prompt containing question and schema. The base models are Qwen2.5-Coder 0.5B, 1.5B, and 3B (Hoang et al., 5 May 2026).

In the second stage, the distilled policy is optimized with GRPO under a goal-augmented partially observable Markov decision process:

$q$0

A rollout $q$1 is sampled from policy $q$2, and the optimization target is

$q$3

GRPO operates on groups of sampled rollouts for the same prompt, computing relative advantages from group reward statistics rather than from a learned critic. The paper describes this as critic-free optimization. This design matters because dense but heterogeneous rewards become more usable when normalized relative to sibling samples rather than interpreted in absolute scale (Hoang et al., 5 May 2026).

The training configuration reported for the 3B model uses 32 rollouts per sample, learning rate $q$4, global batch size 32, bfloat16 precision with FlashAttention, and Qwen3-Embedding-0.6B for reasoning embeddings. SFT is run for 2 epochs with learning rate $q$5. Training is reported on 2 × NVIDIA A6000 GPUs, while inference measurements are reported on a single A5000 24GB GPU (Hoang et al., 5 May 2026).

3. Fine-grained execution feedback

The defining feature of FINER-SQL is its composite reward:

$q$6

Each component contributes a different granularity of supervision (Hoang et al., 5 May 2026).

The format reward enforces the output protocol. It is defined as

$q$7

This is directly tied to the paper’s diagnosis that small models have poor instruction following and often emit malformed mixed-format outputs.

The execution reward provides coarse semantic supervision:

$q$8

Unlike binary execution reward, this version distinguishes between syntax/runtime failure and executable but semantically incorrect SQL.

The memory reward aligns reasoning with a bank of verified reasoning traces. For a generated reasoning trace $q$9 and question $S$0, cross-database neighbors are retrieved, their embeddings are averaged into a centroid $S$1, and the reward is

$S$2

where $S$3 is the normalized embedding of the current reasoning and $S$4 is the normalized centroid of retrieved successful traces. This supplies a dense signal even when SQL execution fails (Hoang et al., 5 May 2026).

The atomic reward supplies structural partial credit at the SQL-operator level. Each SQL is parsed into a set of atomic operations, including FROM, JOIN, ON_EQ, SELECT_COL, SELECT_AGG, WHERE_PRED, GROUP_BY, ORDER_BY, and LIMIT. For a predicted SQL $S$5 and a pool of reference SQLs $S$6, the system computes

$S$7

then takes

$S$8

and applies the shaping function

$S$9

to define

$K$0

The purpose is to reward operation-level overlap while avoiding saturation near perfect but still incorrect SQL (Hoang et al., 5 May 2026).

A common misconception is to treat FINER-SQL as a process reward model in the usual sense. The reported framework does not train a separate heavy verifier or require human step annotations. Its dense reward is assembled from execution, AST decomposition, and vector retrieval, which the paper presents as cheaper and more reusable than PRM-style supervision (Hoang et al., 5 May 2026).

4. Reasoning memory, reference enrichment, and inference procedure

The reasoning memory is implemented as a vector database storing reasoning traces whose SQL executed correctly. Each entry contains the reasoning text, its embedding, and metadata such as sample ID and database ID. Memory is initialized from teacher traces, then updated during RL whenever the current policy produces a correct SQL whose reasoning passes length, schema-mention, lexical-diversity, and redundancy filters. Redundancy is controlled by nearest-neighbor cosine similarity thresholding below 0.9 (Hoang et al., 5 May 2026).

Retrieval is explicitly cross-database. For a current question $K$1, retrieved traces are required to come from databases different from $K$2. The paper reports that cross-DB retrieval outperforms same-only and mixed retrieval regimes, and that increasing retrieval top-$K$3 improves execution accuracy up to about $K$4 before diminishing returns set in. This suggests that FINER-SQL is not merely memorizing question-specific templates; it is shaping reasoning toward database-agnostic prototype manifolds, although that implication is inferential rather than explicitly claimed (Hoang et al., 5 May 2026).

Reference enrichment is equally important for the atomic reward. The gold SQL is augmented by sampling $K$5 candidates from the distilled 3B model at temperature $K$6, executing them, and retaining those whose denotation matches the gold. The resulting reference pool contains multiple correct structural realizations of the same intent. This guards the atomic reward against penalizing harmless SQL variation and makes the reward closer in spirit to functionality-aware evaluation methods such as FuncEvalGMN, which also target semantic rather than surface equivalence (Zhan et al., 2024).

At inference time, FINER-SQL is paired with GRAST-SQL, a 0.6B schema ranker used to retrieve the top-30 relevant columns. The model then samples typically 30 candidates, sometimes up to 50, with temperature around 1.0. Each candidate SQL is executed; candidates are grouped by execution result; and the final prediction is selected by majority voting over execution groups. The paper reports that accuracy improves steadily with the number of candidates and saturates beyond roughly 30, while higher temperatures improve diversity and Pass@K relative to greedy decoding (Hoang et al., 5 May 2026).

5. Empirical performance, ablations, and efficiency

On the reported development benchmarks, FINER-SQL 3B achieves 67.73% execution accuracy on BIRD Dev and 85.0% on Spider Dev (Hoang et al., 5 May 2026).

System	BIRD Dev EX	Spider Dev EX
FINER-SQL 3B	67.73%	85.0%
CodeS-15B	58.47%	84.9%
Reasoning-SQL 14B	65.31%	81.4%
SQL-R1 14B	67.10%	86.7%
Alpha-SQL 14B	68.70%	87.0%
ExCoT 70B	68.51%	—

These results are used in the paper to argue that a 3B model can match or closely approach much larger 14B–70B systems on BIRD, and remain competitive on Spider (Hoang et al., 5 May 2026). This suggests that reward design can substitute for some portion of raw scale in Text-to-SQL, though that broader interpretation extends beyond the strictly reported benchmark numbers.

The RL gains over the SFT initialization are monotonic in the published training curves. On BIRD, the 0.5B model improves from 42.9% to 49.3% EX, the 1.5B model from 59.4% to 63.1%, and the 3B model from 63.4% to 67.7%. Pass@K improves in parallel: 55.9% to 66.3% for 0.5B, 75.3% to 80.1% for 1.5B, and 78.3% to 81.6% for 3B (Hoang et al., 5 May 2026).

Ablations isolate the contribution of the dense reward terms. Removing memory reward lowers EX by about 1.8–2.2 points and sharply increases syntax error rate; for the 0.5B model the syntax error rate rises from 19.6% to 31.5%. Removing atomic reward lowers EX by about 2–3.3 points. Removing both produces the largest degradation, including a reported 4.44% EX drop for the 3B model (Hoang et al., 5 May 2026). The paper also reports that memory reward increases Self-BLEU and reduces the mean number of distinct execution groups per query, which it interprets as more stable reasoning and more coherent candidate sets.

The efficiency profile is central to the framework’s practical identity. On BIRD Dev with 30 candidates, FINER-SQL 3B is reported at 5.57 s/sample with about 10 GB VRAM on a single 24GB A5000; the 1.5B and 0.5B variants run at 3.25 s/sample and 2.60 s/sample with about 6 GB and 3 GB VRAM, respectively. By contrast, Alpha-SQL 14B is reported at 1802 s/sample with about 38 GB VRAM, and Alpha-SQL 32B at 2512 s/sample with about 80 GB VRAM; API pipelines such as CHESS+GPT-4 are reported at 118–156 s/sample (Hoang et al., 5 May 2026). Within the evidence provided, FINER-SQL’s empirical identity is therefore inseparable from its deployment economics.

6. Position within the Text-to-SQL literature

FINER-SQL belongs to a broader movement toward explicit intermediate structure in Text-to-SQL, but it occupies a distinct point in that design space. Earlier fine-grained work made semantic roles explicit at the query token/span level. The 2022 NER–NEL–NSP pipeline jointly modeled query and database, used typed entity recognition and neural entity linking, and injected column-role embeddings into a grammar-based parser, reaching 56.8% execution accuracy on the WikiTableQuestions test set and improving compositional generalization on nested queries (Wang et al., 2022). FINER-SQL shares the emphasis on explicit intermediate signals, but its intermediate objects are reasoning traces, retrieved prototypes, and SQL operation sets rather than typed spans.

Other systems addressed large-schema complexity through focused context construction rather than reward design. DFIN-SQL inserted a focused schema stage before DIN-SQL, combining LLM-based table linking with embedding-based column retrieval and reaching 51.69 EX on BIRD dev (Volvovsky et al., 2024). Blar-SQL decomposed schema linking and SQL generation, added schema chunking for 4K-context open-source models, and reported 46.68% execution accuracy on BIRD dev in its best non-trusting configuration (Domínguez et al., 2024). Dubo-SQL pursued fine-tuning and diverse retrieval-augmented prompting, reporting 60.71% EX on BIRD test for v1 and 61.47% EX on BIRD dev for v2 (Thorpe et al., 2024). These methods are orthogonal to FINER-SQL: they mostly attack schema exposure, prompting, or retrieval, whereas FINER-SQL attacks optimization signal.

Reasoning-centric and RL-centric contemporaries provide an even closer backdrop. STaR-SQL reframed Text-to-SQL as rationale-plus-SQL generation and used an outcome-supervised reward model to rank best-of-$K$7 candidates, reaching 86.6% execution accuracy on Spider dev with ORM@16 (He et al., 19 Feb 2025). FineStep, in turn, treated tool-integrated Text-to-SQL as sequential decision making and introduced step-level credit assignment with independent process rewards, improving average EX by 3.25% over GRPO at the 4B scale (Dai et al., 6 May 2026). Relative to these systems, FINER-SQL retains critic-free RL but avoids tool-step supervision and instead densifies reward through reasoning-memory alignment and SQL-atomic overlap (Hoang et al., 5 May 2026, Dai et al., 6 May 2026).

Evaluation research also clarifies FINER-SQL’s design. FuncEvalGMN argued that exact-match and execution metrics are insufficient proxies for functional correctness, and proposed a graph-based equivalence predictor over optimized relational operator trees (Zhan et al., 2024). FINER-SQL’s atomic reward is not an equivalence metric of that kind, but it similarly rejects a strict string-level notion of correctness and awards partial structural credit. A plausible implication is that future FINER-SQL-style systems could replace or augment Jaccard-over-atoms with learned functionality-aware metrics, though this extension is not reported in the paper.

In summary, FINER-SQL denotes a specific 2026 framework for boosting small Text-to-SQL models by dense execution-grounded reinforcement learning. Its distinctive contribution is not schema serialization, prompt decomposition, or stepwise tool use in isolation, but the conversion of weak, sparse success signals into a composite reward over formatting, execution, reasoning-memory alignment, and operation-level SQL overlap. Within the supplied literature, it represents a convergence point between fine-grained semantic structuring and deployment-oriented small-model optimization (Hoang et al., 5 May 2026).