SquRL: Adaptive Text-to-SQL Workflows

• SquRL is a reinforcement learning framework that adaptively constructs workflows for Text-to-SQL queries, replacing static pipelines with a dynamic, context-driven policy.
• It integrates a rule-based reward function, dynamic actor masking, and pseudo rewards with GRPO to optimize workflow efficiency and accuracy.
• Empirical results demonstrate SquRL’s superiority over static workflows on benchmarks like Spider and SynSQL, especially for complex, out-of-distribution queries.

SquRL is a reinforcement learning framework for adaptive workflow construction in Text-to-SQL, introduced to address the limitation of single static workflows in real-world settings. The central claim is that Text-to-SQL systems should not require users to select a suitable method through extensive experimentation; instead, they should adaptively construct workflows at inference time. The framework is motivated by theoretical and empirical analysis showing that optimal dynamic policies consistently outperform the best static workflow, with gains driven by heterogeneity across candidate workflows. SquRL combines a rule-based reward function with dynamic actor masking and pseudo rewards, and is trained after an initial supervised fine-tuning stage using GRPO (Group Relative PPO) (Wang et al., 17 Feb 2026).

1. Problem Setting and Conceptual Basis

SquRL casts the decision of “which workflow to run on this Text-to-SQL query?” as a learning problem over workflows rather than as a fixed pipeline-selection problem. In the formulation given in the paper, each state is the full Text-to-SQL context,

$$s = (q, S_{\text{schema}}, K_{\text{ext}}, A_{\text{pool}}),$$

where $q$ is the natural-language question, $S_{\text{schema}}$ the database schema, $K_{\text{ext}}$ any external knowledge, and $A_{\text{pool}}$ the current actor pool. Each action is a complete workflow $W$ chosen from the combinatorial set $\Omega$ of all valid (template, actor-assignment) pairs; equivalently, actions may be viewed as the token sequence $y$ that describes a workflow (Wang et al., 17 Feb 2026).

This framing makes workflow construction itself the object of optimization. The policy $\pi_\theta(a \mid s)$ is a parametric distribution, instantiated as a LLM with parameters $\theta$, over workflows conditioned on context. In its practical implementation, each rollout is treated as a single-step decision with discount factor $q$0, so the optimization target is

$q$1

The motivation for this formulation is explicitly tied to out-of-distribution and long-tail scenarios. Static workflows are described as fundamentally limiting scalability in such cases. A plausible implication is that workflow diversity is not merely a convenience but a structural requirement when query distributions are heterogeneous.

2. Reinforcement-Learning Formulation

The general reinforcement-learning objective is written as

$q$2

with $q$3. Because SquRL treats each rollout as a one-shot decision, the paper specializes this to the single-step case above (Wang et al., 17 Feb 2026).

A central component is the rule-based “real” reward:

$q$4

The five terms are designed to provide a denser signal and to allow early termination.

The individual components are:

• Format reward:

$q$5

• Timeout penalty (5 min cap):

$q$6

• Execution reward:

$q$7

• Result reward:

$q$8

• Time reward:

$q$9

where time is the actual execution time in seconds and timeout is the 5 min threshold in seconds.

The reward decomposition is notable because it does not rely solely on a terminal binary correctness signal. This suggests that SquRL is designed to optimize not only final execution accuracy but also workflow validity and efficiency during training.

3. Dynamic Actor Masking and Pseudo Rewards

SquRL introduces two training mechanisms intended to improve exploration and training efficiency: dynamic actor masking and pseudo rewards (Wang et al., 17 Feb 2026).

Dynamic actor masking is used to prevent the policy from collapsing onto a few high-reward workflows. If the full actor pool contains $S_{\text{schema}}$0 actors $S_{\text{schema}}$1, the method samples a binary mask

$S_{\text{schema}}$2

where $S_{\text{schema}}$3 is the retention rate. The mask induces a reduced available-workflow set

$S_{\text{schema}}$4

This is implemented through a masking operator on action probabilities,

$S_{\text{schema}}$5

leading to the masked policy

$S_{\text{schema}}$6

The effect is to force exploration of workflows that do not rely on the full actor set.

Pseudo rewards are introduced because full execution feedback, especially for complex workflows, can be costly. With probability $S_{\text{schema}}$7, SquRL uses the real reward $S_{\text{schema}}$8. With probability $S_{\text{schema}}$9, a strong LLM compares the candidate workflow $K_{\text{ext}}$0 to a baseline workflow $K_{\text{ext}}$1 and returns a preference judgment, “BETTER” or “NOT_BETTER,” together with a confidence score $K_{\text{ext}}$2. The pseudo reward is

$K_{\text{ext}}$3

The reward actually used for reinforcement learning is

$K_{\text{ext}}$4

where $K_{\text{ext}}$5 is sampled so that $K_{\text{ext}}$6.

Taken together, these mechanisms target two distinct failure modes: over-reliance on a narrow actor subset and the high cost of obtaining execution-based supervision.

4. Training Procedure

Training proceeds in two stages. First, the model is initialized by supervised fine-tuning on known $K_{\text{ext}}$7 pairs so that it learns to output valid workflow structures. Second, SquRL applies GRPO to refine $K_{\text{ext}}$8 using execution or pseudo feedback (Wang et al., 17 Feb 2026).

At a high level, the GRPO procedure samples a batch $K_{\text{ext}}$9. For each sample, it draws a mask $A_{\text{pool}}$0, generates $A_{\text{pool}}$1 candidate workflows from $A_{\text{pool}}$2 under that mask, assigns either real or pseudo reward to each candidate, computes the group mean $A_{\text{pool}}$3 and standard deviation $A_{\text{pool}}$4 of rewards, and normalizes each candidate reward into an advantage

$A_{\text{pool}}$5

The optimization target is the GRPO loss

$A_{\text{pool}}$6

Here

$A_{\text{pool}}$7

$A_{\text{pool}}$8 is the PPO clip, and $A_{\text{pool}}$9 is the KL penalty weight.

The described division of labor between supervised fine-tuning and reinforcement learning is explicit in the ablation analysis: supervised fine-tuning is used to learn valid workflows, whereas reinforcement learning refines the model toward efficiency and accuracy.

5. Empirical Evaluation

The empirical study uses widely-used Text-to-SQL benchmarks: Spider (dev/test), Spider 2.0 (“Lite”), BIRD (dev), and SynSQL (Easy/Moderate/Complex/Highly Complex/All). Static-workflow baselines, all implemented via Squrve, are DIN-SQL, MAC-SQL, LinkAlign, RSL-SQL, and CHESS. The reported metric is execution accuracy (EX) (Wang et al., 17 Feb 2026).

The main quantitative results for SquRL 7B are as follows:

Benchmark	SquRL 7B	Best static
Spider-Dev	86.27%	84.30%
Spider-Test	86.58%	83.74%
BIRD-Dev	67.60%	65.82%
SynSQL–All	64.42%	57.42%

These correspond respectively to gains of $W$0, $W$1, $W$2, and $W$3. The best static workflow on Spider-Dev and SynSQL–All is identified as RSL-SQL.

The gains increase with query complexity. On Highly Complex SynSQL, SquRL-7B achieves 46.51% versus RSL-SQL at 38.54%, a gain of $W$4. On the out-of-distribution Spider 2.0-Lite benchmark, SquRL-7B + Deepseek-R1 achieves 49.18% versus the best static result of 33.09%, a gain of $W$5 (Wang et al., 17 Feb 2026).

These results are presented as evidence that dynamic workflow construction consistently outperforms the best static workflow methods, with especially pronounced gains on complex and out-of-distribution queries. A plausible implication is that adaptive workflow selection matters most when query difficulty and distribution shift increase the variance in workflow effectiveness.

6. Theoretical Result, Ablations, and Interpretation

The paper states a theoretical result as Theorem 1. Let

$W$6

Then $W$7 always, and it is strictly greater whenever two workflows succeed on disjoint query subsets with nonzero probability (Wang et al., 17 Feb 2026).

This theorem formalizes the paper’s core premise: the best fixed workflow is not generally optimal when the query distribution exhibits heterogeneity. The empirical counterpart is an oracle dynamic selector that achieves 81.5% accuracy on SynSQL versus $W$8 for any static pipeline.

The ablation results further specify the contribution of each component.

For dynamic masking, the retention rate is varied over $W$9. With dynamic masking, SquRL remains robust down to $\Omega$0; without masking, performance collapses rapidly as $\Omega$1 shrinks. The paper interprets this as confirmation that masking prevents over-reliance on a small actor subset.

For pseudo rewards, Table 7 reports:

• True only ($\Omega$2): Spider-Dev 82.71%, BIRD-Dev 63.61%
• 10% pseudo ($\Omega$3): Spider-Dev 85.10%, BIRD-Dev 64.47%
• 30% pseudo: Spider-Dev 80.30%, BIRD-Dev 61.50%

The best setting is the 10% pseudo-reward condition; the 30% condition is described as suffering from too much noise.

For supervised fine-tuning and reinforcement learning, Table 8 reports:

• Full SquRL: Acc 83.27%, Format 0.29%, Time 4.41s
• w/o SFT: Acc 78.94%, Format 1.26%, Time 5.26s
• w/o RL: Acc 77.37%, Format 3.87%, Time 10.51s
• w/o both: Acc 18.47%, Format 77.37%, Time 7.19s

The paper’s summary interpretation is explicit: SFT is crucial to learn valid workflows, RL refines toward efficiency and accuracy, dynamic masking broadens exploration, and a modest amount of pseudo reward accelerates training without degrading final performance.

7. Position Within Text-to-SQL Workflow Research

SquRL is situated against a background in which Text-to-SQL has achieved impressive progress but remains difficult to apply effectively in real-world scenarios because of reliance on single static workflows (Wang et al., 17 Feb 2026). Its comparison set is composed of static-workflow methods implemented via Squrve: DIN-SQL, MAC-SQL, LinkAlign, RSL-SQL, and CHESS.

The framework’s distinguishing feature is not a new static pipeline but a learned dynamic policy for workflow construction at inference time. In the paper’s own terms, it empirically verifies that learning a dynamic workflow-selection policy via RL—augmented with rule-based rewards, actor masking, and pseudo rewards—yields consistent and substantial gains over any fixed Text-to-SQL pipeline, particularly on complex and out-of-distribution queries.

From an encyclopedia perspective, SquRL can therefore be characterized as a Text-to-SQL framework that shifts the unit of optimization from SQL generation within a preselected pipeline to adaptive selection and composition of workflows themselves. This suggests a broader methodological distinction between static pipeline design and inference-time policy learning over workflow spaces, with SquRL representing the latter.