AST-driven Sub-SQL Augmentation

• The paper introduces a method that uses AST perturbations to generate negative SQL examples while maintaining syntactic validity, leading to improved semantic validation.
• The strategy systematically transforms SQL subtrees using rules like operator inversion and identifier substitution, ensuring each change produces semantically distinct queries.
• It integrates into training pipelines for NL2SQL systems, yielding notable gains in AUPRC and AUROC across datasets such as BIRD, Spider, and EHRSQL.

The AST-driven Sub-SQL Augmentation Strategy is a structured, algorithmic approach for generating high-quality, fine-grained negative SQL examples by systematically perturbing the Abstract Syntax Tree (AST) of gold-standard SQL queries. This technique is central to robust semantic validation in text-to-SQL and NL2SQL systems, enabling models to detect not only syntactic but also fine-grained semantic inconsistencies between a user query and a generated SQL statement. AST-driven augmentation underpins both data generation for training semantic validators and the construction of complex reasoning pipelines in LLM-based NL2SQL, as demonstrated in HEROSQL and LearNAT (Qiu et al., 28 Dec 2025, Liao et al., 3 Apr 2025).

1. Formalism and Theoretical Foundation

AST-driven augmentation treats the SQL query as a tree-structured object, where each node in the AST corresponds to an atomic syntactic component: clause nodes (e.g., SELECT, WHERE), operator nodes (e.g., AND, >, AVG), and operand nodes (e.g., table/column names, literals). Given a gold SQL $s^+$ and its AST $$\mathcal{G}_\mathrm{AST} = (V_\mathrm{AST}, E_\mathrm{AST})$$, any subtree rooted at $v \in V_\mathrm{AST}$ corresponds to a valid sub-SQL fragment.

Perturbation is operationalized through a set $\mathcal{T} = \{T_1, \dots, T_K\}$ of AST-level transformation rules. Applying such transformations to targeted subtrees yields perturbed SQL statements, $s^- = T(s^+)$, with the specific aim of introducing semantic errors while preserving syntactic validity (Qiu et al., 28 Dec 2025).

2. Algorithmic Procedure

The AST-driven sub-SQL augmentation workflow consists of three main phases: subtree sampling, transformation, and re-serialization with semantic filtering.

1. Subtree Sampling: The AST is flattened to a node list; a node $v$ is sampled according to a distribution $P_\mathrm{sample}(v)$, typically uniform or schema-aware (biased towards predicates or other node categories).
2. Transformation: One or more transformation rules $T \in \mathcal{T}$ are applied to the selected subtree. Typical rules include:
   • Operator inversion (e.g., changing $>$ to $\leq$, AND to OR)
   • Identifier substitution (replacing a column or table name)
   • Constant replacement (altering a literal)
   • Aggregation mutation (e.g., AVG $\rightarrow$ MAX)
3. Re-serialization and Filtering: The modified subtree is incorporated back into the AST, and the structure is serialized to produce candidate SQL $s^-$. $s^-$ is retained only if its execution result on the target database differs from $s^+$, ensuring the introduced perturbation is semantically significant.

The process is formalized by the following pseudocode (Qiu et al., 28 Dec 2025):

Input: gold SQL s⁺, its AST (V, E), db Output: set of perturbed SQLs D_AST D_AST ← ∅ for i in 1…N_aug: sample v ∈ V with prob P_sample(v) S_sub ← Subtree(v) choose rule T ∈ 𝒯 modify S_sub ← T(S_sub) s⁻ ← Serialize(full AST with S_sub replaced) if Exec(s⁻) ≠ Exec(s⁺): D_AST ← D_AST ∪ {(q, s⁻)} return D_AST

3. Subtree Selection and Transformation Probabilities

The node sampling distribution $P_\mathrm{sample}(v)$ is crucial for the diversity and utility of generated examples. A standard approach is uniform sampling:

$P_\mathrm{sample}(v) = \frac{1}{|V_\mathrm{AST}|}$

Alternatively, researchers may bias the process:

$$P(v)\propto \alpha\cdot \mathbf{1}\{\mathrm{v\ is\ a\ predicate}\} + (1-\alpha)\cdot\mathbf{1}\{\mathrm{else}\}$$

This allows targeted augmentation of specific logical constructs within SQL statements, such as predicates likely to yield challenging negative samples.

4. Illustrative Examples

Consider the following canonical transformations:

• Operator Inversion:

Original:

SELECT name FROM students WHERE age > 20;

Perturbed:

SELECT name FROM students WHERE age ≤ 20;

• Aggregation Mutation:

Original:

SELECT dept, AVG(salary) FROM employees GROUP BY dept;

Perturbed:

SELECT dept, MAX(salary) FROM employees GROUP BY dept;

Both examples preserve syntactic well-formedness but break semantic alignment with the original user question, creating negative samples suitable for model training and evaluation (Qiu et al., 28 Dec 2025).

5. Integration into Training Pipelines

Once negative samples $\{(q_i, s_i^-)\}$ are constructed via AST-driven augmentation, they are incorporated with the gold pairs $\{(q_i, s_i^+)\}$ to compose the complete training set:

$$\mathcal{D}_\mathrm{train} = \mathcal{D}_\mathrm{gold} \cup \mathcal{D}_\mathrm{AST}$$

Semantic validators—typically binary classifiers $f(q, s)$—are then trained using binary cross-entropy loss:

$$\mathcal{L} = -\frac{1}{N} \sum_{i=1}^N \left[ y_i \log \hat y_i + (1-y_i)\log(1-\hat y_i) \right]$$

where $y_i=1$ if $(q_i,s_i)$ is a perturbed sample and $y_i=0$ for gold.

This approach extends beyond the classification task: the model can backpropagate loss at the LP node embedding associated with the mutated AST node, promoting fine-grained error localization (Qiu et al., 28 Dec 2025).

6. Empirical Impact and Comparative Analysis

Ablation studies indicate that removing AST-driven sub-SQL augmentation leads to consistent performance degradation in semantic validation. Reported results include:

Dataset	AUPRC with AST-aug	AUPRC no augmentation	AUPRC Difference
BIRD	67.39%	62.70%	-4.69
Spider	51.92%	48.91%	-3.01
EHRSQL	89.07%	88.62%	-0.45
Spider 2.0	92.59%	90.73%	-1.86

Averaged across benchmarks, the augmentation strategy yields a 3–4% absolute AUPRC gain and a 2–5% absolute AUROC gain (Qiu et al., 28 Dec 2025). This demonstrates the utility of AST-driven sub-SQL augmentation in promoting models’ ability to identify nuanced, fine-grained semantic errors.

7. Significance in Contemporary NL2SQL Systems

AST-driven sub-SQL augmentation is a foundational module in current NL2SQL pipelines for both validation (e.g., HEROSQL) and in LLM-based task decomposition frameworks (e.g., LearNAT). It provides an automated, scalable alternative to manual annotation, enabling dense sampling of semantic errors that inform and regularize model learning. Furthermore, this method delivers samples that are both syntactically correct and challenging in semantic space, supporting the robust evaluation and training of both binary validators and stepwise semantic decomposition architectures (Qiu et al., 28 Dec 2025, Liao et al., 3 Apr 2025). A plausible implication is that continued adaptation of AST-driven methods can address even more subtle schema-level or intent-level misalignments in future text-to-SQL research.