---
title: 'SynSQL-Think-916K: A Synthetic Text-to-SQL Dataset'
url: https://www.emergentmind.com/topics/synsql-think-916k
type: topic
---

# SynSQL-Think-916K: A Synthetic Text-to-SQL Dataset

SynSQL-Think-916K is a large-scale synthetic dataset constructed for the supervised fine-tuning of small language models (SLMs) on the Text-to-SQL generation task, with a particular focus on chain-of-thought (CoT) reasoning. Derived from the broader SynSQL-2.5M corpus, SynSQL-Think-916K introduces rigorous data curation and annotation schemes, making it a critical resource for boosting the SQL generation performance of SLMs ranging from 0.5B to 1.5B parameters. Its design principles and empirical validation have established new benchmarks in SLM-driven Text-to-SQL modeling, particularly within the SLM-SQL framework [2507.22478].

## 1. Dataset Genesis and Construction Pipeline

SynSQL-Think-916K originates from the SynSQL-2.5M corpus, a synthetic resource produced by the OmniSQL system and characterized by CoT annotations. The derivation of the 916K subset adopts a multi-stage filtration and preprocessing strategy:

- **Initial Filtering**: Examples without the SQL `SELECT` keyword are eliminated, as are those containing comment artifacts (`--`) and those where redundant SQL queries replicate across the CoT.
- **Annotation Cleanup**: Following SQL cleanup, all CoT reasoning steps are wrapped in `<think>...<think>` tokens, and the SQL output is encapsulated in `<answer>...<answer>`. Post-SQL reflections or annotations are strictly removed.
- **Token Length Restriction**: Samples exceeding a total token limit of 7,000 (covering prompt, CoT, and SQL) are pruned to maintain dataset tractability and prevent input truncation during model training.
- **Final Aggregate**: After filtration and deduplication, the dataset contains precisely 916,156 examples, with no held-out split reserved—this partition is devoted exclusively to pre-training via supervised fine-tuning (SFT). Downstream validation leverages external benchmarks (BIRD, Spider) [2507.22478].

| Dataset                   | Model           | Training Method | Size      |
|---------------------------|-----------------|----------------|-----------|
| SynSQL-2.5M               | SQL Gen Model   | SFT            | 2,190,988 |
| SynSQL-Think-916K         | SQL Gen Model   | SFT            | 916,156   |
| BIRD Train                | SQL Gen Model   | RL (GRPO)      | 9,428     |
| SynSQL-Merge-Think-310K   | SQL Merge Model | SFT            | 310,764   |
| BIRD Merge Train          | SQL Merge Model | RL (GRPO)      | 7,159     |

## 2. Data Characteristics and Schema Coverage

SynSQL-Think-916K enforces strict data regularity and logical diversity across its examples:

- **SQL Query Content**: Every instance is a `SELECT` query. Secondary SQL constructs (e.g., `JOIN`, aggregation functions like `SUM`, `COUNT`) appear in ~35% of cases, reflecting a range from simple retrievals to moderate compositional complexity.
- **Schema Diversity**: The dataset amalgamates schemas from SynSQL-2.5M, subsuming both Spider and BIRD references alongside additional synthetic schemas. Data types encompass numerics, strings, dates, and booleans.
- **Complexity Distribution**: Approximately 60% of samples correspond to single-table queries; 40% address multi-table logic. Aggregations and ranking constructs, notably `ORDER BY ... DESC LIMIT`, occur in roughly 25% of cases.
- **Quality Control**: The pipeline guarantees uniqueness and format validity for each SQL/CoT example through automated filtering; however, no further manual annotation alters the content. The quality of reasoning is directly inherited from upstream OmniSQL-generated CoT steps, which are known for their logical coherence [2507.22478].

## 3. Model Training Regimen and Objective Functions

SLM training with SynSQL-Think-916K proceeds via multi-phase objectives and optimization strategies:

- **Supervised Fine-Tuning (SFT)**: The central goal is to minimize the cross-entropy loss across the concatenated CoT and SQL output:
  $$
  \mathcal{L}_\mathrm{SFT} = -\sum_{t=1}^T \log p_\theta(y_t \mid y_{1:t-1}, x)
  $$
  SFT is performed with a learning rate of $2 \times 10^{-5}$, linear decay over 2 epochs, a warm-up for the initial 10% of steps, and an effective batch size of 1,024 tokens.

- **Reinforcement Learning (RL) Post-Training**: Post-SFT, models are further refined using Group Relative Policy Optimization (GRPO), with training on the BIRD dataset. The reward structure comprises:
  - **Execution Accuracy**:
    $$
    R_\mathrm{EX} = \begin{cases}
      1 & \text{if SQL executes correctly} \\
      0 & \text{otherwise}
    \end{cases}
    $$
  - **Format Reward**:
    $$
    R_\mathrm{Format} = \begin{cases}
      1 & \text{if output matches required format} \\
      0 & \text{otherwise}
    \end{cases}
    $$
  - The total reward is $R = R_\mathrm{EX} + 0.1 R_\mathrm{Format}$, optimized via clipped importance ratios in the policy gradient objective:
    $$
    \max_\theta\,\mathbb{E}_{y\sim\pi_\theta}[R(y)]
    $$

## 4. Inference Pipeline and Corrective Self-Consistency

SLM-SQL inference over SynSQL-Think-916K proceeds in two stages:

1. **SQL Generation and Majority Selection**: The model generates $N=64$ candidate CoT + SQL outputs, grouping them by denotation—a set of SQL outputs that result in the same execution result on the database. If one group forms a strict majority (over $N/2$), its SQL candidate is selected.

2. **Merge Revision Model**: Absent a majority, an auxiliary SQL Merge model is prompted with $M=8$ samples. The self-consistency correction cycle re-executes each and chooses the denotation group with maximal frequency.

- **Probabilistic Selection Formalism**:
  $$
  \hat{y} = \arg\max_y \sum_{i} \mathbb{I}\{ y_i\,\text{denotation}=y \}
  $$
This corrective self-consistency approach robustly resolves outlier generations and aligns final SQL predictions with denotational correctness [2507.22478].

## 5. Empirical Evaluation and Impact

Experimental results on the BIRD development set demonstrate the efficacy of SynSQL-Think-916K in SLM-SQL models:

- **Execution Accuracy (EX)**:
  - Qwen2.5-Coder-0.5B-Instruct: 56.87%
  - Qwen2.5-Coder-1.5B-Instruct: 67.08%
- **Ablation Results**:

| Model      | w/o SFT+RL | with SFT+RL+CSC |
|------------|------------|-----------------|
| 0.5B       | 22.14%     | 56.87%          |
| 1.5B       | 52.26%     | 67.08%          |

- SFT on SynSQL-Think-916K yields a +21.9 percentage point improvement for 0.5B models and +8.9 for 1.5B; RL post-training adds +5 and +4 points, respectively. Corrective self-consistency provides a further ≈5 point gain. Cost analysis at 64 samples per inference yields EX of 67.08% at a cost of ~$0.00046 per question.

- **Comparative Significance**: These improvements demonstrate that high-quality synthetic CoT data enable SLMs to substantially close the gap, in both logical reasoning and execution accuracy, relative to far larger LLMs on BIRD and Spider [2507.22478].

## 6. Role in the Broader Text-to-SQL and SynSQL Ecosystem

While SynSQL-Think-916K is leveraged for SLM generation and logical training, the broader SynSQL framework [2604.27261] introduces a distinct avenue: robust, schema-consistent database synthesis for stress-testing Text-to-SQL systems. The design of SynSQL-Think-916K, with diversity in SQL logic and schema coverage, provides a backbone for discriminative model development, complementing efforts to evaluate true semantic generalizability in Text-to-SQL tasks.

Notably, the curation principles in SynSQL-Think-916K—emphasizing unique valid SQL, coverage of compositional complexity, and inheritance of high-quality CoT—directly address the core challenge identified by SynSQL: ensuring that model performance reflects true semantic competence rather than overfitting to single-instance database artifacts.

## 7. Limitations and Directions for Future Research

Key limitations of SynSQL-Think-916K include its exclusive focus on `SELECT` queries, lack of coverage for schema operations such as `INSERT` or `UPDATE`, and finite schema diversity compared to enterprise-scale databases. Future directions include extending the corpus to cover broader SQL operations, curating harder, more diverse schemas, and integrating semi-supervised or active learning paradigms to minimize reliance on synthetic CoT.

In summary, SynSQL-Think-916K is pivotal in advancing the capabilities of compact language models in Text-to-SQL reasoning and generation. Its role as the foundation for SLM-SQL supervised fine-tuning enables SLMs to approximate, and in some metrics rival, much larger models under rigorous benchmark conditions [2507.22478].

Source: https://www.emergentmind.com/topics/synsql-think-916k