SingSQL-LM: In-Domain Text-to-SQL Models

• SingSQL-LM are compact in-domain text-to-SQL models fine-tuned on synthetic data generated by the SING-SQL framework for proprietary schemas.
• The framework automates a two-stage synthetic supervision pipeline using hierarchical schema partitioning and rigorous validation to ensure comprehensive coverage.
• Evaluation reveals that SingSQL-LM significantly outperforms larger models on enterprise benchmarks, achieving notable gains in Soft F1 and Execution Accuracy.

Searching arXiv for the cited SingSQL-LM paper and closely related context. SingSQL-LM is a family of compact, in-domain Text-to-SQL LLMs fine-tuned on synthetic supervision produced by SING-SQL, a fully automated two-stage framework for generating high-quality, high-coverage Text-to-SQL data for any target database without relying on SQL logs or manual annotations. The framework is designed for enterprise settings in which the central requirement is high accuracy on a single proprietary schema rather than broad cross-domain robustness. Its pipeline combines hierarchical schema partitioning, controlled SQL synthesis across multiple complexity levels, LLM-as-a-judge validation, executability checks, automatic repair, and column balancing; the resulting models are released at 1.5B and 3B scales and are reported to achieve strong in-domain generalization on the California Schools subset of BIRD and on synthetic evaluation sets (Caferoğlu et al., 30 Sep 2025).

1. Problem setting and scope

SingSQL-LM is situated in a specific Text-to-SQL regime: enterprise deployment on one target database. The underlying motivation is that Text-to-SQL systems in practice often do not need broad cross-domain generalization; instead, they need high accuracy on one proprietary schema, while organizations also need to evaluate their systems on their own databases. The SING-SQL framework addresses the gap left by synthetic-data approaches that emphasize cross-domain robustness by generating data tailored to a single target schema and by enforcing comprehensive coverage using only schema information (Caferoğlu et al., 30 Sep 2025).

This design choice is consequential because it avoids dependencies on operational SQL logs and manual annotation. The framework requires only the schema, including primary and foreign keys, and when foreign key metadata is incomplete, manual annotations can be added to restore valid join paths. A common misconception in the Text-to-SQL literature is that synthetic supervision is primarily valuable for domain transfer across many schemas; SingSQL-LM instead formalizes a different objective, namely specialization to one database under coverage constraints. This suggests a shift from benchmark-oriented breadth to deployment-oriented depth.

The associated model family is explicitly described as compact and in-domain. Rather than training large general-purpose Text-to-SQL models, the approach fine-tunes Qwen2.5-Coder-Instruct backbones at 1.5B and 3B scale on synthetic data generated for the target database. The reported result is a model line intended to be practical for enterprise adaptation while remaining competitive on official BIRD metrics.

2. Synthetic supervision pipeline in SING-SQL

SING-SQL models a relational schema as a graph whose vertices are tables and whose edges represent foreign key relationships. On top of this graph, it constructs sub-schemas hierarchically. At the table level, it enumerates combinations of tables such that every table in a combination is transitively joinable with the others, while constraining the maximum number of tables per sub-schema to keep contexts realistic and tractable. At the column level, each table contributes connection columns—primary keys and foreign-key-participating columns—together with windowed subsets of non-connection columns obtained by random shuffling and a sliding window of size $w$ and stride $s$ (Caferoğlu et al., 30 Sep 2025).

Formally, for a table $t$, the framework distinguishes $C_{\mathrm{conn}}(t)$ and $C_{\mathrm{non}}(t)$, shuffles $C_{\mathrm{non}}(t)$, forms windowed parts $C_{\mathrm{win}}^i(t)$, and then constructs table-specific subsets of the form $C_{\mathrm{conn}}(t) \cup C_{\mathrm{win}}^i(t)$. The Cartesian product of these subsets across the tables in a joinable table-level schema yields the column-level sub-schemas. The paper names the relevant procedures ConstructSubSchemas(w, s, tc), GenTableLevelSubSchemas(tc), and GenColumnLevelSubSchemas(w, s).

For each column-level sub-schema, SING-SQL adopts SQL-to-Text generation rather than Text-to-SQL generation, motivated by greater reliability for naturalness and alignment. In the published configuration, an LLM—Gemini-2.5-Flash in this work—is prompted to synthesize $N$ SQL queries for each of four complexity levels: Simple, Moderate, Challenging, and Window. The Window level explicitly enforces a distribution of queries using window functions, which the paper notes are often underproduced without guidance. With $N=3$ per level, each sub-schema produces 12 SQL–Text examples.

The framework then applies a quality-aware pipeline. Generated SQL–Text pairs are filtered by LLM-as-a-judge validation for logical validity and semantic alignment relative to the sub-schema. Each SQL query is executed; failures trigger automatic repair via the LLM, and persistent failures are discarded. For valid, executable pairs, the system also generates step-by-step reasoning traces using a divide-and-conquer strategy as additional supervised targets. After an initial synthesis round, column balancing measures per-column usage frequency $s$0 and triggers a second pass focused on underrepresented columns below a threshold $s$1. In the published California Schools configuration, the minimum per-column example count was set to 400, which the paper states effectively eliminates unused columns and produces balanced coverage.

The framework does not add a separate paraphrasing module. Diversity instead arises from hierarchical partitioning, multi-level SQL synthesis, and randomization in column windowing. This is significant because it frames coverage and operator diversity, rather than surface paraphrase diversity, as the main lever for in-domain Text-to-SQL data generation.

3. Model family, training recipe, and data regime

SingSQL-LM comprises four named variants: SingSQL-LM-1.5B-R32, SingSQL-LM-1.5B-R64, SingSQL-LM-3B-R32, and SingSQL-LM-3B-R64. The backbones are Qwen2.5-Coder-1.5B Instruct and Qwen2.5-Coder-3B Instruct. The suffixes “R32” and “R64” denote the LoRA rank used during supervised fine-tuning rather than the number of candidates or any retrieval size (Caferoğlu et al., 30 Sep 2025).

The fine-tuning recipe is specified at the level of optimizer and hardware. Training uses the Unsloth library with a cosine schedule and a 0.1 warm-up ratio. For rank 32, the configuration is LoRA Alpha = 32, learning rate $s$2, effective batch size 8, and 2 epochs. For rank 64, it is LoRA Alpha = 64, learning rate $s$3, effective batch size 8, and 2 epochs. The reported hardware is a single NVIDIA A40 with 48 GB VRAM. Tokenizer and prompt formatting follow base-model defaults; prompts include schema context at inference and optionally few-shots, while fine-tuning uses synthetic SQL–Text pairs produced by SING-SQL, with reasoning traces available.

For the California Schools database from BIRD dev, the pipeline produced 2,249 sub-schemas at $s$4, $s$5, and a maximum of 3 tables per sub-schema. The resulting synthetic dataset contains 39,734 examples including the column-focused second pass. The published split used for evaluation is 34,266 synthetic training examples, 1,124 synthetic development examples, and 1,124 synthetic test examples. The synthetic sets achieve complete column coverage; by comparison, BIRD-Dev leaves 15 columns unused, or 16.85%, whereas SING-SQL achieves 0 unused columns in its splits.

This training regime is notable because it couples parameter-efficient adaptation with a deliberately coverage-oriented synthetic corpus. A plausible implication is that the model family’s competitiveness depends at least as much on the data-generation and balancing process as on backbone choice alone.

4. Context management and inference strategy

A central empirical finding associated with SingSQL-LM concerns context management. The paper compares schema-free fine-tuning, denoted T2S, against fine-tuning with schema context, denoted T2SWS, and evaluates several inference contexts: schema-only; schema plus few-shots; and schema plus few-shots with reasoning. Across these settings, schema-only inference is reported as the most robust, and schema-free fine-tuning combined with schema-only inference yields the strongest performance (Caferoğlu et al., 30 Sep 2025).

The reported ablation for Qwen2.5-Coder-3B is especially explicit. With 8 candidates, T2S and schema-only inference reach 60.67% EX upper bound and 72.35% Soft F1 upper bound at rank 32, and 64.04% EX upper bound and 73.61% Soft F1 upper bound at rank 64. Variants that include few-shot demonstrations at inference perform worse, and the degradation is often more pronounced when reasoning traces are included. This directly contradicts the common assumption that adding demonstrations and explicit reasoning at inference is uniformly beneficial.

The paper’s interpretation is that schema-free fine-tuning allows the model to learn stable SQL patterns and task behavior without overfitting to a specific schema prompt format, while schema-only grounding at inference supplies the relational context needed for the target database. Fine-tuning with schema context yields intermediate performance and can hinder transfer by encouraging reliance on training-time schema phrasing. This suggests that, for compact in-domain Text-to-SQL models, the main role of schema is runtime grounding rather than supervised conditioning during adaptation.

The evaluation protocol also distinguishes upper bound and lower bound performance over multiple candidates. Upper bound is the best achievable score if an ideal selector chose the best candidate among those generated, typically for $s$6; lower bound is the baseline score without selection or refinement. The paper emphasizes that this isolates raw SQL generation quality, rather than full-system self-correction or reranking behavior.

5. Evaluation on BIRD and synthetic benchmarks

Evaluation uses the official BIRD metrics Execution Accuracy (EX) and Soft F1. EX measures the proportion of predicted SQL queries whose execution result exactly matches the gold query’s result. Soft F1 compares overlap between predicted and gold result tables in a precision–recall framework and tolerates benign differences such as column reordering (Caferoğlu et al., 30 Sep 2025).

On the California Schools subset of BIRD, SingSQL-LM-3B-R64 reaches 82.87% Soft F1 upper bound and 73.03% EX upper bound with 32 candidates. The paper reports that this outperforms the best 3B-scale baseline by +16.21 in Soft F1 and +12.36 in EX. At the 1.5B scale, SingSQL-LM-1.5B-R64 reaches 73.82% Soft F1 upper bound and 64.04% EX upper bound with 32 candidates, improving over prior systems at similar scale by +9.30 in Soft F1 and +4.49 in EX.

On synthetic evaluation sets, the same model family is also reported to exceed prior systems by wide margins. At candidate count 8, SingSQL-LM-3B-R64 achieves 75.33% Soft F1 upper bound and 65.21% EX upper bound on SING-Dev, and 72.37% Soft F1 upper bound and 64.08% EX upper bound on SING-Test. SingSQL-LM-1.5B-R64 records 68.05% Soft F1 upper bound and 56.85% EX upper bound on SING-Dev, and 66.59% Soft F1 upper bound and 58.10% EX upper bound on SING-Test. The paper states that these results significantly surpass larger baselines, including 7B systems below 40% Soft F1 and 31% EX on the synthetic sets.

Setting	Soft F1 UB	EX UB
BIRD, SingSQL-LM-3B-R64, 32 candidates	82.87%	73.03%
BIRD, SingSQL-LM-1.5B-R64, 32 candidates	73.82%	64.04%
SING-Dev, SingSQL-LM-3B-R64, 8 candidates	75.33%	65.21%
SING-Test, SingSQL-LM-3B-R64, 8 candidates	72.37%	64.08%

The paper further notes that lower-bound values remain near zero because no refinement or selection is applied. This is an important interpretive constraint: the high upper-bound scores quantify generation potential under best-of-$s$7 evaluation, whereas deployed systems still require a selector or reranker to approximate that potential in practice.

6. Deployment profile, limitations, and broader significance

The operational workflow for SingSQL-LM begins with schema ingestion. The database schema, including primary and foreign keys, is provided; if foreign keys are missing, manual annotations are added so tables are transitively joinable. Partitioning parameters are then chosen, including the maximum number of tables per table-level sub-schema and the sliding-window parameters $s$8 and $s$9 for non-connection columns, after which ConstructSubSchemas(w, s, tc) is run. For each resulting sub-schema, the synthesis configuration generates $t$0 SQL queries per complexity level over the four levels Simple, Moderate, Challenging, and Window. The generated data are filtered by LLM-as-a-judge validation, checked for executability, repaired when possible, and balanced by per-column frequency before final assembly and fine-tuning (Caferoğlu et al., 30 Sep 2025).

At inference time, the recommended configuration is schema-only input, using the full or filtered schema. Few-shots are optional, but the paper reports that they often reduce performance, particularly when reasoning traces are included. Candidate counts from 8 to 32 are used in evaluation, and the paper states that decoding hyperparameters are fixed for consistency rather than ablated. It also explicitly notes that the reported bounds do not include a selection module; enterprises are expected to add a selector, such as execution-based heuristics or a learned reranker, in deployed pipelines.

Several limitations are identified. Validation depends on LLM-as-a-judge and therefore can pass illogical pairs or reject rare operator usage. Coverage gaps may remain for rarely used constructs, including some window functions, despite the explicit Window level. Column partitioning can separate semantically co-dependent fields, and the paper identifies automatic column grouping via LLM-assisted metadata analysis as a future direction. Very large schemas require careful tuning of the maximum tables per sub-schema and of $t$1 and $t$2 to avoid combinatorial explosion. Multilingual generation is not studied. Synthetic data generation uses Gemini-2.5-Flash, and future work is said to include decoding ablations and RL post-training such as GRPO.

The release includes a public repository at https://github.com/HasanAlpCaferoglu/SING-SQL, along with code to construct sub-schemas and generate and validate synthetic SQL–Text pairs, prompt templates, synthetic datasets for California Schools, and SingSQL-LM checkpoints. In a broader research landscape, SingSQL-LM occupies the model-specialization and synthetic-supervision layer of SQL–LLM integration, whereas systems such as iPDB focus on native ML and LLM inferencing inside the relational engine through extended SQL syntax and semantic query optimization (Kumarasinghe et al., 23 Jan 2026). A plausible implication is that these lines of work are complementary: SingSQL-LM addresses how to build accurate in-domain Text-to-SQL models for a target schema, while in-database systems address how learned operators are represented, optimized, and executed within SQL runtimes.

SingSQL-LM therefore represents a database-agnostic but strongly in-domain paradigm: synthetic data are generated with explicit schema coverage guarantees, compact backbones are adapted with LoRA, and schema-free fine-tuning plus schema-only inference emerges as the most robust configuration. Within the evidence reported for California Schools and the associated synthetic benchmarks, it is presented as a practical route to enterprise-grade Text-to-SQL without SQL logs or manual labels.