SQL-to-Schema Pipelines: Concepts and Methods

• SQL-to-Schema pipelines are algorithmic systems that transform SQL queries and related artifacts into structured schema representations, enabling effective schema linking and query mapping.
• These pipelines integrate methods such as schema linking, mapping, and refinement to mitigate errors and improve semantic parsing in multi-domain database queries.
• They leverage iterative LLM feedback, self-consistency voting, and execution-based verification to enhance SQL generation accuracy and support robust data preparation.

SQL-to-Schema pipelines refer to algorithmic and learning-based systems that transform SQL queries and related artifacts—such as natural-language questions, code, and database outputs—into structured representations of the underlying database schema. These pipelines are now foundational in the text-to-SQL literature, supporting tasks such as schema linking, schema transfer, schema refinement, multi-domain adaptation, automatic pipeline synthesis, and complex query mapping. Over the past five years, several rigorously evaluated approaches have established SQL-to-Schema as both a subfield and a core technical primitive within neural semantic parsing and automated data preparation.

1. Conceptual Foundations and Taxonomy

Traditional text-to-SQL systems employ a pipeline in which a natural-language question $Q$ is mapped to a valid SQL query $q$ using an explicit schema representation $S$ of the target database. The SQL-to-Schema paradigm reverses or expands this process in several ways:

• Schema Linking: Identifying the subset $(T',C')$ of tables and columns in $S$ relevant to $Q$ or the partially generated SQL.
• Schema Mapping: Aligning and transferring SQL queries between heterogeneous schemas while preserving query semantics.
• Schema Refinement: Modifying or renaming schema elements to improve downstream query generation accuracy.
• Pipeline Synthesis: Inferring an entire sequence of data preparation steps from inputs and a target schema.

Prominent approaches formalize these subproblems as either two-step schema-linking (Yang et al., 2024), cross-schema query mapping (Daviran et al., 9 Aug 2025), greedy-constrained schema refinement (Wang et al., 1 May 2026), or by-target pipeline synthesis (Yang et al., 2021). Each of these instantiates a different interpretation of SQL-to-Schema pipelines with unique algorithmic and theoretical properties.

2. Two-Step Schema Linking: SQL-to-Schema Pipeline

The SQL-to-Schema pipeline in (Yang et al., 2024) defines a robust two-step schema-linking methodology aimed at mitigating schema-linking, join, nested, and group-by errors in text-to-SQL tasks:

1. Initial SQL Generation (ISG): The LLM (e.g., CodeLlama-34B or GPT-4) receives the natural-language question $Q$ and the full schema $S_\text{full}$, producing a preliminary SQL query $SQL_0$.
2. SQL-based Schema Extraction (SQL Parse, SP): $SQL_0$ is parsed to extract the precise set of tables $q$0 and columns $q$1 it references: $S$9 This step filters extraneous schema elements, producing a compact linking schema suited for accurate SQL generation.

Iterative Feedback and Voting: The pipeline feeds the reduced schema back into the LLM for further SQL generation, repeating the parse/generate loop up to three times. The most frequent SQL (by exact match) across passes is selected via self-consistency voting.

Formal Metric: Table-recall@4 quantifies whether all gold tables are in the linkage set of size at most 4 (matching Spider query structure), and execution accuracy measures downstream correctness.

3. Cross-Schema Query Mapping and Template Transformation

SQL-Exchange (Daviran et al., 9 Aug 2025) generalizes the SQL-to-Schema pipeline to map SQL queries from a source schema $q$2 to a target schema $q$3. This is realized via:

• Template Abstraction: Extracting the “skeleton” $q$4 of $q$5—a clause and join structure with schema elements abstracted as placeholders.
• Domain Alignment: Computing a bijection $q$6 mapping placeholders to concrete tables and columns in $q$7, constrained by primary/foreign key graphs.
• Query Reconstruction: Filling the template with mappings and sampled constants from $q$8, then validating the reconstructed $q$9 via real execution.
• Evaluation: Metrics include Generation Success, Structural Alignment, Execution Validity, NL Meaningfulness, and SQL–NL Alignment.

The mapping is feasible if there exists a homomorphism between the join graph of $S$0 and the schema graph $S$1, and empirical results show these mapped queries improve the execution accuracy of in-context text-to-SQL prompts on both BIRD and SPIDER benchmarks.

4. Schema Refinement via Constrained Optimization

EGRefine (Wang et al., 1 May 2026) reframes schema selection and manipulation as a constrained optimization problem: find a renaming/refinement function $S$2 over columns $S$3 such that average downstream text-to-SQL execution accuracy is maximized. The constraints are:

• Non-destructive equivalence: Any query over the refined schema must be semantically and operationally equivalent to the original via view-based materialization.
• Column-local non-degradation: No renamed column degrades query execution accuracy on queries referencing it.

Pipeline Phases:

1. Ambiguous-Column Screening: LLM-driven, flagging columns liable to confuse text-to-SQL models.
2. Context-Aware Candidate Generation: LLM proposes renaming candidates, conditioned on context and query examples.
3. Execution-Grounded Verification: For each candidate, create a temporary view-layer schema and empirically verify accuracy using one or more text-to-SQL models; select the best candidate only if it yields a strictly positive improvement above a threshold.
4. View Layer Materialization: Non-destructive schema changes instantiated as SQL views, ensuring all read-only queries on the refined schema have directly equivalent rewrites on the base tables.

Cross-benchmark evaluation establishes that EGRefine achieves robust gains, abstains where improvements are non-discriminable, and supports “refine-once, serve-many” deployment.

5. Full-Schema Versus Selective Schema Pipelines

Recent work has revisited the need for explicit schema linking in light of the capabilities of modern LLMs (Maamari et al., 2024). Empirical analysis demonstrates that state-of-the-art LLMs (e.g., ft-GPT-4o, Gemini 1.5) can absorb large, augmented schemas in their context window and reliably focus on relevant elements during SQL generation, obviating explicit schema filtering when context size permits. Instead, quality is improved via:

• Schema Augmentation: Enhancing column descriptions and adding chain-of-thought prompts.
• Self-Consistency Selection: Generating multiple SQL candidates and selecting the most frequent normalized structure.
• Iterative Correction: Execution-guided LLM feedback to repair generation errors.

On BIRD, the full-schema approach attains 71.83% execution accuracy using ft-GPT-4o, surpassing any pipeline with explicit schema linking steps.

Observed Trade-offs: Schema linking remains advantageous for resource-constrained or small models, but with growing context windows and model reasoning capabilities, its utility diminishes for large-scale LLMs.

6. Pipeline Synthesis in Data Preparation: The By-Target Paradigm

Auto-Pipeline (Yang et al., 2021) generalizes SQL-to-Schema pipelines to end-to-end data preparation, where the goal is to synthesize an operator pipeline from raw tables $S$4 to a target table $S$5, honoring all implicitly extracted schema constraints (FDs, keys, mappings). Probabilistic and RL-based search through a space of operators ($S$6 including joins, group-bys, pivots, etc.) is guided by constraint satisfaction.

• Constraint Extraction: Target table $S$7 is profiled for FDs and key structure; mapping from intermediate outputs to $S$8 is checked at each synthesis step.
• Synthesis Mechanisms: Both beam-search and Deep Q-learning (DQN) agents are used to optimize for pipelines whose output covers schema and functional constraints.
• Empirical Results: Auto-Pipeline achieves 60–70% synthesis success for pipelines up to 10 steps, outperforming by-example synthesis (Yang et al., 2021).

The by-target approach unifies schema reasoning, multi-table/column mapping, and formalizes the “what you see is what you want” interaction for data pipeline induction.

7. Challenges, Limitations, and Open Directions

SQL-to-Schema pipelines encounter inherent challenges:

• Schema Mismatch and Infeasibility: Not all queries or mapping tasks have feasible homomorphic alignments between source and target schemas. When the join graph in the query cannot be matched, the mapping procedure fails gracefully (Daviran et al., 9 Aug 2025).
• Combinatorial Search Spaces: Schema refinement and pipeline synthesis are provably NP-hard under realistic renaming and scope-collision constraints (Wang et al., 1 May 2026), necessitating greedy decomposition and pruning via verification.
• Model-Brittleness and Coverage: Full-schema approaches can overwhelm small models, while schema-linking approaches carry the risk of pruning essential columns or tables (Maamari et al., 2024).
• Error Diagnosis and Safe Refinement: EGRefine’s design guarantees that no refinement worsens execution accuracy at the column-level, but interactions between columns and prompt-level context are only empirically bounded.

Future work prioritizes improved schema-level validation, dynamic prompt adaptation, learned heuristics for aliasing and complex joins, more expressive template-based mappings, and extension to non-relational targets and query modalities. Cross-benchmark transfer, efficient in-context demonstration selection, and the unification of schema reasoning with error correction remain active areas of development.