MCI-SQL: Text-to-SQL with Metadata-Complete Context and Intermediate Correction

Abstract: Text-to-SQL aims to translate natural language queries into SQL statements. Existing methods typically follow a pipeline of pre-processing, schema linking, candidate SQL generation, SQL alignment, and target SQL selection. However, these methods face significant challenges. First, they often struggle with column filtering during schema linking due to difficulties in comprehending raw metadata. Also, the candidate SQL generation process often suffers from reasoning errors, which limits accuracy improvements. To address these limitations, we propose a framework, called MCI-SQL, to efficiently and precisely generate SQL queries. Specifically, we assign metadata-complete contexts to each column, which significantly improves the accuracy of column filtering for schema linking. Also, for candidate SQL generation, we propose an intermediate correction mechanism that validates SQL queries and revises errors in a timely way. Moreover, we also propose effective optimizations in subsequent SQL alignment and selection phases, which further enhance the performance. Experiments on the widely-used BIRD benchmark show that MCI-SQL achieves execution accuracy of 74.45% on the development set and 76.41% on the test set, surpassing current published state-of-the-art results. In addition, we manually identify and correct 412 samples in the BIRD dataset, forming a new version named BIRD-clear, which is released together with our code on GitHub. We also evaluate our methods on BIRD-clear and find that MCI-SQL outperforms baselines by 8.47 percentage points in execution accuracy, further demonstrating the effectiveness and reliability of our framework.

• The paper introduces MCI-SQL, a prompt-based framework that combines metadata-complete schema context, draft-SQL schema linking, and execution-guided correction to improve database query generation.
• MCI-SQL achieves 74.45% execution accuracy on BIRD development, 76.41% on test, and 88.7% on Spider test, while using nine candidates compared with larger ensemble-based systems.
• The paper releases BIRD-clear, a manually corrected BIRD subset showing that annotation errors can substantially change results, with MCI-SQL improving from 44.66% to 65.78% on 412 corrected samples.

MCI-SQL is a prompt-based, LLM-driven Text-to-SQL framework that addresses two persistent failure modes in existing pipelines: poor column filtering caused by incomplete database metadata, and late-stage error correction that ignores reasoning details during SQL generation. The framework achieves 74.45% execution accuracy (EX) on the BIRD development set and 76.41% on the test set, and the authors additionally release BIRD-clear, a manually corrected version of the BIRD dev set in which 412 annotation errors were fixed (2603.13390).

Motivation and problem setting

The paper targets the standard five-phase prompt-based Text-to-SQL workflow: pre-processing, schema linking, candidate SQL generation, SQL alignment, and SQL selection. The authors identify two bottlenecks. First, prior systems (DIN-SQL, MAC-SQL, OpenSearch-SQL, among others) make limited use of database metadata, producing a marked gap between LLMs and domain experts in column filtering. Second, correction is typically performed post hoc by a separate refinement module that consumes only the final erroneous SQL and execution errors, discarding intermediate reasoning state. MCI-SQL's design responds directly to both: it constructs rich per-column contexts before schema linking, and embeds validation-and-correction inside the generation loop itself.

Metadata-complete context

The central pre-processing contribution is a systematic taxonomy of database metadata along two dimensions: information source (structure, semantics, statistics) and granularity (column-level, inter-column-level, table-level). For each column, the framework aggregates this metadata into a natural-language description $NL(\Sigma, D)$ that serves as the column's context.

At the column level, the system uses column names, data types, descriptions (reused from TA-SQL), value ranges computed via SQL execution, BM25-retrieved question-relevant example values, and value patterns summarized by an LLM from 200 sampled values of non-numeric columns.

The inter-column level is the most distinctive component. The framework distinguishes duplicate columns (identical data content) from similar columns (semantically and statistically related but not identical), using a sentence encoder for retrieval, an LLM to identify join paths, and SQL queries to verify value consistency. It also mines functional dependencies with TANE to derive cardinality relationships (1:1, N:1, N:M) between columns; these constraints inform decisions about DISTINCT, GROUP BY, and nested queries. This inter-column analysis is largely absent from prior work.

Table-level descriptions are generated by an LLM from schemas enriched with column-level metadata, capturing table function, key columns, and usage scenarios.

A notable concession: not all modeled metadata is used at inference time. The authors prune to a subset because current LLMs degrade with excessive context, and they explicitly frame this as temporary — whether richer metadata helps as models improve remains open.

Schema linking via draft SQL

Rather than asking the LLM to enumerate relevant columns directly, MCI-SQL generates a draft SQL query $SQL_d$ over the enriched schema and parses it to extract referenced columns, motivated by evidence that LLMs generate SQL more reliably than they identify relevant columns explicitly. The candidate set $\mathcal{C}$ is then augmented with similar and duplicate columns discovered through inter-column metadata, yielding the filtered schema $schema_f$.

Intermediate correction SQL generation

Generation proceeds through adaptive prompt chaining: a single LLM performs a semantic check on $SQL_d$ (yes/no intent match), followed either by polishing (fine-grained fixes such as DISTINCT handling and NULL treatment) or rewriting (decomposition into sub-questions and subqueries assembled progressively). Prompts are injected dynamically based on the model's reasoning state rather than presented statically, exploiting positional attention sensitivity to reduce instruction drift in long chains.

The core mechanism validates each subquery against the SQL executor and injects state-specific corrective instructions across four states:

• Execution Success: truncated result rows returned; the model decides next actions autonomously.
• Execution None: treated as a strong signal of invalid logic; instructions explicitly discourage naive IS NOT NULL filtering (which typically violates user intent) and push reconsideration of column selection.
• Execution Empty: instructions target data-format mismatches, fuzzy matching (LOWER, LIKE) before strict matching, and join-domain verification.
• Execution Failure: raw SQLite error messages are returned for syntax debugging.

This design integrates generation and correction into one loop, preserving semantic continuity between them — a deliberate contrast with multi-agent designs where separate agents lack shared context.

Alignment and selection

Two lightweight alignment stages follow. Function alignment applies manually distilled rules mapping equivalent function implementations to canonical forms (e.g., date handling conventions) plus rule-generated constraints propagated from the generation phase: condition rules protecting critical filters, table rules specifying essential joins, and negative constraints pruning unnecessary conditions. Output alignment uses nine few-shot examples to restrict SELECT clauses to exactly what the user requested. Both are logic-preserving and pluggable.

For selection, candidates are generated under two metadata granularities — metadata-partial (descriptions plus examples) and metadata-complete — with temperature-diversified sampling (9 candidates total), and the final SQL chosen by majority voting over execution results. This yields fewer candidates than Chase-SQL's 21-query ensemble while remaining competitive.

Main results

On BIRD, MCI-SQL reaches 74.45% EX (dev) and 76.41% EX (test). Chase-SQL scores slightly higher on dev (74.90%) but requires three generation strategies and 21 candidates; without voting, MCI-SQL achieves 72.03% on dev, which the authors claim outperforms all published single-SQL methods and any single Chase-SQL strategy. On Spider test, MCI-SQL attains 88.7% EX without any Spider training data, behind MCS-SQL (89.6%) but ahead of CHESS, OpenSearch-SQL, and Chase-SQL.

Method	BIRD dev	BIRD test	Spider test
GPT-4	46.35	54.89	83.9
DIN-SQL+GPT-4	50.72	55.90	85.3
DAIL-SQL+GPT-4	54.76	57.41	86.6
CHESS+Gemini-1.5-pro	68.31	66.53	87.2
OpenSearch-SQL	69.30	72.28	87.1
Chase-SQL+Gemini	74.90	76.02	87.6
MCI-SQL	74.45	76.41	88.7
MCI-SQL w/o vote	72.03	–	87.6

The ablation study attributes gains cumulatively: schema linking adds +7.87 EX_single over the GPT-4o baseline (58.67 → 66.56); metadata-complete contexts add +2.80 single / +2.29 vote; intermediate correction adds +1.69 / +1.75; function alignment, output alignment, and rule-generated alignment add smaller increments (+0.13/+0.35/+0.56 single), reaching 72.03/74.45. Metadata analysis on BIRD MiniDev shows removing pattern and range metadata causes the largest drop (EX 67.5 → 62.0), while adding rows/null-value/size metadata slightly hurts, confirming the pruning decision given current model capabilities.

Cost analysis reports ~92 seconds and ~$0.10 per question online, dominated by the SQL generation phase (~80 s, ~6,700 output tokens); offline metadata construction costs ~3,700 s and$0.231 per database but amortizes across reuse. Voting gain diminishes with question difficulty, since candidate diversity collapses on hard questions — the authors note learned selection models could help here.

BIRD-clear

During evaluation the authors identified 412 problematic samples in the BIRD dev set. Ten graduate students corrected them over 15 days through individual annotation, cross review, and final inspection, categorizing errors as incorrect SQL, ambiguous questions, or unanswerable questions. Notably, noisy or misleading evidence hints were retained except where ambiguity required revision, and original SQL was kept when revised SQL produced identical results — choices intended to preserve realistic retrieval noise and respect execution-based scoring.

Results substantiate the underestimation concern: on the 412 corrected samples ($\text{BIRD}_c$-412), MCI-SQL improves from 44.66% EX (original) to 65.78%, while Arctic-Text2SQL-R1-7B improves from 41.99% to 43.93%. Overall BIRD-clear EX rises to 77.70% for MCI-SQL versus 69.23% for Arctic-SQL, an 8.47-point margin. The implication is concrete: benchmark annotation quality materially distorts cross-method comparisons, and the corrected set is released alongside code.

Error attribution across pipeline phases shows SQL generation accounts for the largest share of residual failures (40.86%), driven mainly by logical errors and insufficient business-logic understanding; schema linking contributes 26.88%, mostly missed relevant columns.

Limitations and open questions

Several limitations are stated plainly. The metadata taxonomy is representative rather than exhaustive, and exhaustive enumeration is acknowledged as infeasible. Metadata pruning is an explicit concession to current LLM context limitations, leaving open whether full metadata utilization would help stronger models. Voting gains shrink on challenging questions, and the paper does not evaluate learned selection models within its own framework. The BIRD-clear corrections involve human judgment; the paper does not report inter-annotator agreement statistics, so some residual subjectivity in the 412 revisions cannot be ruled out. Finally, cost figures depend on external API pricing and response times, limiting generalizability of the efficiency claims.

Conclusion

MCI-SQL demonstrates that enriching per-column context with inter-column relationships and embedding execution-feedback-driven correction inside SQL generation yield measurable accuracy gains at moderate cost, achieving state-of-the-art published results on BIRD (76.41% test EX) with far fewer candidates than ensemble-heavy competitors. Its secondary contribution, BIRD-clear, provides evidence that annotation errors in widely used benchmarks can shift measured performance by double-digit margins on affected subsets, making dataset curation a first-order concern for Text-to-SQL evaluation.