FlexSQL: Flexible Exploration and Execution Make Better Text-to-SQL Agents

Abstract: Text-to-SQL over large analytical databases requires navigating complex schemas, resolving ambiguous queries, and grounding decisions in actual data. Most current systems follow a fixed pipeline where schema elements are retrieved once upfront and the database is only revisited for post-hoc repair, limiting recovery from early mistakes. We present FlexSQL, a text-to-SQL agent whose core design principle is flexible database interaction: the agent can explore schema structure, inspect data values, and run verification queries at any point during reasoning. FlexSQL generates diverse execution plans to cover multiple query interpretations, implements each plan in either SQL or Python depending on the task, and uses a two-tiered repair mechanism that can backtrack from code-level errors to plan-level revisions. On Spider2-Snow, using gpt-oss-120b, FlexSQL achieves a 65.4\% score, outperforming strong open-source baselines that use stronger, larger models such as gpt-o3 and DeepSeek-R1. When integrated into a general-purpose coding agent (as skills in Claude Code), our approach yields over 10\% relative improvement on Spider2-Snow. Further analysis shows that flexible exploration and flexible execution jointly contribute to the effectiveness of our approach, highlighting flexibility as a key design principle. Our code is available at: https://github.com/StringNLPLAB/FlexSQL

• The paper’s main contribution is a flexible, agentic architecture that incrementally interacts with large databases for dynamic schema exploration and error repair.
• It employs bilingual program synthesis to generate both SQL and Python code, overcoming the rigidity of traditional fixed-pipeline systems.
• Empirical results on Spider2-Snow demonstrate significant accuracy gains, with ablation studies highlighting the crucial roles of diversity enforcement and backtracking.

FlexSQL: A Flexible Agentic Paradigm for Text-to-SQL over Large Analytical Databases

Problem Setting and Limitations of Fixed-Pipeline Systems

Text-to-SQL for large-scale analytical databases poses fundamental challenges: complex schema navigation, multi-step reasoning, query ambiguity, and the requirement of grounding in actual data values. Typical data warehouse settings (e.g., Snowflake, BigQuery) expose hundreds of tables and tens of thousands of columns; even identifying relevant schema elements and valid context is non-trivial. Most extant systems implement rigid pipeline architectures: schema retrieval is performed once (upfront), SQL is synthesized in a monolithic step, and only terminal post-hoc repair (usually syntax and execution error recovery) is invoked. This approach fails to address upstream misinterpretations, ambiguous queries requiring exploratory analysis, and non-trivial mapping between natural language and schema primitives.

FlexSQL Architecture: Agentic, Tool-Driven, and Bilingual Reasoning

FlexSQL introduces a highly flexible agent that, throughout its reasoning process, incrementally interacts with the database via a tool suite purposely designed for dynamic schema and data exploration. Critically, this mechanism is available at any reasoning step rather than being constrained to fixed points in a pipeline.

The framework consists of three core components (summarized in Figure 1):

Figure 1: FlexSQL system overview: the agent utilizes incremental database exploration (Plan Generation), bilingual program synthesis (Program Generation), and majority vote aggregation over $K$ diverse execution plans.

• Plan Generation dynamically explores the schema (via user-defined tools like GetSchema, GetTableCol, GetColValues, FindRows), generating $K$ diverse plans that encode different interpretations of the user query. Diversity is enforced through verbalized sampling and batch-wise negative prompting. Refinement is possible via iterative tool feedback.
• Program Generation synthesizes executable code (in SQL or Python) for each plan. A repair loop addresses code-level errors, and, crucially, supports plan-level backtracking—enabling the agent to revise higher-level strategies when errors originate from incorrect schema assumptions or misinterpreted requirements.
• Majority Voting aggregates consensus answers, merging outputs from solutions regardless of implementation language. If Python code is selected, a module translates it to SQL, guaranteeing that the final output is a SQL program.

Agentic Schema Discovery and Execution

Key to FlexSQL is its unrestricted, agentic exploration loop. At any time during the solution process, the agent can explore schema hierarchies, inspect sample data, and run partial code to empirically ground column value mappings, resolve ambiguous joins, and verify constraints. The impact of this paradigm is illustrated (Figure 2):

Figure 2: Example trace of FlexSQL: progressive schema discovery, empirical value grounding, and execution of verification and final queries yield robust plan diversity and accurate handling of ambiguous cases.

For queries ambiguous in their natural language specification (e.g., “find patents filed in Q1 2014 in materials science, and count how many earlier patents each one cites”), FlexSQL decomposes the problem into iterative sub-steps: locating the right schema entity by data inspection, confirming filter semantics via exploratory queries, and covering multiple interpretations of ambiguous predicates or join paths across diverse execution plans.

Diversity Enforcement and Bilingual Generation

The explicit promotion of diverse candidate plans is implemented via batch-wise negative prompting, which encourages the agent to cover alternative schema mappings, filter expressions, and join strategies. This diversity is critical, as execution ambiguity is prevalent in enterprise data environments. The system enables program synthesis in both SQL and Python for each plan, leveraging Python’s procedural expressiveness when declarative SQL is limiting (e.g., multi-stage transformations, conditional aggregations, or stateful computations), and subsequently transporting correct Python plans to SQL for interface requirements.

Strong Test-Time Performance and Scaling

FlexSQL achieves significant numerical gains on Spider2.0 leaderboards—especially on Spider2-Snow (enterprise Snowflake databases):

Figure 3: Test-time Pass@$K$ performance of FlexSQL vs. ReFoRCE on Spider2-Snow; FlexSQL exhibits reliable and monotonic improvement as sampling budget increases, surpassing baselines at all $K$.

• FlexSQL with gpt-oss-120b achieves Pass@1 of 55.2% and Majority@8 of 59.7% on Spider2-Snow, outperforming both ReFoRCE and DSR-SQL, even as those systems use larger backbones (DeepSeek-R1, gpt-o3).
• Scaling diversity to $K=16$ yields 65.4% Majority@K, the leading open-source result in the evaluation.
• The method remains parameter-efficient, with FlexSQL using gpt-oss-20b matching/surpassing 120b-class baselines.

These results confirm that not only does flexibility enable stronger single-shot performance, but it also compounds the advantage as inference budgets (test-time sampling) increase—an avenue well-aligned with trends in inference-time compute allocation for high-stakes, ambiguous database querying.

Ablation Study: Mechanism Attribution

Ablation experiments pinpoint the unique contributions of each design choice:

• Removing diversity enforcement reduces Majority@8 by 8–10%, indicating that plan diversity is a first-order effect.
• Disabling Python synthesis (SQL-only) causes the steepest accuracy drop—by over 10–12% absolute—demonstrating the essentiality of procedural expressiveness, especially in multi-phase analytical queries.
• Repair and backtracking are also critical: without them, both code-level and high-level errors persist, degrading aggregate execution accuracy.

Schema Linking Quality

Incremental, agentic schema acquisition achieves high precision and recall on table-level schema linking, with best-of-8 plan F1 over 95%, substantially better than context-pruned or monolithic schema retrieval strategies. This is achieved without increasing model context length, promoting efficiency.

Practical Integration and Broader Impact

The flexible exploration and execution paradigm of FlexSQL generalizes to mainstream coding agents (e.g., Claude Code): integration as skill modules for Claude produces >7% absolute Pass@1 gains, underscoring the paradigm’s architectural agnosticism and broad applicability in code-centric LLM systems.

Implications and Future Directions

Practically, FlexSQL demonstrates that flexible, agentic program synthesis equipped with rich tool interfaces unlocks superior performance in structured data domains compared to fixed pipeline architectures. Theoretically, this paradigm prescribes future directions:

• Deeper integration of agentic cognitive architectures with database exploration and dynamic grounding
• Adaptive, context-sensitive plan generation that scales to arbitrarily large schemas and multi-modal databases
• Broader application of bilingual and multi-representation program generation, especially for systems requiring interoperability between declarative and procedural code
• Exploitation of majority voting and diversity scaling for robust ambiguity resolution under high query uncertainty

Conclusion

FlexSQL operationalizes flexibility as a central principle in text-to-SQL, employing incremental tool-based exploration, diversity-driven planning, and bilingual generation with robust repair and backtracking. This architecture demonstrably and consistently outperforms strong open-source baselines at all model and compute scales. Its empirical results, ablation analyses, and demonstrated transferability indicate that future text-to-SQL and code generation systems for complex data environments will benefit from integrating similar agentic, tool-driven, and flexible pipelines.

---

References: (2605.02815) (see paper for additional references to related work)