RubikSQL: Industrial NL2SQL System

• RubikSQL is a knowledge-centric NL2SQL system that integrates lifelong learning with a continuously updated enterprise knowledge base to address implicit intents and domain-specific terminology.
• It employs a multi-agent architecture with database profiling, structured information extraction, rule mining, and Chain-of-Thought reasoning to generate accurate SQL queries.
• Benchmark results on KaggleDBQA, BIRD Mini-Dev, and RubikBench demonstrate its state-of-the-art execution accuracy and robustness in complex industrial scenarios.

RubikSQL is a knowledge-centric NL2SQL system for industrial deployment that addresses implicit intents and domain-specific terminology by framing NL2SQL as a lifelong learning task requiring both Knowledge Base (KB) maintenance and SQL generation. The system systematically builds and refines its KB through database profiling, structured information extraction, agentic rule mining, and Chain-of-Thought (CoT)-enhanced SQL profiling, and then employs a multi-agent workflow to generate accurate SQLs. Reported results place it at SOTA on KaggleDBQA and BIRD Mini-Dev, and the work introduces RubikBench, a benchmark designed to capture vital traits of industrial NL2SQL scenarios (Chen et al., 25 Aug 2025).

1. Industrial problem setting

RubikSQL is tailored for industrial (enterprise) deployment rather than the narrower settings emphasized by many public academic benchmarks. The system is designed around four barriers: implicit intent, private domain knowledge, wide table schema, and context sensitivity. Implicit intent arises when users pose terse or vague queries such as “XXX revenue last month?”, requiring inference of business logic or metric defaults. Private domain knowledge includes specialized terms, aliases, abbreviations, company-specific metrics, computation rules, and context-dependent terminology. Wide table schema refers to large, denormalized, or “wide” tables, schema variations, and nontrivial column/value organization. Context sensitivity includes temporal, user, and context-driven query differences such as currency preferences, fiscal time calculations, and user roles (Chen et al., 25 Aug 2025).

The paper’s key insight is that practical NL2SQL deployment cannot be treated as isolated per-query inference. Instead, it requires lifelong learning: an evolving agentic KB that accumulates semantic, contextual, and usage knowledge over time. This framing also bears on a common misconception in enterprise NL2SQL—namely, that most failures are primarily failures of LLM reasoning. RubikSQL’s later error analysis reports that less than 12% of errors stem from LLM reasoning, while the majority are due to incomplete or ambiguous knowledge. This suggests that knowledge maintenance is a primary systems concern rather than an auxiliary optimization.

2. System architecture and semantic layer

RubikSQL adopts a four-stage, multi-agent, knowledge-oriented workflow. The stages are decoupled by a semantic layer, Unified Knowledge Format (UKF 1.0), which provides a generalized schema for representing any domain, language, or context (Chen et al., 25 Aug 2025).

Stage	Function
Database Context Engineering	Converts raw database, documentation, and history into structured knowledge via profiling and extraction
User Query Augmentation	Enriches and refines historical NL-SQL pairs and user logs, leveraging CoT reasoning/annotation
Knowledge Base Indexing	Stores, organizes, and surfaces all structured knowledge via advanced, hybrid indexing strategies
Knowledge Distillation	Distills learned experiences and reasoning traces into smaller models for efficient, cost-effective SQL generation

UKF 1.0 functions as the storage and representation standard for any knowledge artifact. Its schema includes Metadata (name, type, short/long desc, version, variant), Content (main knowledge, auxiliary resources, serialization functions), Provenance (source, owner, workspace, parents), Retrieval (tags, synonyms, triggers, priority), Relationships (knowledge graph tuples), and Life-cycle (timestamp, versioning, expiration, deprecation marks). The format includes built-in templates for table, column, enum, predicate, metric, and synonym, and is extended dynamically for domain-specific facts and rules.

Architecturally, the system separates knowledge extraction and maintenance, indexing, utilization, and distillation. This separation is significant because it permits KB growth, retrieval, generation, and compression into smaller models to proceed as interlocking but distinct functions.

3. KB construction, maintenance, and indexing

RubikSQL’s lifelong-learning behavior depends on systematic KB maintenance. Database profiling agents automatically extract and characterize tables, columns, value types, statistics, and missing semantic descriptions, using LLMs as summarizers. The resulting toolkit supports downstream agents with fuzzy column and enum search, column type inference, and dynamic prompt serialization (Chen et al., 25 Aug 2025).

Structured Information Extraction (SIE) converts documentation, logs, and text into UKF entries. This includes automatic mapping of free-form language such as explanations and how-to documents into structured, retrievable knowledge artifacts. Agentic rule/context mining operates over both labeled and unlabeled data. From labeled queries, it extracts correspondences between NL phrases and SQL elements, synonyms, and predicates. From unlabeled logs, it uses powerful offline agents with majority voting, CoT, TTS, and verification against existing knowledge to hypothesize context and update UKF.

User Query Augmentation, described as CoT-enhanced SQL profiling, enriches each NL-SQL pair with query time, user profile, expected result schema, detailed comment headers outlining reasoning, edge cases, and formulas, as well as inline clause explanations within SQL. This process generates NL-CoT-SQL tuples for model training and curriculum. If needed, the reasoning trace is rectified by comparing system output versus ground truth and updating knowledge if improvement is found.

The KB is further expanded by data synthesis and query transfer. Query synthesis generates NL-SQL tuples from schema or ontology by composition or decomposition of existing queries. Query transfer ports queries from similar databases, adapting logic and substituting entities using the current KB. These mechanisms operationalize the lifelong-learning claim by making the KB cumulative rather than static.

Retrieval is implemented through multiple overlapping indices. The string-based index uses the novel LLM-augmented DAAC (Double Array Aho-Corasick), with complexity $O(|Q| + r)$ for query length $|Q|$ and $r$ matches. The remaining indices are faceted search, multi-vector index, graph-based traversal, and autonomous search, in which agents combine indices, apply triggers, and summarize retrieved knowledge for downstream consumption. The resulting design is not a single retriever but an indexed memory substrate for enterprise NL2SQL.

4. Lifelong learning formulation and agentic execution

RubikSQL formulates NL2SQL as a coupled knowledge-update and SQL-generation process. Given a database $$\mathcal D = (\mathcal S, \mathcal V, \tilde{\mathcal K}^0)$$, knowledge base state at time $t$, and query $q_t$, the paper defines KB update and SQL generation as follows (Chen et al., 25 Aug 2025):

$$\Delta \mathcal K^t = \mathcal M_k\left(q_t, \mathcal{S}, \mathcal V, \mathcal K^{t-1} \right)$$

$$s_t = \mathcal M_s\left(q_t, \mathcal S, \mathcal{K}^t \right)$$

The objective is to maximize EXECUTION ACCURACY over a batch of queries. Operationally, this formulation is realized through three agents. The RAG Agent retrieves and summarizes relevant knowledge and context from the KB using indices and toolkits. The SQL Gen Agent generates CoT-annotated SQL statements conditioned on that context. The SQL Refine Agent verifies execution, corrects errors, and fine-tunes output, auto-correcting structural or logical issues based on runtime results.

The execution workflow is described in six steps: retrieval, prompt construction, CoT reasoning and generation, SQL generation and profiling, refinement and verification, and KB update. The final step is especially important: new experience, including CoT, commentary, and actual outcome, is stored back in the KB so that future queries can exploit accumulated experience. In this sense, inference and memory maintenance are part of a single loop.

RubikSQL also incorporates test-time scaling and cascading. In test-time scaling, multiple SQL outputs are ensembled and selected via majority voting on execution results, while unexecutable outputs are discarded. In cascading, parallel LLMs of ascending size are run, and execution stops as soon as two outputs agree on the executed result, reducing latency while preserving accuracy. For efficient deployment, the system performs knowledge distillation from high-quality NL-CoT-SQL tuples into smaller models. Tuple selection uses the score

$$\mathcal{S} = \alpha\mathcal{H} + \beta\mathcal{Q} + \gamma\mathcal{V} + \mathcal B$$

where $\mathcal{H}$ is hardness, $|Q|$0 is quality, $|Q|$1 is variety or diversity, and $|Q|$2 is a tie-breaker.

5. Benchmarks, metrics, and empirical findings

The empirical evaluation covers KaggleDBQA, BIRD Mini-Dev, and RubikBench. KaggleDBQA contains 8 databases and enterprise-like queries. BIRD Mini-Dev contains 19 databases and is described as focused query density. RubikBench is introduced as a realistic, anonymized financial database with approximately 5000 queries over wide tables of approximately 70 columns, reflecting real business line-of-questioning and context-sensitive, user-profile-aware queries. It includes knowledge resources such as user profiles, jargons, and policies, and its construction consists of synthetic seed generation, real-world query transfer, augmentation/composition, and manual verification by experts (Chen et al., 25 Aug 2025).

The reported execution-accuracy results are as follows:

Benchmark	Method	EX (%)
KaggleDBQA	RAT-SQL	26.8
KaggleDBQA	ZeroNL2SQL	44.9
KaggleDBQA	ODIS-Codex	54.8
KaggleDBQA	RubikSQL (n=8)	58.9
BIRD Mini-Dev	AskData	73.0/75.36
BIRD Mini-Dev	CHASE-SQL	74.9
BIRD Mini-Dev	RubikSQL (n=8)	77.3

The benchmark suite uses Execution Accuracy (EX) and Bipartite $|Q|$3-score (BF$|Q|$4). BF$|Q|$5 compares execution results via best-match assignment between predicted and ground-truth rows, using weighted bipartite matching with the Hungarian algorithm for unordered outputs and dynamic programming for ordered outputs. The paper characterizes this metric as providing partial credit and robustness to output-format mismatches.

The error analysis on BIRD Mini-Dev is notable. Predicate errors account for approximately 40% and include extra or missing conditions, wrong enumerations, and date processing. Intent misinterpretation accounts for approximately 20% and includes wrong columns, extra or missing outputs, and improper post-processing. Content issues account for approximately 16% and include NULL handling and different ways of calculation. Benchmark or LLM errors form the remaining fraction. The reported finding that less than 12% of errors stem from LLM reasoning is central to the paper’s interpretation of industrial NL2SQL: the dominant bottleneck is incomplete or ambiguous knowledge rather than reasoning in isolation.

6. Relation to adjacent text-to-SQL directions

RubikSQL sits at the intersection of retrieval, agentic workflows, and enterprise memory. A useful comparison is RB-SQL, a retrieval-based LLM framework for text-to-SQL that constructs prompts through three modules: a concise table retriever, a concise column retriever, and an in-context example retriever. RB-SQL is designed to reduce prompt scope, mitigate context-length limitations, and improve SQL generation on large databases, and it reports better performance than several competitive baselines on BIRD and Spider (Wu et al., 2024). RubikSQL shares the emphasis on retrieval and prompt construction, but it extends that emphasis into explicit KB maintenance, UKF-based representation, agentic rule mining, KB update, and knowledge distillation. This suggests a shift from per-query prompt engineering to persistent enterprise knowledge infrastructure (Chen et al., 25 Aug 2025).

A second point of comparison arises in evaluation research. The later work RuCo-C argues that text-to-SQL evaluation has been constrained by pairwise comparison, binary signals such as exact match or execution match, and dependence on human-labeled gold SQL. It proposes automated query-specific rubrics, interpretable critiques, and fine-grained reinforcement learning rewards via progressive exploration. In that paper, “RubikSQL” is referenced externally as denoting fine-grained, rubric-based, interpretable, and automated evaluation frameworks for text-to-SQL, and RuCo-C is described as embodying that philosophy by automatically generating query-adapted rubrics and critique-based reward signals without human-labeled gold SQL (Wang et al., 27 Nov 2025).

This dual usage is analytically important. In the narrower sense, RubikSQL is the industrial NL2SQL system defined by its lifelong-learning agentic KB architecture. In a broader methodological sense, later work associates the term with rubric-centric, interpretable, and automated evaluation. A plausible implication is that “RubikSQL” now names both a specific system and a wider orientation within text-to-SQL research: scalable handling of enterprise knowledge, explicit interpretability, and evaluation or training signals that move beyond coarse query-level judgments.