---
title: 'SSEV: Self-Refinement & Ensemble Voting for Text-to-SQL'
url: https://www.emergentmind.com/topics/single-agent-self-refinement-with-ensemble-voting-ssev
type: topic
---

# SSEV: Self-Refinement & Ensemble Voting for Text-to-SQL

Single-Agent Self-Refinement with Ensemble Voting (SSEV) is an inference-time methodology for improving robustness and execution accuracy in Text-to-SQL agents. The approach integrates execution-guided iterative refinement and an ensemble voting mechanism, supporting scalable, production-grade natural language semantic parsing over structured databases. SSEV has been embedded in leading frameworks including ReFoRCE [2502.00675] and PET-SQL derivatives [2601.17942], and achieves SOTA or near-SOTA results on benchmarks such as Spider 2.0-Lite and BIRD.

## 1. Conceptual Foundation

SSEV is motivated by the high variance and instability often observed in single-pass LLM-based SQL generation under real-world constraints: large database schemas, SQL dialect variation, and ambiguous user queries. The methodology is founded on two principles:

- **Single-agent self-refinement**: Iteratively repair and redirect candidate SQL queries in response to execution feedback (both syntactic and semantic), mediated by LLM prompting. This loop continues until a valid, consistent output is obtained or a maximal number of refinements is reached.
- **Ensemble voting**: Parallel execution of multiple independent self-refinement threads (or, in multi-expert extensions, candidate LLMs) whose outputs are aggregated via statistically rigorous voting schemes (majority, weighted majority, randomized weighted majority).

This hybridization addresses two critical failure modes: brittle one-shot prediction and the lack of reliable agent self-critique in the absence of gold-standard data at inference time [2502.00675, 2601.17942].

## 2. Detailed Methodology

SSEV separates into two mutually reinforcing modules: self-refinement and ensemble voting. Below, "Editor’s term" is used for operational clarity.

### 2.1 Self-Refinement Module

At each iteration $t$, the agent maintains $(S^{(t)}, R^{(t)})$ where $S^{(t)}$ is a SQL candidate and $R^{(t)} = \text{EXEC\_SQL}(S^{(t)})$. If $R^{(t)}$ is invalid due to a syntax, type, or semantic error, the LLM receives an error-augmented prompt and is tasked to produce $S^{(t+1)}$ correcting failure causes:

$$ 
S^{(t+1)} = \text{RefineLLM}(S^{(t)}, \Delta_t, P_{format}) 
$$

where $\Delta_t$ encodes the error message and schema context, and $P_{format}$ enforces the required output form. This procedure is iterated up to $T_{\text{max}}$ steps. Termination occurs when self-consistency is detected—i.e., a non-error, non-empty $R^{(k)}$ is seen two or more times—or by reaching $T_{\text{max}}$ without convergence [2502.00675].

### 2.2 Ensemble Voting Mechanism

Multiple self-refinement threads (typically $W=3$–$8$), each initialized with identical inputs but stochastic LLM behavior, are executed in parallel. Upon completion:

- Each thread outputs its final $(S_i, R_i)$, where $R_i$ is the execution result or $\perp$ if failure.
- All results $\{R_i\}$ are tallied and the consensus answer $R_{vote}$ is chosen:

$$
R_{vote} = \arg\max_{r \neq \perp} \sum_{i=1}^W \mathbb{I}[R_i = r]
$$

A strict majority threshold $\theta = \left\lfloor W/2 \right\rfloor + 1$ is required to issue $R_{vote}$; otherwise, ambiguity is declared and the example is deferred [2502.00675]. In multi-expert setups, deterministic or randomized Weighted Majority Algorithms (WMA/RWMA) compute weights $w_i$, aggregate per-candidate votes $W(s)$, and select $\hat{s} = \arg\max_s W(s)$, enabling adaptive expert specialization without ground-truth supervision [2601.17942].

## 3. Mathematical Formulation and Execution

The key mathematical operations in SSEV are as follows:

| Symbol/Step                                     | Role                      | Source           |
|-------------------------------------------------|---------------------------|------------------|
| $S^{(t+1)} = \text{argmin}_S L_{\text{err}}(\cdot)$  | Iterative error repair    | [2502.00675]     |
| $W(s) = \sum_{i: s \in S_i} w_i$                | Candidate weight tally    | [2601.17942]     |
| $w_i^{(t+1)} = w_i^{(t)} \times (1 - \epsilon)$ | Online weight update      | [2601.17942]     |
| $M_T \leq 2 M^* + 4\sqrt{T \ln N}$              | WMA mistake bound         | [2601.17942]     |
| $\mathbb{E}[M_T] \leq M^* + 2 \sqrt{T \ln N}$   | RWMA expected bound       | [2601.17942]     |
| $\text{Conf}(S) = \#$threads producing $R(S)/W$ | Answer confidence metric  | [2502.00675]     |

Self-refinement in SSEV is not based on loss backpropagation but on minimizing execution failure rates via error-guided LLM feedback. Ensemble voting employs online-learning-style regret minimization to approximate oracle selection across multiple candidate generators without the need for reference SQL.

## 4. Prompt Engineering and Schema Linking

The efficacy of SSEV is critically dependent on rigorous prompt construction and schema representation:

- **PreSQL prompts**: Present the full database DDL, sample values, and relevant task instructions (e.g., “Minimize SQL execution time”), often with few-shot demonstrations selected by embedding similarity.
- **PostSQL prompts**: Restrict schema context to only the columns/tables linked in the PreSQL phase.
- **Self-refinement prompts**: Supply the original question, preceding SQL, execution error message/type, correction strategies (e.g., simplify clause, adjust join), and strict output format constraints (e.g., “Action: BIGQUERY_EXEC_SQL(sql_query=...)”).
- **Schema linking**: Use regex-based DDL parsing or explicit mentions to minimize prompt length and focus the LLM on relevant schema fragments [2601.17942].

For large or complex schemas, schema compression and context pruning are necessary to mitigate LLM context window limitations, a feature implemented in ReFoRCE [2502.00675].

## 5. Empirical Evaluation and Ablation Studies

SSEV exhibits measurable gains on several datasets and under multiple LLM backbones. Notably:

| Dataset          | SSEV (WMA) EX   | Best Individual EX | Gain        | Source        |
|------------------|----------------:|------------------:|------------:|--------------|
| Spider 1.0-Dev   | 85.5%           | 84.9% (Gemini)    | +0.6%       | [2601.17942]  |
| Spider 1.0-Test  | 86.4%           | 85.7% (Gemini)    | +0.7%       | [2601.17942]  |
| BIRD-Dev         | 66.3%           | 65.97%            | +0.33%      | [2601.17942]  |
| Spider 2.0-Snow  | 26.7% (W=3)     | ~20% (single run) | +6.7%       | [2502.00675]  |

Ablation studies confirm that:

- Self-refinement contributes more substantially on complex/real-world data and with less capable LLMs.
- Limiting refinement to $M=2$–3 rounds suffices for maximal utility–most recoverable errors are corrected early.
- Schema linking and output format enforcement further regularize agent predictions, especially in high-column-count settings [2601.17942].

## 6. Implementation and Practical Considerations

SSEV has minimal deployment cost increase over naive LLM inference due to parallelization and bounded refinement loops. Each thread incurs up to $T_{\text{max}}$ LLM calls and SQL executions, and all candidates are batched for voting. Production notes include:

- LLM ensemble: $N=8$ APIs (e.g., Llama-3-70B, Gemini-2.5, Qwen2.5) [2601.17942].
- Execution: Pipelines modularized in Python, LLM calls distributed, embeddings cached, and results persisted for bootstrapping.
- Confidence deferral: Results below confidence threshold $\tau$ are flagged as “Ambiguous” for human or further computational review [2502.00675].

A plausible implication is that SSEV preserves $O(1)$ inference latency under sufficient hardware parallelism and enables practical deployment on datasets with large, cross-dialect schema variance.

## 7. Extensions, Limitations, and Future Directions

While SSEV is robust for single-hop query construction and schema-centric Text-to-SQL, it does not natively support multi-step reasoning or leverage external world knowledge. Recent work proposes ReCAPAgent-SQL, which decomposes the refinement-critique-act-plan cycle over specialized LLM agents. Moreover, future enhancements may include:

- Integrating semantic self-critique in addition to execution feedback.
- Calibrating ensemble weights dynamically per domain or user profile.
- Addressing prompt context overflow via progressive schema partitioning [2601.17942].

Overall, SSEV exemplifies a class of methods coupling LLM self-improvement with statistically grounded consensus estimation, demonstrably raising the reliability ceiling of scalable, industry-strength semantic parsing agents [2502.00675, 2601.17942].

Source: https://www.emergentmind.com/topics/single-agent-self-refinement-with-ensemble-voting-ssev