IberLEF 2025 PRESTA Table QA
- The paper introduces a multi-step architecture that decomposes table reasoning into schema selection, JSON-based planning, and Python code execution, achieving 85% accuracy.
- PRESTA is a Spanish table QA task that emphasizes handling large, heterogeneous tables through structured control flow and precise column filtering.
- The approach employs modular components—such as the column selector, coder, and fuzzy matching with error recovery—to balance efficiency and accuracy in table-based question answering.
Searching arXiv for IberLEF 2025 PRESTA papers and official task descriptions. PRESTA, expanded as Preguntas y Respuestas sobre Tablas en Español, was an IberLEF 2025 shared task centered on answering questions in Spanish from tabular data. In the participant system described by MRT, the task was characterized by large, heterogeneous tables with many columns, mixed data types, ambiguous headers, and categorical values whose surface forms do not necessarily match question wording. The resulting formulation differs from conventional free-text QA: instead of direct answer generation, the reported approach treated each table as a Pandas DataFrame and decomposed inference into table understanding, column selection, natural-language planning, Python code generation, execution, and iterative recovery from errors. Using that multi-step design, MRT reported 85% accuracy and third place in PRESTA (Lagos et al., 17 Jul 2025).
1. Task definition and position within IberLEF 2025
PRESTA was one of the IberLEF 2025 shared tasks. Its objective was to answer questions in Spanish using information contained in tables, rather than in unstructured prose. The paper emphasizes that the central difficulty is not only linguistic: the tables are large and heterogeneous, with many columns, mixed data types, semantically unclear column names, and values that may appear in forms different from those used in the question. Ordinary end-to-end LLM prompting is presented as insufficient under these conditions because prompts can become too large, irrelevant columns can dominate the context, and direct numerical or categorical operations may become brittle (Lagos et al., 17 Jul 2025).
This framing places PRESTA in the TableQA family, but with a specifically Spanish and schema-intensive profile. A common misconception is to reduce the task to generic multilingual QA. The MRT description argues instead that PRESTA is fundamentally a table reasoning problem in which schema selection, value normalization, and executable operations are first-order concerns. This suggests that success depends at least as much on structured control flow as on language modeling.
2. Dataset regime and evaluation criterion
The MRT paper reports the following splits for PRESTA: train: 6 tables, 150 questions; dev: 4 tables, 100 questions; test: 10 tables, 100 questions. The authors did not train new base models; the splits were used for development and evaluation. For hardware, experiments ran on an NVIDIA RTX A6000 with 48 GB VRAM. To reduce overhead, questions were processed in batches, outputs from the column descriptor were cached, and model loading and unloading were minimized (Lagos et al., 17 Jul 2025).
The reported evaluation metric was accuracy. On dev or validation, the MRT system obtained an overall score of 0.71. Most answer types ranged from 0.75 to 0.80, while numerical answers were more difficult and scored 0.50. On test, three submitted variants obtained 85%, 85%, and 83%, respectively, so the best result was 0.85 accuracy (Lagos et al., 17 Jul 2025).
The paper also notes that the test set may have been simpler on average than validation, which partly explains why test accuracy exceeded dev accuracy by about 15 points. This suggests caution in interpreting the headline score as a complete characterization of PRESTA difficulty, especially given the modest evaluation set size.
3. MRT as a multi-step architecture
The PRESTA system was an evolved version of MRT (Maximizing Recovery from Tables with Multiple Steps), originally developed for the related SemEval 2025 TableQA task. The earlier MRT workflow already contained a modular structure—Column descriptor, Explainer, Coder, Runner, Interpreter, and Formatter—whose purpose was to solve table questions by generating and executing Python code in a structured and traceable way. For PRESTA, the authors added or strengthened a column selector, rules to remove uninformative columns, structured JSON outputs, Levenshtein-based corrections, custom functions, fuzzy matching, and stronger retry/error handling (Lagos et al., 17 Jul 2025).
| Module | Role | Notable detail |
|---|---|---|
| Column descriptor | Analyze columns | Cached per table |
| Column selector | Reduce schema | Groups of 25 columns |
| Explainer | Produce plan | JSON output |
| Coder | Generate Python | Uses Qwen 2.5 Coder 14B Instruct |
| Runner / Interpreter / Formatter | Execute and normalize | Supports retries |
The architectural significance of MRT lies in its refusal to treat answer generation as a monolithic inference step. The system instead implements a sequential divide-and-conquer strategy over a DataFrame. This makes intermediate artifacts inspectable and localizes failure modes: column selection can fail independently of code generation, and formatting can fail independently of reasoning.
4. PRESTA-specific adaptations for large and noisy tables
A central PRESTA adaptation was the column selector. The motivation was quantitative: PRESTA tables had an average of 174.1 columns, compared with 24.8 in the SemEval task. Including all columns in prompts would risk exceeding context limits, increase LLM failure rates, and raise the probability of wrong-column selection. The selector therefore used an LLM to choose potentially relevant columns from groups of 25, with prompts that included the question, the column names, and their descriptions. Its instruction was deliberately conservative: if in doubt, include the column (Lagos et al., 17 Jul 2025).
The system also removed columns considered uninformative or dangerous, including misleadingly described columns such as N_R_ and near-duplicate enumeration-style columns such as Ns_Nc_0 and Ns_Nc_1. This was intended not only to improve efficiency but also to prevent downstream modules from anchoring on irrelevant schema artifacts.
The explainer then converted the question and filtered context into a step-by-step plan. A major PRESTA change was the shift from raw text to JSON with the fields instructions, columns, and filter_values. Post-processing corrected misspelled column names using Levenshtein distance and added clarification notes instead of blindly substituting categorical values. The reported form of the clarification was: “Be careful! The value <old value> appears in the database with the following format: <new value>.” The explainer also appended type and example-value information for non-numeric columns. In the paper’s example, a question containing “enero” was explicitly linked to the database value “Enero” and to the fact that the month column was of type object (Lagos et al., 17 Jul 2025).
These changes are important because they recast schema linking as an explicit stage rather than an implicit side effect of prompting. A plausible implication is that PRESTA’s difficulty resides partly in cross-representation alignment: question language, schema names, and cell values must be synchronized before reasoning becomes reliable.
5. Code generation, execution, and recovery mechanisms
For code generation, MRT used Qwen 2.5 Coder 14B Instruct. For the other modules it used Qwen 2.5 14B Instruct, and it also evaluated Qwen 3 14B in the explainer only. The coder translated natural-language instructions into executable Python over the Pandas DataFrame. To reduce code-generation burden, the authors exposed a library of pre-coded generic functions that the model could invoke conceptually, in a design described as similar in spirit to tool-calling through prompt engineering. The listed functions included operations for numeric filtering, substring filtering, counting, frequency estimation, sorting, deleting rows, checking existence, and extracting numeric content, with examples such as filter_rows_by_column_equals_or_less_than_numeric_value, filter_rows_that_contain_column_value, count_elements_equal_to_value_in_column, find_most_frequent_element_in_column_subset, and extract_numeric (Lagos et al., 17 Jul 2025).
Execution was handled by the runner, which applied the generated program to the DataFrame. If execution or answer parsing raised an exception, the system returned to the coder, regenerated code, retried, and stopped after a fixed number of attempts. The interpreter then extracted the result in the expected type, and the formatter converted it into the benchmark’s required output format.
A particularly important robustness mechanism was fuzzy matching for categorical values. The appendix gives the similarity score as
and describes a workflow in which exact filtering is attempted first, followed by fuzzy search when the column is string-typed and exact matching fails. The appendix reports a threshold of 90 in _best_fuzzy_match, while the filtering function uses 75 for the overall fuzzy step. This mechanism was introduced to handle discrepancies such as capitalization, pluralization, minor typos, and other surface-form mismatches (Lagos et al., 17 Jul 2025).
The paper reports that all experiments were repeated 8 times, with the final answer selected by majority vote. That choice reflects the stochasticity of LLM-based pipelines and turns run-level variance into an explicit ensembling strategy.
6. Results, failure modes, and significance
The three submitted PRESTA variants consisted of a main system using Qwen 2.5 + Qwen 2.5 Coder, the same system but selecting the interpreter output, and a version using Qwen 3 14B only in the explainer. Their reported test scores were 85%, 85%, and 83%. The best result was therefore 0.85 accuracy, and the paper states that this placed the team third in the task. The authors attribute the result to several interacting factors: modular decomposition, column selection, clarified instructions, custom functions, fuzzy matching, the retry loop, and majority voting (Lagos et al., 17 Jul 2025).
The manual error analysis identifies the dominant failure modes as wrong instruction generation in the explainer and wrong column filtering in the selector, especially for long and semantically dense survey questions. Less common issues involved output formatting, such as +65 becoming 65, 18-24 becoming 1824, or parenthetical content being removed from answers such as PP (Partido Popular). The paper explicitly states that, compared with the earlier SemEval version, wrong cell filtering is no longer a major issue, and code errors and exceptions have been notably reduced.
The ablation study removed the column selector, custom functions, explainer correction, coder retries, and fuzzy substitution. The resulting validation scores remained in a narrow band of roughly 0.69 to 0.74, which led the authors to argue that the validation set was probably too small and insufficiently diverse for strong statistical conclusions. Even so, they report that removing fuzzy substitution hurt performance, that removing custom functions affected some categories, and that column filtering improved efficiency by about 3×, although it could hurt accuracy if a useful column was mistakenly removed (Lagos et al., 17 Jul 2025).
Taken together, these results portray PRESTA as a task in which control over schema exposure and execution fidelity matters more than single-shot fluency. The MRT paper does not present the problem as solved: it notes that ambiguous columns can still be missed, the explainer remains the main source of errors, numerical answers are harder than categorical ones, and the observed gains from individual modules are not always statistically clear. Within those limits, PRESTA stands as a representative IberLEF 2025 benchmark for Spanish table QA in which LLM-guided program synthesis, rather than pure generative answering, emerged as a highly effective operating paradigm.
