PanelTR: Zero-Shot Multi-Agent Table Reasoning

• PanelTR is a zero-shot multi-agent table reasoning framework that mimics a scientific workflow using Investigation, Self-Review, and Peer-Review.
• It addresses fact verification, question answering, and NL-to-SQL tasks through prompt engineering and semantic-level transfer.
• The framework achieves strong performance on benchmarks like FEVEROUS, TAT-QA, WikiSQL, and SEM-TAB-FACTS without fine-tuning or specialized training data.

PanelTR is a zero-shot, multi-agent table reasoning framework in which a single pre-trained LLM is instantiated as multiple “scientist agents” that follow a structured scientific workflow—Investigation, Self-Review, and Peer-Review—to solve table reasoning tasks without fine-tuning, task-specific synthetic data generation, or parametric updates (Ma, 8 Aug 2025). In the paper, “table reasoning” is treated as an umbrella term covering fact verification over tables, question answering over tables or table-plus-text inputs, and natural-language-to-SQL generation. The framework is designed to obtain supervised-like robustness through prompt engineering and multi-agent orchestration alone, and the paper presents this as a form of semantic-level transfer across heterogeneous tasks.

1. Problem formulation and scope

The paper defines table reasoning as a mapping from a table $\tau$ and a query or claim $\xi$ to an output $y$, written for table-only settings as $y = f(\tau, \xi)$ and for hybrid table-plus-text settings as $y = f(\tau, P, \xi)$, where $P$ denotes surrounding text (Ma, 8 Aug 2025). The output space is task-dependent: it may be a verification label, a textual answer, a numerical answer, or an SQL query.

Within this formulation, the paper covers three task families. The first is fact verification over tables, exemplified by FEVEROUS and SEM-TAB-FACTS, where a model must determine whether a claim is supported, refuted, or not verifiable, sometimes with evidence retrieval. The second is question answering over tables or table-plus-text inputs, exemplified by TAT-QA, where numerical and textual answers often require multi-step arithmetic over financial reports. The third is NL-to-SQL over tables, exemplified by WikiSQL, where the goal is to generate an executable query or an equivalent output whose denotation matches the answer.

The motivation for PanelTR is framed against three limitations in prior approaches. Supervised neural and PLM-based models such as TAPAS, TAPEX, and TAGOP achieve strong task-specific performance but require large annotated in-domain datasets and do not transfer easily across domains or table styles. Unsupervised, self-training, and synthetic-data approaches such as MQA-QG and UCTR-ST reduce direct annotation requirements but depend on complex data augmentation pipelines, substantial engineering, and often a warm start. Direct use of LLMs with simple prompting avoids retraining but often underperforms specialized supervised systems on structured table tasks, tends to answer without systematic table inspection, and remains fragile on numerical reasoning and faithful grounding.

In this setting, “zero-shot table reasoning” means using a pre-trained LLM as is: no fine-tuning, no task-specific label training, no parametric updates, and no synthetic-data retraining loop. The difficulty arises from arbitrary table schemas, mixed structured and unstructured evidence, multiple output formats, and multi-step numerical and logical reasoning grounded in table entries. PanelTR is proposed specifically to impose a scientific-method-style structure on inference in this zero-shot regime.

2. Scientific-method workflow

PanelTR organizes inference into three stages: Investigation, Self-Review, and Peer-Review (Ma, 8 Aug 2025). All agents are copies of the same base LLM; differentiation arises entirely from prompt instructions and personas rather than from distinct model parameters.

The Investigation stage is independent for each scientist. It begins with problem analysis, which assesses the difficulty of the instance and identifies key analytical points. The paper formalizes this with an assessment function $$\mathcal{A} : \mathcal{X} \rightarrow \mathcal{D} \times \mathcal{G}$$, where the perceived complexity $\delta$ belongs to $$\{\text{basic}, \text{intermediate}, \text{complex}\}$$ and $\gamma$ is a set of analytical points such as unit conversion, multi-row aggregation, or a needed join with text. Investigation then proceeds to solution formulation through a strategy function $\xi$0 that uses the assessment to produce an initial solution, including both an answer and a reasoning trace. In prompt terms, each scientist is instructed to classify complexity, enumerate key factors, plan a solution, and execute that plan.

Self-Review is an iterative verification pass by the same scientist over its own answer. The paper defines a verification function $\xi$1, where the status is either “uncertain” or “validated.” If the answer is marked uncertain, the scientist re-runs assessment and strategy and may revise once; in the experiments, the maximum number of self-refinement iterations is set to $\xi$2. This stage is explicitly narrower than unconstrained chain-of-thought generation: the model must check consistency with the table and question, diagnose failure, and issue a targeted revision.

Peer-Review is the multi-agent stage. The panel is $\xi$3, corresponding to Albert Einstein, Isaac Newton, Marie Curie, Alan Turing, and Nikola Tesla. Each scientist first produces a validated solution through the Investigation-plus-Self-Review pipeline. If all answers are identical, PanelTR short-circuits and returns the unanimous answer. If there is disagreement, the system enters iterative discussion. In the experiments, the maximum number of discussion rounds is also $\xi$4; if consensus is still absent after that round, the final answer is selected by majority vote over the scientists’ last answers.

This workflow is the framework’s central methodological claim. Rather than treating reasoning as a single free-form generation, PanelTR decomposes it into assessment, planning, self-validation, and collective deliberation. The paper characterizes this as a structured scientific approach at inference time, with hypothesis formation, checking, and peer review serving as explicit reasoning priors.

3. Personas, prompting, and orchestration

The five scientist personas are realized purely in prompt text. The paper reports that each persona is associated with a distinct prompt focus:

Scientist	Prompt focus
Albert Einstein	Explore alternative interpretations and conceptual frameworks
Isaac Newton	Verify numerical relationships and logical consistency
Marie Curie	Validate with experimental evidence and practical tests
Alan Turing	Analyze problem structure and optimize solution efficiency
Nikola Tesla	Synthesize diverse perspectives into coherent solutions

The paper’s ablations show that the specific historical identities are not crucial. Using random roles or alternative professions—Doctor, Artist, Researcher, Social Influencer, Entrepreneur—yields similar performance. A common misconception is therefore that the gains come from the named scientist personas themselves. The reported evidence indicates instead that the benefit comes from multiple diverse agents following the structured methodology.

Prompting is organized around atomic functions. Separate prompts are assigned to $\xi$5 for complexity assessment and analytical points, $\xi$6 for plan construction and answer generation, and $\xi$7 for review and validation, in addition to prompts for Individual Presentation and Collective Deliberation (Ma, 8 Aug 2025). These prompts are implemented through nested generation calls with rigid output formats; regex is used to extract the complexity label, analytical points, answer, and validation status. The paper also states that no explicit numeric confidence scores are used beyond the internal “validated/uncertain” labels in Self-Review and the majority-vote mechanism in Peer-Review.

The prompting strategy explicitly encourages decomposition, natural-language reasoning explanations, hypothesis testing, skepticism, and revision. Self-Review emphasizes identifying flaws or missing steps rather than assuming the first answer is correct. Peer-Review emphasizes maintaining dissent unless convincing evidence warrants a change, which is intended to avoid forced consensus.

Implementation details reflect the textual interface of the base model. Tables are flattened into plain strings, such as header-row pairs or row-wise lists. Benchmark-specific task descriptions are then added: FEVEROUS and SEM-TAB-FACTS prompts define fact-verification labels and evidence; TAT-QA prompts define numerical QA over financial reports and permissible answer formats; WikiSQL prompts specify natural-language-to-SQL mapping and require SQL output. For WikiSQL, no SQL execution engine or external symbolic tool is used; evaluation is by denotation accuracy rather than by query execution within the system.

The paper uses deepseek-v3 via the deepseek-chat API as the primary base model with temperature $\xi$8, and AutoGen v0.2.40 is used to orchestrate the multi-agent conversations. Agent presentation order during Peer-Review is randomized to avoid systematic bias.

4. Benchmark coverage and empirical performance

PanelTR is evaluated on four benchmarks: FEVEROUS, TAT-QA, WikiSQL, and SEM-TAB-FACTS (Ma, 8 Aug 2025). FEVEROUS is a fact-verification task over Wikipedia pages containing text and tables, measured by label accuracy and FEVEROUS score; the system uses the official baseline retriever and focuses on reasoning and classification. TAT-QA is a financial QA benchmark over hybrid tables and text, evaluated by Exact Match and F1. WikiSQL is a question-answering and SQL-generation benchmark over Wikipedia tables, evaluated here by denotation accuracy because no SQL executor is used within PanelTR. SEM-TAB-FACTS is SemEval 2021 Task 9 on fact verification over tables from scientific documents, evaluated by three-way micro F1.

The experimental setup is explicitly zero-shot: no supervised fine-tuning, no synthetic training data, and no self-training loops. The principal vanilla LLM baselines are deepseek-v3 in single-agent direct-prompt form and gpt-4o-mini in a comparable zero-shot setting. Supervised and unsupervised baselines differ by benchmark and include TAPAS, NumNet+, TAGOP, FinMath, UniPCQA, TAPEX, MQA-QG, UCTR-ST, and combinations such as TAGOP+UCTR-ST, TAPAS+UCTR-ST, and TAPEX+UCTR-ST.

On TAT-QA dev, PanelTR achieves 67.2 EM and 74.8 F1. These results exceed all listed supervised baselines, including UniPCQA at 64.7 EM and 72.0 F1, and improve over vanilla deepseek-v3 by +9.2 EM and +8.3 F1. On SEM-TAB-FACTS, PanelTR reaches 87.1 dev and 90.8 test micro F1, substantially above supervised TAPAS at 66.7 dev and 62.4 test and above vanilla deepseek-v3 at 74.3 dev and 83.3 test. On WikiSQL, PanelTR records 87.2 dev and 87.3 test denotation accuracy, slightly exceeding supervised TAPEX on test accuracy, where TAPEX reaches 87.0, and improving modestly over vanilla deepseek-v3 at 85.6 dev and 85.4 test. On FEVEROUS dev, PanelTR achieves 75.5 accuracy and 24.1 FEVEROUS score. Its label accuracy remains below the best supervised baseline at 86.0, but its FEVEROUS score exceeds all listed baselines, including the supervised full baseline at 20.2 and vanilla deepseek-v3 at 23.5.

These results are interpreted in the paper as task-sensitive. The largest gains occur on TAT-QA and SEM-TAB-FACTS, both of which require substantial multi-step reasoning and interpretation. WikiSQL is more constrained, so the gains are smaller but consistent. FEVEROUS presents a trade-off: PanelTR is not best on raw label accuracy, yet it is strongest on the FEVEROUS score, which combines correctness of label and evidence and therefore places more weight on reasoning-intensive cases.

5. Ablation findings, misconceptions, and limitations

The ablation study isolates the contributions of Self-Review, Peer-Review, and the full Investigation-plus-Review pipeline (Ma, 8 Aug 2025). Adding Self-Review to the vanilla model improves TAT-QA, SEM-TAB-FACTS, and WikiSQL but slightly hurts FEVEROUS. Adding Peer-Review improves TAT-QA and SEM-TAB-FACTS but hurts FEVEROUS and slightly hurts WikiSQL. Combining Self-Review and Peer-Review yields larger gains on TAT-QA and SEM-TAB-FACTS, a drop on FEVEROUS, and a moderate gain on WikiSQL. Investigation alone improves over vanilla on most tasks, and the full configuration—Investigation, Self-Review, and Peer-Review—produces the best overall results.

A second misconception addressed by the paper is that more discussion turns should monotonically improve performance. The iteration-turn study reports the opposite: increasing the number of panel discussion turns hurts performance, especially for fact verification. The stated interpretation is that additional iterations induce oscillating answers, overthinking, and drift away from the original evidence. The paper’s practical conclusion is “Less is more”: one investigation pass and one discussion round are sufficient, while further deliberation is counterproductive.

The paper identifies several limitations. PanelTR depends on the capabilities of the underlying pre-trained LLM and cannot exceed its core knowledge or numerical abilities; it only orchestrates them. The five-agent, multi-stage workflow increases latency and cost relative to single-shot prompting. Evaluation by EM and F1 may penalize semantically correct but differently phrased outputs, particularly in open-ended QA settings such as TAT-QA and WikiSQL. The paper also notes that it does not include direct comparisons with other multi-agent-system frameworks such as ReAct, DEPS, or MACT because different designs and benchmarks make fair head-to-head evaluation non-trivial. More generally, multi-agent systems remain subject to task mis-specification, inter-agent misalignment, and inadequate verification and termination conditions; PanelTR is presented as a mitigation rather than a complete solution.

6. Transfer properties and prospective extensions

A central concept in the paper is semantic-level transfer: the same core process—complexity assessment, key-point extraction, self-checking, and peer discussion—is applied across fact verification, numerical QA, and SQL generation, with only the task description prompt and required output format changing (Ma, 8 Aug 2025). This is the basis for the framework’s claim to flexibility across pure-table and table-plus-text settings and across scientific, financial, and encyclopedic tables.

The paper argues that the scientific panel methodology is not specific to tables. Its explicit components—problem complexity assessment, hypothesis or plan formulation, self-checking, and multi-agent critique and synthesis—are framed as a generic reasoning scaffold. This suggests, in the paper’s formulation, that structured scientific methodology can handle complex tasks beyond table reasoning in a zero-shot context. A plausible implication is that the framework’s principal contribution is not a table-specific operator library but an orchestration pattern for prompt-based reasoning.

Several future directions are identified. One is improved evaluation for open-ended LLM outputs beyond EM and F1, so that semantically equivalent answers are not unduly penalized. Another is the development of standardized benchmarks specifically for multi-agent reasoning frameworks, enabling more direct comparison of different MAS designs. The paper also points to hybrid architectures that combine PanelTR’s orchestration with specialized parametric components, external calculators, SQL executors, or tool-augmented agents. Multimodal extensions are also proposed, especially settings in which tables must be integrated with figures and images such as charts embedded in financial or scientific documents.

Taken together, the paper presents PanelTR as a task-agnostic inference scaffold rather than a trained table model. Its reported empirical contribution is that structured assessment, validation, and deliberation can substantially improve zero-shot performance over a strong base LLM and, on several benchmarks, rival or exceed supervised systems, while remaining independent of training data and model-weight updates (Ma, 8 Aug 2025).