A Multi-Agent Approach for Claim Verification from Tabular Data Documents

Abstract: We present a novel approach for claim verification from tabular data documents. Recent LLM-based approaches either employ complex pretraining/fine-tuning or decompose verification into subtasks, often lacking comprehensive explanations and generalizability. To address these limitations, we propose a Multi-Agentic framework for Claim verification (MACE) consisting of three specialized agents: Planner, Executor, and Verifier. Instead of elaborate finetuning, each agent employs a zero-shot Chain-of-Thought setup to perform its tasks. MACE produces interpretable verification traces, with the Planner generating explicit reasoning strategies, the Executor providing detailed computation steps, and the Verifier validating the logic. Experiments demonstrate that MACE achieves state-of-the-art (SOTA) performance on two datasets and performs on par with the best models on two others, while achieving 80--100\% of best performance with substantially smaller models: 27--92B parameters versus 235B. This combination of competitive performance, memory efficiency, and transparent reasoning highlights our framework's effectiveness.

• The paper introduces MACE, a multi-agent framework that decomposes claim verification into planning, executing, and verifying steps, circumventing fine-tuning limitations.
• It achieves state-of-the-art results across closed- and open-domain datasets through modular agent collaboration and detailed reasoning traceability.
• The system provides interpretable, step-by-step decision logs that mitigate error cascades and enhance transparency in structured data analysis.

Multi-Agentic Framework for Claim Verification from Tabular Data Documents

Motivation and Problem Setting

Claim verification over tabular data is gaining prominence within the broader fact-checking domain, particularly due to the prevalence of structured data in technical, scientific, and financial documents. Existing LLM-based techniques predominantly depend on complex pretraining/fine-tuning—often yielding brittle generalization—or rely on subtask decomposition pipelines that suffer from limited transparency and error propagation. These issues are exacerbated by diverse table formats, the necessity for compositional and relational reasoning, and the requirement for explainable verdicts. The described work, "A Multi-Agent Approach for Claim Verification from Tabular Data Documents" (2604.17225), addresses these gaps by proposing a multi-agent framework that eschews model fine-tuning in favor of zero-shot, interpretable, agentic collaboration.

Multi-Agent Framework Architecture

The proposed system, denoted as MACE, leverages explicit task modularization through distinct, specialized agents: Planner, Executor, and Verifier. Instantiated via the AutoGen library in a group-chat paradigm, the architecture supports closed- and open-domain claim verification settings. The workflow sequence is as follows: the user submits a claim and evidence; the Planner generates a stepwise reasoning plan; the Executor implements the plan with detailed computations and provisional verdict; the Verifier validates explanation-verdict consistency and correctness. Crucially, MACE incorporates bi-directional feedback, enabling the Executor and Verifier to request clarifications or iterations from upstream agents—effectively mitigating classical pipeline error cascades.

Figure 1: A schematic of the agentic workflow, illustrating principal and feedback interactions for claim verification.

This agentic decomposition directly supports transparent decision-making by producing interpretable traces: the Planner documents its reasoning strategy, the Executor details the computation pipeline and data extraction points, and the Verifier audits logical and evidentiary alignment.

Agent Functionality and Reasoning Traces

Planner Agent

Given a claim and supporting tables (or mixed tabular-textual evidence), the Planner formulates an explicit, executable verification plan. It decouples reasoning into unambiguous extraction, computation, and comparison steps, attends to linguistic negations and ambiguous quantifiers, and structures sub-claim and dependency chains with fine-grained granularity. Its output includes both a natural language rationale (<explanation>) and actionable instructions (<plan>).

Executor Agent

The Executor operationalizes the Planner’s blueprint, performing structured data extraction and all computations required for claim checking. This agent is responsible for distinguishing among “support,” “refute,” and “not enough info” outcomes, rigorously handling partial evidence and ambiguous phrasing. If the plan is imprecise, the Executor can trigger a “revise” loop for clarification, thereby enhancing robustness. Detailed execution traces illustrate cell, row, and operation selection for full auditability.

Verifier Agent

The Verifier agent provides critical post hoc validation, checking the Executor’s rationale and verdict for logical and evidential consistency. If misalignment or errors are found, the Verifier triggers correction rather than forwarding an erroneous output downstream. The system thus provides strong guarantees on the correctness and auditability of claim verification outputs prior to user delivery.

Experimental Evaluation

Datasets and Baselines

The system is evaluated on both closed-domain (SciTab, SemTab) and open-domain (FinDVer, SciTab-OD) datasets, encompassing a spectrum of table types, domains, and claim granularities. Baselines span programmatic pipelines, pre-trained/fine-tuned architectures (e.g., TART [lu2024tart], ProTrix [Wu2024ProTrixBM]), and LLMs of varied scale. Standard benchmarks and metrics (Macro F1, Micro F1, Accuracy) are used for direct comparability.

Quantitative Results

MACE manifests multiple notable trends:

• SOTA performance: Achieves best-in-class results on SciTab and SemTab, and matches leading methods on FinDVer and SciTab-OD.
• Strong results with compact models: Configurations with 27–92B parameters (e.g., Llama-8B Agent, Qwen-72B Agent) reach 80–96% of 235B model performance using only 11.5–39.1% memory, with detailed, step-by-step explainability.
• Memory and runtime trade-offs: Multi-agent coordination introduces a 2–27× runtime overhead relative to single-model inference, but is justified in use cases prioritizing transparency and veracity over latency.
• End-to-end transparency: Unlike previous pipelines, MACE provides interpretable selection of rows, columns, and mathematical operations—enabling full verification step replayability.

Closed-domain performance comparison

MACE yields Macro F1 of 0.71 on SciTab (Qwen-235B) and 0.86–0.90 Micro F1 on SemTab (Llama-8B, Qwen-72B, Qwen-235B), surpassing competitors including GPT-4 and fine-tuned programmatic models.

Open-domain results

On FinDVer and SciTab-OD, MACE attains or matches best accuracy with Qwen-235B and Qwen-72B configurations. Smaller models (Llama-8B, Mistral-7B) remain competitive with significantly reduced hardware footprint.

Agent composition ablations

Analysis highlights that independent verification (e.g., using a distinct GPT-20B Verifier) can benefit small models, but larger models achieve stronger consistency by sharing all agent weights to maintain uniform reasoning protocols and avoid inter-model policy drift.

Qualitative Analysis: Traces and Plan Executions

Sample Planner and Executor outputs for complex claims (e.g., “Reused/recycled water made up 55.82% of all operational water use across all years”) demonstrate the system’s capacity for hierarchical reasoning, precise numerical computation, and principled comparison with domain-grounded thresholds.

Figure 1: Action plan and execution trace visualizations exemplify the Planner’s decomposition and Executor’s cell-specific operations for aggregate percentage computation.

Error Analysis

Failure cases primarily cluster around three phenomena: excessive numerical rigidity in independent Verifier configurations; over-strictness on comparative/compound claims (e.g., requiring strict superiority across all sub-claims for “outperform”); and retrieval ambiguity/inference gaps in small models when handling open-domain evidence. These highlight that while independent Verifiers can bring external scrutiny, they may introduce failure modes if reasoning conventions diverge from base agent logic. Maintaining model-consistent agent families can mitigate such errors.

Implications and Future Directions

Practical and Theoretical Impact

The multi-agent decomposition of claim verification over tabular data offers several advantages:

• Deployability of verification pipelines on modest hardware due to parameter-efficient models
• Modular integration in applications requiring explainable, auditable verification (e.g., scientific publishing, financial compliance)
• Enhanced generalizability versus dataset-specific fine-tuning paradigms

From a theoretical perspective, MACE exemplifies a compositional protocol that isolates subtask competencies, supports structured feedback, and minimizes error propagation—an important architectural motif for robust agentic LLM systems.

Limitations and Prospective Enhancements

Key limitations include increased runtime overhead and limited handling of extremely large tables or context lengths. Future enhancements may involve: incorporating specialized mathematical agents, more advanced table segmentation/summarization methods for context window management, and tighter integration of retrieval-verification feedback loops to further improve NEI (Not Enough Info) handling.

Conclusion

This work establishes that multi-agentic orchestration—with explicit task division, structured feedback, and traceable reasoning—can deliver state-of-the-art claim verification against tabular data, leveraging zero-shot prompting and interpretable output. The approach democratizes high-quality verification capabilities to models with much smaller parameter counts and paves the way for modular, explainable, and scalable fact-checking systems in domains reliant on complex structured data (2604.17225).