When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors

Abstract: While LLMs perform well on table tasks, they still make data referencing errors (DREs), i.e., incorrectly citing or omitting table values, despite understanding the table structure. Beyond final-answer accuracy, DREs directly compromise the correctness and reliability of intermediate reasoning steps. Yet prior studies have only offered limited, small-scale analyses. In this work, we present the first systematic evaluation of tabular data referencing errors across different models and tasks. Our results show that DREs occur across all tested models (1.7B to 20B parameters). Furthermore, we demonstrate that incorporating data referencing as a critic significantly improves answer accuracy up to 12.0%, through critic-based filtering and rejection sampling. Finally, we trained a lightweight 4B-parameter critic model that achieves an average F1 score of 78.2% in detecting both in-distribution and out-of-distribution DREs, and effectively assists inference for larger models.

• The paper demonstrates a critic-driven framework to automatically detect and reduce data referencing errors in LLM table reasoning.
• It evaluates candidate filtering and segment-level rejection sampling, achieving accuracy gains and lowering error rates by up to 15.45%.
• The study introduces a lightweight 4B-parameter open-source critic that generalizes across models and datasets to enhance factual reliability.

Measuring and Mitigating Data Referencing Errors in LLM Table Reasoning

Introduction

This paper systematically investigates data referencing errors (DREs) in LLMs performing table-related tasks, emphasizing that such errors compromise the reliability of both intermediate reasoning and final answers. While LLMs have shown substantial improvements in tabular understanding, the authors demonstrate that even strong models frequently commit avoidable mistakes by inaccurately citing, misattributing, or omitting table values. The study rigorously analyzes the phenomenon, introduces methodologies for automatic detection and reduction of DREs, and provides empirical evidence of performance gains when these methods are applied.

Characterization and Prevalence of Data Referencing Errors

The investigation formalizes two principal classes of DREs in tabular tasks: (1) Incorrect Citation, referring to individual value misattribution or hallucination, and (2) Omitted Information, relating to necessary cells or rows being missed during information extraction. Both types degrade both the trustworthiness and factuality of LLM outputs, regardless of the apparent structural understanding of the table.

Figure 1: Illustration of DREs—incorrect citation (column confusion) and omission (missing a required row).

Empirical evaluations, employing the "LLM-as-a-Judge" paradigm with Sonnet-3.7, reveal that DREs are persistent across a spectrum of models (1.7B–20B parameters) and datasets covering QA, claim verification, and table-to-text. Notably, the DRE rate persists irrespective of prompt-based mitigations or the inclusion of self-reflection mechanisms. For instance, Qwen3-8B records a 14.04% DRE rate on the WTQ benchmark, dropping only marginally to 12.50% with explicit prompting; models such as Llama4-Scout and various LLaMA/Distill variants exhibit even higher DRE rates.

These errors are not limited to incorrect final answers. The "Correct-in-DRE" metric shows numerous instances where a response arrives at the correct final answer despite containing DREs in the reasoning steps, especially for binary decision tasks (e.g., SciTab).

Critic-Driven Mitigation of DREs

The study evaluates two principal critic-based strategies for mitigating DREs without requiring alteration to the generation model:

Critic-Based Filtering

This approach samples multiple candidate outputs per input and uses a dedicated critic to select only those responses free (or with fewer) DREs. The filtered subset, when subjected to post-hoc strategies such as majority voting, leads to consistently higher final accuracy than baseline, vanilla majority-voting, or random candidate selection.

Rejection Sampling

Segment-level rejection sampling is proposed for longer, step-wise reasoning model responses. Here, generation proceeds incrementally, and segments flagged by the critic for DREs are resampled individually, which both lowers sampling cost and constrains the error propagation typical in LLM chain-of-thought outputs. Across multiple datasets (WTQ, TableBench, FinQA) and models, this method achieves accuracy gains up to 11.96% in challenging DRE settings. Importantly, these improvements result from inference procedures rather than retraining, confirming that many DREs are easily avoided given robust output validation.

Lightweight Critic Models for Scalable DRE Detection

Despite the effectiveness of strong proprietary LLM critics, their black-box nature and inference cost limit practical applicability. To address this, the authors introduce a 4B-parameter open-source model (Qwen3-4B-Instruct) trained via supervised fine-tuning and reinforcement learning with verified reward (RLVR). This lightweight critic is distilled from Sonnet-3.7 outputs and further augmented with synthetic DRE cases to boost generalization.

Figure 2: Critic F1 scores for DRE detection across model-dataset pairs.

The trained Critic-4B achieves an overall F1 score of 78.16% in flagging DREs, significantly outperforming its untrained baseline (69.51%). It generalizes across reasoning/non-reasoning models and domain-shifted datasets. Rejection sampling guided by Critic-4B yields tangible reductions in DRE incidence (e.g., DRE rate drops by up to 15.45% for Distill-Qwen-7B on WTQ), with accuracy consistently improved compared to unfiltered outputs, albeit with slightly lower gains than the full-size LLM critic.

Implications and Future Directions

This work highlights that DREs represent a distinct, quantifiable failure mode in LLM-based table reasoning that is not captured by final-answer accuracy or indiscriminate process-level evaluation metrics. Critic-driven mitigation methods are both practically feasible and effective, decoupling step-wise factuality assessment from end-to-end answer correctness. Empirical findings invalidate the hypothesis that DREs are fundamental to model architecture or training data: instead, they are largely avoidable through better output adjudication.

For practitioners, integrating lightweight DRE critics can directly enhance factual reliability in high-stakes domains like finance, healthcare, and scientific analysis, with minimal computational overhead. Theoretically, these results motivate a research program toward developing specialized process-level reward models, scalable reference-based verifiers, and improved attention mechanisms for tabular reasoning in LLMs.

Further research should explore (1) DREs in non-tabular and multimodal contexts, (2) model interpretability frameworks diagnosing DRE genesis, and (3) the design of proactive self-correction architectures that preemptively mitigate DREs during generation.

Conclusion

The analysis establishes DREs as prevalent and consequential errors in LLM table understanding, independent of model size or training paradigm. The proposed critic-based mitigation framework—especially when coupled with lightweight models—substantially reduces DRE frequency and elevates answer accuracy. Data referencing fidelity should be adopted as a primary evaluation metric in developing and benchmarking robust LLMs for structured data applications (2606.32029).