SimpleToolHalluBench Benchmark

• SimpleToolHalluBench is a benchmark that measures tool hallucination by testing LLM responses when required tools are absent or irrelevant.
• It employs 296 distinct tool-conditioned queries to isolate two failure modes—No-Tool-Available and Distractor-Tool—and supports causal analyses across various reasoning enhancements.
• Empirical findings show that enhanced chain-of-thought reasoning correlates with increased tool hallucination, highlighting a critical reliability–capability trade-off.

Searching arXiv for the cited paper and related benchmark context. SimpleToolHalluBench is a diagnostic benchmark for measuring tool hallucination in LLMs under conditions where external tool use is required but unsupported or misleadingly scaffolded. It was introduced in "The Reasoning Trap: How Enhancing LLM Reasoning Amplifies Tool Hallucination" as the principal instrument for testing whether strengthening reasoning increases tool hallucination (Yin et al., 27 Oct 2025). The benchmark isolates two failure modes—No-Tool-Available and Distractor-Tool—and was designed to support controlled causal analyses across reinforcement learning, supervised distillation, and inference-time chain-of-thought settings. In that role, it serves not as a general-purpose agent benchmark, but as a targeted probe of reliability failures that arise when models are encouraged to "think then act."

1. Definition and failure modes

SimpleToolHalluBench measures hallucination in two settings in which a user query explicitly requires an external tool, but the available environment either omits the required tool or supplies only irrelevant ones (Yin et al., 27 Oct 2025). The benchmark therefore operationalizes tool hallucination as an error of tool-grounding rather than as a generic factual mistake.

Setting	Environment condition	Hallucination criterion
No-Tool-Available (NTA)	The environment provides zero tools	The model fabricates a tool name, a function signature, or tool outputs it could not have obtained
Distractor-Tool (DT)	The environment provides one or more irrelevant tools	The model attempts to use the distractor or invents a new, unprovided tool

In the NTA setting, hallucination occurs if the model fabricates any of three elements: a tool name, a function signature, or tool outputs it could not have obtained. In the DT setting, hallucination occurs if the model either attempts to use the distractor or invents a new, unprovided tool (Yin et al., 27 Oct 2025).

This design makes the benchmark sharply diagnostic. Rather than asking whether a model can successfully complete a tool-using task under standard assumptions, it asks whether the model can correctly recognize that the requisite tool support is absent. This suggests that the benchmark is especially suited to studying failures of abstention, solvability recognition, and toolset calibration.

2. Construction and task composition

The benchmark is built from 296 distinct tools sampled from AgentSafetyBench. For each tool, a single user query is generated via ChatGPT-4o, with the constraint that no query can be answered without the correct tool. For every query, two evaluation instances are created, one for NTA and one for DT, yielding 592 total test examples (Yin et al., 27 Oct 2025).

The inputs are natural-language user queries that, in realistic deployment, would trigger a function call. Tool interfaces are described in JSONSchema. In the DT setting, one irrelevant schema—the distractor—is appended to the system prompt. This construction preserves the original task semantics while altering only the tool availability conditions.

The composition strategy is deliberately minimal. Each of the 296 tool-conditioned queries is duplicated into two controlled counterfactual environments: one with no tools and one with an irrelevant tool. A plausible implication is that the benchmark reduces confounding variation in task content, making it easier to attribute behavioral differences to reasoning enhancement or prompt conditions rather than to dataset heterogeneity.

3. Evaluation protocol and measured quantities

For each setting $s \in \{\mathrm{NTA}, \mathrm{DT}\}$, the hallucination rate is defined as

$R_s = \frac{H_s}{N_s},$

where $N_s$ is the total number of examples in setting $s$ and $H_s$ is the number judged hallucinatory. In SimpleToolHalluBench, $N_s = 296$ for both NTA and DT (Yin et al., 27 Oct 2025).

The benchmark is also used alongside additional metrics in the surrounding experimental program. For tool-reasoning tasks on SynTool, task reward is reported as a scalar in $[0,1]$. For mathematical reasoning on GSM8K, the reported quantity is standard percentage accuracy. For mechanistic analysis, the paper additionally uses a discrimination score over activations (Yin et al., 27 Oct 2025).

During inference, models produce internal reasoning within explicit tags and then emit tool calls in <tool>…</tool> format. This explicit separation between reasoning traces and tool invocation is important for interpreting hallucination events, because it exposes the model’s intermediate deliberation while preserving a distinct action channel (Yin et al., 27 Oct 2025).

4. Experimental use in reasoning-enhancement studies

SimpleToolHalluBench is used to compare multiple reasoning-enhancement methods: tool-specific RL, non-agentic RL, supervised distillation, and inference-only chain-of-thought. In tool-specific RL, Qwen2.5-7B-Instruct is fine-tuned via Group Relative Policy Optimization on ReCall’s SynTool setup to interleave reasoning with tool calls, with checkpoints saved every 100 steps. In non-agentic RL, the same algorithm is applied to GSM8K mathematics problems without tools. Supervised distillation is represented by DeepSeek-R1-Distill-Qwen-7B, and inference-only reasoning is tested with Qwen3-8B and Qwen3-32B by turning on "thinking" mode (Yin et al., 27 Oct 2025).

The central empirical finding is a causal relationship between stronger reasoning and higher tool hallucination. Under ReCall on SynTool, validation reward rises from approximately $0.3$ to approximately $0.55$, while $R_{\mathrm{NTA}}$ climbs from $R_s = \frac{H_s}{N_s},$0 to over $R_s = \frac{H_s}{N_s},$1 and $R_s = \frac{H_s}{N_s},$2 rises from $R_s = \frac{H_s}{N_s},$3 to $R_s = \frac{H_s}{N_s},$4 (Yin et al., 27 Oct 2025). Under GRPO on GSM8K, accuracy increases from approximately $R_s = \frac{H_s}{N_s},$5 to approximately $R_s = \frac{H_s}{N_s},$6, yet both hallucination rates again increase monotonically despite the absence of any tool supervision.

The method-agnostic pattern is also visible in fixed model comparisons.

Model	Configuration	$R_s = \frac{H_s}{N_s},$7	$R_s = \frac{H_s}{N_s},$8
Qwen2.5-7B-Instruct	Base	34.8	54.7
Qwen2.5-7B (DeepSeek-R1 SFT)	Distilled	74.3	78.7
Qwen3-8B	Thinking Disabled	4.1	36.2
Qwen3-8B	Thinking Enabled	5.4	56.8
Qwen3-32B	Thinking Disabled	5.1	46.6
Qwen3-32B	Thinking Enabled	8.8	50.7

Across these comparisons, enhanced reasoning always yields higher hallucination rates on both tasks (Yin et al., 27 Oct 2025). The experimental significance is that the effect is not restricted to one training regime. It appears when reasoning is optimized through RL, inherited through supervised fine-tuning, and merely elicited at inference by switching from direct answers to step-by-step thinking. This suggests that the benchmark is probing a general failure mode associated with reasoning enhancement rather than an idiosyncrasy of a single pipeline.

5. Mitigation attempts and the reliability–capability trade-off

The benchmark is also used to test mitigation strategies. One intervention is prompt engineering via the system instruction: "You must not use any tools that are not explicitly provided." This produces only marginal benefit: $R_s = \frac{H_s}{N_s},$9 drops from $N_s$0 to $N_s$1, $N_s$2 from $N_s$3 to $N_s$4, and SynTool reward changes from approximately $N_s$5 to approximately $N_s$6 (Yin et al., 27 Oct 2025).

A stronger intervention is Direct Preference Optimization. The preference pairs are constructed as honest abstention versus hallucination when tools are missing, and correct tool use versus needless refusal when tools are available. After DPO, hallucination is reduced more substantially, to $N_s$7 and $N_s$8, but SynTool reward falls from $N_s$9 to $s$0 (Yin et al., 27 Oct 2025).

These outcomes are presented as a fundamental reliability–capability trade-off. In the reported experiments, aggressive mitigation reduces hallucination at the cost of core tool-using utility. The benchmark is therefore not only diagnostic but also intervention-sensitive: it can register whether a method improves abstention while simultaneously showing whether that improvement erodes task reward.

A common misconception is that tool hallucination can be removed simply by adding an explicit prohibition against unprovided tools. The reported prompt-engineering result does not support that view. Another misconception is that hallucination is only a consequence of tool-specific overfitting. The GSM8K experiments, in which reasoning is strengthened on non-tool mathematics tasks and hallucination still increases, directly contradict that interpretation (Yin et al., 27 Oct 2025).

6. Mechanistic interpretation and relation to adjacent benchmarks

The paper associates SimpleToolHalluBench with two mechanistic findings. First, comparing post-RL Qwen2.5-7B against the base model using Centered Kernel Alignment, the authors report that in-distribution mathematics examples maintain CKA greater than $s$1 across all layers, whereas tool examples from SimpleToolHalluBench drop below $s$2 in early and middle layers. This is interpreted as a collapse of tool-related representations under reasoning RL (Yin et al., 27 Oct 2025).

Second, activation-level localization is performed by collecting vectors for correct versus hallucinated tool calls and training a linear classifier on attention outputs, MLP outputs, and residual streams. The discrimination score is defined as

$s$3

The strongest separability appears in late-layer residual streams, specifically layers 20–32, which achieve scores greater than $s$4, while attention and MLP outputs remain at approximately $s$5–$s$6 (Yin et al., 27 Oct 2025). The reported interpretation is that small differences accumulate through the residual pathway and ultimately diverge when the model hallucinates.

In comparative context, SimpleToolHalluBench is narrower than ToolBeHonest. ToolBeHonest evaluates hallucination through two perspectives—depth and breadth—using a multi-level diagnostic process across missing necessary tools, potential tools, and limited functionality tools, and it contains seven tasks with 700 evaluation samples (Zhang et al., 2024). By contrast, SimpleToolHalluBench focuses on two tightly controlled failure modes, NTA and DT, with 592 total examples (Yin et al., 27 Oct 2025). A plausible interpretation is that ToolBeHonest emphasizes broad diagnostic decomposition of tool-use failure, whereas SimpleToolHalluBench emphasizes causal sensitivity to reasoning enhancement under experimentally controlled conditions.

Taken together, the benchmark’s role is twofold. Empirically, it provides evidence that enhancing multi-step, chain-of-thought reasoning—via RL, distillation, or inference prompting—coincides with increased tool hallucination in the tested settings. Methodologically, it offers a compact diagnostic substrate for studying whether models can preserve tool fidelity when reasoning competence improves (Yin et al., 27 Oct 2025).