BFCL-V3: Berkeley Function-Calling Benchmark

• BFCL-V3 is a benchmark for evaluating multi-turn function calling, emphasizing state tracking and real-world API simulation.
• It tests tool selection, argument correctness, and error decomposition across single-turn and multi-turn conditions.
• The benchmark spurs debates on evaluation methodology, semantic correctness, and procedural robustness in tool-use systems.

Berkeley Function-Calling Leaderboard version 3 (BFCL-V3) is a benchmark suite for evaluating function-calling and agentic tool-use systems under both single-turn and multi-turn conditions. In recent literature, it is used to test whether a model can select relevant tools, fill arguments correctly, coordinate multi-step invocations, preserve conversation state, and handle long-context and missing-function scenarios. At the same time, BFCL-V3 has become a focal point for methodological debate, because several studies argue that AST-based or reference-trajectory-based scoring can diverge from semantic correctness, final task success, and procedural robustness (Ghugarkar et al., 29 May 2026, Chen et al., 5 Mar 2026, Xu et al., 28 Oct 2025).

1. Benchmark scope and task structure

BFCL-V3 is presented in later work as a benchmark for realistic function calling rather than isolated syntax production. In the multi-turn regime, it is described as a set of 200 curated, state-dependent tasks inspired by real-world APIs, with success scored by the final environment state rather than by a single response string (Waqas et al., 29 Nov 2025). Other papers treat BFCL-V3 as a broader leaderboard with multi-turn, non-live, live, relevance, and irrelevance components, and with published breakdowns that expose different failure modes and interaction settings (Li et al., 29 Dec 2025).

A published task breakdown used for BFCL-V3-style evaluation is summarized below.

Block	Subcategories	Reported role
Multi-Turn	Base; Miss Func; Miss Param; Long Cont.	Stateful multi-turn evaluation
Non-Live	Simple; Multiple; Parallel; Multi Para.	AST-based evaluation
Live	Simple; Multiple; Parallel; Multi Para.	Live or real-record evaluation
Auxiliary	Relevance; Irrelevance	Hallucination-oriented behavior

At a finer granularity, the BFCL-V3 Multiple split is described as a set of 200 contiguous tasks in which each task contains a user query plus 2–4 candidate function schemas, and the model must emit a JSON function call with the correct function name and arguments (Qi, 2 Apr 2026). This complements the multi-turn setting by isolating function routing and argument selection under a constrained candidate set.

Recent work also characterizes BFCL-V3 as an extension of earlier BFCL versions. One paper states that BFCL v3 “extends prior versions by incorporating long-context reasoning and missing-function scenarios,” which helps explain why it is routinely used as a stress test for agentic interaction rather than merely for single-call formatting (Ghugarkar et al., 29 May 2026).

2. Capability profile and what BFCL-V3 is designed to expose

BFCL-V3 is repeatedly framed as a benchmark for multi-turn, multi-step function calling with explicit state tracking. In that framing, success is not only a matter of producing a correct final call; it depends on composing tool use over time, preserving intermediate state, and managing interaction structure across turns (Ghugarkar et al., 29 May 2026). This is why scaffolded systems such as MAVEN emphasize a loop of context buffering, atomic subtask synthesis, tool selection, execution, storage of outputs and diagnostics, and revision when verification fails: these are precisely the operations BFCL-V3 is meant to stress (Ghugarkar et al., 29 May 2026).

The benchmark also exposes distinct subproblems inside function calling. On the BFCL-V3 Multiple split, a systematic chain-of-thought budget study reports a three-way error decomposition into hallucinated function names, wrong valid function selection, and wrong arguments given the right function. For Qwen2.5-1.5B-Instruct, accuracy is 44.0% at $d=0$, rises to 64.0% at $d=32$, then collapses to 25.0% at $d=256$. The dominant error at $d=0$ is wrong valid function selection: 30.5% of tasks; brief reasoning reduces this to 1.5%, while long reasoning reverses the gain and raises hallucinated functions to 18.0% at $d=256$ (Qi, 2 Apr 2026). Within the benchmark’s own logic, this indicates that BFCL-V3 is especially sensitive to function routing, and only secondarily to downstream argument instantiation.

FunReason-MT’s use of BFCL-V3 supports the same interpretation from the multi-turn side. Its reported submetrics—Base, Miss Func, Miss Param, and Long Context—treat failures in missing function selection, missing parameter completion, and context retention as distinct benchmark phenomena rather than as a single aggregate error class (Xu et al., 28 Oct 2025).

3. Evaluation methodology and disputes over semantic validity

A persistent criticism of BFCL-V3 concerns the gap between surface-form matching and actual functional correctness. MAVEN explicitly notes that BFCL-V3’s AST-based evaluation may not fully capture semantic correctness in real-world tool use (Ghugarkar et al., 29 May 2026). EigenData goes further and argues that, in the original benchmark, evaluation often rewarded matching a reference transcript rather than achieving the correct outcome (Chen et al., 5 Mar 2026).

EigenData’s audit of the multi-turn BFCL-V3 suite reports three major error classes.

Issue class	Affected entries	Reported examples
Function schema issues	39 (19.5%)	Type mismatches; missing required parameters; contradictory descriptions
Function implementation bugs	91 (45.5%)	Logic errors; input-handling defects; state-management bugs
Trajectory and evaluation problems	82 (41.0%)	Overly rigid matching; incorrect labels; missing alternative paths

After deduplication, EigenData concludes that 143 of 200 multi-turn BFCL-V3 entries (71.5%) had at least one such problem (Chen et al., 5 Mar 2026). The significance of this claim is methodological rather than incidental: if schemas are inconsistent, executable environments are buggy, or alternative valid trajectories are rejected, then leaderboard scores become a noisy proxy for real tool-use competence.

To address this, the repaired benchmark replaces trace-equivalent scoring with outcome-aware evaluation. In the repaired protocol, correctness is assessed through state-based checking of whether predicted database changes match ground truth, whether key functions were correctly invoked, and whether critical information was correctly handled and conveyed; these checks are synthesized by a VerificationFunctionAgent (Chen et al., 5 Mar 2026). The paper reports that this repaired All Three metric is better aligned with human judgments than the original BFCL leaderboard. Two examples are especially illustrative: GPT-5.2 had an original BFCL multi-turn score of 28.12% but a human pass rate of 65.0%, whereas GLM-4.6 had an original BFCL multi-turn score of 68.00% but a human pass rate of 54.8% (Chen et al., 5 Mar 2026).

4. BFCL-V3 as a target for model design, decoding, and data generation

Because BFCL-V3 spans multi-turn interaction, live and non-live execution, and relevance behavior, it has become a common target for methods that claim improved tool-calling generalization. Reported scores are heterogeneous and not directly comparable across all rows below, because some refer to total leaderboard accuracy and others to split-specific results.

Method or setup	BFCL-V3 setting	Reported score
MAVEN	BFCL v3 Multi-Turn	58.50
InfTool-32B	BFCL v3 Total	70.9
FunReason-MT (Qwen3-4B, RL)	Multi-Turn Overall	56.50
Qwen2.5-1.5B with 32-token CoT	BFCL-V3 Multiple	64.0

MAVEN treats BFCL-V3 as a transfer benchmark for cross-benchmark robustness rather than as a retraining target. In a paired table on BFCL Multi-Turn, it reports GPT-5: 33.5 versus MAVEN (GPT-5): 51.5, and Llama-4-Maverick: 23.5 versus MAVEN (Llama-4-Maverick): 46. These correspond to gains of +18.0 and +22.5 points, respectively, under the same benchmark setting (Ghugarkar et al., 29 May 2026). The paper’s explanation is architectural: externally enforced decomposition and verification reduce error accumulation in long-horizon tool-use tasks.

InfTool uses BFCL-V3 as its primary benchmark because of its support for advanced multi-turn and multi-step assessment, including state inconsistencies. It reports 70.9 Total for InfTool-32B and 61.7 Total for InfTool-7B, and states that the base 32B model improves from 19.8% to 70.9% on BFCL. For the 32B system it further highlights 59.0 on Multi-turn and 87.5 on live parallel execution (Li et al., 29 Dec 2025). The same paper also notes a versioning caveat: although BFCL had released version 4, the authors were unable to reproduce most BFCL-v4 results and therefore restricted evaluation to BFCL v3 (Li et al., 29 Dec 2025).

FunReason-MT uses BFCL-V3 to evaluate multi-turn data synthesis for a Qwen3-4B-Instruct-2507 backbone. It reports Multi-Turn Overall: 15.75 for the base model, 46.90 after FunReason-MT (SFT), and 56.50 after FunReason-MT (RL). On the same benchmark, Single-Turn Overall increases from 78.19 to 81.97 to 85.02 (Xu et al., 28 Oct 2025). Here BFCL-V3 functions as a measure of whether synthetic trajectories improve both stateful interaction and simpler single-turn function calling.

BFCL-V3 has also been used to study decoding behavior directly. On the Multiple split, brief reasoning budgets outperform both zero-CoT and long-CoT conditions, with a pronounced optimum near 8–32 tokens. The paper proposes Function-Routing CoT (FR-CoT), which achieves 64.0% on Qwen2.5-1.5B and 83.0% on Qwen2.5-7B, while reducing hallucinated function names to 0.0% (Qi, 2 Apr 2026). This establishes BFCL-V3 as a benchmark not only for model training but also for test-time control of reasoning.

5. Robustness, procedural compliance, and security

A central misconception in reading BFCL-V3 scores is that high end-task accuracy implies robust deployment behavior. Several papers explicitly reject that conclusion. The A-CC framework is introduced as a complement to BFCL and BFCL-V3, not a replacement, precisely because BFCL-style leaderboards can rank a model highly while remaining blind to procedural robustness (Waqas et al., 29 Nov 2025).

A-CC defines assertion-conditioned compliance through a compliance rate (CR): whether the model invokes the asserted function at least once within the same turn in which the assertion is made. It distinguishes user-sourced assertions (USAs) from function-sourced assertions (FSAs) and decomposes outcomes into S→S, S→F, F→S, and F→F. On the BFCL multi-turn backbone, baseline success ranges from 43.1% to 80.7%, with a macro-average of 63.2%. Under assertions, the macro-averaged CR is 36.3% for confident USAs and 27.7% for hedged USAs; for FSAs it is 31.4% confident and 22.1% hedged. The paper reports worst-case success drops of up to 23.4 percentage points and an average worst-case degradation of 16.9 points (Waqas et al., 29 Nov 2025). The important implication is that BFCL final-state success and procedural compliance are only partially coupled.

A different robustness study builds a 200-case benchmark from a challenging single-turn BFCL subset and perturbs it in two ways: meaning-preserving query rephrasing and toolkit expansion with semantically related tools. The original toolkit has 2.7 tools on average; the expanded one has 5.6 tools on average. Reported AST accuracy drops by 8%–19% under paraphrase and by 1%–8% under toolkit expansion across the tested models (Rabinovich et al., 1 Apr 2025). However, the same paper argues that 70–90% of errors in the rephrasing condition arise from parameter value mismatch, suggesting that part of the apparent robustness loss is a metric artifact caused by exact matching of semantically equivalent parameter values (Rabinovich et al., 1 Apr 2025).

Security work extends the critique further by showing that the benchmarked tool-selection process itself is attackable. A function hijacking attack evaluated on 200 BFCL payloads, each with 2 to 4 available functions, reaches Function Name ASR between 0.70 and 1.00 and Slot Filling ASR between 0.65 and 0.92 across the tested models (Belkhiter et al., 22 Apr 2026). In that setting, BFCL no longer functions only as an accuracy benchmark; it becomes a measurement surface for adversarial manipulation of tool choice.

6. Broader research role and emerging directions

BFCL-V3 is increasingly used not only to rank models but also to study orchestration, efficiency, and deployment constraints. AsyncLM evaluates on v3-base-multi-turn and on a derived v3-multi-step-parallel set, and reports 1.6x–5.4x reductions in end-to-end task completion latency relative to synchronous function calling (Gim et al., 2024). The same work argues that BFCL-style evaluation should eventually distinguish between synchronous correctness and asynchronous orchestration competence, since two systems with similar call accuracy can have very different completion latency profiles (Gim et al., 2024).

Edge-device studies use BFCL-style multi-turn evaluation to expose scale sensitivity under constrained deployment. One such study reports 65.74% overall accuracy and 55.62% multi-turn accuracy for xLAM-2-3b-fc-r, compared with 45.76% overall and 1.38% multi-turn for Qwen3-0.6B, and 0.00% multi-turn for TinyLlama-1.1B-32k-Inst and TinyAgent-1.1B (Haque et al., 27 Nov 2025). In this line of work, BFCL-V3-style multi-turn evaluation serves as a stringent threshold for determining whether compact models are suitable for on-device agents.

Taken together, these findings suggest that BFCL-V3 now occupies several roles at once. It remains a leaderboard for tool selection, argument correctness, and interaction success, but it also functions as a substrate for benchmark repair, provenance-aware robustness analysis, adversarial security evaluation, decoding studies, and latency-aware systems work (Chen et al., 5 Mar 2026, Waqas et al., 29 Nov 2025, Gim et al., 2024). A plausible implication is that BFCL-V3’s long-term importance lies less in any single score than in the way it organizes disputes over what should count as correct, robust, and semantically faithful tool use.