---
title: 'BFCL-V3: Berkeley Function-Calling Benchmark'
url: https://www.emergentmind.com/topics/berkeley-function-calling-leaderboard-bfcl-v3
type: topic
---

# BFCL-V3: Berkeley Function-Calling Benchmark

Berkeley Function-Calling Leaderboard version 3 (BFCL-V3) is a benchmark suite for evaluating function-calling and agentic tool-use systems under both single-turn and multi-turn conditions. In recent literature, it is used to test whether a model can select relevant tools, fill arguments correctly, coordinate multi-step invocations, preserve conversation state, and handle long-context and missing-function scenarios. At the same time, BFCL-V3 has become a focal point for methodological debate, because several studies argue that AST-based or reference-trajectory-based scoring can diverge from semantic correctness, final task success, and procedural robustness [2605.30738][2603.05553][2510.24645].

## 1. Benchmark scope and task structure

BFCL-V3 is presented in later work as a benchmark for realistic function calling rather than isolated syntax production. In the multi-turn regime, it is described as a set of **200 curated, state-dependent tasks inspired by real-world APIs**, with success scored by the **final environment state** rather than by a single response string [2512.00332]. Other papers treat BFCL-V3 as a broader leaderboard with multi-turn, non-live, live, relevance, and irrelevance components, and with published breakdowns that expose different failure modes and interaction settings [2512.23611].

A published task breakdown used for BFCL-V3-style evaluation is summarized below.

| Block | Subcategories | Reported role |
|---|---|---|
| Multi-Turn | Base; Miss Func; Miss Param; Long Cont. | Stateful multi-turn evaluation |
| Non-Live | Simple; Multiple; Parallel; Multi Para. | AST-based evaluation |
| Live | Simple; Multiple; Parallel; Multi Para. | Live or real-record evaluation |
| Auxiliary | Relevance; Irrelevance | Hallucination-oriented behavior |

At a finer granularity, the **BFCL-V3 Multiple** split is described as a set of **200 contiguous tasks** in which each task contains a user query plus **2–4 candidate function schemas**, and the model must emit a JSON function call with the correct function name and arguments [2604.02155]. This complements the multi-turn setting by isolating function routing and argument selection under a constrained candidate set.

Recent work also characterizes BFCL-V3 as an extension of earlier BFCL versions. One paper states that BFCL v3 **“extends prior versions by incorporating long-context reasoning and missing-function scenarios,”** which helps explain why it is routinely used as a stress test for agentic interaction rather than merely for single-call formatting [2605.30738].

## 2. Capability profile and what BFCL-V3 is designed to expose

BFCL-V3 is repeatedly framed as a benchmark for **multi-turn, multi-step function calling with explicit state tracking**. In that framing, success is not only a matter of producing a correct final call; it depends on composing tool use over time, preserving intermediate state, and managing interaction structure across turns [2605.30738]. This is why scaffolded systems such as MAVEN emphasize a loop of context buffering, atomic subtask synthesis, tool selection, execution, storage of outputs and diagnostics, and revision when verification fails: these are precisely the operations BFCL-V3 is meant to stress [2605.30738].

The benchmark also exposes distinct subproblems inside function calling. On the **BFCL-V3 Multiple** split, a systematic chain-of-thought budget study reports a three-way error decomposition into **hallucinated function names**, **wrong valid function** selection, and **wrong arguments** given the right function. For **Qwen2.5-1.5B-Instruct**, accuracy is **44.0%** at **\(d=0\)**, rises to **64.0%** at **\(d=32\)**, then collapses to **25.0%** at **\(d=256\)**. The dominant error at **\(d=0\)** is **wrong valid function selection: 30.5%** of tasks; brief reasoning reduces this to **1.5%**, while long reasoning reverses the gain and raises hallucinated functions to **18.0%** at **\(d=256\)** [2604.02155]. Within the benchmark’s own logic, this indicates that BFCL-V3 is especially sensitive to function routing, and only secondarily to downstream argument instantiation.

FunReason-MT’s use of BFCL-V3 supports the same interpretation from the multi-turn side. Its reported submetrics—**Base**, **Miss Func**, **Miss Param**, and **Long Context**—treat failures in missing function selection, missing parameter completion, and context retention as distinct benchmark phenomena rather than as a single aggregate error class [2510.24645].

## 3. Evaluation methodology and disputes over semantic validity

A persistent criticism of BFCL-V3 concerns the gap between surface-form matching and actual functional correctness. MAVEN explicitly notes that BFCL-V3’s **AST-based evaluation may not fully capture semantic correctness** in real-world tool use [2605.30738]. EigenData goes further and argues that, in the original benchmark, evaluation often rewarded matching a reference transcript rather than achieving the correct outcome [2603.05553].

EigenData’s audit of the multi-turn BFCL-V3 suite reports three major error classes.

| Issue class | Affected entries | Reported examples |
|---|---:|---|
| Function schema issues | 39 (19.5%) | Type mismatches; missing required parameters; contradictory descriptions |
| Function implementation bugs | 91 (45.5%) | Logic errors; input-handling defects; state-management bugs |
| Trajectory and evaluation problems | 82 (41.0%) | Overly rigid matching; incorrect labels; missing alternative paths |

After deduplication, EigenData concludes that **143 of 200 multi-turn BFCL-V3 entries (71.5%)** had at least one such problem [2603.05553]. The significance of this claim is methodological rather than incidental: if schemas are inconsistent, executable environments are buggy, or alternative valid trajectories are rejected, then leaderboard scores become a noisy proxy for real tool-use competence.

To address this, the repaired benchmark replaces trace-equivalent scoring with **outcome-aware evaluation**. In the repaired protocol, correctness is assessed through state-based checking of whether **predicted database changes match ground truth**, whether **key functions were correctly invoked**, and whether **critical information was correctly handled and conveyed**; these checks are synthesized by a **VerificationFunctionAgent** [2603.05553]. The paper reports that this repaired **All Three** metric is better aligned with human judgments than the original BFCL leaderboard. Two examples are especially illustrative: **GPT-5.2** had an original BFCL multi-turn score of **28.12%** but a human pass rate of **65.0%**, whereas **GLM-4.6** had an original BFCL multi-turn score of **68.00%** but a human pass rate of **54.8%** [2603.05553].

## 4. BFCL-V3 as a target for model design, decoding, and data generation

Because BFCL-V3 spans multi-turn interaction, live and non-live execution, and relevance behavior, it has become a common target for methods that claim improved tool-calling generalization. Reported scores are heterogeneous and not directly comparable across all rows below, because some refer to total leaderboard accuracy and others to split-specific results.

| Method or setup | BFCL-V3 setting | Reported score |
|---|---|---:|
| MAVEN | BFCL v3 Multi-Turn | 58.50 |
| InfTool-32B | BFCL v3 Total | 70.9 |
| FunReason-MT (Qwen3-4B, RL) | Multi-Turn Overall | 56.50 |
| Qwen2.5-1.5B with 32-token CoT | BFCL-V3 Multiple | 64.0 |

MAVEN treats BFCL-V3 as a transfer benchmark for **cross-benchmark robustness** rather than as a retraining target. In a paired table on **BFCL Multi-Turn**, it reports **GPT-5: 33.5** versus **MAVEN (GPT-5): 51.5**, and **Llama-4-Maverick: 23.5** versus **MAVEN (Llama-4-Maverick): 46**. These correspond to gains of **+18.0** and **+22.5** points, respectively, under the same benchmark setting [2605.30738]. The paper’s explanation is architectural: externally enforced decomposition and verification reduce error accumulation in long-horizon tool-use tasks.

InfTool uses BFCL-V3 as its primary benchmark because of its support for advanced multi-turn and multi-step assessment, including state inconsistencies. It reports **70.9 Total** for **InfTool-32B** and **61.7 Total** for **InfTool-7B**, and states that the base **32B** model improves from **19.8%** to **70.9%** on BFCL. For the 32B system it further highlights **59.0** on **Multi-turn** and **87.5** on **live parallel execution** [2512.23611]. The same paper also notes a versioning caveat: although BFCL had released **version 4**, the authors were unable to reproduce most BFCL-v4 results and therefore restricted evaluation to BFCL v3 [2512.23611].

FunReason-MT uses BFCL-V3 to evaluate multi-turn data synthesis for a **Qwen3-4B-Instruct-2507** backbone. It reports **Multi-Turn Overall: 15.75** for the base model, **46.90** after **FunReason-MT (SFT)**, and **56.50** after **FunReason-MT (RL)**. On the same benchmark, **Single-Turn Overall** increases from **78.19** to **81.97** to **85.02** [2510.24645]. Here BFCL-V3 functions as a measure of whether synthetic trajectories improve both stateful interaction and simpler single-turn function calling.

BFCL-V3 has also been used to study decoding behavior directly. On the **Multiple** split, brief reasoning budgets outperform both zero-CoT and long-CoT conditions, with a pronounced optimum near **8–32 tokens**. The paper proposes **Function-Routing CoT (FR-CoT)**, which achieves **64.0%** on **Qwen2.5-1.5B** and **83.0%** on **Qwen2.5-7B**, while reducing hallucinated function names to **0.0%** [2604.02155]. This establishes BFCL-V3 as a benchmark not only for model training but also for test-time control of reasoning.

## 5. Robustness, procedural compliance, and security

A central misconception in reading BFCL-V3 scores is that high end-task accuracy implies robust deployment behavior. Several papers explicitly reject that conclusion. The A-CC framework is introduced as a **complement to BFCL and BFCL-V3, not a replacement**, precisely because BFCL-style leaderboards can rank a model highly while remaining **blind to procedural robustness** [2512.00332].

A-CC defines **assertion-conditioned compliance** through a **compliance rate (CR)**: whether the model **invokes the asserted function at least once within the same turn in which the assertion is made**. It distinguishes **user-sourced assertions (USAs)** from **function-sourced assertions (FSAs)** and decomposes outcomes into **S→S**, **S→F**, **F→S**, and **F→F**. On the BFCL multi-turn backbone, baseline success ranges from **43.1%** to **80.7%**, with a macro-average of **63.2%**. Under assertions, the macro-averaged CR is **36.3%** for confident USAs and **27.7%** for hedged USAs; for FSAs it is **31.4%** confident and **22.1%** hedged. The paper reports worst-case success drops of up to **23.4 percentage points** and an average worst-case degradation of **16.9 points** [2512.00332]. The important implication is that BFCL final-state success and procedural compliance are only partially coupled.

A different robustness study builds a **200-case** benchmark from a challenging single-turn BFCL subset and perturbs it in two ways: meaning-preserving query rephrasing and toolkit expansion with semantically related tools. The original toolkit has **2.7 tools on average**; the expanded one has **5.6 tools on average**. Reported AST accuracy drops by **8%–19%** under paraphrase and by **1%–8%** under toolkit expansion across the tested models [2504.00914]. However, the same paper argues that **70–90% of errors** in the rephrasing condition arise from **parameter value mismatch**, suggesting that part of the apparent robustness loss is a metric artifact caused by exact matching of semantically equivalent parameter values [2504.00914].

Security work extends the critique further by showing that the benchmarked tool-selection process itself is attackable. A function hijacking attack evaluated on **200 BFCL payloads**, each with **2 to 4** available functions, reaches **Function Name ASR** between **0.70** and **1.00** and **Slot Filling ASR** between **0.65** and **0.92** across the tested models [2604.20994]. In that setting, BFCL no longer functions only as an accuracy benchmark; it becomes a measurement surface for adversarial manipulation of tool choice.

## 6. Broader research role and emerging directions

BFCL-V3 is increasingly used not only to rank models but also to study orchestration, efficiency, and deployment constraints. AsyncLM evaluates on **v3-base-multi-turn** and on a derived **v3-multi-step-parallel** set, and reports **1.6x–5.4x** reductions in end-to-end task completion latency relative to synchronous function calling [2412.07017]. The same work argues that BFCL-style evaluation should eventually distinguish between synchronous correctness and asynchronous orchestration competence, since two systems with similar call accuracy can have very different completion latency profiles [2412.07017].

Edge-device studies use BFCL-style multi-turn evaluation to expose scale sensitivity under constrained deployment. One such study reports **65.74% overall accuracy** and **55.62% multi-turn accuracy** for **xLAM-2-3b-fc-r**, compared with **45.76% overall** and **1.38% multi-turn** for **Qwen3-0.6B**, and **0.00% multi-turn** for **TinyLlama-1.1B-32k-Inst** and **TinyAgent-1.1B** [2511.22138]. In this line of work, BFCL-V3-style multi-turn evaluation serves as a stringent threshold for determining whether compact models are suitable for on-device agents.

Taken together, these findings suggest that BFCL-V3 now occupies several roles at once. It remains a leaderboard for tool selection, argument correctness, and interaction success, but it also functions as a substrate for benchmark repair, provenance-aware robustness analysis, adversarial security evaluation, decoding studies, and latency-aware systems work [2603.05553][2512.00332][2412.07017]. A plausible implication is that BFCL-V3’s long-term importance lies less in any single score than in the way it organizes disputes over what should count as correct, robust, and semantically faithful tool use.

Source: https://www.emergentmind.com/topics/berkeley-function-calling-leaderboard-bfcl-v3