LABBench2: AI Benchmark for Practical Biology
- LABBench2 is a multi-category benchmark designed to evaluate real-world AI capabilities in practical biology research tasks.
- It comprises 1,912 tasks across literature retrieval, data access, protocol troubleshooting, molecular biology assistance, and experiment planning.
- The benchmark emphasizes realistic inputs and evaluation protocols to mimic genuine research workflows and improve scientific AI performance.
LABBench2 is a large, multi-category benchmark designed to measure how well AI systems can perform practical biology research tasks rather than answer textbook questions. Introduced as an evolution of the original Language Agent Biology Benchmark (LAB-Bench), it is, for the most part, a continuation of LAB-Bench that measures similar capabilities in more realistic contexts, using predominantly open-response tasks, heterogeneous scientific artifacts, and tool-oriented workflows. The benchmark comprises 1,912 tasks spanning literature retrieval and understanding, data access, protocol troubleshooting, molecular biology assistance, and experiment planning, and is presented as a benchmark for measuring the real-world capabilities of AI systems performing useful scientific tasks in biology (Laurent et al., 4 Feb 2026).
1. Conceptual motivation and scope
LABBench2 is situated within a broader shift in AI for science from static question answering toward systems that retrieve information, run tools, and orchestrate multi-step workflows. The paper places it in a landscape where current applications of AI in scientific research range from training dedicated foundation models on scientific data to agentic autonomous hypothesis generation systems to AI-driven autonomous labs. Against that background, the stated need is to measure progress not only on rote knowledge or reasoning, but on the ability to perform meaningful work in scientific domains (Laurent et al., 4 Feb 2026).
The benchmark is explicitly designed around practical biology research tasks. Its central motivation is that benchmarks should test “real-world capabilities” such as finding and reading primary literature, including figures, tables, and supplements; accessing and using specialized biological databases; designing and troubleshooting wet-lab protocols; and manipulating long DNA sequences while planning experiments. This design moves evaluation closer to actual research workflows, where the primary difficulty is often source identification, document navigation, format heterogeneity, and exact execution rather than isolated factual recall (Laurent et al., 4 Feb 2026).
Relative to LAB-Bench, LABBench2 changes both form and target. LAB-Bench relied heavily on multiple-choice questions and simplified contexts, whereas LABBench2 uses open-response tasks almost everywhere, embeds questions in realistic research settings, adds new task families such as patents, clinical trials, and source-quality assessment, and expands the benchmark to more than 1,900 tasks. The authors present this as an attempt to preserve continuity with the earlier benchmark while restoring headroom for progress after substantial model improvements on LAB-Bench (Laurent et al., 4 Feb 2026).
2. Benchmark structure and task families
LABBench2 comprises 1,912 tasks organized into five broad capability families with finer subcategories. The five families are literature retrieval and understanding, data access, protocol troubleshooting, molecular biology assistance, and experiment planning. Literature retrieval and understanding includes LitQA3, FigQA2, TableQA2, SuppQA2, PatentQA, TrialQA, and SourceQualQA; data access is represented by DbQA2; protocol troubleshooting by ProtocolQA2; molecular biology assistance by SeqQA2 and CloningQA; and experiment planning is primarily embodied in CloningQA (Laurent et al., 4 Feb 2026).
| Category | Variant | Count |
|---|---|---|
| LitQA3 | – | 168 |
| PatentQA | – | 121 |
| TrialQA | – | 120 |
| SourceQuality (SourceQualQA) | – | 150 |
| DbQA2 | – | 86 |
| SuppQA2 | – | 125 |
| ProtocolQA2 | – | 125 |
| SeqQA2 | – | 400* |
| CloningQA | – | 14* |
| FigQA2 | retrieve | 101 |
| FigQA2 | pdf (paper) | 101 |
| FigQA2 | img (image) | 101 |
| TableQA2 | retrieve | 100 |
| TableQA2 | pdf (paper) | 100 |
| TableQA2 | img (image) | 100 |
| Total | 1912 |
*SeqQA2 and CloningQA have multiple “sequence delivery” modes; the counts refer to base items, and the same logical item can be run under multiple input-modality conditions (Laurent et al., 4 Feb 2026).
Across these families, LABBench2 targets literature reasoning and retrieval, data access and database navigation, protocol understanding and troubleshooting, molecular biology sequence skills, and experiment planning and protocol synthesis. The benchmark therefore covers both information-seeking tasks, such as finding the correct paper, patent, trial, or database record, and exact manipulation tasks, such as designing primers, performing sequence operations, and specifying detailed cloning workflows in a structured format (Laurent et al., 4 Feb 2026).
A distinctive feature is the inclusion of SourceQualQA, which measures whether a system can explain why a study is not appropriate evidence for a particular research question. This broadens the scope from document retrieval and extraction to scientific discernment. Another distinctive feature is CloningQA, which turns experiment planning into a directly evaluable task by requiring structured protocol design rather than informal narrative answers (Laurent et al., 4 Feb 2026).
3. Realism of inputs, outputs, and contexts
LABBench2 emphasizes realistic, heterogeneous inputs. Inputs include free-text prompts, full PDFs of scientific papers including figures and tables, image files for figures and tables, supplementary files such as PDFs, spreadsheets, and CSVs, sequence files, database records accessed by tools, and patent and clinical trial documents. This diversity is a deliberate departure from simplified prompts that isolate relevant evidence in advance (Laurent et al., 4 Feb 2026).
The output side is likewise oriented toward research practice. Most tasks require open-response short answers. Agentic setups may involve multi-step reasoning and retrieval actions, although the benchmark measures final answer correctness. SeqQA2 uses unconstrained text graded via custom verifiers, while CloningQA uses a custom domain-specific language describing reagents and protocol steps in a structured, machine-parseable format. SourceQualQA requires explanations of exclusion rationale rather than categorical labels alone (Laurent et al., 4 Feb 2026).
Several design choices are explicitly intended to increase realism relative to LAB-Bench. In FigQA2 and TableQA2 “paper” mode, the task is to find and interpret the correct figure or table within a full PDF containing distracting context. In LitQA3, PatentQA, and TrialQA, no context is given, so the system must retrieve the correct document and then answer. SuppQA2 includes heterogeneous supplementary artifacts such as PDFs, Excel files, CSVs, figures, and non-standard formats. Sequence tasks may involve inputs that are often 3,000+ bp and may arrive through files or retrieval rather than inline text (Laurent et al., 4 Feb 2026).
This design is meant to mimic actual scientific workflows, where the main challenge is often not a lack of conceptual knowledge but the need to locate the right source, parse complex artifacts, and perform precise operations with tool support. By moving away from multiple-choice and toward open response under realistic context, LABBench2 makes answer generation, retrieval robustness, and exactness central to evaluation (Laurent et al., 4 Feb 2026).
4. Task construction methodology
The benchmark uses different construction pipelines for different task families. Literature tasks other than SourceQualQA—namely LitQA3, FigQA2, TableQA2, PatentQA, TrialQA, and SuppQA2—were constructed by contracted domain experts, described as biologists with or pursuing PhDs, using a custom web platform. Experts were provided with collections of papers, figures, and tables, and could also contribute their own chosen papers. Questions had to be answerable only using the specified source or source fragment, and answers could not be contradicted by any other part of the same document or by other sources. Retrieval-style tasks had to contain enough detail that a domain expert could reliably identify the unique target document. Multiple rounds of review by external experts and internal scientific experts at Edison Scientific/FutureHouse checked correctness, source fidelity, and uniqueness of retrieval questions (Laurent et al., 4 Feb 2026).
SourceQualQA is constructed from 100 open-access systematic reviews, each addressing a specific evidence-based medicine question. For each review, the authors extracted a precise causal research question and used excluded studies together with human-written exclusion justifications to create prompts asking why a study did not provide appropriate evidence for that question. The final set was manually checked to ensure that an expert could answer using the research question together with the study design and content, and that the exclusion rationale was epistemically salient rather than purely checklist-mechanical. This framing makes SourceQualQA a test of scientific discernment rather than rubric compliance alone (Laurent et al., 4 Feb 2026).
DbQA2 measures data access across specialized biological databases. Its construction began with automatic analysis, using an LLM, of approximately 1,000 recent bioRxiv papers to detect where input data came from and where output data was deposited. The most common data sources were ranked by frequency, then manually curated and extended by internal experts, yielding 43 specific databases. Internal and contracted experts then wrote questions that targeted non-trivial and often obscure fields, with the stated aim of emphasizing information unlikely to be memorized during LLM pretraining. As with the literature tasks, multiple review rounds were used to ensure answerability, uniqueness, and correctness (Laurent et al., 4 Feb 2026).
SeqQA2 is largely programmatically generated. It comprises 20 distinct sequence-related subtasks defined by an in-house biologist via question templates, with each template instantiated into 20 variant questions using different sequences, either designed or real genes, for a total of 400 items. The tasks are delivered in three input modalities: inject mode, where sequences are embedded inline; file mode, where sequences are supplied via attached files; and retrieval mode, where the model must obtain sequences from an external source through tools. For subtasks with many valid answers, such as PCR primer design, evaluation uses custom verification functions, for example by running an in silico PCR to determine whether the returned primers amplify the correct target region (Laurent et al., 4 Feb 2026).
ProtocolQA2 extends LAB-Bench’s ProtocolQA by sourcing protocols from public repositories such as protocols.io and STAR Methods, as well as expert-submitted protocols, including some unpublished ones. Experts inject a single critical error, such as an incorrect incubation temperature, so that the protocol would unambiguously fail, and then frame a scenario in which a hypothetical experimental outcome must be explained by identifying one important error. CloningQA generalizes the earlier CloningScenarios into full protocol design tasks covering restriction/ligation cloning, Gibson assembly, and Golden Gate assembly. It requires the design of an entire cloning strategy, including DNA fragments, primers, enzymes, and experimental steps, all encoded in a machine-parseable domain-specific language that can be verified in silico (Laurent et al., 4 Feb 2026).
5. Capability dimensions and evaluation protocol
LABBench2 operationalizes “real-world capabilities” through several dimensions: information retrieval and document navigation, scientific comprehension and reasoning, database literacy, experimental troubleshooting, molecular design and exact manipulation, and scientific discernment. These dimensions are not treated as separate theoretical constructs only; they are tied directly to task families and scoring procedures. For example, document navigation appears in figure and table retrieval within full PDFs, database literacy appears in DbQA2 through schema and identifier navigation, and molecular exactness appears in SeqQA2 and CloningQA through sequence fidelity and structured design (Laurent et al., 4 Feb 2026).
The evaluation is accuracy-based, with custom verifiers where needed. Many tasks are graded against reference answers using string or pattern matching, possibly with normalization. SeqQA2 and CloningQA depart from pure string matching: correctness is determined functionally. In SeqQA2, primer design can be checked with in silico PCR; in CloningQA, the DSL is parsed and the cloning plan is simulated to verify that the designed constructs and steps produce the desired result. Aggregated metrics are reported per subtask, per mode, and per broad family (Laurent et al., 4 Feb 2026).
The paper describes the standard accuracy measure as
Figures report family-level summaries that behave as macro-like averages across items within each family. The paper also explicitly states that there is no mention of LLM-as-a-judge or rubric-based scoring; grading is rule-based or verifier-based. This is methodologically important because it constrains evaluation to explicit answer criteria and executable validation rather than post hoc model judgment (Laurent et al., 4 Feb 2026).
A recurrent misconception would be to treat LABBench2 as a benchmark of biology knowledge alone. Its design instead binds knowledge to retrieval, localization, tool use, format handling, and exact manipulation. Another misconception would be to interpret the benchmark as evaluating full scientific autonomy. The paper is explicit that many tasks remain granular and that the benchmark does not include full wet-lab execution or long-horizon iterative research cycles (Laurent et al., 4 Feb 2026).
6. Empirical findings and benchmark significance
The paper evaluates “current frontier models” under two regimes: base models without tools and tool-augmented models with web search and code execution. The figures explicitly mention Gemini 3 Pro, Claude Opus 4.5, and GPT-5.2 Pro, although the main emphasis is on comparative behavior across task families and modes rather than exhaustive system descriptions. The central empirical result is that LABBench2 is substantially harder than LAB-Bench: across all shared families and models, accuracy drops by 26–46 percentage points when moving from LAB-Bench to LABBench2 on corresponding task families (Laurent et al., 4 Feb 2026).
Performance patterns differ sharply by family. Tool augmentation gives strong gains on retrieval-heavy literature tasks such as LitQA3, PatentQA, and TrialQA. SuppQA2 remains notably lower than main-text retrieval, even with tools, likely because of heterogeneous supplementary-information formats. DbQA2 remains one of the hardest families: even with tools, models struggle to choose the right database, navigate complex interfaces and schemas, and extract precise fields without confusion. In FigQA2 and TableQA2, image-only mode is best, paper mode is harder, and retrieval mode is hardest, indicating a substantial gap between interpreting a provided figure and locating the correct figure within a retrieved full paper (Laurent et al., 4 Feb 2026).
SeqQA2 and CloningQA reveal strong modality effects. Inline mode is typically the best or near-best configuration. For SeqQA2, file mode is nearly equivalent to inline for models that support file input correctly. For CloningQA, file-based tasks can be easier than injected tasks for some tool-augmented models, specifically Gemini 3 Pro and Claude Opus 4.5, likely because long sequences are easier to handle as files than as inline strings. Retrieval mode is very poor for all models, highlighting limitations in retrieving specific sequences from external sources. Within SeqQA2, the hardest subtasks are those requiring exact subsequence handling, such as primer design and amino acid identification from nucleotide sequence, whereas global operations and calculation-heavy tasks such as GC content and enzyme kinetics benefit substantially from code execution (Laurent et al., 4 Feb 2026).
The discussion identifies three broad failure modes. First, retrieval and localization failures: systems often retrieve the wrong document, trial, or patent, fail to locate the correct figure or table within a large PDF, or miss relevant supplementary content. Second, fragile handling of exact inputs: even conceptually simple sequence operations can fail when strings are long, copied incorrectly, truncated, or mishandled by tools. Third, scientific discernment failures: SourceQualQA suggests that systems trained to follow checklists or structured rubrics may not surface the most important reason a study is inappropriate, because true judgment requires combining methodological critique with relevance assessment. Taken together, these observations explain why high performance on earlier, less realistic benchmarks does not transfer directly to LABBench2 (Laurent et al., 4 Feb 2026).
7. Use cases, limitations, and relation to adjacent benchmarks
LABBench2 is intended for model developers building foundation models specialized for biology, tool-augmented LLM agents for scientific research, agent framework designers working on autonomous hypothesis generation and AI-driven autonomous labs, and evaluation researchers who need de facto standard benchmarks for real biology capabilities. The authors explicitly intend it to continue LAB-Bench’s role as a de facto benchmark for AI scientific research capabilities. They recommend evaluating both base and tool-augmented systems, reporting results by family, subtask, and input modality, and using the released evaluation harness to avoid ad hoc grading. The dataset is hosted at https://huggingface.co/datasets/futurehouse/labbench2, and the public evaluation harness is hosted at https://github.com/EdisonScientific/labbench2 (Laurent et al., 4 Feb 2026).
The benchmark’s limitations are stated directly. LABBench2 does not evaluate full wet-lab execution; tasks stop at planning and troubleshooting. Most tasks are granular rather than long-horizon and do not model iterative research over days or weeks. Automated verifiers and grading rules remain imperfect for tasks with multiple valid solutions, partial credit, or ambiguous outcomes. The scope is substantial but not exhaustive, focusing on literature, data access, molecular biology, and protocols while leaving areas such as complex systems biology, advanced modeling, and clinical decision-making out of scope. The paper also notes possible biases arising from expert choice of source papers, protocols, and datasets, as well as from PubMed and bioRxiv sampling for database identification (Laurent et al., 4 Feb 2026).
Future work is framed around long-horizon, multi-step workflows, improved evaluation for ambiguous and open-ended tasks, subdomain-focused extensions such as drug development, and eventual integration with physical or robotic labs. A plausible implication is that LABBench2 is intended less as a terminal benchmark than as an extensible foundation for increasingly composite “AI Scientist” evaluations. In a related but distinct direction, LabOSBench benchmarks computer-use agents for scientific instrument control through browser-native instrument simulators and execution-based evaluation. It does not explicitly mention LABBench2, but it is conceptually complementary: LABBench2 concentrates on literature, data access, protocol troubleshooting, and molecular biology assistance, whereas LabOSBench focuses on feedback-driven scientific-instrument GUI control. This suggests a broader emerging benchmark ecosystem for AI systems performing laboratory work at different layers of abstraction (Zou et al., 15 Jun 2026).