Finch Benchmark: Finance & Accounting Workflows

Updated 8 June 2026

Finch Benchmark is a comprehensive suite that defines real-world F&A workflows by replicating complex, multimodal spreadsheet-centric tasks encountered by enterprise analysts.
It integrates authentic datasets like Enron emails, financial spreadsheets, and PDFs to provide a realistic testbed for evaluating AI agents on financial operations.
It employs a rigorous two-stage, LLM-assisted and expert-verified methodology, revealing performance metrics such as a 38.4% success rate on complex workflows.

The term "Finch Benchmark" encompasses several domain-specific datasets, evaluation suites, programming frameworks, and hypotheses central to distinct fields, particularly finance & accounting AI agent evaluation, sparse tensor computation, financial Text-to-SQL, and RNN-based sequence models. This article focuses on the "Finch" benchmark for finance & accounting workflow evaluation as defined in "Finch: Benchmarking Finance & Accounting across Spreadsheet-Centric Enterprise Workflows" (Dong et al., 15 Dec 2025), with contextual reference to related specialized Finch-named evaluation artifacts where clarity is essential.

1. Definition and Motivation

The Finch benchmark is a comprehensive suite for evaluating AI agents on end-to-end, real-world finance and accounting (F&A) workflows. Unlike table QA or simplified spreadsheet benchmarks, Finch is explicitly constructed to replicate the composite, multimodal, and long-horizon workflows of practicing enterprise financial analysts. This involves tasks such as data entry, data structuring, formula updates, modeling, cross-file retrieval, validation, reporting, and translation under real enterprise constraints.

Motivation arises from the mismatch between the complexity of actual F&A work—interleaved across spreadsheets, emails, PDFs, charts, and code—and the artificial simplicity of prior benchmarks, which have generally emphasized small-scale, isolated QA or arithmetic tasks. Finch establishes a rigorous testbed reflecting genuine business processes and multimodal data flows encountered by F&A professionals in organizations (e.g., budgeting, asset management, trading), capturing the "in-the-wild" noise and messiness absent from laboratory-style datasets (Dong et al., 15 Dec 2025).

2. Dataset Composition and Structure

Finch derives its corpus from authentic enterprise financial environments, notably:

Enron Email & Spreadsheet Corpus: ~15,000 .xls/.xlsx files, 500,000 emails across 150 employees.
EUSES Spreadsheet Corpus: ~450 additional financial workbooks.
World Bank, Government Agency, Private Financial Data: PDFs, XML reports, proprietary artifacts.
WideSearch and DABStep: 13 labeled workflows adapted from domain-adjacent tasks.

Artifacts include spreadsheets with raw data, formulas, pivot tables, charts, images, PDFs, Word documents, markdowns, emails with attachments, and web snapshot tasks. The dataset comprises:

Statistic	Value
Workflows	172 end-to-end (composite)
Distinct Tasks	384
Involved Spreadsheets	1,710
Cells (inputs+outputs)	~27 million
Median cells/workflow	15,000 (mean 157,000; max 3.7 million)
Formula counts/workflow	median 212, mean 21.5k, up to 100k+
Multisheet/Multi-file	92.4% ≥2 sheets (avg 8, max 91); 86.6% >1 file (max 14)

This scale far exceeds prior "tabular QA" benchmarks, positioning Finch as genuinely enterprise-grade (Dong et al., 15 Dec 2025).

3. Workflow Construction and Annotation Methodology

Finch workflows are generated via a two-stage, LLM-assisted and expert-verified process:

LLM-Assisted Discovery:
- Email-thread mining: GPT-5 is prompted to surface threads specifying business goals tied to spreadsheets.
- Version-history differencing: Consecutive versions of spreadsheets are differenced, and GPT-5 drafts workflow type tags and instructions.
Expert Annotation and Editing:
- Human domain experts rewrite instructions, align input/reference files, and reconstruct missing artifacts.
- Inter-annotator agreement and a secondary LLM-as-judge (ChatGPT 5.1 Pro, Claude Sonnet 4.5) ensure instruction-output consistency.
- Workflows accepted only if: business goal is explicit and attachments align above strict cell alignment threshold (θ=1.0 for "strongly grounded").

This rigorous protocol ensures that each workflow embodies intentional, auditable analyst actions with precisely delineated inputs, outputs, and task instructions requiring over 700 expert hours for annotation.

4. Task Taxonomy, Compositionality, and Pipeline Complexity

Finch defines a taxonomy encompassing ten high-level task types:

Data Entry / Import (44 workflows)
Structuring / Formatting (86)
Calculation (119)
Cross-Sheet/Cross-File Retrieval (36)
Validation / Review (37)
Summary / Visualization (33)
Financial Modeling (15)
Web Search (11)
Translation (3)
Reporting / QA (implicit in many workflows)

Workflows exhibit compositionality: 21.5% are single-task; 78.5% are multi-task (up to 5 tasks), often chaining multiple modalities and requiring persistent state across interleaved steps. Typical pipelines may traverse data import → structuring → retrieval → calculation → visualization or narrative, and agents must manage cross-artifact context and state maintenance throughout (Dong et al., 15 Dec 2025).

5. Agent Evaluation Protocol and Metrics

Finch evaluates both interactive (product-style) and API-based (code-generation) AI agents:

Web/interactive (GPT 5.1 Pro, Claude Sonnet 4.5): Multi-turn conversations with web browsing enabled.
API-based (e.g., GPT 5.1, Gemini 3 Pro, Qwen 3 Max): One-shot code-generation using Python (openpyxl, pandas); execution in sandbox with no retries.

Input and outputs are encoded via Address–Value–Type–Formula tuples (for spreadsheets) and Markdown tables; PDFs/images handled natively or with OCR text extraction.

Primary metric: Workflow Success Rate (SR = N_pass / N_total), a binary pass/fail at the workflow level. Automated LLM-as-judge (GPT-5-mini) compares concise diffs between model and reference outputs, with rationale. Final adjudication relies on human expert grading.

Agent	Workflows Passed	Success Rate (%)
GPT 5.1 Pro (web)	66 / 172	38.4
Claude Sonnet 4.5 (web)	43 / 172	25.0
API GPT 5.1 (auto)	32.0	(automated)
API Claude 4.5, others	13–20	(range)

Automated judge-human agreement for web agents is high (GPT 5.1 Pro: judge/human acc = 82.1%, recall = 83.3%, precision = 73.6%). Pass rates decline sharply with workflow complexity: ≤2 task workflows pass at 44.3%, while >2 task chains drop to 23.5% (GPT 5.1 Pro). By task type, financial modeling, calculation, and reporting/QA outperform (up to 50%), while translation, data entry/import, and structuring/formatting consistently underperform (10–35%) (Dong et al., 15 Dec 2025).

6. Error Analysis and Key Findings

Qualitative errors for frontier models (Claude Sonnet 4.5 web, for instance):

Task Misunderstanding: 10%. Agents miss implicit domain context.
Data Retrieval Errors: 25%. Wrong sheet or cell range used.
Formula Reasoning: 35%. Logical/syntactic formula errors, dropped calculations.
Code Generation: 25%. Python code misaligned with workbook structure.
Rendering/Formatting: 5%. Unintended changes to artifact structure.

Average runtime per workflow for GPT 5.1 Pro is 16.8 minutes, increasing with task count. For 4–5 task workflows, time increases but most are not solved.

Finch surfaces the limitations of current agents on enterprise-grade F&A: even with 48 hours of GPT 5.1 Pro run time, only 38.4% of workflows are successfully navigated. Successes are skewed toward simpler or more formulaic tasks, while compositionality and artifact diversity remain major bottlenecks (Dong et al., 15 Dec 2025).

7. Challenges, Open Problems, and Prospective Advances

Major technical challenges identified include:

Long-horizon composition and error accumulation.
Large-scale cross-sheet/cross-file navigation.
Dense, jargon-laden, semantically nuanced financial contexts.
Complex table structures: Merged headers, blank rows, subtotals.
Formula semantic extraction: Business logic encoded in formulas is routinely missed by agents that only track cell values.
Multimodal artifact reasoning: Integration of spreadsheet, PDF, and image content.

Recommended research directions:

Improved spreadsheet-centric pretraining; cell-level provenance tracking.
Hybrid agent frameworks coupling LLMs with spreadsheet solvers and vision models.
Interactive code execution-feedback-correction protocols.
Continued domain adaptation using actual F&A corpora (email threads, version histories).
Explicit formal verification, such as consistency checks and subtotals unit tests.

A plausible implication is that advances in spreadsheet state tracking, formula graph reasoning, and multimodal embedding-alignment could meaningfully increase agent headroom on Finch.

8. Position within the FinanceAI Benchmarking Ecosystem

Finch should be distinguished from:

BigFinanceBench (Wang et al., 2 Jun 2026): Targets open-ended financial research and audit-chained derivations rather than spreadsheet workflows.
FINCH (Text-to-SQL) (Singh et al., 2 Oct 2025): A financial NL-to-SQL benchmark; focuses on schema complex query translation, with its own FINCH Score metric.
Finch (Tensor Programming) (Ahrens et al., 2024): Not F&A; rather, a programming language and benchmarking suite for sparse/structured tensor computation.
Finch (RWKV-6) (Peng et al., 2024): Refers to a recurrent sequence model, unrelated to finance and accounting tasks.

Each shares only the nominal "Finch" branding, and direct comparison of results or task scope should only occur within their respective domains.

In summary, Finch (Dong et al., 15 Dec 2025) is the first benchmark providing a rigorous, in-the-wild measurement of AI agent capabilities across real-world, spreadsheet-centric finance and accounting workflows. It exposes both the persistent headroom and technical barriers remaining for robust enterprise deployment of AI in F&A, while offering a foundation for future work in multimodal, artifact-aware, compositional agent design.