Spreadsheet Gym: Structured Spreadsheet Evaluation

• Spreadsheet Gym is a standardized environment that defines spreadsheet tasks, interfaces, and evaluation criteria to support training, benchmarking, and operational control.
• It integrates multi-turn interactive Excel sessions with code sandboxes for reinforcement learning, improving model generation accuracy on benchmark tasks.
• It underpins various benchmark stations—like NL2Formula, SpreadsheetBench, and SpreadsheetArena—that test formula synthesis, manipulation, and full workbook generation.

Spreadsheet Gym is used across recent spreadsheet research to denote a structured setting for spreadsheet-centered reasoning, automation, documentation, and governance. In the most explicit contemporary usage, it is a multi-turn interactive Microsoft Excel environment integrated with a code sandbox for reinforcement learning; in adjacent work, it appears as a workflow-level testbed for spreadsheet agents, a focused station for spreadsheet-operations documentation, and a broader benchmark-and-evaluation paradigm for end-to-end workbook generation (Chi et al., 21 May 2026, Zhu et al., 29 Jun 2026, Indika et al., 22 Oct 2025, Kundurthy et al., 16 Feb 2026). Earlier systems foreshadow the same idea through centralized spreadsheet services, web-based spreadsheet platforms, component repositories, model-driven generators, and composition platforms that replace ad hoc file handling with controlled interfaces and auditable workflows (0908.1584, Guthrie et al., 2011, 0809.3584, 0803.1875, Baglietto et al., 2011). This suggests that Spreadsheet Gym is best understood as an umbrella concept: a standardized environment in which spreadsheet tasks, interfaces, constraints, and evaluation criteria are made explicit enough to support training, benchmarking, and operational control.

1. Origins in spreadsheet risk, complexity, and end-user modeling

The intellectual background of Spreadsheet Gym lies in longstanding work on spreadsheet risk. One strand emphasizes that risk is not confined to model construction: even when a master spreadsheet is well engineered, routine usage by end-users remains risky because of data entry errors, formula overwrites, version confusion, broken links, aggregation mistakes, lack of data centralization, and exposure of proprietary logic (0908.1584). Another strand argues that many of these difficulties stem from the traditional “cell-matrix” interaction paradigm, which is appropriate for presentation but becomes increasingly problematic for creation, maintenance, and comprehension once spreadsheets pass a “minimal threshold of complexity” (0803.1875).

Spreadsheet style research makes this risk quantitative. Bewig’s formulation treats correctness as a process of design, testing, documentation, and review rather than a matter of careful typing, and gives the cascade relation

$P(\text{error}) = 1 - (1 - e)^n,$

where $e$ is the per-cell error probability and $n$ is the length of a dependency chain (Bewig, 2013). That perspective aligns naturally with gym-like training: the environment should not only test final outputs, but also expose whether a model or user can manage cascades, validation, assertions, and review discipline.

A complementary line of work treats the spreadsheet grid itself as a modeling surface rather than merely a ledger. In Excel-based Sudoku modeling, overlapping ranges encode a “three dimensional model in a two dimensional space,” and the spreadsheet is designed so that “a reviewer can evaluate each step in the process” (Allen, 2014). This is relevant because Spreadsheet Gym is not limited to finance or formula filling; it also inherits the broader idea that spreadsheet competence includes spatial reasoning, explicit intermediate states, and transparent constraint propagation.

Formula understandability research adds another human-centered layer. Across experiments with spreadsheet professionals, the number of ranges, nesting depth, presence of conditional operations, reverse references, same-column alignment, and calculation-chain length were all shown to matter for comprehension in different ways, whereas simple reference count alone was not strongly predictive (Hermans et al., 2012). A plausible implication is that a mature Spreadsheet Gym must evaluate not only functional correctness, but also whether generated or maintained formulas remain inspectable by future users.

2. Pre-agent architectural patterns

Long before LLM-centric environments, several systems instantiated core Spreadsheet Gym principles by constraining interaction, centralizing models, and separating roles. EASA wrapped critical spreadsheets in a web-based application layer so that users interacted through validated forms rather than raw workbooks; authors used a codeless application builder, underlying spreadsheets remained centralized on servers, and the case study reports that a typical EASAP took 2–8 hours to build and that Amlin’s RDS process was “error-free since the deployment with EASA” (0908.1584). The architectural logic is gym-like: constrain the input space, hide model internals, and enforce version control and auditability.

Hypernumbers generalized that logic into a server-centric spreadsheet platform. It treated spreadsheets as web pages in a hierarchy, exposed role-specific views—spreadsheet view for makers, wikipage view for data inputers, webpage view for consumers—and integrated templates, permissions, logging, backups, and hierarchical “z-queries” such as =sum(/some/page/[a1 > 44]/b7) (Guthrie et al., 2011). The platform reportedly supported deployments with 10,000+ spreadsheet pages used by tens of users. Here the gym principle is not merely safer interaction, but a redefinition of spreadsheets as governed multi-user applications rather than files.

Model-driven alternatives pursued the same goal from the opposite direction. Sunsight Modeller replaced direct cell editing with a Business Algebra Model written in plain-language business terms—time frames, category hierarchies, reports, and equations—from which spreadsheets were generated automatically (0803.1875). The spreadsheet became the presentation layer; the textual model became the source of truth. A related modularization effort proposed a repository of reusable spreadsheet components represented in Excelsior, where users supplied input and output ranges and the component was reshaped to fit the target spreadsheet (0809.3584). In both cases, Spreadsheet Gym appears as structured reuse: train users or agents to compose models from specifications and components instead of improvising cell logic.

Other systems emphasized distributed composition and multi-dimensional modeling. DISCOM defined spreadsheet composition in terms of exported and imported cell ranges, intermediate spreadsheets, spaces, and server-side propagation using XML/SOAP/HTTP, SpringWS, Apache POI, and MySQL (Baglietto et al., 2011). PivotModel reframed complex Excel workbooks as multi-dimensional models with explicit dimensions, hierarchies, and rule logic; in the illustrative case, a $14 \times 5 \times 5 \times 9 \times 4 = 12{,}600$-cell logical space contained 1,728 data cells and 10,872 calculation cells, yet the rule engine centralized the logic into 12 rules (Litt, 2018). These systems show that Spreadsheet Gym can mean not only “benchmark,” but also an operational regime in which spreadsheet state transitions are mediated by explicit metadata, model structure, and propagation rules.

A more programmatic lineage appears in the batch spreadsheet system “SS,” which treated spreadsheets as batch-processed, C-flavored numerical laboratories. SS used plain text command files, a separate symbol table, cyclic formulas, iterative evaluation, and user-defined functions in C loaded through a dynamic link library (Perry, 2012). That design anticipates modern agent environments by making spreadsheet execution scriptable, repeatable, and tightly coupled to an external toolchain.

3. Benchmark stations and task taxonomies

Recent work decomposes Spreadsheet Gym into benchmark stations targeting distinct capabilities: formula synthesis, spreadsheet manipulation, workflow completion, documentation, end-to-end workbook generation, and RL training in a real spreadsheet runtime. The scope of the major resources is summarized below.

Resource	Primary task	Stated scope
NL2Formula	NL query + table $\rightarrow$ executable formula	70,799 pairs; 21,670 tables; 37 function types
SODBench	xwAPI code $\rightarrow$ natural-language documentation	111 validated tasks; evaluation on $N=107$
SpreadsheetBench	Real-world manipulation from forum questions	912 instructions; 2,729 spreadsheet files; avg 3 test cases
SpreadsheetBench 2	End-to-end business workflows	321 tasks; avg 11.8 worksheets; 593.5 modified cells
SpreadsheetArena	Blind pairwise evaluation of workbook generation	436 prompts; 4,357 pairwise votes; 16 LLMs
Spreadsheet-RL / Spreadsheet Gym	Multi-turn RL in Excel 365	Pass@1 on SpreadsheetBench improved from 12.0% to 23.4%

These resources span complementary task definitions (Zhao et al., 2024, Indika et al., 22 Oct 2025, Ma et al., 2024, Zhu et al., 29 Jun 2026, Kundurthy et al., 16 Feb 2026, Chi et al., 21 May 2026). NL2Formula formalizes grounded semantic parsing over spreadsheets with

$F = f_{\theta}(N; T),$

where $N$ is the natural-language query, $T$ is the spreadsheet table, and $e$0 is the target Excel formula (Zhao et al., 2024). Its dataset contains 70,799 paired NL queries and formulas, covering 21,670 tables and 37 types of formula functions, and establishes formula generation as one foundational gym task.

SpreadsheetBench shifts from isolated formulas to real-world spreadsheet manipulation. It is built from 912 real questions gathered from online Excel forums, paired with 2,729 spreadsheet files, and uses multiple test cases per instruction in an online-judge style protocol (Ma et al., 2024). SpreadsheetBench 2 moves further to workflow-level tasks in generation, debugging, and visualization. Its 321 tasks are built from authentic business data, average 11.8 worksheets and 593.5 modified cells, and explicitly target cross-sheet reasoning, business modeling, and long-horizon planning (Zhu et al., 29 Jun 2026).

SODBench, introduced in “SODBench: A LLM Approach to Documenting Spreadsheet Operations,” formalizes Spreadsheet Operations Documentation as the reverse of typical spreadsheet automation: executed spreadsheet operations, captured as xwAPI code, are translated into brief stepwise natural-language explanations (Indika et al., 22 Oct 2025). The paper explicitly characterizes SODBench as a first, focused “Spreadsheet Gym station,” because it isolates one capability—code-to-documentation—within a standardized input-output and scoring setup. SpreadsheetArena broadens the gym to open-ended generation by asking models to produce full workbooks that are judged through blind pairwise preference battles over SheetSpec@2 outputs (Kundurthy et al., 16 Feb 2026).

Taken together, these benchmark stations imply that Spreadsheet Gym is not a single task family. It is a layered suite in which forward execution, reverse documentation, debugging, visualization, preference-sensitive generation, and robust generalization are all separate but interoperable competencies.

4. Interfaces, representations, and environment design

A defining feature of Spreadsheet Gym is the use of canonical representations that bridge heterogeneous spreadsheet environments. In SODBench, spreadsheet operations are expressed in xwAPI, the atomic spreadsheet action API from SheetCopilot, with calls such as Write, AutoFill, Filter, CreateChart, SetChartLegend, CreateSheet, and CopyPaste; the paper argues that Excel VBA, Google Apps Script / JavaScript, and the Excel JavaScript API can be mapped to xwAPI through simple syntax substitution (Indika et al., 22 Oct 2025). This makes xwAPI an interface layer suitable for both forward execution and backward explanation.

SpreadsheetArena addresses a different interface problem by standardizing workbook generation through the SheetSpec@2 schema. A SheetSpec workbook contains sheets, cells, optional namedRanges, optional conditionalFormats, outputs, and rules, and all participating models must return valid JSON conforming to that schema (Kundurthy et al., 16 Feb 2026). The schema-based interface enables pairwise preference evaluation over complete workbooks rather than raw text.

Interactive agent benchmarks rely on tool harnesses. SpreadsheetBench 2 uses a unified multi-turn scaffold with bash, view_xlsx, and submit inside a sandboxed Docker environment with Python 3.11, NumPy, Pandas, Matplotlib, openpyxl, LibreOffice UNO, and a 50-turn episode cap (Zhu et al., 29 Jun 2026). The primary inspection tool, view_xlsx, exposes sheet names and content ranges, while bash lets the agent manipulate .xlsx files through scripts. Spreadsheet-RL makes that loop spreadsheet-native and Excel-specific: its Spreadsheet Gym runs Microsoft Excel 365 on Windows as the authoritative engine, integrates a Python sandbox, and exposes inspect_range, find_cells, fill_formula, clear_range, delete_rows, delete_columns, recalculate_and_read, and code_interpreter, together with tool-routing rules that serialize write actions and allow up to 20 read-only tool calls per assistant turn (Chi et al., 21 May 2026).

These environment choices matter because spreadsheet semantics are not reducible to plain tables. Spreadsheet-RL explicitly relies on real Excel to support modern dynamic-array functions such as FILTER, UNIQUE, SORT, TAKE, and MAP, which alternative engines may miss or implement differently (Chi et al., 21 May 2026). SpreadsheetBench 2 likewise treats spreadsheet inspection and modification as tool-mediated actions over full workbooks rather than as static prompt context (Zhu et al., 29 Jun 2026). Historical systems reach similar conclusions through different means: Hypernumbers introduced hierarchy-aware formulas and structural functions, DISCOM standardized import/export ranges and propagation, Excelsior separated abstract tables from physical layout, and SS used scripted batch evaluation with cyclic formulas and symbols (Guthrie et al., 2011, Baglietto et al., 2011, 0809.3584, Perry, 2012). Across these designs, Spreadsheet Gym emerges as an interface problem as much as a modeling problem.

5. Evaluation paradigms and empirical findings

Spreadsheet Gym research uses several distinct evaluation paradigms, reflecting the heterogeneity of spreadsheet work. SpreadsheetBench formalizes robust manipulation with online-judge style soft and hard restrictions:

$e$1

$e$2

so that a solution is rewarded not only for solving one spreadsheet instance but for generalizing across perturbed test cases (Ma et al., 2024). On this benchmark, GPT-4o reaches 18.35 soft and 15.02 hard in the single-round setting, while human performance on the evaluated subset reaches 71.33 soft and 62.00 hard (Ma et al., 2024). This gap is one of the clearest empirical arguments for Spreadsheet Gym as a nontrivial training regime.

Workflow-level evaluation adds stricter notions of end-to-end completion. SpreadsheetBench 2 defines Modification as the fraction of target cells whose computed value matches gold and Accuracy as a task-level all-cells match criterion, with visualization scored by rubric pass rate under a VLM judge (Zhu et al., 29 Jun 2026). The best reported model, Claude Opus 4.6, achieves 34.89% overall task accuracy and 77.20 overall Modification, while debugging accuracy is only 12.00%. The paper’s failure taxonomy identifies insufficient spreadsheet inspection and wrong target selection as dominant bottlenecks, suggesting that spreadsheet agency fails more often at locating and scoping edits than at generating local operations correctly (Zhu et al., 29 Jun 2026).

Formula-level and documentation-level stations use different metrics. NL2Formula evaluates with Formula EM and Execution Result Assessment; Coder-Large reaches 70.6 Formula EM and 77.1 ERA, substantially outperforming GPT-3.5 10-shot prompting on the same benchmark (Zhao et al., 2024). SODBench evaluates five LLMs using BLEU, GLEU, ROUGE-L, and METEOR over $e$3 task instances; GPT-4o-mini, GPT-4o, and LLaMA-3.3-70B form the top tier, and Appendix E reports no statistically significant differences among those three top models $e$4 (Indika et al., 22 Oct 2025). The SOD authors explicitly note that these are text-similarity metrics rather than task-success metrics, and propose more semantic or behavioral objectives for future work.

Preference-based workbook generation introduces yet another evaluation geometry. SpreadsheetArena uses Bradley–Terry / Elo estimation, with win probability

$e$5

and a feature-augmented variant

$e$6

to decompose preferences into observable spreadsheet features (Kundurthy et al., 16 Feb 2026). The results show that pct_text, pct_fill, compute_pct_numeric, log_col_count, and pct_number_format positively affect global preferences, while compute_error_rate, log_aspect_ratio, largest_table_pct, and log_num_blank_rows hurt them. Yet the finance expert study finds average overall quality around 2.87 on a 1–5 rubric and agreement with arena winners in only about 56% of decisive comparisons, indicating that crowd preference and domain-specific best practice are not equivalent (Kundurthy et al., 16 Feb 2026).

Spreadsheet-RL turns spreadsheet evaluation into an RL reward signal. Its terminal reward is

$e$7

and RL fine-tuning improves Qwen3-4B-Thinking-2507’s Pass@1 on SpreadsheetBench from 12.0% to 23.4%, and on Domain-Spreadsheet from 8.4% to 17.2% (Chi et al., 21 May 2026). During training, mean reward rises from approximately 0.21 to approximately 0.33 while mean response length and mean turns per episode decrease, indicating more efficient interaction. Taken together, these results show that Spreadsheet Gym already supports exact-match, execution-based, preference-based, rubric-based, and reward-based evaluation, but no single metric family fully captures spreadsheet quality.

6. Applications, misconceptions, and future directions

Spreadsheet Gym has both research and operational applications. In documentation, SOD is proposed to support reproducibility, audits, collaboration, knowledge transfer, and training of spreadsheet-intelligent agents; the paper also describes a RAG pipeline that scrapes Excel JavaScript API documentation, builds a Chroma vector store with 1129 HTML docs, uses GPT-4o-mini plus LangChain, and attains Exec@1 = 60% and Pass@1 = 20% on 20 benchmark tasks (Indika et al., 22 Oct 2025). In enterprise control, EASA, Hypernumbers, and DISCOM use centralized spreadsheets, validated interfaces, visibility controls, logging, spaces, or role-specific views to reduce risks from routine spreadsheet use (0908.1584, Guthrie et al., 2011, Baglietto et al., 2011). In planning and forecasting, PivotModel uses dimensions, hierarchies, and rule logic to support write-back, ragged reporting, and diagnostics within Excel (Litt, 2018). In modular engineering, component libraries and model-driven generation aim to replace copy-paste with reusable, parameterized spreadsheet structures (0809.3584, 0803.1875).

Several misconceptions recur in the literature. First, Spreadsheet Gym is not synonymous with one RL environment: recent work uses the term for a realistic Microsoft Excel runtime, but earlier and parallel work uses gym-like architectures for web-based spreadsheet services, component composition, documentation stations, and workflow benchmarks (Chi et al., 21 May 2026, 0908.1584, Indika et al., 22 Oct 2025). Second, it is not restricted to forward automation from language to formulas or actions; reverse-direction tasks such as Spreadsheet Operations Documentation are treated as core capabilities because reproducibility, audits, and collaboration depend on explaining what was done, not merely on doing it (Indika et al., 22 Oct 2025). Third, high preference scores do not imply adherence to domain-specific spreadsheet practice: SpreadsheetArena shows that even highly ranked models do not reliably produce spreadsheets aligned with finance conventions (Kundurthy et al., 16 Feb 2026). Fourth, model size alone is not determinative: in SODBench, GPT-4o-mini matches or exceeds larger open models, and the top three models are statistically indistinguishable on the reported metrics (Indika et al., 22 Oct 2025).

The stated future directions are broad but coherent. NL2Formula calls for more string and text functions, horizontal and vertical tables, multi-table reasoning, larger inputs, and support for lambda functions and DAX (Zhao et al., 2024). SpreadsheetBench 2 calls for better inspection strategies, stronger planning and memory, richer visualization tasks, and expansion beyond finance and business into scientific and engineering spreadsheets (Zhu et al., 29 Jun 2026). Spreadsheet-RL highlights scaling to larger models, extending tool coverage to pivot tables, charts, conditional formatting, and macros, and improving reward shaping beyond binary terminal outcomes (Chi et al., 21 May 2026). SpreadsheetArena points toward larger preference datasets, stronger automated evaluators, and tighter integration between crowd preference and expert rubrics (Kundurthy et al., 16 Feb 2026). Older platform work adds further enterprise directions: XML schema validation, data versioning, ECM integration, and richer governance of distributed spreadsheet composition (Baglietto et al., 2011).

In aggregate, Spreadsheet Gym names a convergence. Spreadsheet research is moving from isolated formulas and ad hoc spreadsheet use toward explicit environments in which tasks are typed, interfaces are standardized, intermediate states are inspectable, and success is measured under reproducible protocols. Whether the immediate goal is safer human use, better spreadsheet engineering, or stronger LLM agents, the common principle is the same: spreadsheet competence becomes a trainable and testable discipline rather than an informal by-product of manipulating cells.