LeanPhysBench: Verified Scientific and Physics Benchmark

• LeanPhysBench is a benchmark framework that delivers machine-checked formal verification for both scientific computing and physics reasoning using Lean 4.
• Its scientific computing instantiation, LeanLJ, rigorously validates molecular simulation energy calculations with a proof-to-program bridge over real and floating-point numbers.
• The Lean4PHYS variant curates 200 physics problems that mix textbook and competition levels, emphasizing unit-aware reasoning and domain-specific formalization.

Searching arXiv for LeanPhysBench and closely related Lean benchmarks to ground the article in current papers. LeanPhysBench denotes a Lean 4–based benchmark paradigm for machine-checked scientific and physical reasoning, but the published literature uses the name in two distinct senses. In "Benchmarking Energy Calculations Using Formal Proofs," LeanPhysBench is instantiated as a benchmark for formally verified scientific computing centered on molecular simulation energy calculations, with LeanLJ as the concrete artifact and the NIST Standard Reference Simulation Website as the numerical reference point (Ugwuanyi et al., 14 May 2025). In "Lean4Physics: Comprehensive Reasoning Framework for College-level Physics in Lean4," LeanPhysBench is a college-level benchmark for formal physics reasoning in Lean4, embedded in the broader Lean4PHYS framework and coupled to the domain library PhysLib (Li et al., 30 Oct 2025). This suggests that the term currently functions less as the name of a single canonical dataset than as a benchmark label for Lean-based, physics-adjacent formalization efforts.

1. Scope, nomenclature, and published instantiations

The two published uses of the name differ in both benchmark object and evaluation protocol.

Usage	Defining paper	Core object
LeanPhysBench as formally verified scientific computing	"Benchmarking Energy Calculations Using Formal Proofs" (Ugwuanyi et al., 14 May 2025)	Verified molecular simulation energy calculations in Lean 4
LeanPhysBench within Lean4PHYS	"Lean4Physics: Comprehensive Reasoning Framework for College-level Physics in Lean4" (Li et al., 30 Oct 2025)	200 hand-crafted and peer-reviewed Lean4 physics statements

In the first usage, the benchmark centers on a pure numerical kernel whose mathematical properties are proved in Lean and whose executable outputs are compared against a gold-standard scientific reference. In the second, the benchmark centers on theorem statements derived from physics textbooks and competition problems, with success measured by Lean-compilable proof generation. The common denominator is not a fixed task format but a commitment to Lean-level verification as the primary correctness criterion.

This naming overlap is potentially misleading. A common misconception is that LeanPhysBench denotes a single, stable benchmark analogous to a standardized shared task. The current literature does not support that reading. Instead, it presents two benchmark instantiations that share domain ambition—formal reasoning for physics or scientific computation in Lean—but operationalize that ambition differently (Ugwuanyi et al., 14 May 2025).

2. LeanPhysBench as formally verified molecular simulation

In its 2025 scientific-computing instantiation, LeanPhysBench targets the computation of pairwise interaction energy for particles in a cubic box under periodic boundary conditions, together with the long-range correction tail contribution. The underlying model is the Lennard-Jones pair potential with cutoff radius $r_c$:

$$V(r)= \begin{cases} 4\varepsilon \left[\left(\frac{\sigma}{r}\right)^{12}-\left(\frac{\sigma}{r}\right)^6\right], & r\le r_c\ 0, & r>r_c \end{cases}$$

with total pair energy

$$U_{\text{pair}}=\sum_{i=1}^{N}\sum_{j=i+1}^{N}V(r_{ij}), \quad r_{ij}\le r_c.$$

Periodic wrapping is defined coordinate-wise, for example

$$x_{i,\text{wrapped}} = x_i - L\cdot \mathrm{round}\!\left(\frac{x_i}{L}\right),$$

and the minimum-image distance is

$$r_{ij}=\sqrt{ \left(\Delta x - L\cdot \mathrm{round}\!\left(\frac{\Delta x}{L}\right)\right)^2+ \left(\Delta y - L\cdot \mathrm{round}\!\left(\frac{\Delta y}{L}\right)\right)^2+ \left(\Delta z - L\cdot \mathrm{round}\!\left(\frac{\Delta z}{L}\right)\right)^2}.$$

The long-range correction is given by

$$U_{LRC}=\frac{1}{2}4\pi\rho\int_{r_c}^{\infty}r^2V(r)\,dr,$$

and the paper derives the closed-form expression implemented in Lean. The executable system realizing this benchmark is LeanLJ, described as a Lean package of definitions, proofs, and executable code for the Lennard-Jones pair potential under periodic boundary conditions (Ugwuanyi et al., 14 May 2025).

The central technical feature is the proof-to-program bridge between exact real-number reasoning and floating-point execution. The authors define a Real-valued noncomputable function lj_Real and prove its equivalence to a simpler Real-valued version lj through

$$\texttt{lj\_eq}:\quad lj\_Real\ r\ r_c\ \varepsilon\ \sigma = lj\ r\ r_c\ \varepsilon\ \sigma.$$

They then introduce a polymorphic function lj_p parameterized by a generic type $\alpha$ carrying a RealLike typeclass. RealLike packages the operations needed by the benchmark: addition, subtraction, multiplication, division, negation, exponentiation by naturals, comparison, zero and one, conversion from integers and naturals, square root, and rounding. The same abstract definition can therefore be instantiated for $\mathbb{R}$ in proofs and for Float in execution. The paper emphasizes HSub, HMul, HDiv, HasRound, and HasSqrt as the mechanism by which Lean knows that the type supports the required operations.

The formal guarantees are expressed by named theorems. For the potential itself, cutoff_behavior states that if $r>r_c$, then the potential is zero; ljp_eq_le states that for $$V(r)= \begin{cases} 4\varepsilon \left[\left(\frac{\sigma}{r}\right)^{12}-\left(\frac{\sigma}{r}\right)^6\right], & r\le r_c\ 0, & r>r_c \end{cases}$$0, the Lennard-Jones form is exactly $$V(r)= \begin{cases} 4\varepsilon \left[\left(\frac{\sigma}{r}\right)^{12}-\left(\frac{\sigma}{r}\right)^6\right], & r\le r_c\ 0, & r>r_c \end{cases}$$1; and lj_p_continuous_closed_domain establishes continuity on $$V(r)= \begin{cases} 4\varepsilon \left[\left(\frac{\sigma}{r}\right)^{12}-\left(\frac{\sigma}{r}\right)^6\right], & r\le r_c\ 0, & r>r_c \end{cases}$$2. For periodic wrapping, abs_pbc_le proves that the wrapped coordinate lies in $$V(r)= \begin{cases} 4\varepsilon \left[\left(\frac{\sigma}{r}\right)^{12}-\left(\frac{\sigma}{r}\right)^6\right], & r\le r_c\ 0, & r>r_c \end{cases}$$3 for positive $$V(r)= \begin{cases} 4\varepsilon \left[\left(\frac{\sigma}{r}\right)^{12}-\left(\frac{\sigma}{r}\right)^6\right], & r\le r_c\ 0, & r>r_c \end{cases}$$4, and squaredminImageDistance_theorem proves equivalence between computing the distance between arbitrary points and computing it after wrapping both points into the box. minImageDistance_real_nonneg proves nonnegativity, minImageDistance_real_self proves that a particle’s distance to itself is zero, and long_range_correction_equality proves that the integral definition of the tail correction equals the closed-form U_LRC_Real implementation.

Executable implementation choices are also part of the benchmark definition. The paper discusses three versions of total pair energy: a loop-like version using Id.run, do, and mutable state; an explicit recursive version; and a proof-friendly version based on a pairs helper and foldl. The helper generates all index pairs and satisfies

$$V(r)= \begin{cases} 4\varepsilon \left[\left(\frac{\sigma}{r}\right)^{12}-\left(\frac{\sigma}{r}\right)^6\right], & r\le r_c\ 0, & r>r_c \end{cases}$$5

certifying that the code visits exactly the expected number of distinct particle pairs. The final executable function total_energy_pairs uses this pair list and evaluates the Lennard-Jones energy using floating-point functions.

Validation against NIST SRSW is reported for four systems with 30, 200, 400, and 800 particles. LeanLJ’s computed pairwise interaction energy and long-range correction agree with NIST values to the digits shown; for 30 particles, the paper gives $$V(r)= \begin{cases} 4\varepsilon \left[\left(\frac{\sigma}{r}\right)^{12}-\left(\frac{\sigma}{r}\right)^6\right], & r\le r_c\ 0, & r>r_c \end{cases}$$6 as Lean $$V(r)= \begin{cases} 4\varepsilon \left[\left(\frac{\sigma}{r}\right)^{12}-\left(\frac{\sigma}{r}\right)^6\right], & r\le r_c\ 0, & r>r_c \end{cases}$$7 and NIST $$V(r)= \begin{cases} 4\varepsilon \left[\left(\frac{\sigma}{r}\right)^{12}-\left(\frac{\sigma}{r}\right)^6\right], & r\le r_c\ 0, & r>r_c \end{cases}$$8, and the long-range correction as Lean $$V(r)= \begin{cases} 4\varepsilon \left[\left(\frac{\sigma}{r}\right)^{12}-\left(\frac{\sigma}{r}\right)^6\right], & r\le r_c\ 0, & r>r_c \end{cases}$$9 and NIST $$U_{\text{pair}}=\sum_{i=1}^{N}\sum_{j=i+1}^{N}V(r_{ij}), \quad r_{ij}\le r_c.$$0. The authors are explicit that this empirical agreement is not their main source of confidence: confidence is grounded in the Lean theorems certifying the mathematical properties of the code, while the NIST comparison functions as external numerical validation rather than the sole correctness argument (Ugwuanyi et al., 14 May 2025).

3. LeanPhysBench within Lean4PHYS

In Lean4PHYS, LeanPhysBench is a college-level physics benchmark formalized in Lean4 and designed to evaluate formal physics reasoning rather than validated numerical kernels. It contains 200 hand-crafted and peer-reviewed Lean4 statements, divided into 104 college-level and 96 competition-level problems, with the competition subset further partitioned into 62 competition-easy and 34 competition-hard instances (Li et al., 30 Oct 2025).

The source material is heterogeneous. Textbook-level questions are adapted from University Physics with Modern Physics and from the UGPhysics dataset. Olympiad-easy questions are derived from the intermediate solution steps of competition problems, and Olympiad-hard questions are based on problem-solving ideas from Selected Advanced Physics Problems, Volume II. The paper notes that textbook problems are rephrased and rewritten rather than copied verbatim, and that source figures are manually converted into natural-language context when needed.

Topic coverage follows the organization of PhysLib and spans six major areas: mechanics, waves and acoustics, thermodynamics, electromagnetism, optics, and modern physics. The benchmark’s design deliberately mixes topics and difficulty levels. The paper characterizes college problems as often emphasizing unit-aware numerical computation, competition-easy problems as being closer to short symbolic derivations, and competition-hard problems as requiring longer chains of symbolic reasoning, quantifiers, functional relationships, and sometimes calculus notions such as limits or continuity.

The formalization pipeline is explicitly bottom-up. Natural-language problems are first aligned from question-answer format into proof-style targets, inspired by autoformalization work in mathematics. Relevant physical laws and conditions are then extracted from step-by-step solutions, after which the proving target is defined as one of three types: a numerical value, a physical expression, or a logical formula describing a physical state. Lean4 code is then written, with missing theorems added to PhysLib when necessary, and each statement is formalized by one expert and verified by at least two experts in physics and Lean. The resulting benchmark is therefore highly curated rather than automatically generated.

Examples in the paper illustrate the benchmark’s formal style. A college-level mechanics problem includes assumptions such as t_0 = 0 * second, t_1 = 4 * second, a = (v_1 - v_0) / dt, v_1 = vf dt, and xf t = (a * t**2)/2 + v_0 * t, with a target like a = 6 * meter / second**2 ∧ v_0 = -2 * meter / second. A competition-level example formalizes friction on an inclined path with trigonometric terms and force balance, concluding with a formula of the form v**2 = .... Another appendix example treats Coulomb-force reasoning using units such as SI.nano (1 * [coulomb](https://www.emergentmind.com/topics/canonical-gauge-coulomb)), meter, and newton. Across these examples, unit correctness is not decorative; it is a central formal object.

PhysLib is the enabling infrastructure. It is a community-driven Lean4 repository extending Mathlib4 and the Lean4 UnitSystem kernel. The paper states that PhysLib defines the seven SI base units—time, length, mass, electric current, temperature, amount of substance, and luminous intensity—and also implements basic algebraic computation, calculus and derivative support, interchangeability between physics quantities and mathematical quantities, and dimension casts that preserve mathematical well-formedness. This library-level support is one reason the benchmark is framed as a benchmark for physics reasoning rather than merely for symbol manipulation.

Evaluation is reported under both with-PhysLib and without-PhysLib prompt settings, using Pass@16. The detailed results summary includes the following overall figures:

Model	With PhysLib	Without PhysLib
DeepSeek-Prover-V2-7B	14.50%	11.50%
Claude-Sonnet-4	34.50%	2.00%
Gemini-2.5-pro	39.50%	7.50%

The paper also states that the best expert prover reaches only 15.00%, while the best large general model reaches 39.50%, and it reports an average performance gain of 11.75% from adding PhysLib. Its abstract presents the benchmark as challenging enough that DeepSeek-Prover-V2-7B achieves only 16% and Claude-Sonnet-4 achieves 35%, a summary-level presentation consistent in spirit with the more specific Pass@16 figures in the detailed results (Li et al., 30 Oct 2025).

The reported failure modes are domain-specific. Without PhysLib, models only manage basic tactics such as constructor, rw, abel, exact, and aesop. Unit handling is identified as a major bottleneck on college-level problems. Expert provers may overthink by producing long, tedious proofs, while closed-source models are reported as weak on competition-hard symbolic derivations, where all closed-source models in the table score 0.00%. The benchmark is thus used to support the paper’s larger claim that physics reasoning does not transfer automatically from Lean mathematics reasoning.

4. Shared technical pattern: proofs over reals, execution over floats

A defining methodological pattern across LeanPhysBench-style scientific benchmarks is the separation between proof over idealized mathematics and execution over computational numerics. In LeanLJ, that separation is realized through noncomputable Real-valued definitions, polymorphic functions such as lj_p, and the RealLike typeclass, allowing one shared algorithmic structure to serve both theorem proving and floating-point computation (Ugwuanyi et al., 14 May 2025).

A closely related example is LeanBET, a fully executable and formally verified BET analysis pipeline in Lean 4. LeanBET formalizes the BETSI-style workflow for estimating surface area from adsorption isotherms, including window enumeration, monotonicity checks, knee selection, and linear regression, and it carries out computations in floating-point arithmetic while developing corresponding correctness proofs over the real numbers through a shared polymorphic implementation (Ugwuanyi et al., 15 May 2026). The underlying BET equation is formalized as

$$U_{\text{pair}}=\sum_{i=1}^{N}\sum_{j=i+1}^{N}V(r_{ij}), \quad r_{ij}\le r_c.$$1

with linearized form

$$U_{\text{pair}}=\sum_{i=1}^{N}\sum_{j=i+1}^{N}V(r_{ij}), \quad r_{ij}\le r_c.$$2

where the executable analysis computes transformed pairs

$$U_{\text{pair}}=\sum_{i=1}^{N}\sum_{j=i+1}^{N}V(r_{ij}), \quad r_{ij}\le r_c.$$3

performs ordinary least-squares regression using

$$U_{\text{pair}}=\sum_{i=1}^{N}\sum_{j=i+1}^{N}V(r_{ij}), \quad r_{ij}\le r_c.$$4

and

$$U_{\text{pair}}=\sum_{i=1}^{N}\sum_{j=i+1}^{N}V(r_{ij}), \quad r_{ij}\le r_c.$$5

then extracts

$$U_{\text{pair}}=\sum_{i=1}^{N}\sum_{j=i+1}^{N}V(r_{ij}), \quad r_{ij}\le r_c.$$6

The formal claims mirror the executable stages: the regression theorem states that the returned $$U_{\text{pair}}=\sum_{i=1}^{N}\sum_{j=i+1}^{N}V(r_{ij}), \quad r_{ij}\le r_c.$$7 are a least-squares minimizer under stated assumptions; parameter extraction is proved to agree with the specification-level formulas; window enumeration is proved sound and complete; admissibility checks are proved sound; and knee selection is proved to satisfy its specification. Experimentally, LeanBET agrees with BETSI to machine precision for 18 of the 19 benchmark isotherms, with only a $$U_{\text{pair}}=\sum_{i=1}^{N}\sum_{j=i+1}^{N}V(r_{ij}), \quad r_{ij}\le r_c.$$8 deviation for UiO-66 (Ugwuanyi et al., 15 May 2026).

The importance of LeanBET for LeanPhysBench is methodological rather than terminological. It shows that the pattern visible in LeanLJ is not peculiar to Lennard-Jones energy: one can formalize the scientific derivation itself, express the algorithm generically over a numeric type $$U_{\text{pair}}=\sum_{i=1}^{N}\sum_{j=i+1}^{N}V(r_{ij}), \quad r_{ij}\le r_c.$$9, prove properties over the reals, instantiate the same pipeline for floating-point execution, and validate against an external reference implementation. This suggests that, in its scientific-computing sense, LeanPhysBench is best understood as a benchmark family for executable, domain-grounded, theorem-backed workflows rather than only for isolated theorem statements.

5. Relation to the broader Lean benchmark ecosystem

LeanPhysBench sits within a rapidly diversifying landscape of Lean benchmarks, but its domain commitments distinguish it from several adjacent efforts.

Lean Workbook is a large-scale Lean 4 autoformalization and theorem-proving dataset built from natural-language Olympiad-style mathematics problems. Its main contribution is infrastructural: an iterative active-learning pipeline with compile checking, back-translation, natural-language inference filtering, and human diagnostics, yielding about 57K formal-informal pairs and searched proofs (Ying et al., 2024). It is relevant to LeanPhysBench as a data-generation and benchmark-construction resource, not as a physics or scientific-computing benchmark. Its methodological lesson is that benchmark construction in Lean benefits from semantic filtering in addition to syntactic compilation.

CAM-Bench is a Lean 4 theorem-proving benchmark of 1,000 proof targets in computational and applied mathematics, derived from 778 textbook exercises and centered on optimization, numerical linear algebra, and numerical analysis (Long et al., 17 May 2026). It introduces a dependency-recovery pipeline because many textbook applied-mathematics problems are not standalone theorem statements. This is highly relevant by contrast: CAM-Bench shows how a Lean benchmark can target applied-math content while preserving textbook-local context, but it remains a theorem-proving benchmark rather than an executable scientific workflow or a domain-specific physics benchmark.

VeriSoftBench pushes in a different direction, toward repository-scale software-verification proofs. It contains 500 Lean 4 proof obligations from 23 open-source repositories and preserves realistic repository context and cross-file dependencies (Xin et al., 20 Feb 2026). Its main finding—that transitive repository dependence is moderately negatively correlated with proof success, with Spearman r = -0.359, p < 0.001—is salient for Lean benchmark design, but the domain is formal methods rather than science. The connection to LeanPhysBench is structural: it argues that benchmark difficulty often resides in local abstractions and project-specific libraries, a point echoed by the role of PhysLib in Lean4PHYS.

CLEVER addresses end-to-end verified code generation rather than science or textbook theorem proving. It contains 161 Lean problems adapted from HumanEval, each requiring generation of a formal specification, a proof of specification equivalence to a hidden ground truth, an implementation, and a proof of correctness against the ground-truth specification (Thakur et al., 20 May 2025). Its use of non-computable specifications and its near-zero end-to-end success rate—only one fully verified HumanEval-derived task solved at best—make it a strong benchmark for verified synthesis. Relative to LeanPhysBench, CLEVER clarifies a conceptual boundary: LeanPhysBench is not principally about code synthesis from natural language, but about domain-grounded formal reasoning or verified scientific computation.

Taken together, these comparisons indicate that LeanPhysBench occupies a hybrid niche. It is more domain-specific than general-purpose theorem-proving corpora, more scientifically grounded than Olympiad autoformalization datasets, and more concerned with semantic physical correctness than generic verified code-generation benchmarks.

6. Significance, limitations, and likely extensions

The significance of LeanPhysBench lies in the shift from output checking to machine-checked semantic guarantees. In the molecular-simulation instantiation, traditional software validation through openness, transparency, unit testing, and human oversight is explicitly contrasted with formal logic and proofs of correctness, and the benchmark is presented as demonstrating “not just empirical agreement, but formal correctness guarantees” for the core numerical kernel (Ugwuanyi et al., 14 May 2025). In the Lean4PHYS instantiation, the significance lies in making intermediate reasoning visible and checkable in Lean4 rather than awarding credit for a final numerical answer, thereby exposing the current limits of formal physics reasoning by LLMs (Li et al., 30 Oct 2025).

The limitations are equally important. In LeanLJ, the proofs do not cover the I/O layer; Lean’s IO monad is used to read CSV benchmark configurations and obtain user inputs, but file parsing and input handling are validated by tests rather than by theorem proving. The design philosophy is therefore explicitly “prove what can be proved, test what must interact with the outside world” (Ugwuanyi et al., 14 May 2025). In LeanBET, the executable pipeline runs on floating-point numbers, so rare threshold-level discrepancies between float behavior and exact real-number reasoning remain possible, even though the paper argues that this is negligible relative to experimental noise for the datasets considered (Ugwuanyi et al., 15 May 2026). In Lean4PHYS, PhysLib is described as still incomplete and most fully developed for mechanics, and the performance collapse on competition-hard tasks shows that the benchmark is not close to saturation (Li et al., 30 Oct 2025).

A further limitation is terminological. Because the same name denotes both a verified scientific-computing benchmark centered on executable kernels and a theorem-proving benchmark centered on physics statements, the literature currently lacks a single standardized reference object called LeanPhysBench. This suggests that future work may either consolidate the term around a broader benchmark suite or continue to use it for distinct domain-specific instantiations.

The published extension paths are concrete. For the LeanLJ line, the authors list triclinic simulation boxes, Coulomb interactions and Ewald summation, neighbor lists and cutoffs, molecular dynamics time integration via Newton’s equations, higher-body potentials such as 3-body terms, and more complex molecular exclusions such as 1–4 interactions (Ugwuanyi et al., 14 May 2025). For Lean4PHYS, the stated directions include expanding PhysLib beyond its current topics and theorem set, growing the benchmark and library collaboratively, extending Lean formalization beyond physics to other natural sciences, and developing adaptive reasoning methods that avoid overthinking and better match task difficulty (Li et al., 30 Oct 2025). LeanBET supplies an additional template for expansion: derivation-as-specification, polymorphic shared implementation, and validation against an external scientific reference method (Ugwuanyi et al., 15 May 2026).

In that broader sense, LeanPhysBench is most accurately understood as an emerging program in Lean-based scientific benchmarking. Its published forms differ in task design, but both insist that benchmark success in scientific or physical reasoning should be grounded in theorem-checked semantics rather than solely in numerical coincidence or informal plausibility.