How Powerful are LLMs in Generating Formal Program Specifications?

Abstract: Formal verification provides strong guarantees of software correctness, but its adoption is limited by the high cost of writing precise formal specifications. While recent LLMs have shown strong capabilities in theorem proving and verified code generation, their true ability to generate program specifications remains unclear. Existing evaluations require either verifying implementation conformance or proving semantic equivalence between specifications, both of which are formidably difficult and may conflate proof difficulty with specification quality. To address this problem, we introduce Coins, a Rocq based evaluation framework that assesses specification quality by instantiating specifications under evaluation on trusted test cases and generating concrete proof obligations. This design aligns with the asymmetric nature of formal reasoning, where successful proofs provide reliable evidence while proof failures are inherently ambiguous. Using Coins, we conduct a large scale study on HumanEval with a curated set of human written Rocq specifications. Our results show that specification generation remains a formidable challenge, and that verification complexity can obscure genuine differences in specification quality. Overall, we find that accurate specification evaluation, rather than model scaling alone, is central to understanding the power of LLMs for specification synthesis, and that test case based formal reasoning offers a more faithful and discriminative measure of progress.

• The paper introduces Coins, a Rocq-based evaluation framework that tests whether generated specifications accept trusted positive cases and reject 1,640 mutated negative cases without requiring full semantic-equivalence proofs.
• Experiments on all 164 HumanEval problems show Gemini 3 Pro Preview leading with a 28.05% Reject_all score, while syntax errors—especially type mismatches, missing imports, and invalid recursive definitions—remain major obstacles.
• The results show executable Fixpoint specifications improve proof success and readability, while equivalence-based benchmarks can underestimate quality because only 31.0% of mutually valid human and LLM specifications have provably isomorphic counterparts.

The evaluation of LLMs for formal program specification has been hampered by a methodological problem: existing benchmarks either check whether an implementation satisfies a generated specification, which permits over-approximation, or demand full semantic equivalence proofs against ground-truth specifications, which conflate specification quality with prover capability. This paper introduces Coins (COq-based INstantiated Specification evaluation), a Rocq-based framework that assesses specification quality by instantiating specifications on trusted test cases and discharging the resulting proof obligations, and uses it to conduct a large-scale empirical study of specification generation on HumanEval (2608.13077).

Motivation and problem statement

The authors argue that traditional verification pipelines are oriented toward establishing that an implementation satisfies a specification, and therefore provide limited evidence about whether the specification precisely captures intended semantics. A specification may suffice to verify a particular implementation yet remain overly permissive, admitting incorrect implementations. End-to-end benchmarks such as Clever, which require joint generation of specifications, implementations, and equivalence proofs in Lean, enforce a strong notion of correctness but yield sparse, hard-to-interpret evaluation signals: failures cannot be attributed to the specification, the implementation, or the prover.

The paper's central methodological claim is an evaluation duality: when a specification fails to verify, it is unclear whether the fault lies in the specification or in the prover. The authors formalize this in an appendix by modeling an LLM prover as a probabilistic oracle whose success probability $p_{S,i,o}$ varies with specification style, proof complexity, and test-case difficulty. A theorem states that verification failure is observationally indistinguishable from a true counterexample when $p_{S,i,o}$ is unknown. The consequence is a fundamental asymmetry: successful proofs constitute sound positive evidence (by Rocq's kernel soundness), while failed proofs carry no definitive information. Coins is designed around this asymmetry.

The Coins framework

Coins evaluates a specification by instantiating it on concrete input–output pairs and asking whether the induced proof obligations can be discharged in Rocq. The framework proceeds in two phases:

• Preparation. The authors manually authored ground-truth Rocq specifications for all 164 HumanEval problems, cross-reviewed by experts over several person-weeks—the first complete formal specification suite for HumanEval in Rocq. Positive test cases are drawn from HumanEval+ (averaging 755.98 cases per problem). Negative cases (10 per problem, 1,640 total) are generated by mutation testing: AST-level operator mutations supplemented by LLM-guided semantic mutations, with input-variant strategies to ensure each mutant is killed.
• Evaluation. Generated specifications pass through a multi-stage pipeline: syntactic compilation, acceptance of the first positive case (Pass$_{\text{first}}$), acceptance of all positive cases (Pass$_{\text{all}}$), and rejection of all negative cases (Reject$_{\text{all}}$). Reject$_{\text{all}}$ is the core metric: a specification is a candidate only if it provably accepts all positive cases and no negative case can be proven acceptable. Failure to prove acceptance is treated as inconclusive rather than as rejection, reflecting the soundness asymmetry.

Notably, the methodology does not require proving semantic equivalence between specifications; the authors report that among specifications passing all test-based criteria, only 31.0% admit provably isomorphic counterparts, supporting their claim that equivalence proving systematically underestimates specification quality.

Experimental setup and verifier selection

Six models are evaluated: GPT-4o, GPT-5, Claude 3.7 Sonnet, Claude 4.5 Opus, DeepSeek-V3.1, and Gemini 3 Pro Preview. Because evaluation depends on proof capability, the authors first select the strongest verifier using their human-written specifications. Gemini 3 Pro Preview achieves 29.88% Pass$_{\text{all}}$ on human specifications—substantially ahead of Claude 4.5 Opus (19.51%) and GPT-5 (16.64%), and far above GPT-4o (7.32%) and DeepSeek-V3.1 (0.00%)—and is adopted as the primary prover. Even this best verifier succeeds on fewer than a third of the human specifications, underscoring how far current LLM proof synthesis is from reliable automated theorem proving.

Specification generation results

The headline result is that specification generation remains a formidable challenge, with Coins scores ranging from 28.05% (Gemini 3 Pro Preview) to 1.22% (DeepSeek-V3.1):

Model	Syntax	Pass$_{\text{first}}$	Reject$_{\text{all}}$ (core metric)
Gemini 3 Pro Preview	78.05%	60.98%	28.05%
GPT-5	67.07%	60.98%	15.24%
Claude 4.5 Opus	59.76%	45.12%	14.63%
Claude 3.7 Sonnet	19.51%	15.24%	1.83%
GPT-4o	14.63%	12.20%	4.27%
DeepSeek-V3.1	7.93%	4.27%	1.22%

Several findings stand out. Syntax is the first bottleneck: for these algorithmic tasks, the dominant failure mode is not wrong behavioral intent but hallucinated invalid Rocq syntax or ill-typed constructs. An error analysis of 156 compilation failures attributes 45% to type mismatches (chiefly conflating nat and Z, an artifact of pretraining on dynamically typed languages), 22% to missing Require Import statements, and 17% to Fixpoint definitions violating the guard checker's structural termination requirements. The sharp drop from Pass$_{\text{first}}$ to Pass$p_{S,i,o}$0 (e.g., 60.98% → 28.05% for Gemini) shows that individual test cases are insufficient and that comprehensive suites are essential. Self-verification experiments rule out a self-verification bias: strong models achieve comparable or lower Pass$p_{S,i,o}$1 when verifying their own outputs. The close alignment between Pass$p_{S,i,o}$2 and Reject$p_{S,i,o}$3 indicates that specifications passing all positive tests are typically precise enough to reject mutants.

An ablation disentangling the evaluation duality finds that substituting human-written specifications for self-generated ones improves performance by +5.01% on average, while replacing self-verification with the stronger Gemini verifier contributes +3.05%—both factors materially affect measured performance, confirming that verification complexity can obscure genuine differences in specification quality.

Specification style: executability

The paper challenges the requirement in Clever that specifications be purely non-executable. Comparing Fixpoint-based (executable) components against purely relational Inductive predicates, the authors find that LLMs use Fixpoint at rates (31–44%) comparable to human experts (~40%), and that prohibiting Fixpoint causes substantial drops in proof success, longer specifications, and lower readability. The conclusion is that executable components reduce both specification complexity and verification difficulty, and that rigid non-executability constraints—justified in Clever by concerns about specification leakage in code synthesis—are counterproductive when the goal is to evaluate specification quality itself.

Comparison against human specifications and other benchmarks

Aggregating candidates via priority-based selection yields 60 LLM-generated specifications covering 36.6% of the benchmark, versus 50 validated human specifications (30.5%)—evidence that frontier models approach human experts in coverage rate, though on largely disjoint problem subsets. Of the 29 problems where both sources produced valid specifications, only 9 (31.0%) admit provably equivalent specifications; manual inspection of single-direction cases confirmed semantic equivalence with prover failure, reinforcing that equivalence-based metrics systematically underestimate quality.

A cross-benchmark comparison yields a notable divergence: DeepSeek-V3.1 scores 66.0% on SWE-bench Verified—comparable to GPT-5 (65.0%)—yet achieves the lowest syntax score (7.93%) and Coins score (1.22%). Meanwhile, the two models strongest on AIME 2025 (Gemini at 95.7%, GPT-5 at 91.7%) also lead on specification quality, suggesting formal specification generation correlates more with mathematical reasoning than with software engineering capability. Even on HumanEval, where code generation is near saturation, Coins differentiates top-tier models by 18.3% on syntax and 11.6% on the core metric—discriminative power comparable to far harder benchmarks.

Limitations and open questions

The authors are explicit about scope constraints. The study covers only 164 HumanEval problems in Rocq; the annotation effort (several person-weeks of expert specification writing) rather than methodology limits scale, and extending to other proof assistants or benchmarks demands comparable effort. The human-written "ground-truth" specifications are carefully reviewed but not formally verified against implementations—the authors concede this anchoring is a belief-based assumption, not an absolute guarantee. The evaluation also depends on the chosen prover; even the strongest verifier achieves under 30% Pass$p_{S,i,o}$4, so some failures attributed to specifications may still reflect prover limitations, and the inconclusive treatment of failed acceptance proofs means candidate specifications may still be incorrect in ways the test suite does not capture. The paper leaves open whether test-case-based evaluation scales to specifications of programs with more complex semantics, and whether the observed correlation with mathematical reasoning holds beyond HumanEval-style tasks.

Conclusion

This paper reframes LLM specification evaluation around what formal tools can establish reliably: provable behavior on concrete test cases, rather than all-or-nothing equivalence proofs. Its contributions are a principled evaluation framework grounded in the soundness asymmetry of proof assistants, the first complete Rocq specification suite for HumanEval, and an empirical study showing that specification generation remains far from solved (best model: 28.05%) while exposing capability differences that conventional code benchmarks no longer discriminate. Its central argument—that accurate evaluation, not model scaling alone, is central to measuring specification synthesis capability—is supported by the ablation showing both specification quality and prover strength materially shift measured outcomes.