Lean Finder: Semantic Search for Lean

• Lean Finder is a retrieval system in the Lean ecosystem that enables semantic search and premise selection, facilitating efficient theorem discovery.
• It leverages synthetic user queries and human-aligned preference tuning to bridge the gap between formal statements and real user queries in proof development.
• Deployed as a web service and VS Code extension, Lean Finder integrates seamlessly into interactive theorem proving workflows with sub-second latency.

Searching arXiv for the cited Lean Finder papers and closely related Lean retrieval/prover work. Lean Finder is a retrieval-oriented line of work in the Lean ecosystem concerned with theorem discovery and premise selection under realistic proof-development conditions. In the most recent formulation, Lean Finder is a semantic search engine for Lean and mathlib that “understands and aligns with the intents of mathematicians,” addressing the difficulty of locating relevant theorems in Lean 4/mathlib4 and the mismatch between real user queries and the neutral paraphrases used by prior search systems (Lu et al., 8 Oct 2025). In an earlier formulation associated with machine-learned premise selection, the same name is used for a Lean 4 tool that suggests relevant premises directly inside interactive proof development, emphasizing tight integration, ease of installation, and sub-second latency (Piotrowski et al., 2023).

1. Scope, nomenclature, and problem formulation

The literature summarized under the name Lean Finder spans two closely related retrieval tasks in formal mathematics. One task is semantic search over Lean and mathlib, including natural-language queries, formal statements, and proof states. The other is interactive premise selection, where a proof assistant ranks lemmas likely to be useful for the current goal. Both address theorem discovery as a bottleneck in formalization, but they do so at different points in the workflow.

Two intertwined bottlenecks motivate the semantic-search formulation. First, locating relevant lemmas among 230 K+ theorems and 110 K+ definitions is described as “extremely frustrating”; naming conventions drift, library_search/#find/Loogle often fail, and high-quality examples are scarce. Second, Lean’s dependent-type syntax and tactic language impose a steep learning curve even on experienced programmers and mathematicians. Existing semantic search engines are characterized as relying primarily on informalizations—natural-language translations of formal statements—while largely overlooking the mismatch with real-world user queries (Lu et al., 8 Oct 2025).

The earlier premise-selection formulation frames the problem somewhat differently. Interactive theorem proving in Lean often requires recall or discovery of lemmas from mathlib, yet textual searches such as #find, manual browsing, and suggest / library_search tactics either interrupt proof development or only propose lemmas that exactly match the current goal. This establishes a narrower but operationally important problem: ranking nearby but not exact matches in the editor, without leaving the proof context (Piotrowski et al., 2023).

Work	Primary task	Characteristic method
Lean Finder (Lu et al., 8 Oct 2025)	Semantic search for Lean and mathlib	Public Lean discussions, synthetic user queries, contrastive retrieval, DPO
“Machine-Learned Premise Selection for Lean” (Piotrowski et al., 2023)	Interactive premise selection	Sparse features, k-NN baseline, online random forest, suggest_premises

A common misconception is that theorem retrieval is adequately modeled by embedding informalized statements alone. The semantic-search work explicitly argues otherwise: LLM informalizations capture a neutral paraphrase of a statement, whereas practitioners ask questions driven by specific intents, such as “do these minimal polynomials coincide?” versus “does algEquiv send x to y?” This suggests that retrieval quality depends not only on semantic similarity to the theorem statement, but also on alignment with the user’s proof intent.

2. User-intent-centered semantic retrieval pipeline

Lean Finder’s semantic-search pipeline has three core stages: data synthesis and augmentation, contrastive retrieval-model fine-tuning, and human-aligned preference tuning. The data stage begins with public Lean discussions from Zulip Chat and GitHub issues, from which 693 human-asked queries are extracted. These are clustered into five semantic intent groups: lemma search, metaprogramming/tactic-writing questions, type-class/instance guidance, proof engineering, and library design/refactoring discussions (Lu et al., 8 Oct 2025).

For each intent group and for each formal Lean statement drawn from mathlib4 + research-linked repos (1.4 M total statements), GPT-4o is prompted to generate user-style queries grounded in that intent, yielding 582 K synthetic user queries. In parallel, three additional modalities are produced from the same formal statements. These are 244 K informalized statements generated with rich context, 337 K augmented proof states derived from proof-step transitions $(S^{(i-1)}, T^{(i)}, S^{(i)})$ and paired as $(S^{(i-1)}, \text{description})$, and 244 K formal statements stripped of proofs to support partial recall (Lu et al., 8 Oct 2025).

The role of these modalities is structurally important. Synthetic user queries target the domain gap between theorem paraphrases and actual questions. Informalized statements preserve the prevailing retrieval formulation used by prior tools. Augmented proof states extend retrieval from theorem lookup to proof-state-aware assistance. Proof-stripped formal statements broaden matching under partial information. A plausible implication is that Lean Finder is designed less as a single-query retriever than as a unified retrieval substrate across several interaction regimes.

This user-centered design distinguishes Lean Finder from search engines that rely primarily on the informalized statement as the surrogate query. In the formulation given here, the central methodological claim is that the semantics of public Lean discussions should shape the query distribution seen during training. The resulting retriever is therefore optimized not merely for theorem-description similarity, but for the query forms mathematicians actually produce.

3. Retrieval model, objectives, and preference alignment

The base retrieval model is DeepSeek-Prover-V1.5-RL (7B decoder-only). As an encoder, it uses the final hidden state of the last token. Contrastive learning is performed on batches of size $B$ and group size $G$: for each query $q_i$, there is one positive code $c_i^+$, $G-1$ negatives, and all in-batch negatives. The objective is

$$\mathcal{L}_{\text{contrastive}} = -\frac{1}{B}\sum_{i=1}^B \log \frac{\exp\big(\mathrm{Sim}(\tilde q_i,c_i^+)/\tau\big)} {\sum_{c\in\mathcal C_b}\exp\big(\mathrm{Sim}(\tilde q_i,c)/\tau\big)},$$

where $\tilde q_i$ is token-augmented and $\mathrm{Sim}$ is cosine similarity (Lu et al., 8 Oct 2025).

Preference alignment is then introduced through a retrieval-adapted Direct Preference Optimization objective. Preference tuples $(S^{(i-1)}, \text{description})$0 are collected from three sources: retrieval-level votes on the web service, blind A/B arena comparisons between Lean Finder and Lean Search with user and GPT-4o judgments, and GPT-4o annotations on Zulip queries. The DPO objective is

$(S^{(i-1)}, \text{description})$1

where

$(S^{(i-1)}, \text{description})$2

Joint training combines the two objectives as

$(S^{(i-1)}, \text{description})$3

This architecture makes explicit that retrieval quality is not treated as a purely supervised matching problem. It is also a preference-learning problem in which human judgments, blind comparisons, and synthetic annotations shape ranking behavior. The paper describes this as aligning Lean Finder with mathematicians’ preferences using diverse feedback signals and encoding “a rich awareness of their goals from multiple perspectives” (Lu et al., 8 Oct 2025).

4. Interfaces, deployment, and integration with theorem provers

Lean Finder is deployed as a live web service at https://leanfinder.github.io and as a VS Code extension. The interface provides a query box, a selector for Lean 4 + mathlib4 context, and results containing formal code + informalization. The live service also exposes arena mode, which collects pairwise feedback through randomized A/B display. The API provides JSONL endpoints returning top-k $(S^{(i-1)}, \text{description})$4, and REST endpoints return top-k statement IDs and code, making the system consumable by any client (Lu et al., 8 Oct 2025).

The deployment model is explicitly oriented toward interoperability. Supported platforms include modern browsers, Lean 4 REPL via HTTP calls, and any RAG-enabled prover. The GitHub release includes full code, model weights (LoRA adapters), dataset (JSONL) under CC BY-SA 4.0. The VS Code extension is described as in development and is intended to embed retrieval inline (Lu et al., 8 Oct 2025).

A notable design claim is LLM-Prover Compatibility. In plug-in style retrieval for RAG workflows, whole-proof generators such as Gödel Prover and DeepSeek-Prover ingest Lean Finder’s top 10 lemmas as context, yielding consistent/MAP gains of up to +0.5 pp on MiniF2F and ProofNet. A step-wise prover, REALProver, can replace its native retriever with Lean Finder while maintaining comparable performance on MiniF2F/ProofNet/FATE-M. This situates Lean Finder not only as a human-facing search engine, but also as a retrieval component in automated formal reasoning pipelines (Lu et al., 8 Oct 2025).

The earlier premise-selection system is integrated more tightly into Lean 4 itself. It is implemented entirely in Lean 4’s metaprogramming framework, with no external Python or OCaml server, distributable as a Lake/Lean 4 package, and exposed as a single tactic, suggest_premises, callable in tactic mode. Asynchronous RPC calls from the editor infoview try lightweight tactics such as apply, rw, and simp only on each suggestion without blocking the main Lean process (Piotrowski et al., 2023). This suggests two integration philosophies under the same name: one web/API-centric and retrieval-general, the other editor-native and tactic-centric.

5. Evaluation, benchmarks, and ablations

The semantic-search evaluation uses test sets comprising 1 000 informalized statements, 1 000 formal statements (20% token noise), 1 000 synthetic user queries, and 2 224 proof states (raw and augmented). Metrics are Recall@1/5/10 and MRR, with GPT-4o (web search + name-stem matching) included as a non-retrieval baseline (Lu et al., 8 Oct 2025).

On informalized statements, Lean Finder reports rec@1 = 64.2% versus Lean Search 49.2% and GPT-4o (full) 14.8%, described as a +30% relative improvement over Lean Search. On synthetic user queries, it reports rec@1 = 54.4% versus 47.1% for Lean Search and 13.6% for GPT-4o (full), a +16% gain. On augmented proof states, Lean Finder reaches rec@1 = 24.6%, compared with Lean State Search 4.99%, RealProver 8.0%, and GPT-4o 7.4%, corresponding to +392% relative improvement over Lean State Search. On raw proof states, Lean Finder obtains rec@1 = 8.3% versus baselines in the 3–7% range (Lu et al., 8 Oct 2025).

The real-query user study is conducted with 5 Lean 4 experts on 128 GitHub queries under blind ranking among Lean Finder, Lean Search, and GPT-4o. Lean Finder appears in the top-3 on 81.6% of ballots, compared with 56.9% for Lean Search and 54.1% for GPT-4o. Its Normalized Borda score is 0.67, versus 0.41 and 0.40. These results are significant because they directly target the stated domain gap between surrogate queries and real user questions (Lu et al., 8 Oct 2025).

The ablations reinforce that interpretation. A model without DPO shows small retrieval gains on synthetic inputs but drops in user preference to Borda 0.63. A model without synthetic queries + DPO suffers a major drop on user queries to Borda 0.48, and on informalized inputs rec@1 from 64.2 → 61.1. The paper states that this “demonstrates critical role of synthetic user queries for capturing real-style intent” (Lu et al., 8 Oct 2025).

6. Relation to machine-learned premise selection in Lean 4

The earlier Lean Finder-associated work in premise selection pursues a different technical route. The central task is: given a goal state with hypotheses and target, featurize it and produce a ranking of premises likely to be useful next. Feature extraction uses sparse binary features of three kinds: names (bag-of-names), bigrams as pairs $(S^{(i-1)}, \text{description})$5 along syntax-tree edges, and trigrams as triples $(S^{(i-1)}, \text{description})$6 along depth-2 paths. Features are tagged as appearing in the hypothesis or target, for example H: and T: (Piotrowski et al., 2023).

Relevant premises are extracted by traversing the fully elaborated proof term of each theorem in mathlib3port and collecting all constants of propositional type. Three filters are used: all, source, and math. The reported dataset statistics are: all with 96 915 total premises, 41 755 total examples, and 3.12 avg premises/ex.; source with 28 784, 20 571, and 2.35; and math with 67 462, 40 187, and 2.09. An intermediate dataset constructed via LeanInk contains approximately 91 292 examples, 143 165 premises, and avg 1.57 (Piotrowski et al., 2023).

A standard k-nearest neighbours baseline uses weighted overlap similarity

$(S^{(i-1)}, \text{description})$7

with

$(S^{(i-1)}, \text{description})$8

and $(S^{(i-1)}, \text{description})$9. Because prediction scans all training examples, query time is approximately 5–6 s per theorem. The proposed eager model is an online random forest of $B$0 trees, updated incrementally with probability $B$1 over 3 passes of the data. Each node tests for presence of a single feature, leaves store a small bucket of training examples, and voting across the forest yields predictions with ≈ 0.28 s on average (Piotrowski et al., 2023).

Evaluation holds out all theorems in 592 leaf modules of mathlib3port as test data, with the remaining 2436 modules for training. The metrics are Cover and Cover$B$2, where Cover measures how many true premises lie in the first $B$3 suggestions and Cover$B$4 allows the user to scan up to $B$5 suggestions. The best results are obtained by the random forest with names + bigrams, giving Cover 0.57 / 0.29 / 0.26 for the all / source / math filters, respectively. The paper notes that adding trigrams often hurts, likely through overfitting, and reports latency for the source filter + (n+b) of RF ≈ 0.28 s versus k-NN ≈ 5.65 s per query (Piotrowski et al., 2023).

Historically, this earlier system clarifies what Lean retrieval looked like before user-intent semantic search became the central concern. It is lightweight, entirely internal to Lean 4, and optimized for editor assistance. The later semantic-search Lean Finder generalizes the retrieval problem to natural language, proof states, and external APIs, while retaining compatibility with automated provers. Taken together, the two works outline a progression from fast, tightly integrated premise ranking toward broader intent-aware retrieval across human and machine theorem-proving workflows.