---
title: 'Lean Finder: Semantic Search for Lean'
url: https://www.emergentmind.com/topics/lean-finder
type: topic
---

# Lean Finder: Semantic Search for Lean

Searching arXiv for the cited Lean Finder papers and closely related Lean retrieval/prover work.
Lean Finder is a retrieval-oriented line of work in the Lean ecosystem concerned with theorem discovery and premise selection under realistic proof-development conditions. In the most recent formulation, Lean Finder is a semantic search engine for Lean and mathlib that “understands and aligns with the intents of mathematicians,” addressing the difficulty of locating relevant theorems in Lean 4/mathlib4 and the mismatch between real user queries and the neutral paraphrases used by prior search systems [2510.15940]. In an earlier formulation associated with machine-learned premise selection, the same name is used for a Lean 4 tool that suggests relevant premises directly inside interactive proof development, emphasizing tight integration, ease of installation, and sub-second latency [2304.00994].

## 1. Scope, nomenclature, and problem formulation

The literature summarized under the name Lean Finder spans two closely related retrieval tasks in formal mathematics. One task is semantic search over Lean and mathlib, including natural-language queries, formal statements, and proof states. The other is interactive premise selection, where a proof assistant ranks lemmas likely to be useful for the current goal. Both address theorem discovery as a bottleneck in formalization, but they do so at different points in the workflow.

Two intertwined bottlenecks motivate the semantic-search formulation. First, locating relevant lemmas among **230 K+ theorems** and **110 K+ definitions** is described as “extremely frustrating”; naming conventions drift, `library_search`/`#find`/`Loogle` often fail, and high-quality examples are scarce. Second, Lean’s dependent-type syntax and tactic language impose a steep learning curve even on experienced programmers and mathematicians. Existing semantic search engines are characterized as relying primarily on informalizations—natural-language translations of formal statements—while largely overlooking the mismatch with real-world user queries [2510.15940].

The earlier premise-selection formulation frames the problem somewhat differently. Interactive theorem proving in Lean often requires recall or discovery of lemmas from mathlib, yet textual searches such as `#find`, manual browsing, and `suggest` / `library_search` tactics either interrupt proof development or only propose lemmas that exactly match the current goal. This establishes a narrower but operationally important problem: ranking nearby but not exact matches in the editor, without leaving the proof context [2304.00994].

| Work | Primary task | Characteristic method |
|---|---|---|
| Lean Finder [2510.15940] | Semantic search for Lean and mathlib | Public Lean discussions, synthetic user queries, contrastive retrieval, DPO |
| “Machine-Learned Premise Selection for Lean” [2304.00994] | Interactive premise selection | Sparse features, k-NN baseline, online random forest, `suggest_premises` |

A common misconception is that theorem retrieval is adequately modeled by embedding informalized statements alone. The semantic-search work explicitly argues otherwise: LLM informalizations capture a neutral paraphrase of a statement, whereas practitioners ask questions driven by specific intents, such as “do these minimal polynomials coincide?” versus “does algEquiv send x to y?” This suggests that retrieval quality depends not only on semantic similarity to the theorem statement, but also on alignment with the user’s proof intent.

## 2. User-intent-centered semantic retrieval pipeline

Lean Finder’s semantic-search pipeline has three core stages: data synthesis and augmentation, contrastive retrieval-model fine-tuning, and human-aligned preference tuning. The data stage begins with public Lean discussions from Zulip Chat and GitHub issues, from which **693 human-asked queries** are extracted. These are clustered into **five semantic intent groups**: lemma search, metaprogramming/tactic-writing questions, type-class/instance guidance, proof engineering, and library design/refactoring discussions [2510.15940].

For each intent group and for each formal Lean statement drawn from **mathlib4 + research-linked repos (1.4 M total statements)**, GPT-4o is prompted to generate user-style queries grounded in that intent, yielding **582 K synthetic user queries**. In parallel, three additional modalities are produced from the same formal statements. These are **244 K informalized statements** generated with rich context, **337 K augmented proof states** derived from proof-step transitions \((S^{(i-1)}, T^{(i)}, S^{(i)})\) and paired as \((S^{(i-1)}, \text{description})\), and **244 K formal statements stripped of proofs** to support partial recall [2510.15940].

The role of these modalities is structurally important. Synthetic user queries target the domain gap between theorem paraphrases and actual questions. Informalized statements preserve the prevailing retrieval formulation used by prior tools. Augmented proof states extend retrieval from theorem lookup to proof-state-aware assistance. Proof-stripped formal statements broaden matching under partial information. A plausible implication is that Lean Finder is designed less as a single-query retriever than as a unified retrieval substrate across several interaction regimes.

This user-centered design distinguishes Lean Finder from search engines that rely primarily on the informalized statement as the surrogate query. In the formulation given here, the central methodological claim is that the semantics of public Lean discussions should shape the query distribution seen during training. The resulting retriever is therefore optimized not merely for theorem-description similarity, but for the query forms mathematicians actually produce.

## 3. Retrieval model, objectives, and preference alignment

The base retrieval model is **DeepSeek-Prover-V1.5-RL (7B decoder-only)**. As an encoder, it uses the **final hidden state of the last token**. Contrastive learning is performed on batches of size \(B\) and group size \(G\): for each query \(q_i\), there is one positive code \(c_i^+\), \(G-1\) negatives, and all in-batch negatives. The objective is

$$
\mathcal{L}_{\text{contrastive}}
= -\frac{1}{B}\sum_{i=1}^B
\log
\frac{\exp\big(\mathrm{Sim}(\tilde q_i,c_i^+)/\tau\big)}
{\sum_{c\in\mathcal C_b}\exp\big(\mathrm{Sim}(\tilde q_i,c)/\tau\big)},
$$

where \(\tilde q_i\) is token-augmented and \(\mathrm{Sim}\) is cosine similarity [2510.15940].

Preference alignment is then introduced through a retrieval-adapted **Direct Preference Optimization** objective. Preference tuples \(\mathcal P=\{(q,c^+,c^-)\}\) are collected from three sources: retrieval-level votes on the web service, blind A/B arena comparisons between Lean Finder and Lean Search with user and GPT-4o judgments, and GPT-4o annotations on Zulip queries. The DPO objective is

$$
\mathcal{L}_{\mathrm{DPO}}
= -\mathbb{E}_{(q,c^+,c^-)\sim\mathcal P}
\log \sigma\!\Bigl[\beta\bigl(\Delta_\theta(q,c^+,c^-)-\Delta_{\theta_{\mathrm{ref}}}(q,c^+,c^-)\bigr)\Bigr],
$$

where

$$
\Delta_\theta(q,c^+,c^-)=\mathrm{Sim}_\theta(q,c^+)-\mathrm{Sim}_\theta(q,c^-).
$$

Joint training combines the two objectives as

$$
\mathcal{L} = \mathcal{L}_{\mathrm{DPO}} + \lambda\,\mathcal{L}_{\text{contrastive}}.
$$

This architecture makes explicit that retrieval quality is not treated as a purely supervised matching problem. It is also a preference-learning problem in which human judgments, blind comparisons, and synthetic annotations shape ranking behavior. The paper describes this as aligning Lean Finder with mathematicians’ preferences using diverse feedback signals and encoding “a rich awareness of their goals from multiple perspectives” [2510.15940].

## 4. Interfaces, deployment, and integration with theorem provers

Lean Finder is deployed as a live web service at **https://leanfinder.github.io** and as a **VS Code extension**. The interface provides a query box, a selector for **Lean 4 + mathlib4 context**, and results containing **formal code + informalization**. The live service also exposes **arena mode**, which collects pairwise feedback through randomized A/B display. The API provides **JSONL endpoints returning top-k \(\{ \text{name}, \text{code}, \text{informalization} \}\)**, and REST endpoints return top-k statement IDs and code, making the system consumable by any client [2510.15940].

The deployment model is explicitly oriented toward interoperability. Supported platforms include **modern browsers**, **Lean 4 REPL via HTTP calls**, and **any RAG-enabled prover**. The GitHub release includes **full code, model weights (LoRA adapters), dataset (JSONL) under CC BY-SA 4.0**. The VS Code extension is described as **in development** and is intended to embed retrieval inline [2510.15940].

A notable design claim is **LLM-Prover Compatibility**. In plug-in style retrieval for RAG workflows, **whole-proof generators** such as **Gödel Prover** and **DeepSeek-Prover** ingest Lean Finder’s **top 10 lemmas** as context, yielding consistent/MAP gains of **up to +0.5 pp on MiniF2F and ProofNet**. A **step-wise prover**, **REALProver**, can replace its native retriever with Lean Finder while maintaining comparable performance on **MiniF2F/ProofNet/FATE-M**. This situates Lean Finder not only as a human-facing search engine, but also as a retrieval component in automated formal reasoning pipelines [2510.15940].

The earlier premise-selection system is integrated more tightly into Lean 4 itself. It is implemented entirely in Lean 4’s metaprogramming framework, with **no external Python or OCaml server**, distributable as a **Lake/Lean 4 package**, and exposed as a single tactic, **`suggest_premises`**, callable in tactic mode. Asynchronous RPC calls from the editor infoview try lightweight tactics such as `apply`, `rw`, and `simp only` on each suggestion without blocking the main Lean process [2304.00994]. This suggests two integration philosophies under the same name: one web/API-centric and retrieval-general, the other editor-native and tactic-centric.

## 5. Evaluation, benchmarks, and ablations

The semantic-search evaluation uses test sets comprising **1 000 informalized statements**, **1 000 formal statements (20% token noise)**, **1 000 synthetic user queries**, and **2 224 proof states (raw and augmented)**. Metrics are **Recall@1/5/10** and **MRR**, with **GPT-4o (web search + name-stem matching)** included as a non-retrieval baseline [2510.15940].

On **informalized statements**, Lean Finder reports **rec@1 = 64.2%** versus **Lean Search 49.2%** and **GPT-4o (full) 14.8%**, described as a **+30% relative improvement** over Lean Search. On **synthetic user queries**, it reports **rec@1 = 54.4%** versus **47.1%** for Lean Search and **13.6%** for GPT-4o (full), a **+16%** gain. On **augmented proof states**, Lean Finder reaches **rec@1 = 24.6%**, compared with **Lean State Search 4.99%**, **RealProver 8.0%**, and **GPT-4o 7.4%**, corresponding to **+392%** relative improvement over Lean State Search. On **raw proof states**, Lean Finder obtains **rec@1 = 8.3%** versus baselines in the **3–7%** range [2510.15940].

The real-query user study is conducted with **5 Lean 4 experts** on **128 GitHub queries** under blind ranking among Lean Finder, Lean Search, and GPT-4o. Lean Finder appears in the **top-3 on 81.6% of ballots**, compared with **56.9%** for Lean Search and **54.1%** for GPT-4o. Its **Normalized Borda** score is **0.67**, versus **0.41** and **0.40**. These results are significant because they directly target the stated domain gap between surrogate queries and real user questions [2510.15940].

The ablations reinforce that interpretation. A model **without DPO** shows small retrieval gains on synthetic inputs but drops in user preference to **Borda 0.63**. A model **without synthetic queries + DPO** suffers a major drop on user queries to **Borda 0.48**, and on informalized inputs **rec@1 from 64.2 → 61.1**. The paper states that this “demonstrates critical role of synthetic user queries for capturing real-style intent” [2510.15940].

## 6. Relation to machine-learned premise selection in Lean 4

The earlier Lean Finder-associated work in premise selection pursues a different technical route. The central task is: given a goal state with hypotheses and target, featurize it and produce a ranking of premises likely to be useful next. Feature extraction uses **sparse binary** features of three kinds: **names** (bag-of-names), **bigrams** as pairs \((\text{head}, \text{arg})\) along syntax-tree edges, and **trigrams** as triples \((\text{grandparent}, \text{parent}, \text{child})\) along depth-2 paths. Features are tagged as appearing in the hypothesis or target, for example `H:` and `T:` [2304.00994].

Relevant premises are extracted by traversing the fully elaborated proof term of each theorem in **mathlib3port** and collecting all constants of propositional type. Three filters are used: **all**, **source**, and **math**. The reported dataset statistics are: **all** with **96 915 total premises**, **41 755 total examples**, and **3.12 avg premises/ex.**; **source** with **28 784**, **20 571**, and **2.35**; and **math** with **67 462**, **40 187**, and **2.09**. An **intermediate dataset** constructed via LeanInk contains approximately **91 292 examples**, **143 165 premises**, and **avg 1.57** [2304.00994].

A standard **k-nearest neighbours** baseline uses weighted overlap similarity

$$
M(x_1,x_2)=
\frac{\sum_{f\in F_1\cap F_2} t(f)}
{\sum_{f\in F_1} t(f)+\sum_{f\in F_2} t(f)-\sum_{f\in F_1\cap F_2} t(f)},
$$

with

$$
t(f)=\bigl(\log(|D|/|D_f|)\bigr)^2,
$$

and **\(k=100\)**. Because prediction scans all training examples, query time is **approximately 5–6 s per theorem**. The proposed eager model is an **online random forest of \(T=300\) trees**, updated incrementally with probability **\(p=0.3\)** over **3 passes** of the data. Each node tests for presence of a single feature, leaves store a small bucket of training examples, and voting across the forest yields predictions with **≈ 0.28 s on average** [2304.00994].

Evaluation holds out all theorems in **592 leaf modules** of mathlib3port as test data, with the remaining **2436 modules** for training. The metrics are **Cover** and **Cover\(_+\)**, where Cover measures how many true premises lie in the first \(n\) suggestions and Cover\(_+\) allows the user to scan up to \(n+10\) suggestions. The best results are obtained by the **random forest with names + bigrams**, giving **Cover 0.57 / 0.29 / 0.26** for the **all / source / math** filters, respectively. The paper notes that **adding trigrams often hurts**, likely through overfitting, and reports latency for the **source filter + (n+b)** of **RF ≈ 0.28 s** versus **k-NN ≈ 5.65 s per query** [2304.00994].

Historically, this earlier system clarifies what Lean retrieval looked like before user-intent semantic search became the central concern. It is lightweight, entirely internal to Lean 4, and optimized for editor assistance. The later semantic-search Lean Finder generalizes the retrieval problem to natural language, proof states, and external APIs, while retaining compatibility with automated provers. Taken together, the two works outline a progression from fast, tightly integrated premise ranking toward broader intent-aware retrieval across human and machine theorem-proving workflows.

Source: https://www.emergentmind.com/topics/lean-finder