Ablation and the Meno: Tools for Empirical Metamathematics

Abstract: We present the results from Meno, a simple autoformalizer that proves theorems in Lean by systematically exploring the space of both formal and informal proofs, and tactic ablation, a new method for exploring mathematical creativity under constraint. We show these tools in action on simple theorems found in Terrence Tao's Analysis I, selectively ablating solution paths associated with non-constructive proofs, and analyze the properties of the resulting population using Goedel Prover embeddings. Among other things, our analysis of this novel population reveals that they lie on low (one or two) dimensional submanifolds of the much higher-dimensional representation space, and far away from their corresponding human constructions.

• The paper introduces Meno as a human-like autoformalization framework for generating and formalizing mathematical proofs in Lean.
• It employs tactic ablation to constrain formal toolkits, revealing the structure of proof strategy clusters and the influence of logical constraints.
• The empirical study shows that human-authored proofs diverge from autoformalized ones, highlighting implications for constructive and automated reasoning.

Empirical Metamathematics with Meno and Tactic Ablation

Introduction

"Ablation and the Meno: Tools for Empirical Metamathematics" (2604.22519) introduces Meno, an autoformalization framework for exploring the empirical landscape of mathematical proofs in Lean, and presents tactic ablation as a method for investigating mathematical creativity under formal constraint. The research is motivated by a dual focus: first, to systematically survey the space of formal and informal proof constructions, and second, to probe the effect of logical and heuristic constraints on plausible solution strategies. By leveraging both a human-inspired tactic orchestration pipeline and selective tactic removal grounded in the structure of Lean's proof system, the paper frames a new empirical approach to metamathematics.

Meno: Human-Like Autoformalization

Meno operationalizes a multi-phase pipeline inspired by the cognitive processes of mathematicians. Each phase—Set-up, Evaluate, Strategize, Decompose, Scaffold, Implement, Iterate—targets a core subproblem, facilitating both the generation and formalization of informal proofs. Meno is implemented as a server-driven orchestration protocol (MCP), coordinating multiple LLM agents with Lean 4 as the underlying proof engine. Integrating open-source modules such as lean4-skills and lean4-lsp-mcp, the system maintains a functional separation between high-level procedural logic and lower-level LLM-enabled tasks. This architecture enables both deterministic manipulation of proof states and stochastic generation of tactics, thereby capturing non-trivial proof behaviors.

A notable aspect of Meno is its interoperability with tactic ablation, allowing for dynamic alteration of the available formal toolkit during proof search. The research identifies significant cost and infrastructure concerns associated with orchestrating large LLMs (e.g., Claude Haiku/Sonnet), emphasizing the need for future improvements in modularity and efficiency.

Tactic Ablation: Principled Constraint on Proof Discovery

Tactic ablation is formalized as the removal of accessible Lean tactics along three orthogonal dimensions: abstraction level, functional category, and axiom dependency. Abstraction levels range from atomic (term-level primitives) to domain-specific, with functional categories reflecting both core Lean and mathlib module structure. The axiom tier analysis distinguishes tactics by constructive content—strongly constructive, weakly constructive, and classical (i.e., may invoke the Law of Excluded Middle or Choice).

Practically, the system employs both static and empirical audits to assign tactics to tiers, cross-validating with probe theorems and axiom tracing in Lean. This granular taxonomy enables fine-grained ablation experiments targeting specific non-constructive techniques or higher-level automation.

A critical insight is that ablation does not categorically preclude solution discovery due to the ontology of Lean's tactic system and foundational interdependencies; instead, it modulates proof difficulty and alters the available search space, mirroring the constraints imposed in intuitionistic or reverse mathematics paradigms.

Measuring and Visualizing the Proof Space

To render the high-dimensional space of proof strategies tractable, the study leverages the internal representations of the Goedel Prover V2-32B model [lin2025goedel]. Layerwise activations (primarily from an intermediate layer) are extracted while the model reads the Lean proof code, yielding 5140-dimensional embedding vectors for each proof. This approach is empirically justified: activations encode contextual semantic information salient to mathematical reasoning, outperforming surface-level syntactic or graph-theoretic descriptors.

Pairwise cosine distances between proof embeddings are computed and dimensionality is reduced using Multi-Dimensional Scaling (MDS), followed by Bayesian Gaussian mixture clustering to graphically represent the structure of proof populations.

Figure 1: The space of proofs as found by Meno, both with and without ablation (open vs. solid circles), for three theorems. Tao's Lean proof is marked as a star.

Quantitative analysis reveals that for three theorems from Tao's "Analysis I" (Russell's Paradox, $|X| < |2^X|$, and real completeness), the proof populations, both with and without ablation, concentrate on low-dimensional submanifolds (one to two effective dimensions). The variance in the high-dimensional embedding space is robustly captured in one to three dimensions, illustrating unexpectedly simple topological structure despite nontrivial syntactic variation.

A salient empirical result is that human-authored Lean proofs (Tao's originals) are distant outliers with respect to the clusters of autoformalized proofs, frequently located near the centroid of the MDS projection. This indicates significant divergence in solution strategy or representational style between human and agentically generated proofs, even when logical content is ostensibly equivalent.

Impact of Constraint and Constructivism

Analysis of tactic ablation (specifically, the removal of tactics introducing classical choice) demonstrates that while certain non-constructive pathways are blocked—thus forcing the system to find constructive alternatives—the overall geometry of the proof space is not dominated by the ablation. Proofs discovered under ablation are often interspersed with non-ablated proofs in the embedding space, reflecting Lean's bias towards constructive reasoning for theorems drawn from basic analysis, as well as the redundancy and flexibility in Meno's search strategy.

The methodology offers a robust empirical analog to foundational questions in proof theory: to what extent do formally equivalent proofs cluster together, how does logical or strategic constraint affect the typology of solutions, and in what ways do human-generated proofs inhabit the mathematical landscape defined by automated theorem provers?

Implications and Future Directions

This work exemplifies a scalable approach to empirical metamathematics, coupling agentic autoformalization with constraint-driven exploration of proof spaces. The main practical contribution is demonstrable: with minimal human intervention, an agent can rapidly generate diverse populations of formally correct proofs while the built-in ablation mechanism allows for structured comparative studies analogous to reverse mathematics.

The observation that most formal proofs inhabit low-dimensional subspaces challenges assumptions about the combinatorial explosion of reasoning strategies and suggests the presence of strong inductive biases in LLM-driven formalization. This insight supports both the study of proof synthesis algorithms and foundational research into the phenomenology of mathematical practice.

Theoretically, the divergence between human and autoformalized proofs raises questions for future investigation, such as the operational and pedagogical value of different proof clusters, the prevalence of "unnatural" agentic strategies, and the prospects for bridging the correspondence problem between informal and formal mathematics [dedeo2026correspondence].

Scaling this approach may enable the systematic empirical study of mathematical creativity, the evolution of proof techniques, and the comparative analysis of foundations under constraint. As model architectures develop and as richer theorem corpora become available, approaches like Meno and tactic ablation will facilitate high-resolution mapping of the epistemic landscape of mathematical reasoning.

Conclusion

The paper provides a concrete methodology for probing the empirical landscape of mathematical proof using agentic autoformalization and principled tactic ablation. Key contributions include the demonstration that proof spaces, even for logically rich theorems, are organized along simple geometric structures, with human and machine-generated proofs often fundamentally separated. Tactic ablation reshapes but does not fragment the space of formal proofs, highlighting both the robustness of constructivist strategies and the flexibility of automated methods. This framework establishes empirical metamathematics as a tractable and promising area for future AI-driven foundational research.