MiniLang: Minimal Proof Language & Model Adaptation

• MiniLang is a minimalist declarative proof language for Isabelle/HOL featuring a canonical, explicit syntax tailored for neural theorem proving.
• It reduces ambiguity, redundancy, and mixed styles using a streamlined command set and explicit structural decomposition.
• The term also denotes a lightweight adaptation pathway for extending pretrained masked language models with reduced computational cost.

Searching arXiv for the cited MiniLang-related paper and closely related Isabelle NTP work. MiniLang denotes, in one current arXiv usage, a redesigned, minimalist declarative proof language for Isabelle/HOL introduced in the IsaMini project, which also includes a cluster-capable Isabelle REPL, a translator from Isar to MiniLang, and an improved proof automation backend, Sledgehammer* (Xu et al., 25 Jul 2025). Its stated purpose is to improve neural theorem proving by replacing human-oriented variability in Isar with a compact, canonical, compositional representation that is more suitable for LLMs. In a separate implementation-oriented usage, the label “MiniLang” is applied to a lightweight pathway based on mini-model adaptation for extending pretrained masked LLMs to new languages with reduced compute (Marchisio et al., 2022). The term therefore spans two distinct technical contexts, but its most fully specified form is the Isabelle/HOL proof language.

1. Problem setting and rationale

MiniLang for Isabelle/HOL is motivated by the observation that prior LLM-based neural theorem proving for Isabelle has typically targeted Isar, while Isar’s human-oriented design hinders machine learning. The cited sources identify three recurrent sources of difficulty: ambiguity, redundancy, and mixed styles. Ambiguity arises from pronouns such as this and that, and from chaining connectives such as from, with, then, thus, hence, also, finally, moreover, and ultimately, whose semantics are subtle and context-sensitive. Redundancy arises because multiple equivalent forms can express the same proof, including legacy overlap with Isabelle’s tactic language and heterogeneous application orders in proof blocks. Mixed styles arise because Isar proofs interleave declarative statements with tactics and legacy constructs, widening the learning space (Xu et al., 25 Jul 2025).

The redesign target is therefore representational rather than merely stylistic. MiniLang is intended to make declarative reasoning canonicalized, explicit, and deterministic. The design goals stated for the language are representational suitability for LLMs, reduced ambiguity and variance, compositionality and determinism, reduced search space, and a better automation interface. These goals are operationalized through explicit references by names, a minimal command set, consistent ordering, uniform scoping rules, standardized subgoal management, and direct hint-based control over Sledgehammer* premise selection.

A central empirical premise of the redesign is that syntax accounts for many failures when generating Isar; the cited study reports that 76% of failures were classified as syntax errors. MiniLang is presented as a response to that brittleness. The source further states that MiniLang reduces both syntax errors and reasoning failures beyond mere syntax, which suggests that the language redesign is intended to change the structure of the prediction problem rather than only its surface form (Xu et al., 25 Jul 2025).

2. Language architecture and formal semantics

A MiniLang script is defined as a sequence of statements, each beginning with a keyword:

$$\text{Proof-Script} \coloneqq (\text{Statement})^+,\qquad \text{Statement} \coloneqq \text{Command}\ (\text{Argument})^*.$$

The command inventory is deliberately small and orthogonal.

Family	Commands	Role
Core structure	INTRO, [HAVE](https://www.emergentmind.com/topics/history-aware-verifier-have), CONSIDER, END, [NEXT](https://www.emergentmind.com/topics/neural-external-torque-estimation-next)	Context introduction, subgoal creation, and ATP-backed closure
Extensions	RULE, SIMPLIFY, UNFOLD, CHOOSE x, INDUCT, LET, NOTATION	Common proof operations
Compatibility	CONFIG, OPEN, APPLY	Corpus compatibility with Isar

The five core commands define the declarative backbone. INTRO moves hypotheses and universally bound variables into the context. HAVE introduces an intermediate lemma as a subgoal and then makes it available in the parent goal’s context. CONSIDER either fixes variables subject to conditions, as in existential reasoning, or splits by cases. END and NEXT discharge the current subgoal via the ATP backend and return to the parent or next sibling. Although END and NEXT are semantically identical, NEXT signals progression to the next sibling subgoal. Both must succeed via ATP; otherwise an error is raised (Xu et al., 25 Jul 2025).

The semantics are given imperatively as transformations of a labeled tree representing the entire proof state:

$$\text{Tree} \coloneqq \text{Leaf} \mid (\text{label: Context}, \text{children: Tree}^+),$$

$$\text{Context} \coloneqq (\text{set of variables}, \text{set of named hypotheses}),\qquad \text{Leaf} \coloneqq \text{Context} \vdash \text{Goal}.$$

Leaves are unproven subgoals. Non-leaf nodes group subgoals sharing a context. The initial state is a single leaf $(\emptyset,\emptyset)\vdash G$. Each statement replaces the leftmost leaf or the leftmost non-leaf node, yielding a deterministic traversal. The language also enforces an invariant that any non-leaf node has at least two children; single-child contexts are flattened by automatic reduction. This leftmost-subgoal discipline is a core part of the language’s compositionality.

The sources emphasize several restrictions as beneficial for LLMs: no pronouns, explicit labels, tactical effects restricted to the leading subgoal, END and NEXT as the only closures, and uniform ordering after translation. Calculation chains are retained without Isar’s also and finally: MiniLang internally maintains chains and adds the derived summary proposition to the context, allowing the result to be referenced directly without extra keywords (Xu et al., 25 Jul 2025).

A canonical illustration is the rendering of the irrationality proof for $\sqrt{2}$. In MiniLang, labels such as A1, A2, B, C, D, and E are explicit, and every use of prior facts is listed with WITH. The corresponding Isar proof instead relies on implicit flows such as “then obtain ... hence ... finally ... with ... thus ...”. MiniLang’s enforcement of labels and WITH lists is one of its principal representational changes.

3. Translation pipeline and automation interface

MiniLang is not only a surface language; it is embedded in a translation and automation stack. The translation pipeline normalizes Isar and maps constructs into MiniLang’s smaller command vocabulary. The reported mappings include obtain and consider to CONSIDER, have and hence to HAVE, intro-like flows to INTRO, simp and auto to SIMPLIFY, unfolding and unfold_tac to UNFOLD, apply to APPLY, and done to END or NEXT. Translation also elaborates sugars and macro variables such as ?thesis and ?case, adds type annotations, names anonymous lemmas, resolves this and that, normalizes connectives into explicit references, rewrites obsolete subgoal structures, and refines APPLY into END WITH/WITHOUT, SIMPLIFY, or UNFOLD where possible (Xu et al., 25 Jul 2025).

The ATP backend is Sledgehammer*, an enhanced version of Sledgehammer. MiniLang exposes premise guidance directly through WITH and WITHOUT clauses in END and NEXT. Missing lemma names are silently ignored. WITH biases premise selection, while WITHOUT penalizes or filters premises. The source describes the resulting ranking conceptually as

$$s(\ell; g) = \lambda_1 \cdot \mathrm{MePo}(g,\ell) + \lambda_2 \cdot \mathrm{kNN}(g,\ell) + \delta_{\mathrm{WITH}}(\ell) - \delta_{\mathrm{WITHOUT}}(\ell).$$

Sledgehammer* adds several other mechanisms. Before invoking Sledgehammer, it runs auto, with fallback to clarsimp on timeout. It runs fastforce in parallel with Sledgehammer. It classifies WITH lemmas heuristically by name and shape as rewrite or introduction rules so that appropriate subsets can be fed to the simplifier or fastforce. It also maintains a local self-learning cache of premise–goal relations, though caches are reset between reported experiments for fairness (Xu et al., 25 Jul 2025).

The operational flow for END and NEXT is summarized in six stages: preprocess the goal via simplification, collect context premises together with WITH and WITHOUT hints, rank premises via augmented MePo and k-NN scoring, launch fastforce in parallel, invoke Sledgehammer with ranked premises, and update the local cache on success. This integration is central to MiniLang’s claim that closure is explicit, hint-guided, and automation-facing rather than hidden inside tactic scripts.

4. Model training, corpora, and evaluation protocol

The reported machine-learning experiments fine-tune two 7B base LLMs specialized for formal reasoning: Llemma and DeepSeek-Prover-Base v1.5. Both have 4K token context windows. Training corpora are drawn from Isabelle HOL libraries for Isabelle 2024 and AFP dated 2025-02-12. The rule-based translator succeeds on 85.28% of AFP proofs, producing approximately 285K MiniLang proofs; the original Isar AFP training set has approximately 332K proofs. Preprocessing removes unreachable code, proofs overlapping the PISA benchmark, and proofs exceeding model context limits. Four corpora are built for ablations: Isar, Isar+SH*, MiniLang, and MiniLang−SH* (Xu et al., 25 Jul 2025).

Prompting follows Baldur’s setup. Prompts contain recent declarations, lemmas, and proofs from the same file, approximately 2K tokens of context, together with the theorem name and statement, approximately 2K tokens for the goal. No repair model is used; sampling is repeated $k$ times to compute pass@k. Supervised fine-tuning uses LLaMA-Factory for 2 epochs with batch size 256, learning rate $2\times 10^{-5}$ with linear decay to zero, on $8\times$ Nvidia H200, requiring approximately 12 hours per model.

Evaluation uses PISA, comprising 2,962 goals after updating the original 3K selection to AFP 2025-02-12 and removing moved or removed goals. The primary metric is pass@k, defined under independence with per-sample success probability $p$ as

$$\text{Tree} \coloneqq \text{Leaf} \mid (\text{label: Context}, \text{children: Tree}^+),$$0

The main reported results are as follows.

Model / setting	pass@1	pass@8
DPSK-PB, MiniLang	69.1%	79.2%
DPSK-PB, Isar+SH*	63.9%	74.3%
DPSK-PB, MiniLang−SH*	35.5%	44.9%
DPSK-PB, Isar	40.2%	50.5%
Llemma, MiniLang	68.0%	78.9%
Llemma, Isar+SH*	63.3%	72.1%
Llemma, MiniLang−SH*	35.2%	44.6%
Llemma, Isar	38.6%	48.6%

These numbers situate MiniLang simultaneously as a language design, a translation target, and a training representation. They also delimit the paper’s scope: whole-proof generation without repair, under a fixed prompting regime, on a benchmark whose caches are reset for comparability (Xu et al., 25 Jul 2025).

5. Empirical significance and comparative position

The central empirical claim is that MiniLang improves success rates over Isar generation and over Isar paired with the same enhanced automation backend. Against generating Isar alone, the reported improvement reaches up to 29% in comparison to Isar proof script generation, exemplified by the DPSK-PB pass@1 increase from 40.2% to 69.1%. Against Isar+SH*, MiniLang still leads by approximately 5% absolute across models and values of $$\text{Tree} \coloneqq \text{Leaf} \mid (\text{label: Context}, \text{children: Tree}^+),$$1. The source interprets this as evidence that the language redesign itself improves declarative neural theorem proving, rather than merely exposing better automation (Xu et al., 25 Jul 2025).

The article’s comparative framing places MiniLang against several Isabelle theorem-proving systems. Thor is described as establishing declarative neural theorem proving for Isabelle by replacing tactics with Sledgehammer across proofs and achieving 57% on PISA. Baldur is described as whole-proof generation and repair of Isar scripts, with 65.7% at pass@64. Magnushammer is described as using contrastive learning for premise selection and reporting 71.0% on PISA with Thor-style automation. Within the reported setup, MiniLang reaches 69.1% at pass@1 and 79.2% at pass@8, exceeding Baldur’s reported pass@64 and the prior state of the art attributed to Magnushammer plus Thor. The source explicitly notes that setups differ, including the absence of repair, a Sledgehammer* timeout of 500s, 16 CPU cores, up to 8 attempts, and cache resets.

The ablations are important for interpretation. MiniLang versus Isar, both without Sledgehammer*, shows that MiniLang is not designed for tactic-heavy settings and trails by approximately 5%. Isar+SH* versus Isar shows large gains from replacing tactics with Sledgehammer* and simplifier flows. The best performance comes from the combination of MiniLang and Sledgehammer*. This suggests that the representation change and the automation interface are complementary rather than interchangeable.

6. Failure modes, scope conditions, and portability

The error analysis compares MiniLang with Isar+SH* at pass@1. Syntax errors are significantly reduced in MiniLang and account for roughly 1% out of the approximately 5% total improvement. Proof operation and Sledgehammer* call failures are also reduced, indicating better logical correctness and better alignment of generated structure with automation. Term language errors become slightly more visible because the sequential error categorization exposes later term-level issues once syntax and structure improve (Xu et al., 25 Jul 2025).

Several limitations are stated explicitly. Translation does not cover the entire AFP corpus: approximately 15% of AFP proofs do not translate, and longer, complex proofs are more likely to fail translation. MiniLang omits some human-friendly constructs, including pronouns and explicit chain keywords, and discourages tactic-heavy idioms. It relies heavily on Sledgehammer*: disabling ATP leads to a substantial performance drop. Reconstruction mismatches are rare but can occur when Sledgehammer* succeeds at the ATP level and proof reconstruction fails inside Isabelle. The work is also restricted to whole-proof generation; it does not incorporate repair or stepwise reinforcement learning.

The broader portability claim is deliberately qualified. The core ideas—explicit structural decomposition via HAVE and CONSIDER, forced closure via END and NEXT, and hint-integrated automation—are said to be conceptually portable to other assistants such as Lean and Coq. However, the source stresses that specific commands such as CONFIG and OPEN are Isabelle-specific, and that porting the core to other assistants would require analogous automation, including SMT or hammers, together with a declarative corpus. A plausible implication is that MiniLang is best understood as an interface design pattern for declarative neural theorem proving rather than merely an Isabelle syntax proposal.

7. Separate usage in mini-model adaptation

A distinct use of the label “MiniLang” appears in the implementation-oriented description attached to “Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Training” (Marchisio et al., 2022). There, “MiniLang” denotes a lightweight pathway for extending pretrained masked LLMs to new target languages by training only target-language embeddings over a shallow, aligned mini-model and then plugging those embeddings back into the full transformer. This usage is unrelated to Isabelle/HOL proof scripting; it belongs to cross-lingual model adaptation.

That pathway builds on two methods. MiniJoint jointly pretrains a 12-layer RoBERTaBASE transformer with two MLM heads, one at the top layer and one at an intermediate layer, typically $$\text{Tree} \coloneqq \text{Leaf} \mid (\text{label: Context}, \text{children: Tree}^+),$$2, minimizing the combined loss $$\text{Tree} \coloneqq \text{Leaf} \mid (\text{label: Context}, \text{children: Tree}^+),$$3 with $$\text{Tree} \coloneqq \text{Leaf} \mid (\text{label: Context}, \text{children: Tree}^+),$$4 by default. MiniPost instead starts from an already pretrained 12-layer model, copies the bottom $$\text{Tree} \coloneqq \text{Leaf} \mid (\text{label: Context}, \text{children: Tree}^+),$$5 layers and the parent MLM head, inserts 2 randomly initialized bridging layers, and trains only those 2 layers in English to align the mini-model. In the subsequent target-language adaptation step, the mini-model transformer is frozen and only the target-language embeddings, tied to the output projection, are trained.

The reported efficiency gains are substantial. For target-language embedding training, BL_BASE with 12 layers costs approximately 54.1 EFLOPs, or approximately 20.9 NVIDIA V100 GPU days; MiniJoint with a 4-layer mini-model costs approximately 21.1 EFLOPs, or approximately 8.1 V100 days; MiniPost with a 6-layer mini-model costs approximately 29.3 EFLOPs, or approximately 11.3 V100 days. Averaged over XNLI, MLQA, and PAWS-X, MiniJoint retains 98.7% of BL_BASE performance at 39% of its Step 2 cost, while MiniPost retains 99.3% at approximately 54% of the cost. Early stopping at 95% of BL_BASE’s final score yields average speedups of approximately $$\text{Tree} \coloneqq \text{Leaf} \mid (\text{label: Context}, \text{children: Tree}^+),$$6 less compute for MiniJoint and approximately $$\text{Tree} \coloneqq \text{Leaf} \mid (\text{label: Context}, \text{children: Tree}^+),$$7 for MiniPost. In this secondary usage, “MiniLang” therefore designates a compute-efficient cross-lingual adaptation workflow rather than a formal language (Marchisio et al., 2022).