ASTactic: Neural Tactic Synthesis

• ASTactic is a deep learning-based model for automated theorem proving in Coq, representing tactic commands as abstract syntax trees.
• It employs a tree-structured encoder–decoder framework to synthesize contextually valid tactics and reduce argument search complexity.
• The integration with Coq and complementary use of traditional tactics, like hammer, achieves state-of-the-art proof success rates.

ASTactic is a deep learning-based model for automated theorem proving in the Coq proof assistant, designed to generate tactic commands as programs represented by abstract syntax trees (ASTs). Developed in the context of the CoqGym environment, ASTactic formalizes the task of tactic synthesis as a conditional program generation problem, where model outputs are structured according to the syntax and semantics of the Ltac tactic language. The architecture leverages tree-structured neural models to encode proof states and decode tactic trees, focusing on producing in-vocabulary tactics with arguments drawn from semantically valid options. The approach combines neural program synthesis with semantic pruning, enabling more efficient and effective search in the space of interactive proofs (Yang et al., 2019).

1. Formalization of Tactic Generation

In interactive theorem proving with Coq, proofs are constructed by iteratively issuing tactics that manipulate a proof state, which itself consists of a global environment $\Gamma$ of named premises and a list of current goals $G_1, G_2, \dots$ with local contexts $\Delta_i$. Tactics $t \in \mathcal{T}$ from Ltac are parameterized commands executed by Coq, updating the current proof state or resulting in failure.

ASTactic’s objective is to learn a conditional distribution $P(t\mid S)$ over the large space of well-formed Coq tactics, given the current proof state $S$. In practice, the tactic space $\mathcal{T}$ is restricted to a context-free grammar of atomic Ltac tactics, with parameters (e.g., identifiers, integers) synthesized only from currently available, syntactically valid options. This constraint drastically reduces the complexity of argument search and aligns tactic generation with Coq’s semantics.

2. Neural Architecture: Encoder–Decoder over Trees

ASTactic adopts an encoder–decoder framework where both the input (proof state) and output (tactic program) are tree-structured.

Encoder: Each goal $G$ and each premise $r \in \Gamma \cup \Delta$ is parsed as a Coq term AST, composed of constructors (App, Lambda, Match, etc.). These ASTs are embedded using a bottom-up child-sum TreeLSTM [Tai et al., 2015]. For each node $v$ with constructor $G_1, G_2, \dots$0 and children $G_1, G_2, \dots$1, the TreeLSTM updates hidden and cell states: $G_1, G_2, \dots$2 where $G_1, G_2, \dots$3 is a learned embedding per constructor. The root state $G_1, G_2, \dots$4 is taken as the embedding for each term. Goal and premise embeddings are tagged to indicate their role, and, for efficiency, only up to $G_1, G_2, \dots$5 premises are encoded per state.

Decoder: The model produces tactic programs as ASTs corresponding to the Ltac grammar, generating each node in depth-first sequence. The decoder maintains a GRU state $G_1, G_2, \dots$6 at each step, which summarizes the current expansion context: $G_1, G_2, \dots$7 where $G_1, G_2, \dots$8 encodes the prior production rule, $G_1, G_2, \dots$9 encodes the parent node and expansion rule, $\Delta_i$0 is the current nonterminal, $\Delta_i$1 the goal embedding, and $\Delta_i$2 an attention-weighted summary of premise embeddings.

At each expansion step, a production rule is selected via a softmax over learned parameters: $\Delta_i$3 For argument synthesis:

• Premise identifiers: Each premise is scored by attention;
• Small integers: Chosen by discrete softmax;
• Quantified variables: Uniform selection among variables in $\Delta_i$4.

3. Probabilistic Modeling of Tactics

Let $\Delta_i$5 be a tactic AST with node set $\Delta_i$6. The probability of generating $\Delta_i$7 in proof state $\Delta_i$8 factors as: $\Delta_i$9 where $t \in \mathcal{T}$0 is the decoder state at node $t \in \mathcal{T}$1. For terminal nodes, the generation further conditions on the corresponding argument classifier, with context restrictions reflecting the current proof environment. This supports both deterministic (highest-probability) and stochastic decoding.

4. Training Strategy and Hyperparameters

ASTactic is trained on pairs $t \in \mathcal{T}$2 with $t \in \mathcal{T}$3 the ground-truth tactic AST executed in state $t \in \mathcal{T}$4 (extracted from 44,000 human-written proofs in CoqGym). Teacher forcing is employed: the target AST is traversed in depth-first order, with the network unrolled along ground-truth expansions, and losses accumulated as cross-entropy on both production rule and argument generation: $t \in \mathcal{T}$5 The optimization uses RMSProp [Tieleman & Hinton, 2012] with learning rate $t \in \mathcal{T}$6, weight decay $t \in \mathcal{T}$7, embedding/hidden sizes $t \in \mathcal{T}$8, over five epochs and batch size $t \in \mathcal{T}$9 (online updates). Training completes in a few days on a single GTX 1080 GPU.

5. Integration and Automated Proof Search

ASTactic is integrated with the Coq execution environment using a beam-search–enhanced depth-first search:

1. Given initial state $P(t\mid S)$0, invoke a recursive DFS-Prove procedure.
2. For each proof state, built-in Coq tactics (trivial, auto, intuition, easy, hammer) are applied first.
3. If none succeed, ASTactic proposes a beam of candidate tactics (via beam search over $P(t\mid S)$1); candidates are executed and resulting states recursed upon up to a maximum search depth (50) or wall time (10 minutes), pruning duplicate states.
4. Communication with Coq uses SerAPI (“Undo; …; t”) and reconstructs new states post-tactic application.

6. Empirical Results on CoqGym and Comparative Evaluation

Experiments on CoqGym use a split of 123 Coq projects: 43,844 proofs for training, 13,875 for validation, 13,137 for testing. Performance is measured by proof success rate—the percentage of theorems fully proved within 300 tactics and 10 minutes.

Method	Success Rate
trivial	2.4%
auto	2.9%
intuition	4.4%
easy	4.9%
hammer (20s)	17.8%
hammer (10 min)	24.8%
ASTactic	12.2%
ASTactic + auto	12.8%
ASTactic + hammer (20s)	23.6%
ASTactic + hammer (10 min)	30.0%

Combining ASTactic with “hammer” results in a new state-of-the-art 30.0% proof success rate (vs. 24.8% for hammer alone). Beam width controls a trade-off between computational time and coverage: width 20 yields 12.2% success at approximately 3.3 seconds per proof. Generated proofs are relatively short (6.0 steps on average vs. 12.5 for humans), and the number of tactics tried per proof is much lower than prior automaton-based systems (200 vs. 10,000 for SEPIA).

7. Strengths, Limitations, and Prospects

Strengths:

• Learns to synthesize in-vocabulary tactics with contextually valid arguments, contrasting with fixed-vocabulary or retrieval techniques.
• Employs semantic constraints to drastically reduce argument search complexity.
• Complements external automated theorem provers (“hammers”), providing a clear additive gain in proof success.
• Proof search is efficient, requiring fewer calls to Coq per proof.

Limitations:

• The tactic grammar is a hand-pruned subset of full Ltac; complex compound tactics (e.g., $P(t\mid S)$2), repetition, and custom tactics are excluded.
• The encoder only attends to a maximum of 10 premises, potentially limiting effectiveness; more comprehensive premise selection remains open.
• Proofs are generated stepwise using a Markovian GRU, making long-range proof planning challenging.

Potential Extensions:

• Extending coverage to the full Ltac language, including compound tactics and repetition.
• Use of end-to-end reinforcement learning, treating ASTactic as a policy to improve proof-space exploration.
• Explicit modeling of premise selection, potentially with learned attention over entire libraries.
• Transfer to other proof assistants, such as Isabelle or HOL Light.

ASTactic exemplifies an overview of neural program generation and symbolic proof search, offering a data-driven framework for interactive theorem proving that leverages both syntax and domain semantics (Yang et al., 2019).