Twitch: Learning Abstractions for Equational Theorem Proving

Abstract: Several successful strategies in automated reasoning rely on human-supplied guidance about which term or clause shapes are interesting. In this paper we aim to discover interesting term shapes automatically. Specifically, we discover abstractions : term patterns that occur over and over again in relevant proofs. We present our tool Twitch which discovers abstractions with the help of Stitch, a tool originally developed for discovering reusable library functions in program synthesis tasks. Twitch can produce abstractions in two ways: (1) from a partial, failed proof of a conjecture; (2) from successful proofs of other theorems in the same domain. We have also extended Twee, an equational theorem prover, to use these abstractions. We evaluate Twitch on a set of unit equality (UEQ) problems from TPTP, and show that it can prove 12 rating-1 problems as well as yielding significant speed-ups on many other problems.

• The paper introduces Twitch, which uses Stitch to learn reusable term patterns from partial or successful proofs and lowers their heuristic weights in Twee.
• Experiments on 1,041 TPTP unit-equality problems show domain abstractions solved 18 hard cases, including 12 rating-1 problems, while combined methods reduced runtimes and solved about 25 additional cases within 300 seconds.
• The approach guides term selection without expanding the inference space, but depends on similar prior proofs, parameter tuning, and careful handling of demodulation and domain selection.

Motivation and overview

Saturation-based equational provers such as Twee face a combinatorial selection problem: a typical proof may generate on the order of $10^5$ rewrite rules and $10^{10}$ candidate critical pairs, of which only a few hundred enter the final proof. Twee's default ranking relies largely on term weight, which is effective but coarse. Rather than supplying human-crafted guidance in the tradition of the weighting, resonance, and hint strategies, Axelrod, Johansson, and Smallbone propose to learn interesting term shapes automatically. Their tool, Twitch, uses Stitch—a library-learning system originally built for program synthesis—to compress sets of proof terms into reusable function definitions, called abstractions. Abstractions are obtained either from a timed-out partial proof attempt of the target conjecture (partial proof abstractions) or from successful proofs of easier problems in the same domain (domain abstractions), the latter following a curriculum-learning methodology. The abstractions are consumed by an extended version of Twee that lowers the computed weight of terms matching abstraction patterns.

The paper reports 12 rating-1 TPTP UEQ problems solved with these techniques (18 hard problems overall by domain abstractions under their hardness criterion), plus broad runtime reductions. The central hypothesis—that recurring term shapes in related proofs are good search guidance for other problems—is plausible but empirical; the paper offers no theoretical justification for why compression-minimal patterns transfer across conjectures.

Discovering abstractions with Stitch

Stitch takes a set of terms and searches for maximally compressive function definitions, maximizing the product of definition size and number of use sites. Since Stitch operates on higher-order $\lambda$-terms, Twitch translates first-order proof terms by treating function symbols as constants and $\lambda$-abstracting free variables; higher-order abstractions produced in practice are discarded. On the motivating problem LAT075-1 (modular ortholattices over the Sheffer stroke), Stitch recovers $g(\alpha) = f(\alpha,\alpha)$—the negation operator—and, from full proofs, also $f(f(x,y), f(x,y))$, corresponding to conjunction. Several learned abstractions are thus semantically interpretable, which the authors note is desirable if proofs are to be reformulated mathematically, though interpretability is not enforced anywhere in the pipeline.

For partial proof abstractions, lemmas extracted from the failed run are ranked by an "interestingness" score $s(l=r) = |T(l=r)|/|l{=}r|^2$—simple statements with long proofs score highest. The top $k$ lemmas' proof terms are fed to Stitch. This scoring is admittedly a rough heuristic; the authors do not claim it matches human judgments of lemma value.

Domain abstractions follow a three-stage procedure: compute per-problem local abstractions via Stitch on each solved "easy" problem; retain only those yielding an empirical speedup above threshold $\tau$ (measured as the ratio of baseline to augmented runtime); then run Stitch again over the retained set to derive domain-level abstractions. Local abstraction generation iterates over strategy combinations (goal flattening on/off × weight factors $\{0, 0.2, 0.5, 0.7\}$) to enlarge the pool of good candidates. A compatibility filter removes abstractions referencing symbols absent from the hard target. Notably, the GRP677-1 success required hand-curating the domain to problems with very similar axioms—the authors themselves conclude that a larger domain is not necessarily better, leaving automatic domain curation open.

Using abstractions inside Twee

Twitch extends Twee's weight function so that any subterm matching an abstraction $10^{10}$0 via substitution $10^{10}$1 costs $10^{10}$2 rather than its full weight. Two properties matter: the abstraction skeleton counts once regardless of nesting, and variables appearing multiple times in $10^{10}$3 have their instantiations counted only once. The latter is essential—for the Sheffer-stroke abstraction $10^{10}$4, the term $10^{10}$5 must weigh the same as a hypothetical unary negation symbol applied to $10^{10}$6. Two schemes for $10^{10}$7 are provided: constant weight $10^{10}$8, and skeleton weight times factor $10^{10}$9. Empirically a weight factor near 0.5 performs best overall, with lower (more aggressive) factors occasionally best on individual problems but more prone to timeouts.

Crucially, abstractions modify only the heuristic ranking, not the search space itself. This contrasts with adding definitional axioms, which enables rewriting toward smaller terms but introduces new inference opportunities. In ablation experiments, definitional axioms scale poorly: increasing from 1 to 10 axioms sharply increases timeouts, whereas the abstraction mechanism degrades gracefully. The abstraction approach therefore yields a larger pool of usable abstractions. However, the comparison is not one-sided: LAT075-1 drops from roughly 250 s to 10 s with the axiom $\lambda$0, versus 130 s with the abstraction mechanism, and on LAT141-1 an abstraction-as-axiom gives ~12 s against ~260 s for the abstraction mechanism. Fifteen problems favor axioms outright, and the authors acknowledge that combining both mechanisms may be worthwhile.

Experimental results

Evaluation uses 1041 unsatisfiable unit-equality TPTP v9.2.1 problems across nine domains (GRP largest at 481). Hardness requires TPTP rating ≥ 0.9 and baseline-Twee failure within 1000 s, giving 70 targets. Key findings:

• Runtime improvements: Domain abstractions alone give modest speedups, but combined with goal flattening they solve roughly 25 additional problems within 300 s, and roughly halve runtime on problems baseline Twee solves within 300 s. The strong interaction with goal flattening is reported but not explained mechanistically.
• Partial proof abstractions solve 11 hard problems within a 1000 s total budget (including a 500 s partial run). Six are LCL Horn-clause encodings where the decisive abstraction $\lambda$1 recurs consistently.
• Domain abstractions solve 18 hard problems, subsuming all partial-proof successes, including LAT075-1 in 32 s (rating 1.00) using five abstractions, of which manual inspection confirms two are crucial. One further problem (LCL351-10) falls only to combining both abstraction kinds.
• Learned abstractions include recognizable algebraic content (negation, conjunction, self-distributivity sites for NAND).

An important caveat: results on hard problems were obtained by iterating over up to eight parameter configurations per problem, so the headline solve counts reflect best-case parameter search rather than a single fixed configuration. Learning time for domain abstractions is excluded from the per-problem timings, justified on grounds of amortization across the domain.

Limitations

The authors are explicit that the system is simple-minded in several respects. Domain construction is crude—restricted to same-theory TPTP problems sharing concrete symbols—with no automatic generation of weakened conjectures. Lemma scoring and abstraction selection use single-objective runtime speedup, ignoring alternatives such as proof length or robustness. A deeper architectural issue is that Twee demodulates critical pairs, which can destroy matches: e.g., with associative $\lambda$2, $\lambda$3 rewrites to $\lambda$4, no longer matching the $\lambda$5 abstraction. The proposed remedy—deriving extra abstractions from overlap critical pairs—produces forms whose fidelity to user intent is unclear and remains unvalidated. Whether compression-derived abstractions generalize beyond syntactically similar domains is likewise unresolved.

Conclusion

Twitch demonstrates that automatically compressing existing or failed proofs into weighted term abstractions is a workable form of search guidance for equational completion: it yields 12 newly solved rating-1 TPTP problems, substantial aggregate speedups when combined with goal flattening, and a mechanism that scales where definitional axioms do not. The method's dependence on available easy proofs, its brittle interaction with demodulation, and its best-case parameter tuning remain the principal obstacles between the current pipeline and a turnkey system.