Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Graph2Tac: Online Representation Learning of Formal Math Concepts (2401.02949v3)

Published 5 Jan 2024 in cs.LG and cs.AI

Abstract: In proof assistants, the physical proximity between two formal mathematical concepts is a strong predictor of their mutual relevance. Furthermore, lemmas with close proximity regularly exhibit similar proof structures. We show that this locality property can be exploited through online learning techniques to obtain solving agents that far surpass offline learners when asked to prove theorems in an unseen mathematical setting. We extensively benchmark two such online solvers implemented in the Tactician platform for the Coq proof assistant: First, Tactician's online $k$-nearest neighbor solver, which can learn from recent proofs, shows a $1.72\times$ improvement in theorems proved over an offline equivalent. Second, we introduce a graph neural network, Graph2Tac, with a novel approach to build hierarchical representations for new definitions. Graph2Tac's online definition task realizes a $1.5\times$ improvement in theorems solved over an offline baseline. The $k$-NN and Graph2Tac solvers rely on orthogonal online data, making them highly complementary. Their combination improves $1.27\times$ over their individual performances. Both solvers outperform all other general-purpose provers for Coq, including CoqHammer, Proverbot9001, and a transformer baseline by at least $1.48\times$ and are available for practical use by end-users.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. TensorFlow: Large-scale machine learning on heterogeneous systems. 2015. URL https://www.tensorflow.org/. Software available from tensorflow.org.
  2. Anonyous. Hashing Modulo Context-Sensitive Alpha-Equivalence. http://64.71.146.254:8000/SFyud_C5TmEq7AwnK9jaLsfTFMzgBl54cQ0pl2FJB-x9o2Hk24F4jO_W75RqGdOJ/papers/sharing-paper.pdf, 2023a.
  3. Anonyous. The Tactician’s Web of Large-Scale Formal Knowledge. http://64.71.146.254:8000/SFyud_C5TmEq7AwnK9jaLsfTFMzgBl54cQ0pl2FJB-x9o2Hk24F4jO_W75RqGdOJ/papers/web-paper.pdf, 2023b.
  4. A modular integration of SAT/SMT solvers to coq through proof witnesses. In Jean-Pierre Jouannaud and Zhong Shao (eds.), Certified Programs and Proofs - First International Conference, CPP 2011, Kenting, Taiwan, December 7-9, 2011. Proceedings, volume 7086 of Lecture Notes in Computer Science, pp.  135–150. Springer, 2011. doi: 10.1007/978-3-642-25379-9_12. URL https://doi.org/10.1007/978-3-642-25379-9_12.
  5. Holist: An environment for machine learning of higher order logic theorem proving. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp.  454–463. PMLR, 2019. URL http://proceedings.mlr.press/v97/bansal19a.html.
  6. Frédéric Besson. Itauto: An extensible intuitionistic SAT solver. In Liron Cohen and Cezary Kaliszyk (eds.), 12th International Conference on Interactive Theorem Proving, ITP 2021, June 29 to July 1, 2021, Rome, Italy (Virtual Conference), volume 193 of LIPIcs, pp. 9:1–9:18. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2021. doi: 10.4230/LIPIcs.ITP.2021.9. URL https://doi.org/10.4230/LIPIcs.ITP.2021.9.
  7. Lasse Blaauwbroek. Tactician’s web of large-scale formal knowledge, December 2023. URL https://doi.org/10.5281/zenodo.10028721.
  8. Tactic learning and proving for the coq proof assistant. In Elvira Albert and Laura Kovács (eds.), LPAR 2020: 23rd International Conference on Logic for Programming, Artificial Intelligence and Reasoning, Alicante, Spain, May 22-27, 2020, volume 73 of EPiC Series in Computing, pp.  138–150. EasyChair, 2020a. doi: 10.29007/wg1q. URL https://doi.org/10.29007/wg1q.
  9. The tactician - A seamless, interactive tactic learner and prover for coq. In Christoph Benzmüller and Bruce R. Miller (eds.), Intelligent Computer Mathematics - 13th International Conference, CICM 2020, Bertinoro, Italy, July 26-31, 2020, Proceedings, volume 12236 of Lecture Notes in Computer Science, pp.  271–277. Springer, 2020b. doi: 10.1007/978-3-030-53518-6_17. URL https://doi.org/10.1007/978-3-030-53518-6_17.
  10. A learning-based fact selector for isabelle/hol. J. Autom. Reason., 57(3):219–244, 2016a. doi: 10.1007/s10817-016-9362-8. URL https://doi.org/10.1007/s10817-016-9362-8.
  11. Hammering towards QED. J. Formaliz. Reason., 9(1):101–148, 2016b. doi: 10.6092/issn.1972-5787/4593. URL https://doi.org/10.6092/issn.1972-5787/4593.
  12. Formal verification of a memory model for C-like imperative languages. In International Conference on Formal Engineering Methods (ICFEM 2005), volume 3785 of Lecture Notes in Computer Science, pp. 280–299. Springer, 2005. URL http://xavierleroy.org/publi/memory-model.pdf.
  13. ENIGMA-NG: efficient neural and gradient-boosted inference guidance for E. In Pascal Fontaine (ed.), Automated Deduction - CADE 27 - 27th International Conference on Automated Deduction, Natal, Brazil, August 27-30, 2019, Proceedings, volume 11716 of Lecture Notes in Computer Science, pp.  197–215. Springer, 2019. doi: 10.1007/978-3-030-29436-6_12. URL https://doi.org/10.1007/978-3-030-29436-6_12.
  14. Lukasz Czajka. Practical proof search for coq by type inhabitation. In Nicolas Peltier and Viorica Sofronie-Stokkermans (eds.), Automated Reasoning - 10th International Joint Conference, IJCAR 2020, Paris, France, July 1-4, 2020, Proceedings, Part II, volume 12167 of Lecture Notes in Computer Science, pp.  28–57. Springer, 2020. doi: 10.1007/978-3-030-51054-1_3. URL https://doi.org/10.1007/978-3-030-51054-1_3.
  15. Hammer for coq: Automation for dependent type theory. J. Autom. Reason., 61(1-4):423–453, 2018. doi: 10.1007/s10817-018-9458-4. URL https://doi.org/10.1007/s10817-018-9458-4.
  16. TF-GNN: graph neural networks in tensorflow. CoRR, abs/2207.03522, 2022. URL http://arxiv.org/abs/2207.03522.
  17. Diversity-driven automated formal verification. In Proceedings of the 44th International Conference on Software Engineering, ICSE ’22, pp.  749–761, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450392211. doi: 10.1145/3510003.3510138. URL https://doi.org/10.1145/3510003.3510138.
  18. Tactok: Semantics-aware proof synthesis. Proc. ACM Program. Lang., 4(OOPSLA), nov 2020. doi: 10.1145/3428299. URL https://doi.org/10.1145/3428299.
  19. Tactictoe: Learning to reason with hol4 tactics. In Thomas Eiter and David Sands (eds.), LPAR-21. 21st International Conference on Logic for Programming, Artificial Intelligence and Reasoning, volume 46 of EPiC Series in Computing, pp.  125–143. EasyChair, 2017. doi: 10.29007/ntlb. URL https://easychair.org/publications/paper/WsM.
  20. The isabelle ENIGMA. In June Andronick and Leonardo de Moura (eds.), 13th International Conference on Interactive Theorem Proving, ITP 2022, August 7-10, 2022, Haifa, Israel, volume 237 of LIPIcs, pp.  16:1–16:21. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022. doi: 10.4230/LIPIcs.ITP.2022.16. URL https://doi.org/10.4230/LIPIcs.ITP.2022.16.
  21. Georges Gonthier. The four colour theorem: Engineering of a formal proof. In Deepak Kapur (ed.), Computer Mathematics, pp.  333–333, Berlin, Heidelberg, 2008. Springer Berlin Heidelberg. ISBN 978-3-540-87827-8.
  22. Sepia: Search for proofs using inferred automata. In Amy P. Felty and Aart Middeldorp (eds.), Automated Deduction - CADE-25, pp.  246–255, Cham, 2015. Springer International Publishing. ISBN 978-3-319-21401-6.
  23. Proof artifact co-training for theorem proving with language models. In International Conference on Learning Representations, 2021.
  24. Y. F. Hu. Efficient and high quality force-directed graph drawing. The Mathematica Journal, 10:37–71, 2005. URL http://www.mathematica-journal.com/issue/v10i1/contents/graph_draw/graph_draw.pdf.
  25. Gamepad: A learning environment for theorem proving. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL https://openreview.net/forum?id=r1xwKoR9Y7.
  26. ENIGMA: efficient learning-based inference guiding machine. In Herman Geuvers, Matthew England, Osman Hasan, Florian Rabe, and Olaf Teschke (eds.), Intelligent Computer Mathematics - 10th International Conference, CICM 2017, Edinburgh, UK, July 17-21, 2017, Proceedings, volume 10383 of Lecture Notes in Computer Science, pp. 292–302. Springer, 2017. doi: 10.1007/978-3-319-62075-6_20. URL https://doi.org/10.1007/978-3-319-62075-6_20.
  27. ENIGMA anonymous: Symbol-independent inference guiding machine (system description). In Nicolas Peltier and Viorica Sofronie-Stokkermans (eds.), Automated Reasoning - 10th International Joint Conference, IJCAR 2020, Paris, France, July 1-4, 2020, Proceedings, Part II, volume 12167 of Lecture Notes in Computer Science, pp.  448–463. Springer, 2020. doi: 10.1007/978-3-030-51054-1_29. URL https://doi.org/10.1007/978-3-030-51054-1_29.
  28. Thor: Wielding hammers to integrate language models and automated theorem provers. In NeurIPS, 2022. URL http://papers.nips.cc/paper_files/paper/2022/hash/377c25312668e48f2e531e2f2c422483-Abstract-Conference.html.
  29. Mizar 40 for mizar 40. Journal of Automated Reasoning, 55(3):245–256, 2015.
  30. Machine learning of coq proof guidance: First experiments. In Temur Kutsia and Andrei Voronkov (eds.), 6th International Symposium on Symbolic Computation in Software Science, SCSS 2014, Gammarth, La Marsa, Tunisia, December 7-8, 2014, volume 30 of EPiC Series in Computing, pp.  27–34. EasyChair, 2014. doi: 10.29007/lmmg. URL https://doi.org/10.29007/lmmg.
  31. Machine learning in proof general. In Proceedings 10th International Workshop On User Interfaces for Theorem Provers (UITP 2012), Electronic Proceedings in Theoretical Computer Science, pp.  15–41, Australia, 2013. Open Publishing Association. doi: 10.4204/EPTCS.118.2.
  32. Mash: Machine learning for sledgehammer. In Sandrine Blazy, Christine Paulin-Mohring, and David Pichardie (eds.), Interactive Theorem Proving - 4th International Conference, ITP 2013, Rennes, France, July 22-26, 2013. Proceedings, volume 7998 of Lecture Notes in Computer Science, pp.  35–50. Springer, 2013. doi: 10.1007/978-3-642-39634-2_6. URL https://doi.org/10.1007/978-3-642-39634-2_6.
  33. Hypertree proof search for neural theorem proving. In NeurIPS, 2022. URL http://papers.nips.cc/paper_files/paper/2022/hash/a8901c5e85fb8e1823bbf0f755053672-Abstract-Conference.html.
  34. Magnushammer: A transformer-based approach to premise selection. arXiv preprint arXiv:2303.04488, 2023.
  35. Christine Paulin-Mohring. Inductive definitions in the system coq - rules and properties. In Marc Bezem and Jan Friso Groote (eds.), Typed Lambda Calculi and Applications, International Conference on Typed Lambda Calculi and Applications, TLCA ’93, Utrecht, The Netherlands, March 16-18, 1993, Proceedings, volume 664 of Lecture Notes in Computer Science, pp. 328–345. Springer, 1993. doi: 10.1007/BFb0037116. URL https://doi.org/10.1007/BFb0037116.
  36. Machine-learned premise selection for lean. In Revantha Ramanayake and Josef Urban (eds.), Automated Reasoning with Analytic Tableaux and Related Methods, pp.  175–186, Cham, 2023. Springer Nature Switzerland. ISBN 978-3-031-43513-3.
  37. Generative language modeling for automated theorem proving. arXiv preprint arXiv:2009.03393, 2020.
  38. Language models are unsupervised multitask learners. 2019.
  39. Artifact for: Graph2Tac: Learning Hierarchical Representations of Math Concepts in Theorem Proving, December 2023. URL https://doi.org/10.5281/zenodo.10410474.
  40. Generating correctness proofs with neural networks. In Proceedings of the 4th ACM SIGPLAN International Workshop on Machine Learning and Programming Languages, MAPL 2020, pp.  1–10, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450379960. doi: 10.1145/3394450.3397466. URL https://doi.org/10.1145/3394450.3397466.
  41. Passport: Improving automated formal verification using identifiers. ACM Trans. Program. Lang. Syst., 45(2), jun 2023. ISSN 0164-0925. doi: 10.1145/3593374. URL https://doi.org/10.1145/3593374.
  42. Josef Urban. Translating mizar for first order theorem provers. In Andrea Asperti, Bruno Buchberger, and James H. Davenport (eds.), Mathematical Knowledge Management, Second International Conference, MKM 2003, Bertinoro, Italy, February 16-18, 2003, Proceedings, volume 2594 of Lecture Notes in Computer Science, pp.  203–215. Springer, 2003. doi: 10.1007/3-540-36469-2_16. URL https://doi.org/10.1007/3-540-36469-2_16.
  43. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp.  38–45, Online, October 2020. Association for Computational Linguistics. URL https://www.aclweb.org/anthology/2020.emnlp-demos.6.
  44. Learning to prove theorems via interacting with proof assistants. In International Conference on Machine Learning (ICML), 2019.
  45. LeanDojo: Theorem proving with retrieval-augmented language models. arXiv preprint arXiv:2306.15626, 2023.
  46. Online machine learning techniques for coq: A comparison. In Fairouz Kamareddine and Claudio Sacerdoti Coen (eds.), Intelligent Computer Mathematics, pp.  67–83, Cham, 2021. Springer International Publishing. ISBN 978-3-030-81097-9.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now