Teaching "Foundations of Mathematics" with the Lean Theorem Prover (2501.03352v3)

Abstract: This study aims to observe if the theorem prover Lean positively influences students' understanding of mathematical proving. To this end, we perform a pilot study concerning freshmen students at the University of Zurich (UZH). While doing so, we apply certain teaching methods and gather data from the volunteer students enrolled in the ``Foundations of Mathematics'' course. After eleven weeks of study covering some exercise questions implemented with Lean, we measure Lean students' performances in proving mathematical statements, compared to other students who are not engaged with Lean. For this measurement, we interview five Lean and four Non-Lean students and we analyze the scores of all students in the final exam. Finally, we check significance by performing a $t$-test for independent samples and the Mann-Whitney $U$-test.

• The paper demonstrates that teaching with the LEAN theorem prover significantly improves freshmen’s ability to construct mathematical proofs, as shown by higher exam scores (p < 0.05).
• The study employed a rigorous methodology using LEAN exercises, interviews, and statistical tests (t-test and Mann-Whitney U-test) to compare LEAN and non-LEAN students' performance.
• The results indicate that LEAN not only motivates students to invest more time in proofs but also supports the integration of automated tactics into undergraduate mathematics education.

The paper "Teaching ``Foundations of Mathematics'' with the LEAN Theorem Prover" investigates the impact of the LEAN theorem prover on students' comprehension of mathematical proofs. The paper focuses on first-year students at the University of Zurich (UZH) enrolled in the "Foundations of Mathematics" course. The researchers taught volunteer students using LEAN and collected data to compare their proving abilities with those of students who did not use LEAN. After eleven weeks of instruction, which included exercises implemented in LEAN, the students' performance in proving mathematical statements was evaluated through interviews and an analysis of final exam scores. Statistical significance was assessed using a $t$-test for independent samples and the Mann-Whitney $U$-test.

The paper begins by highlighting the evolution of automated theorem provers and proof assistants, noting the strengths and weaknesses of systems like Agda, Isabelle, and Coq. LEAN was introduced in 2013 by Leonardo de Moura at Microsoft Research Redmond, and is designed to bridge the gap between proof assistants and automated theorem provers. Although LEAN is based on abstract mathematics, it has become popular in undergraduate education. The paper contributes further evidence of the positive effects of teaching with LEAN on freshman students' performance in mathematics. To conduct the paper, the researchers implemented seven LEAN exercise sheets based on the course content and organized eleven sessions over one semester to teach the foundations of mathematics with LEAN to volunteer students.

The methodology included employing teaching methods, such as defining goals for students, promoting intrinsic motivation, using varied teaching approaches, setting appropriate difficulty levels, differentiating instruction, using digital resources, and creating a learn-efficient class climate. Data was gathered by conducting interviews with both LEAN and Non-LEAN students and comparing their proof writing abilities, as well as analyzing final exam scores. The significance of the results was measured using the $t$-test and the Mann-Whitney $U$-test. A questionnaire with open and closed questions was also used to assess the motivation and contentment of the LEAN students.

The paper provides a detailed overview of the LEAN interface and tactics, focusing on their practical use. The LEAN interface, particularly the infoview, allows users to track the progress of proofs and identify mistakes. Tactics, such as intro, exact, apply, rw (rewrite), constructor, cases', symm, induction', and ring_nf, are introduced as instructions for constructing proofs.

Key tactics discussed include:

• intro: Used to introduce variables and hypotheses when the goal is an implication.
• exact: Applies a proposition or proof to satisfy the current goal.
• apply: Used when a hypothesis contains an implication and the goal matches the conclusion of that implication.
• rw (rewrite): Rewrites statements based on if-and-only-if hypotheses, with options to rewrite specific hypotheses or repeat rewrites.
• constructor: Splits an if-and-only-if goal into two subgoals, requiring both implications to be proved.
• cases': Used when a disjunction ($\vee$) or conjunction ($\wedge$) appears in a hypothesis, splitting the proof into multiple cases.
• symm: Interchanges the left- and right-hand sides of an equality in a goal or hypothesis.
• induction': Initiates proofs by induction over a variable, with options for strong induction.
• ring_nf: Proves statements involving ring arithmetic.

The teaching methods used in the paper were designed to enhance student learning. At the start of each meeting, the session goals were presented to the students. The goals were structured to include both the content and the skills students should develop, based on Mager's method [Mager 1978]. The meetings were structured using the sandwich method [Wahl 2013], alternating between direct instruction and hands-on activities, allowing students to work independently. The difficulty level of the exercises was also carefully considered, aiming for the 'zone of proximal development' [Vygotsky 1978]. Instruction was tailored to each student's needs, and STEM (Science, Technology, Engineering, and Mathematics) skills were fostered by combining technology and mathematics.

The data collection process focused on measuring the quality of proofs and designing interviews and questionnaires for LEAN and Non-LEAN students. The quality of a proof was assessed based on criteria such as the use of definitions, mathematical symbols, logical statements, high-level ideas, modular structure, and examples. The interviews consisted of four questions designed to cover topics from the lecture, including propositional logic, natural numbers and induction, equivalence relations, and set theory. Grading tables were used to assess the students' responses based on the proof quality characteristics outlined. A questionnaire was also administered to gather insights into how LEAN impacted the students.

The results showed that LEAN students performed better in the final exam, with a mean score significantly higher than that of Non-LEAN students ($p < 0.05$). While LEAN students also performed better on average in the interviews, the difference was not statistically significant ($p > 0.05$). A $t$-test for independent samples and the Mann-Whitney $U$-test were used to determine the statistical significance of these results. The paper also analyzed student performance on individual interview questions, noting strengths and weaknesses in areas such as propositional logic and set theory.

The questionnaire results indicated that LEAN motivated students to spend more time on proofs, improved their proving skills, and positively influenced their preparation for the final exam. Most students believed that mathematics should be taught with LEAN in the future, although some noted the challenges associated with learning the language.

The authors also discuss their personal experiences learning and teaching with LEAN, noting the benefits of using LEAN to structure mathematical proofs and the challenges of relying too heavily on automated tactics without fully understanding each step. They suggest that a hybrid approach, combining LEAN with traditional paper-based methods, might be more suitable for undergraduate teaching.