Representativeness of interview non-use rate of Microsoft Copilot

Ascertain whether the higher proportion (50%) of interview participants who did not use Microsoft Copilot reflects the actual rate of generative AI non-use among all students enrolled in MATH-M403, MATH-M404, and MATH-M421 during Spring 2025.

Background

The paper surveyed 17 students (19 total survey responses) and conducted 4 interviews across three upper-level, proof-based undergraduate mathematics courses at Indiana University East in Spring 2025: MATH-M403 (Abstract Algebra I), MATH-M404 (Abstract Algebra II), and MATH-M421 (Topology).

The authors note a discrepancy between survey and interview samples regarding generative AI non-use: 50% of interviewees reported not using Microsoft Copilot, which was higher than the proportion among survey respondents. They explicitly state uncertainty about whether the interview figure represents the true usage rate among all students in the courses, raising a question about sample representativeness and inference validity.

References

Another factor to note is the fact that a higher percentage of interviewees ($50\%$) didn't use Copilot in comparison to the survey participants but it is not clear whether this represents the actual percentage of AI usage among the students in the courses.

— Gen AI in Proof-based Math Courses: A Pilot Study (2509.13570 - Klawa et al., 16 Sep 2025) in Discussion, Summary of Main Findings