Student behavior and test security in online conceptual assessment (1912.05698v1)

Abstract: Historically, the implementation of research-based assessments (RBAs) has been a driver of education change within physics and helped motivate adoption of interactive engagement pedagogies. Until recently, RBAs were given to students exclusively on paper and in-class; however, this approach has important drawbacks including decentralized data collection and the need to sacrifice class time. Recently, some RBAs have been moved to online platforms to address these limitations. Yet, online RBAs present new concerns such as student participation rates, test security, and students' use of outside resources. Here, we report on a pilot study addressing these concerns. We gave two upper-division RBAs to courses at five institutions; the RBAs were hosted online and featured embedded JavaScript code which collected information on students' behaviors (e.g., copying text, printing). With these data, we examine the prevalence of these behaviors, and their correlation with students' scores, to determine if online and paper-based RBAs are comparable. We find that browser loss of focus is the most common online behavior while copying and printing events were rarer.We found no statistically significant correlation between any of these online behaviors and students scores. We also found that participation rates for our upper-division population went up when the RBA was given online. These results indicates that, for our upper-division population, scores on online administrations of these RBAs were comparable to in-class versions.