The challenging task of investigating student thinking: an example from quantum computing

Abstract: Physics education research (PER) studies how students learn physics, yet the nuances of student reasoning can be notoriously difficult to probe even for PER practitioners. This article presents the story of Item 15 on the Quantum Computing Conceptual Survey (QCCS). This assessment item underwent more revision and discussion within the team than the remaining 19 assessment questions combined. This paper provides a behind-the-scenes look at the development of this assessment question: a story that reveals not only interesting findings about student reasoning in quantum computing (specifically, the phenomenon of phase kickback) but provides a cautionary tale for any instructor or researcher attempting to evaluate student reasoning in physics through a multiple-choice test item.