Using a "Study of Studies" to help statistics students assess research findings

Abstract: One learning goal of the introductory statistics course is to develop the ability to make sense of research findings in published papers. The Atlantic magazine regularly publishes a feature called "Study of Studies" that summarizes multiple articles published in a particular domain. We describe a classroom activity to develop this capacity using the "Study of Studies." In this activity, students read capsule summaries of twelve research papers related to restaurants and dining that was published in April 2015. The selected papers report on topics such as how seating arrangement, server posture, plate color and size, and the use of background music relate to revenue, ambiance, and perceived food quality. The students are assigned one of the twelve papers to read and critique as part of a small group. Their group critiques are shared with the class and the instructor. A pilot study was conducted during the 2015-2016 academic year at Amherst College. Students noted that key details were not included in the published summary. They were generally skeptical of the published conclusions. The students often provided additional summarization of information from the journal articles that better describe the results. By independently assessing and comparing the original study conclusions with the capsule summary in the "Study of Studies," students can practice developing judgment and assessing the validity of statistical results.