Validity Verification of the New TOEFL Writing Task Based on Classical Test Theory

Abstract: The TOEFL iBT has introduced the Academic Discussion Task (ADT) to assess test-takers' ability to engage in academic discourse, reflecting the growing emphasis on interactive communication skills in higher education. However, research on the ADT's validity and fairness particularly for culturally and linguistically diverse groups, such as Chinese students, remains limited. This study addresses this gap by employing Classical Test Theory (CTT) to evaluate the psychometric properties of the ADT among Chinese university students. This study finds a robust correlation between the ADT and the CET-6 writing and translation subscores. In addition, there is a high level of expert agreement regarding the construct validity evidence and the appropriateness of the scoring rubric. Furthermore, the results indicate that gender differences in validity indices are minimal. Taken together, these results suggest that the ADT is a valid measure for Chinese test-takers without gender discrimination. However, it is recommended that the cultural sensitivity of the scoring rubric be further refined and that the CET-6 subscores for writing be retained for predictive purposes, in order to better accommodate the needs of diverse test-taker populations. By addressing these issues, this study contributes to the broader discourse on fairness and validity in high-stakes language assessments.