- The paper demonstrates that effective deployment of the AI Assessment Scale is contingent on sustained faculty development and participatory governance.
- It uses qualitative focus groups from institutions in Vietnam and the UK to reveal that institutional conditions and ethical considerations critically affect GenAI integration.
- The study highlights that without careful structural design, frameworks like the AIAS risk evolving into mere compliance artifacts that undermine assessment validity.
Context and Motivation
The proliferation of generative artificial intelligence (GenAI) in higher education has necessitated rapid and substantive assessment reform. The paper scrutinizes the practical deployment of the Artificial Intelligence Assessment Scale (AIAS) within two contrasting institutional contexts—a private university in Vietnam and a public university in the United Kingdom. Rather than focusing solely on the conceptual merits of frameworks like the AIAS, the authors interrogate the institutional and pedagogical conditions that enable or obstruct their effective implementation. The study leverages Critical AI Literacy (CAIL) as an analytic lens to elucidate how issues of integrity, equity, and validity manifest in real-world faculty experiences.
Methodology
The authors adopt a qualitative research protocol anchored in five focus groups comprising 30 academic staff. Data collection employs a hybrid thematic analysis, integrating deductive coding (informed by CAIL and research questions) with inductive theme generation. Cross-site sampling was purposive in Vietnam and self-selected in the UK, reflecting diverse disciplinary perspectives and adoption stages. Rigorous analytic reflexivity and external validation mitigate risks of bias, including those arising from developer involvement in the framework under study.
Thematic Findings
Six interpretive themes encapsulate faculty engagements and institutional dynamics shaping AIAS implementation:
Recognizing and Integrating AI
Faculty broadly perceived the AIAS as legitimizing the use of GenAI, facilitating curiosity-driven experimentation, and providing bounded permission for tool integration into assessment. However, the degree to which the AIAS itself, as opposed to GenAI generally, catalyzed pedagogical change was variable, with uneven adoption and occasional attribution of benefits largely to GenAI rather than the scale.
Facilitating Conditions
Implementation was profoundly contingent on institutional policy clarity, governance structure, workload constraints, and evaluative mechanisms. Rapid, ambiguous rollouts engendered uncertainty, eroded confidence, and invited concerns regarding fairness and capacity to monitor student AI use. Conversely, faculty valued the AIAS as a shared language that could foster clarity and transparency when appropriately embedded.
Building Capacity
Sustained, practice-based training and peer exchange were regarded as indispensable for faculty development and critical awareness. Top-down directives were largely insufficient; the sharing of exemplars and best practices within disciplinary communities supported more robust integration.
Pathways to Adoption
Adoption trajectories diverged; institutional mandates (top-down) expedited rollout but diminished faculty ownership and reflection, whereas department-led (bottom-up) approaches fostered local innovation but precipitated inconsistency. The optimal governance logic appears to blend central policy direction with local championing and participatory feedback loops.
Ethics in Practice
Faculty highlighted ethical imperatives around fairness, critical engagement, and equitable access. Critical AI literacy and questioning were considered essential to responsible GenAI adoption. Inadequate attention to equity risks entrenching disparities, particularly when access to advanced tools is mediated by students’ financial capital.
Reframing Pedagogy
The AIAS rendered GenAI use visible within assessment design, prompting reevaluation of authentic assessment, student accountability, disciplinary alignment, and learning outcome coherence. Notably, misalignment risks emerged: when the AIAS level became the de facto object of design, assessment validity was compromised—assessing compliance with the scale rather than demonstrating achievement of intended learning outcomes.
Numerical Results and Contradictory Claims
The study does not provide conventional numerical results (e.g., statistical significance or effect sizes). Rather, it delivers strong qualitative evidence. One bold claim is that the AIAS, when retrofitted to assessments without structural design changes, can devolve into a compliance artifact, undermining validity and failing to address the integrity problem. The findings also assert that policy statements and rhetorical approaches (e.g., prohibitions and declarations) are inadequate absent substantive changes to assessment design and staff development.
Practical and Theoretical Implications
Practically, the paper substantiates the necessity of embedding frameworks like the AIAS within sustained faculty development, participatory governance, program-level consistency, and equitable tool access. It reveals that the locus of validity in assessment reform is not the framework itself but its enactment—how it is interpreted, resourced, and integrated within institutional and disciplinary contexts.
Theoretically, the research advances sector discourse by reframing assessment reform as a wicked problem oriented around capacity, equity, and institutional pathways, rather than a mere technology adoption challenge. The analysis suggests a shift from policing GenAI use to designing for valid inference about learning, emphasizing authentic, critical, and accountable engagement.
Future Directions
The limitations of the current study signal important avenues for subsequent research—longitudinal investigations, cross-disciplinary comparisons, and controlled studies differentiating versions of the AIAS. There is also scope for evaluating the impact of faculty-led communities of practice, assessment twins, and microlearning strategies on GenAI-era assessment validity and integrity.
Conclusion
Frameworks such as the AI Assessment Scale possess substantive potential to support institutional responses to GenAI-driven assessment challenges in higher education. However, their utility is decisively contingent upon conditions of implementation, including faculty capacity building, consultative governance, and equitable access. In the absence of these conditions, frameworks risk becoming mere compliance artifacts, failing to resolve underlying validity and equity issues. Assessment validity in a GenAI milieu is a function of embedded design, sustained support, and critical engagement—not simply the selection or adoption of a given scale. Institutions must prioritize investment in these infrastructural dimensions to effectuate genuine assessment reform.
