From Co-Design to Metacognitive Laziness: Evaluating Generative AI in Vocational Education (2512.12306v1)

Abstract: This study examines the development and deployment of a Generative AI proof-of-concept (POC) designed to support lecturers in a vocational education setting in Singapore. Employing a user-centred, mixed-methods design process, we co-developed an AI chatbot with lecturers to address recurring instructional challenges during exam preparation, specifically managing repetitive questions and scaling feedback delivery. The POC achieved its primary operational goals: lecturers reported streamlined workflows, reduced cognitive load, and observed improved student confidence in navigating course content. However, the deployment yielded unexpected insights into student learning behaviours. Despite enhanced teaching processes, performance data revealed no significant improvement in overall student assessment outcomes. Deep analysis of interaction logs identified concerning patterns, including self-efficacy-driven dependency, "metacognitive laziness" (cognitive offloading), and divergent usage strategies. While high-ability students leveraged the tool for strategic verification, low-ability students frequently used it to bypass cognitive effort, potentially exacerbating performance gaps. These findings suggest that Generative AI's educational influence extends beyond instructional efficiency to shape cognitive engagement, self-regulation, and learner equity. The study raises consequential design questions regarding how AI tools can be engineered to minimise dependency, scaffold metacognitive development, and calibrate support across varying ability levels. We conclude that while Generative AI can substantially enhance the teaching experience, achieving meaningful learning gains requires rigorous attention to learner behaviour and the equitable design of AI-supported environments.