E3Sense: Head-Confined Multimodal Sensing of Learner Engagement

Abstract: Engagement-aware learning systems could provide hints or adjust pacing when learners struggle. Prior engagement sensing work distributes sensors across or outside the body rather than consolidating them at one site, or reduces engagement to shared affect or a single dimension (from behavioral, emotional, and cognitive engagement). We introduce E3Sense, a head-worn platform that co-locates electroencephalography, eye tracking, and electrodermal activity to personalize engagement measurement. During a lab study we collected 450 ratings of engagement levels on a five-level ordinal scale while participants watched educational videos. For fifteen held-out participants, E3Sense achieved a within-one-level prediction score of 75.0%, compared with 63.0% for always predicting the most common rating. In an exploratory analysis of 18 participants from the same study who defined engagement, conditioning on learners' definitions raised the same measure by 6.9 points, from 64.6% to 71.5%. Our work provides a proof-of-concept of a head-site, personalized multimodal sensing of engagement for adaptive educational interfaces.