ENGAGE: Student Engagement Detection Dataset

• ENGAGE Dataset is a video repository capturing temporally varying student engagement using four-level ordinal labels in natural e-learning environments.
• It comprises 195 videos from 78 subjects recorded in diverse 'in the wild' conditions, ensuring ecological validity through realistic recording challenges.
• The dataset enables research in affective computing and educational data mining by supporting both global engagement prediction and fine-grained segment localization.

The ENGAGE Dataset most directly denotes the “Engagement Detection in the Wild” video database introduced for student engagement recognition and localization in e-learning settings. It was designed for scenarios in which engagement is not treated as a static attribute of a learner but as a temporally varying behavioral state reflected through facial expressions, eye gaze, and head pose while watching instructional material. The dataset contains 195 videos from 78 subjects, totaling about 16.5 hours, and is annotated on a four-level engagement scale. A central contribution of the associated work is to formulate engagement prediction as a weakly supervised learning problem, so that the same resource supports both video-level engagement estimation and localization of engaging versus non-engaging portions of a stimulus video (Kaur et al., 2018).

1. Definition and intended use

ENGAGE was introduced to address a specific gap in educational affect analysis: the absence of a public resource for studying student engagement in realistic, unconstrained environments. In the source paper, it is positioned for e-learning / educational video analytics, especially in relation to MOOCs and Intelligent Tutoring Systems (ITS). The dataset supports three related tasks: engagement prediction at the video level, engagement localization within a video, and weakly supervised learning / Multiple Instance Learning (MIL) when only coarse labels are available (Kaur et al., 2018).

The design premise is that engagement fluctuates while a student watches a lecture or instructional clip. Consequently, ENGAGE is not limited to predicting a single global label; it is intended to make segment-level reasoning possible even when annotation exists only at the whole-video level. This places the dataset at the intersection of affective computing, educational data mining, and weakly supervised temporal modeling.

The paper explicitly associates ENGAGE with observable social cues rather than privileged instrumentation. The main behavioral signals are facial expressions, eye gaze, and head pose / head movement. Body posture and body movement are mentioned as relevant in the introduction, but the principal experiments rely more directly on facial and motion-related descriptors (Kaur et al., 2018).

2. Corpus construction and recording conditions

The dataset contains 195 videos from 78 subjects, with 25 female and 53 male participants aged 19–27 years. The total duration is about 16.5 hours, and each recording is approximately 5 minutes long. Subjects watched educational stimulus videos, specifically “Learn the Korean Language in 5 minutes”, “Tips to learn faster”, and “How to write a research paper.” After watching each stimulus video, each subject also completed a 15-second verbal feedback section discussing whether the video was interesting, whether they would watch it again, which parts were most or least engaging, and how it could be improved (Kaur et al., 2018).

ENGAGE is described as collected “in the wild.” In this context, that means the recordings exhibit variation in background, illumination, pose, and location, including computer labs, hostel rooms, and open ground. The collection pipeline also incorporated a Skype-based video recording setup, which introduced realistic distortions such as network latency and frame drops. This choice increased ecological validity relative to more controlled webcam corpora (Kaur et al., 2018).

The recordings were captured at 640 × 480 resolution and 30 fps using a Microsoft Lifecam wide-angle F2.0 camera. Audio was recorded with the camera’s wideband microphone, and the Skype-based session was captured using Apowersoft Screen Recorder while the subject watched the stimulus video in full-screen mode. The paper states that consent was obtained from all subjects (Kaur et al., 2018).

These design choices make ENGAGE a dataset of response videos, not merely lecture videos. What is recorded is the learner’s observable reaction to educational content under realistic viewing conditions.

3. Annotation protocol and label semantics

ENGAGE uses four engagement levels, following the scheme inspired by Whitehill et al.:

• 0: completely disengaged
• 1: barely engaged
• 2: engaged
• 3: highly engaged

The paper provides behavioral descriptions for each level. Level 0 corresponds to an uninterested subject who frequently looks away. Level 1 includes behaviors such as barely opening the eyes and appearing restless in the chair. Level 2 corresponds to a subject who seems to like the content and interact with the video. Level 3 denotes a highly focused subject who appears “glued to the screen” (Kaur et al., 2018).

The label distribution is reported as 9 videos at level 0, 53 videos at level 1, 82 videos at level 2, and 50 videos at level 3. The labels are assigned at the whole-video level, not at the frame or segment level. However, the modeling framework assumes that each video can be decomposed into temporal segments whose contributions to the overall engagement label differ (Kaur et al., 2018).

Annotation was performed by 5 annotators, who watched the videos without audio and rated engagement intensity from visible facial behavior. Reliability was assessed using weighted Cohen’s $\kappa$ with quadratic weights, appropriate for ordinal labels. Annotators with agreement coefficient below 0.4 were treated as less reliable and their labels were ignored. After this denoising step, the remaining labels were averaged and rounded to the nearest integer to obtain the ground-truth rating for each video (Kaur et al., 2018).

A common misconception is that ENGAGE provides dense temporal labels. It does not. The source labels are video-level ordinal labels; localization is obtained algorithmically through a weakly supervised formulation rather than through manually labeled engaging segments.

4. Weak supervision, features, and modeling formulation

A major methodological feature of ENGAGE is the treatment of engagement prediction as a Multiple Instance Learning problem. Each video is modeled as a bag of temporal instances:

$$B = \{X_i , y_i \}_{i=1}^{N} , \quad X_i = \{x_{ij}\}_{j=1}^{M}, \quad y_i \in Y, \quad Y \in \{ 0,1,2,3\}$$

where $X_i$ is a video bag, $x_{ij}$ are segment instances, and $y_i$ is the video’s engagement label. The paper states that all videos are split into 100 segments for the MIL setting (Kaur et al., 2018).

The preprocessing pipeline uses OpenFace for facial landmark tracking, head pose, and eye gaze, while LBP-TOP is used as a spatio-temporal texture descriptor. The facial region is cropped to 112 × 112, and videos originally recorded at 30 fps are downsampled to 6 fps. For the sequence model, head pose and gaze are summarized per segment using standard deviations and concatenated into a 9-dimensional feature vector (Kaur et al., 2018).

The paper studies both MIL-style aggregation and sequential temporal modeling. In the top-$k$ pooling MIL formulation, instance scores are sorted and averaged over the highest-scoring segments:

$\frac{1}{k}\sum\limits_{j=1}^k r_{ij}'$

with $k \in \{1, 5, 10, 20, 30\}$ and $k=10$ chosen empirically. A second aggregation strategy uses mean pooling over all instances:

$\frac{1}{M}\sum\limits_{j=1}^M r_{ij}$

The paper also proposes an LSTM-based sequence model with 32 hidden units, followed by flattening, three dense layers with sigmoid activations, and average pooling to produce the final estimate (Kaur et al., 2018).

This formulation is significant because it aligns supervision granularity with annotation practicality. Whole-video labels are comparatively feasible to obtain, whereas dense segment-level engagement annotation is expensive and noisy.

5. Benchmarks, localization behavior, and reported limitations

The baseline experiments compare traditional regressors and deep models using both LBP-TOP and head pose + eye gaze features. With LBP-TOP, reported errors include SVR: 0.15, SGDRegressor: 1.20, BayesianRidge: 1.20, DNN: 0.15, and LSTM: 0.10. For deep MIL on LBP-TOP, DNN + max-$$B = \{X_i , y_i \}_{i=1}^{N} , \quad X_i = \{x_{ij}\}_{j=1}^{M}, \quad y_i \in Y, \quad Y \in \{ 0,1,2,3\}$$0 pooling gives error 0.16, PCC 0.003, DNN + mean pooling gives error 0.15, PCC 0.0264, and LSTM + mean pooling gives error 0.10, PCC 0.001 (Kaur et al., 2018).

Using fused head pose + eye gaze features, performance is stronger. For traditional SVR, the best average MSE reported is 0.09 with noisy labels. For deep models, DNN gives avg MSE 0.11, LSTM gives avg MSE 0.10, and mean pooling + LSTM improves to PCC 0.25. The discussion also states that the best model reaches PCC = 0.37, indicating improved correlation between predicted and annotated engagement scores (Kaur et al., 2018).

The paper’s main empirical conclusion is that motion-based features, particularly head pose + eye gaze, are more effective than LBP-TOP in this in-the-wild setting, and that temporal sequence modeling improves performance over static aggregation. The localization mechanism relies on instance-level outputs from the penultimate layer of the sequence or MIL network, which allows the model to identify segments corresponding to different engagement levels. The paper reports that the sequence network differentiates level 0 from levels 1, 2, 3 particularly well (Kaur et al., 2018).

The associated limitations are explicit. The dataset is relatively small by modern deep-learning standards; the label distribution is imbalanced, especially for level 0; LBP-TOP performs poorly in this setting; and fine-grained discrimination between level 2 and level 3 remains difficult. The authors also note that the work does not include body motion / optical flow, and that the subject pool comes from an institute environment, limiting generalization to older or non-institute populations (Kaur et al., 2018).

6. Terminological ambiguity and relation to later engagement datasets

The term “ENGAGE Dataset” is not completely stable across the later literature. In the most direct sense, it refers to the dataset from “Prediction and Localization of Student Engagement in the Wild” (Kaur et al., 2018). However, later work sometimes uses ENGAGE as a query term, a shorthand, or a point of comparison for other engagement resources.

Resource	Setting	Distinctive property
ENGAGE (Kaur et al., 2018)	E-learning response videos	Four-level video labels and weakly supervised localization
ER dataset (Nezami et al., 2018)	Image-based classroom-style engagement	4627 face images with binary engaged/disengaged labels
EduGage (Leng et al., 2 May 2026)	Self-guided video learning with sensors	715 probe-aligned 44-second windows and multimodal sensing

The ER dataset paper states that the new dataset is the Engagement Recognition (ER) dataset and that, in the query, this corresponds to the ENGAGE dataset. It contains 4627 annotated face images, split into 2290 engaged and 2337 disengaged samples, collected from 20 students in an Omosa virtual-world learning environment. Its label is derived from a rule-based combination of behavioral and emotional annotations rather than a direct four-class engagement judgment (Nezami et al., 2018).

A later paper on EduGage states that EduGage is also referred to in the paper as the ENGAGE dataset. EduGage is structurally different from the original ENGAGE resource: it is a multimodal dataset for momentary engagement assessment during self-guided video learning, built from 16 participants, 64 video-viewing sessions, and 715 probe-aligned windows, with a 5-point Likert target reflecting the perceived difficulty of maintaining attention during the preceding segment (Leng et al., 2 May 2026).

This terminological ambiguity matters because engagement datasets differ sharply in label semantics, time scale, and modality. The original ENGAGE dataset is an ordinal video dataset with weak localization; the ER dataset is a binary facial-image dataset; EduGage is a sensor-rich momentary self-report dataset. Treating them as interchangeable obscures the methodological assumptions each benchmark encodes.

7. Position within the engagement-dataset landscape

ENGAGE occupies an early position in the development of engagement benchmarks that emphasize ecological validity. Earlier and contemporaneous work in adjacent directions includes DAiSEE, which provides 9068 in-the-wild video snippets from 112 users with four affective states—boredom, confusion, engagement, and frustration—each annotated on a four-level ordinal scale (Gupta et al., 2016). Later resources diversified along at least three axes: broader web settings, richer modality sets, and domain specialization.

For example, EngageNet introduced a large-scale in-the-wild engagement prediction dataset with 31 hours of data from 127 participants, organized into approximately 11.3K clips of 10 seconds each and labeled with the four-class scheme Not Engaged, Barely Engaged, Engaged, and Highly Engaged (Singh et al., 2023). CMOSE expanded online engagement data with 12,193 segments from 102 participants, psychology-guided annotation, and multimodal visual, speech, and audio features (Wu et al., 2023). Beyond educational video watching, engagement datasets now include industrial human-robot collaboration in MIAM (Mehta et al., 10 Jan 2025), in-person classroom attention and emotion in DIPSER (Marquez-Carpintero et al., 27 Feb 2025), and older-adult virtual rehabilitation in OPEN (Abedi et al., 23 Jul 2025).

Seen against this broader landscape, ENGAGE remains notable for a specific reason: it formalized the idea that educational engagement can be studied not only as coarse classification but also as localization under weak supervision. That formulation remains technically relevant even as later datasets move toward multimodal fusion, self-report probes, behavioral-affective decomposition, or domain-specific engagement constructs.