---
title: 'ENGAGE: Student Engagement Detection Dataset'
url: https://www.emergentmind.com/topics/engage-dataset
type: topic
---

# ENGAGE: Student Engagement Detection Dataset

The **ENGAGE Dataset** most directly denotes the **“Engagement Detection in the Wild”** video database introduced for **student engagement recognition and localization** in **e-learning** settings. It was designed for scenarios in which engagement is not treated as a static attribute of a learner but as a temporally varying behavioral state reflected through **facial expressions, eye gaze, and head pose** while watching instructional material. The dataset contains **195 videos** from **78 subjects**, totaling about **16.5 hours**, and is annotated on a **four-level engagement scale**. A central contribution of the associated work is to formulate engagement prediction as a **weakly supervised learning** problem, so that the same resource supports both **video-level engagement estimation** and **localization of engaging versus non-engaging portions** of a stimulus video [1804.00858].

## 1. Definition and intended use

ENGAGE was introduced to address a specific gap in educational affect analysis: the absence of a public resource for studying **student engagement in realistic, unconstrained environments**. In the source paper, it is positioned for **e-learning / educational video analytics**, especially in relation to **MOOCs** and **Intelligent Tutoring Systems (ITS)**. The dataset supports three related tasks: **engagement prediction** at the video level, **engagement localization** within a video, and **weakly supervised learning / Multiple Instance Learning (MIL)** when only coarse labels are available [1804.00858].

The design premise is that engagement fluctuates while a student watches a lecture or instructional clip. Consequently, ENGAGE is not limited to predicting a single global label; it is intended to make segment-level reasoning possible even when annotation exists only at the whole-video level. This places the dataset at the intersection of affective computing, educational data mining, and weakly supervised temporal modeling.

The paper explicitly associates ENGAGE with observable social cues rather than privileged instrumentation. The main behavioral signals are **facial expressions**, **eye gaze**, and **head pose / head movement**. Body posture and body movement are mentioned as relevant in the introduction, but the principal experiments rely more directly on facial and motion-related descriptors [1804.00858].

## 2. Corpus construction and recording conditions

The dataset contains **195 videos** from **78 subjects**, with **25 female** and **53 male** participants aged **19–27 years**. The total duration is about **16.5 hours**, and each recording is approximately **5 minutes** long. Subjects watched educational stimulus videos, specifically **“Learn the Korean Language in 5 minutes”**, **“Tips to learn faster”**, and **“How to write a research paper.”** After watching each stimulus video, each subject also completed a **15-second verbal feedback section** discussing whether the video was interesting, whether they would watch it again, which parts were most or least engaging, and how it could be improved [1804.00858].

ENGAGE is described as collected **“in the wild.”** In this context, that means the recordings exhibit variation in **background, illumination, pose, and location**, including **computer labs, hostel rooms, and open ground**. The collection pipeline also incorporated a **Skype-based video recording setup**, which introduced realistic distortions such as **network latency and frame drops**. This choice increased ecological validity relative to more controlled webcam corpora [1804.00858].

The recordings were captured at **640 × 480** resolution and **30 fps** using a **Microsoft Lifecam wide-angle F2.0** camera. Audio was recorded with the camera’s **wideband microphone**, and the Skype-based session was captured using **Apowersoft Screen Recorder** while the subject watched the stimulus video in full-screen mode. The paper states that consent was obtained from all subjects [1804.00858].

These design choices make ENGAGE a dataset of **response videos**, not merely lecture videos. What is recorded is the learner’s observable reaction to educational content under realistic viewing conditions.

## 3. Annotation protocol and label semantics

ENGAGE uses **four engagement levels**, following the scheme inspired by Whitehill et al.:

- **0**: completely disengaged  
- **1**: barely engaged  
- **2**: engaged  
- **3**: highly engaged  

The paper provides behavioral descriptions for each level. Level **0** corresponds to an uninterested subject who frequently looks away. Level **1** includes behaviors such as barely opening the eyes and appearing restless in the chair. Level **2** corresponds to a subject who seems to like the content and interact with the video. Level **3** denotes a highly focused subject who appears “glued to the screen” [1804.00858].

The label distribution is reported as **9 videos** at level **0**, **53 videos** at level **1**, **82 videos** at level **2**, and **50 videos** at level **3**. The labels are assigned at the **whole-video level**, not at the frame or segment level. However, the modeling framework assumes that each video can be decomposed into temporal segments whose contributions to the overall engagement label differ [1804.00858].

Annotation was performed by **5 annotators**, who watched the videos **without audio** and rated engagement intensity from visible facial behavior. Reliability was assessed using **weighted Cohen’s $\kappa$ with quadratic weights**, appropriate for ordinal labels. Annotators with agreement coefficient below **0.4** were treated as less reliable and their labels were ignored. After this denoising step, the remaining labels were **averaged and rounded to the nearest integer** to obtain the ground-truth rating for each video [1804.00858].

A common misconception is that ENGAGE provides dense temporal labels. It does not. The source labels are **video-level ordinal labels**; localization is obtained algorithmically through a weakly supervised formulation rather than through manually labeled engaging segments.

## 4. Weak supervision, features, and modeling formulation

A major methodological feature of ENGAGE is the treatment of engagement prediction as a **Multiple Instance Learning** problem. Each video is modeled as a bag of temporal instances:

$$
B = \{X_i , y_i \}_{i=1}^{N} , \quad X_i = \{x_{ij}\}_{j=1}^{M}, \quad y_i \in Y, \quad Y \in \{ 0,1,2,3\}
$$

where $X_i$ is a video bag, $x_{ij}$ are segment instances, and $y_i$ is the video’s engagement label. The paper states that all videos are split into **100 segments** for the MIL setting [1804.00858].

The preprocessing pipeline uses **OpenFace** for facial landmark tracking, **head pose**, and **eye gaze**, while **LBP-TOP** is used as a spatio-temporal texture descriptor. The facial region is cropped to **112 × 112**, and videos originally recorded at **30 fps** are downsampled to **6 fps**. For the sequence model, head pose and gaze are summarized per segment using standard deviations and concatenated into a **9-dimensional feature vector** [1804.00858].

The paper studies both MIL-style aggregation and sequential temporal modeling. In the **top-$k$ pooling** MIL formulation, instance scores are sorted and averaged over the highest-scoring segments:

$$
\frac{1}{k}\sum\limits_{j=1}^k r_{ij}'
$$

with $k \in \{1, 5, 10, 20, 30\}$ and **$k=10$** chosen empirically. A second aggregation strategy uses **mean pooling** over all instances:

$$
\frac{1}{M}\sum\limits_{j=1}^M r_{ij}
$$

The paper also proposes an **LSTM**-based sequence model with **32 hidden units**, followed by flattening, **three dense layers with sigmoid activations**, and average pooling to produce the final estimate [1804.00858].

This formulation is significant because it aligns supervision granularity with annotation practicality. Whole-video labels are comparatively feasible to obtain, whereas dense segment-level engagement annotation is expensive and noisy.

## 5. Benchmarks, localization behavior, and reported limitations

The baseline experiments compare traditional regressors and deep models using both **LBP-TOP** and **head pose + eye gaze** features. With **LBP-TOP**, reported errors include **SVR: 0.15**, **SGDRegressor: 1.20**, **BayesianRidge: 1.20**, **DNN: 0.15**, and **LSTM: 0.10**. For deep MIL on LBP-TOP, **DNN + max-$k$ pooling** gives **error 0.16, PCC 0.003**, **DNN + mean pooling** gives **error 0.15, PCC 0.0264**, and **LSTM + mean pooling** gives **error 0.10, PCC 0.001** [1804.00858].

Using **fused head pose + eye gaze** features, performance is stronger. For traditional **SVR**, the best average **MSE** reported is **0.09** with noisy labels. For deep models, **DNN** gives **avg MSE 0.11**, **LSTM** gives **avg MSE 0.10**, and **mean pooling + LSTM** improves to **PCC 0.25**. The discussion also states that the best model reaches **PCC = 0.37**, indicating improved correlation between predicted and annotated engagement scores [1804.00858].

The paper’s main empirical conclusion is that **motion-based features**, particularly **head pose + eye gaze**, are more effective than **LBP-TOP** in this in-the-wild setting, and that temporal sequence modeling improves performance over static aggregation. The localization mechanism relies on **instance-level outputs** from the penultimate layer of the sequence or MIL network, which allows the model to identify segments corresponding to different engagement levels. The paper reports that the sequence network differentiates **level 0** from **levels 1, 2, 3** particularly well [1804.00858].

The associated limitations are explicit. The dataset is relatively small by modern deep-learning standards; the label distribution is imbalanced, especially for **level 0**; **LBP-TOP** performs poorly in this setting; and fine-grained discrimination between **level 2** and **level 3** remains difficult. The authors also note that the work does **not include body motion / optical flow**, and that the subject pool comes from an institute environment, limiting generalization to older or non-institute populations [1804.00858].

## 6. Terminological ambiguity and relation to later engagement datasets

The term **“ENGAGE Dataset”** is not completely stable across the later literature. In the most direct sense, it refers to the dataset from **“Prediction and Localization of Student Engagement in the Wild”** [1804.00858]. However, later work sometimes uses **ENGAGE** as a query term, a shorthand, or a point of comparison for other engagement resources.

| Resource | Setting | Distinctive property |
|---|---|---|
| **ENGAGE** [1804.00858] | E-learning response videos | Four-level video labels and weakly supervised localization |
| **ER dataset** [1808.02324] | Image-based classroom-style engagement | **4627** face images with binary engaged/disengaged labels |
| **EduGage** [2605.01238] | Self-guided video learning with sensors | **715** probe-aligned **44-second** windows and multimodal sensing |

The **ER dataset** paper states that the new dataset is the **Engagement Recognition (ER) dataset** and that, in the query, this corresponds to the **ENGAGE dataset**. It contains **4627 annotated face images**, split into **2290 engaged** and **2337 disengaged** samples, collected from **20 students** in an **Omosa** virtual-world learning environment. Its label is derived from a rule-based combination of **behavioral** and **emotional** annotations rather than a direct four-class engagement judgment [1808.02324].

A later paper on **EduGage** states that EduGage is also referred to in the paper as the **ENGAGE dataset**. EduGage is structurally different from the original ENGAGE resource: it is a multimodal dataset for **momentary engagement assessment during self-guided video learning**, built from **16 participants**, **64 video-viewing sessions**, and **715 probe-aligned windows**, with a **5-point Likert** target reflecting the perceived difficulty of maintaining attention during the preceding segment [2605.01238].

This terminological ambiguity matters because engagement datasets differ sharply in **label semantics**, **time scale**, and **modality**. The original ENGAGE dataset is an **ordinal video dataset** with **weak localization**; the ER dataset is a **binary facial-image dataset**; EduGage is a **sensor-rich momentary self-report dataset**. Treating them as interchangeable obscures the methodological assumptions each benchmark encodes.

## 7. Position within the engagement-dataset landscape

ENGAGE occupies an early position in the development of engagement benchmarks that emphasize ecological validity. Earlier and contemporaneous work in adjacent directions includes **DAiSEE**, which provides **9068** in-the-wild video snippets from **112 users** with four affective states—**boredom, confusion, engagement, and frustration**—each annotated on a **four-level ordinal scale** [1609.01885]. Later resources diversified along at least three axes: broader web settings, richer modality sets, and domain specialization.

For example, **EngageNet** introduced a large-scale **in-the-wild engagement prediction dataset** with **31 hours** of data from **127 participants**, organized into approximately **11.3K clips** of **10 seconds** each and labeled with the four-class scheme **Not Engaged**, **Barely Engaged**, **Engaged**, and **Highly Engaged** [2302.00431]. **CMOSE** expanded online engagement data with **12,193** segments from **102 participants**, psychology-guided annotation, and multimodal visual, speech, and audio features [2312.09066]. Beyond educational video watching, engagement datasets now include **industrial human-robot collaboration** in **MIAM** [2501.05936], **in-person classroom attention and emotion** in **DIPSER** [2502.20209], and **older-adult virtual rehabilitation** in **OPEN** [2507.17959].

Seen against this broader landscape, ENGAGE remains notable for a specific reason: it formalized the idea that educational engagement can be studied not only as coarse classification but also as **localization under weak supervision**. That formulation remains technically relevant even as later datasets move toward multimodal fusion, self-report probes, behavioral-affective decomposition, or domain-specific engagement constructs.

Source: https://www.emergentmind.com/topics/engage-dataset