---
title: 'NTHU-DDD: Driver Drowsiness Detection Dataset'
url: https://www.emergentmind.com/topics/nthu-driver-drowsiness-detection-dataset-nthu-ddd
type: topic
---

# NTHU-DDD: Driver Drowsiness Detection Dataset

The NTHU Driver Drowsiness Detection Dataset (NTHU-DDD) is a suite of landmark datasets developed by National Tsing Hua University's Computer Vision Lab (Taiwan) for the development and benchmarking of driver drowsiness detection algorithms using computer vision and, in a distinct release, EEG recordings. Widely used in the research community, NTHU-DDD provides controlled laboratory video data and, in a complementary dataset, high-density EEG data capturing both behavioral and physiological signatures of fatigue and vigilance declines during simulated driving. The video-based variants have become a de facto benchmark for spatiotemporal modeling of driver facial dynamics under a range of challenging conditions and occlusions, forming the basis of comparative evaluations for numerous state-of-the-art deep learning frameworks.

## 1. Dataset Design and Variants

NTHU-DDD encompasses several releases differing in sensor modality, scope, and protocols:

- **RGB Video Dataset**: The main NTHU-DDD video corpus, as described in multiple sources [2207.12148], [1910.09722], [1801.02325], [2511.12438], [2209.05269], [2010.06235], consists of video recordings of subjects engaged in simulated driving under controlled laboratory conditions with staged drowsy and non-drowsy behaviors and a wide range of environmental and occlusion scenarios.
- **EEG Dataset**: An independent but related dataset provides 32-channel scalp electroencephalography (EEG) recordings from 27 subjects during a 90-minute sustained-attention highway driving simulation, with precisely timestamped behavioral markers of vigilance [1809.06534].

**Video Dataset Collection Protocol**
- Data from 18–36 subjects (number depending on dataset release and experiment) of diverse ethnic backgrounds. Gender/age breakdowns are variably specified—e.g., [1801.02325] reports 10M/8F in training, 2M/2F in evaluation.
- Each subject controls a simulated steering setup (steering wheel, pedals) in a PC-based driving game, seated in a laboratory environment.
- The camera setup is a dashboard-mounted, frontal RGB (and sometimes infrared) camera; original releases use a resolution of 640×480 (when specified) at 30 fps [1910.09722].
- Environmental and occlusion factors are varied: illumination (day/night), eyewear (none, glasses, sunglasses), and background (lab wall). Each subject is recorded under all combinations.
- Subjects are instructed to enact normal/alert and drowsy states, including specific facial cues: normal driving, slow blinking, yawning, deliberate head nodding (simulated micro-sleep), and laughter (negative control).

**EEG Dataset Protocol**
- Participants operate a motion-platform driving simulator in near-360° VR for 90 continuous minutes of night-time driving. Periodic lane-departure perturbations trigger behavioral corrections, all time-locked to EEG and response events [1809.06534].
- Inclusion criteria: 22–28 years, healthy, free of neurological/psychiatric disorders, normal sleep regimen.

## 2. Ground-Truth Annotations and Labeling Structure

### Video Dataset

**Labels and Channels**
- Binary labels: "drowsy" vs. "non-drowsy" at the frame or image level [2207.12148], [2511.12438].
- Additional annotation channels: head pose (normal/look aside/nodding), mouth state (normal/yawn/talk-laugh), eye state (normal/sleepy) [1910.09722], [1801.02325].
- Per-frame class labels are manually annotated, sometimes leveraging "long-term memory"—i.e., labels for drowsy states may smear across several seconds of video [1801.02325], leading to temporal imprecision.

**Clip-Level Label Aggregation**
- Many protocols assign labels to temporal clips using majority voting. E.g., a 5-frame clip is "drowsy" if at least 3/5 frames are labeled drowsy [1910.09722]:
  $$
  Y = \begin{cases}
      1 & \text{if} \; \sum_{t=1}^T y_t \geq 3 \\
      0 & \text{otherwise}
  \end{cases}
  $$
  where $y_t$ is the frame-level label; $T=5$.
- Not all releases document precise annotation protocols; details such as PERCLOS thresholds or rater instructions are typically unspecified [2207.12148], [2511.12438].

### EEG Dataset

- Automatic trial/event segmentation by behavioral triggers: deviation onset, response onset, response offset.
- Vigilance ground truth inferred behaviorally via reaction time (RT) to lane departure:
  $$
  \text{RT} = t_{\text{resp onset}} - t_{\text{dev onset}}
  $$
  Extended RT is used as a proxy for driver inattention or drowsiness [1809.06534].
- No concurrent subjective drowsiness ratings; EEG-derived vigilance indices are calculated post hoc (e.g., theta/alpha ratio).

## 3. Data Organization, Splits, and Quantitative Characteristics

**Video Corpus Organization**

| Attribute             | Details                               |
|-----------------------|---------------------------------------|
| Training subjects     | 18                                    |
| Evaluation subjects   | 4                                     |
| Test subjects         | 14 (labels not public)                |
| Scenarios             | Day/night × No/Glasses/Sunglasses     |
| Videos/train          | 360 (18×5 scenarios×4 videos)         |
| Videos/eval           | 20 (4×5×1)                            |
| Average frames/video  | ~2000 (train), ~8600 (eval)           |
| Total frames/train    | 722,223                               |
| Format                | AVI, 640x480, 30fps (when specified)  |
| Per-frame labels      | Drowsy, head, eye, mouth, illum/glass |
| Data split protocol   | Official train/eval/test structure    |

*This summary table is drawn from [1910.09722], [1801.02325], [2010.06235], [2209.05269]. Where not specified, listed attributes are drawn from the closest available public description.*

**EEG Corpus Organization**
- 27 subjects, 62 × 90-min sessions, 32 EEG channels + vehicle position, sample rate 500 Hz [1809.06534].
- File formats: raw .cnt, preprocessed .set (EEGLAB), separate event logs.

## 4. Preprocessing and Modeling Pipelines

**Image-Based Approaches**
- Typical steps: face detection/tracking (OpenCV, MTCNN), landmark alignment (Ren et al.), multi-granularity facial patch extraction (eyes, mouth, global face), resizing/normalization ([1801.02325], [2010.06235]).
- Illumination normalization: CLAHE applied to each patch to address lighting variation [2010.06235], [2209.05269].
- Clip generation: Sliding windows (e.g., 5/30/48 frames); majority voting for ground-truth label.
- Data augmentation: Some studies apply random flips, per-channel normalization; others use raw RGB with minimal augmentation [2207.12148], [2511.12438].
- For temporal modeling: LSTM/GRU sequencers, 3D CNNs, and transformer-based methods process per-frame embeddings [1801.02325], [2010.06235], [2207.12148].

**EEG-Based Approaches**
- Zero-phase FIR bandpass filtering (1–50 Hz), down-sampling, ICA-based artifact rejection, epoch extraction time-locked to events [1809.06534].
- Feature extraction: alpha/beta/theta band-power, spectral ratios, and vigilance indices.

## 5. Benchmark Results and Comparative Analysis

**Reported Results on NTHU-DDD (Video)**

| Method/Model                        | Accuracy / AUC | Notable Preprocessing/Architecture              | Citation          |
|-------------------------------------|---------------|------------------------------------------------|-------------------|
| Multi-granularity CNN+LSTM (MCNN)   | 90.05%        | Multi-scale facial patches + LSTM               | [1801.02325]      |
| Two-stream Multi-feature Attention   | 94.46%        | CLAHE + per-patch 3D CNNs + SE Attention        | [2010.06235]      |
| DCNN + OpenCV                       | 99.6%         | 96×96 frame resize, raw face, no alignment      | [2511.12438]      |
| LSTM Autoencoder (AUC)              | 0.8740        | ResNet-34 + CLAHE + sequence anomaly detection  | [2209.05269]      |
| Vision Transformer (Swin, Video)    | 44%           | CNN feature projection + Swin, no augmentation  | [2207.12148]      |

- Notable, higher model capacity or novel architectures (e.g., transformers) do not guarantee improved accuracy—overfitting can occur when data volume is limited, as observed for Swin Transformer on the drowsiness subtask [2207.12148].
- Methodological variations in face alignment, patch extraction, and temporal context modeling are decisive for robustness under occlusion and illumination variation [2010.06235], [1801.02325].

**Reported Results on NTHU-DDD (EEG)**
- Studies using the EEG corpus focus on classification/regression of vigilance states, using event-related potentials, spectral powers, and cross-channel coherence; standardized accuracy metrics are less commonly reported [1809.06534].

## 6. Known Limitations, Variability, and Best Practices

- **Labeling coarseness**: Long-term memory annotation introduces temporal lag in labels, hindering evaluation of rapid state changes (e.g., blinking vs. prolonged closure) [1801.02325].
- **Limited demography/documentation**: Most releases report minimal participant demographic data; no detailed camera lens/FOV/frame-rate metadata [2207.12148], [2511.12438], [1910.09722].
- **Protocol variance**: Number of subjects, amount of footage per condition, and numbers/types of annotated behaviors can vary between different publications using "NTHU-DDD." *A plausible implication is that researchers should specify the exact release and split protocol adopted in benchmarking to ensure comparability.*
- **Realism of drowsiness cues**: Drowsiness is simulated, not spontaneous, and labels depend on staged behaviors, potentially limiting ecological validity [2511.12438], [2207.12148].
- **Evaluation protocol**: Absence of an official cross-validation split for most releases; subject-independent splits are strongly advised to prevent overestimating generalization [2511.12438].

## 7. Applications and Impact in Drowsiness Detection Research

NTHU-DDD's multi-condition, multi-attribute structure has enabled:

- Supervised and semi-supervised classification of driver drowsiness, driving extensive progress in spatial–temporal facial analysis [2010.06235], [1801.02325], [2209.05269].
- Analysis of robustness to illumination and occlusions via per-patch and attention-based architectures [2010.06235].
- Benchmarking anomaly detection frameworks where "normal" only training regimes flag drowsiness as distributional outlier [2209.05269].
- Development and validation of low-latency, real-time drowsiness detection systems suitable for automotive embedded platforms [2511.12438].
- In the EEG corpus, data-driven modeling of cognitive fatigue, online vigilance tracking, and neuroergonomics [1809.06534].

Researchers are advised to consult the specific data descriptor and publication that matches their targeted modality and evaluation protocol, given the diversity of annotation pipelines within the broader NTHU-DDD family. Continued harmonization of ground-truth assignment, transparency in protocol reporting, and release of more naturalistic or spontaneous drowsiness footage are open areas for dataset development.

Source: https://www.emergentmind.com/topics/nthu-driver-drowsiness-detection-dataset-nthu-ddd