DeepSleepNet for EEG Sleep Staging

• DeepSleepNet is a deep learning framework that performs automatic sleep stage scoring from raw single-channel EEG data.
• It integrates dual-scale CNNs for robust feature extraction with BiLSTM modules to capture long-range temporal dependencies.
• DeepSleepNet-Lite, a streamlined variant, reduces parameters by 96% and uses Monte Carlo dropout for effective uncertainty estimation.

DeepSleepNet is a deep learning framework for automatic sleep stage scoring from raw single-channel EEG, notable for its end-to-end architecture that jointly learns feature representations and temporal context without relying on hand-crafted features. The original DeepSleepNet model, introduced by Supratak et al., integrates dual-scale convolutional neural networks (CNNs) for representation learning and bidirectional long short-term memory (BiLSTM) modules for sequence modeling. Its simplified variant, DeepSleepNet-Lite, proposed by Fiorillo et al., omits the recurrent layer, uses a shorter temporal context, and incorporates Monte Carlo dropout for uncertainty estimation, resulting in substantial parameter and computational savings while achieving similar performance metrics. Below, the technical design, methodology, empirical results, and implications of these models are delineated in detail, along with the role of Monte Carlo dropout in model calibration (Fiorillo et al., 2021, Supratak et al., 2017).

1. Architectural Foundations

Both DeepSleepNet and DeepSleepNet-Lite adopt raw, single-channel EEG as input, differentiating themselves from most former approaches that depend on engineered spectral features or rule-based heuristics. The core architectural element is the dual-stream CNN “representation block,” applying two parallel 1D convolutional paths with distinct filter scales to each input epoch, capturing complementary time- and frequency-localized patterns.

a) DeepSleepNet

• Input: 25 consecutive 30-s EEG epochs (totaling 12.5 min, 100 Hz sample rate ⇒ 75,000 time-points).
• Representation Learning (Block 1): Two CNN streams:
  • Small-filter stream (CNNₛ): First conv (kernel = $F_s/2$), stride $F_s/16$, for temporal resolution.
  • Large-filter stream (CNNₗ): First conv (kernel $4F_s$), stride $F_s/2$, for frequency content.
  • Both streams: 4 conv layers, 2 max-pooling, batch normalization, ReLU.
  • Concatenation of stream outputs yields per-epoch features $a_i = h^s_i \Vert h^l_i$.
• Temporal Context (Block 2): Two stacked bidirectional LSTMs (128 units each direction), enabling the model to learn sleep stage transition rules across the 25-epoch sequence.
• Classifier: Time-distributed dense layer followed by softmax over five stages (W, N1, N2, N3, REM).
• Parameter count: $\sim24.7$ million.

b) DeepSleepNet-Lite

• Key Modifications:
  • No LSTM: Eliminates context-aggregating LSTM block.
  • Shorter Context: Processes a three-epoch window (90 s) for sequence-to-epoch modeling: input = epochs $(t−1, t, t+1)$; output = stage at time $t$.
  • Regularization: Heavy dropout ($p=0.5$), label smoothing, early stopping, $L_2$ penalty.
• Layer-wise Summary:

Layer	Small-filter stream (CNNₛ)	Large-filter stream (CNNₗ)
Input	$F_s/16$0	$F_s/16$1
Conv1	kernel=50, filters=64, stride=6	kernel=400, filters=64, stride=50
MaxPool1	pool_size=8, stride=8	pool_size=4, stride=4
Conv2	kernel=8, filters=128, stride=1	kernel=6, filters=128, stride=1
Conv3	kernel=8, filters=128, stride=1	kernel=6, filters=128, stride=1
MaxPool2	pool_size=4, stride=4	pool_size=4, stride=4
Conv4	kernel=8, filters=128, stride=1	kernel=6, filters=128, stride=1
Flatten	$F_s/16$2	$F_s/16$3
Output	Concatenate, dense, softmax	Concatenate, dense, softmax

• Parameter count: $F_s/16$4 million ($F_s/16$5 reduction from original).
• Inference time: Few milliseconds for one 90-s sequence on a single GPU.

2. Mathematical Formulation

Convolutional Operations

Each convolutional layer computes

$F_s/16$6

followed by batch normalization and ReLU activation: $F_s/16$7

LSTM Dynamics (DeepSleepNet only)

The bidirectional LSTM incorporates peephole connections, with the following gates per direction: $F_s/16$8

Final Classification

Output logits per epoch are converted to predicted stage distributions by a softmax: $F_s/16$9 with output stage given by $4F_s$0.

3. Training Paradigm

DeepSleepNet: Two-Step Training

1. Pre-training: The CNN representation block is trained standalone with a temporary softmax on a class-balanced, oversampled set. Adam optimizer, learning rate $4F_s$1, 100 epochs.
2. Fine-tuning: The pretrained CNN is combined with the BiLSTM and final softmax; trained on contiguous, sequential EEG data (sub-sequences of 25 epochs). Dual learning rates ($4F_s$2 for CNN, $4F_s$3 for LSTM/softmax), Adam optimizer, 200 epochs, dropout $4F_s$4, and $4F_s$5 penalty on initial conv filters.

DeepSleepNet-Lite

A strictly feed-forward, end-to-end training regime over short (three-epoch) windows. Dropout $4F_s$6 on all convolutional-ReLU activations, combined with label smoothing and $4F_s$7 weight decay.

4. Monte Carlo Dropout for Uncertainty Estimation

DeepSleepNet-Lite introduces Monte Carlo dropout for uncertainty quantification in sleep stage scoring. At inference, $4F_s$8 stochastic forward passes are conducted with dropout active ($4F_s$9), yielding $F_s/2$0 softmax vectors $F_s/2$1 per input: $F_s/2$2 The final predicted stage is $F_s/2$3. By thresholding mean-confidence or variance, the model can abstain (“reject”) on uncertain epochs. Empirically, rejecting the top 5% of most uncertain predictions increases both accuracy and Cohen's $F_s/2$4 by ~$F_s/2$5 (Fiorillo et al., 2021).

5. Empirical Performance and Benchmarking

The following summarizes principal results of DeepSleepNet and DeepSleepNet-Lite on major open-access datasets:

Model	Dataset	Acc (%)	MF1 (%)	$F_s/2$6	Params (M)
DeepSleepNet	MASS, F4–EOG(L)	86.2	81.7	0.80	24.7
DeepSleepNet	Sleep-EDF, Fpz–Cz	82.0	76.9	0.76	24.7
DeepSleepNet-Lite	EDF v1-2013	84.0	78.0	0.78	0.6
DeepSleepNet-Lite	EDF v2-2018	80.3	75.2	0.73	0.6
DeepSleepNet-Lite (rej 5%)	EDF v1-2013	86.1	79.6	0.81	0.6
DeepSleepNet-Lite (rej 5%)	EDF v2-2018	82.3	76.7	0.76	0.6

Metrics:

• Accuracy ($F_s/2$7) and macro F1-score ($F_s/2$8), per-class F1, and Cohen’s $F_s/2$9 ($a_i = h^s_i \Vert h^l_i$0) calculated on subject-independent folds. Baseline performance for DeepSleepNet-Lite utilizes MC dropout and uniform label smoothing.

Comparison:

Both models yield state-of-the-art or near state-of-the-art performance in single-channel sleep stage scoring and outperform contemporaneous CNN- or RNN-only baselines on the same benchmarks, without domain feature engineering (Fiorillo et al., 2021, Supratak et al., 2017).

6. Limitations, Implications, and Future Directions

DeepSleepNet’s design—dual-scale CNNs with BiLSTM temporal aggregation—enables extraction of both short- and long-range EEG features necessary for robust sleep stage discrimination. However, it is computationally intensive (buffer of 12.5 min, 24.7M parameters) and requires large labeled datasets ($a_i = h^s_i \Vert h^l_i$1 epochs) to avoid overfitting.

DeepSleepNet-Lite demonstrates that comparable accuracy and kappa can be achieved using a purely feed-forward CNN architecture operating on a reduced context window (90 s) and a $a_i = h^s_i \Vert h^l_i$296% parameter reduction, facilitating real-time and embedded applications. The integration of MC dropout offers an accessible uncertainty index: only a small fraction of ambiguous epochs require clinician review for maximal reliability.

Ongoing research directions include:

• Validation on larger and more heterogeneous patient cohorts,
• Extending MC dropout queries to more complex architectures,
• Refinement of label smoothing strategies for calibration,
• Addressing the reduced context window’s potential impacts on rare sequence transition detection,
• Application to multi-channel and alternative biosignal inputs.

DeepSleepNet and its derivatives have established the foundation for end-to-end, efficient, and clinically scalable EEG-based sleep staging without reliance on hand-crafted features (Fiorillo et al., 2021, Supratak et al., 2017).