---
title: Toronto NeuroFace Dataset Overview
url: https://www.emergentmind.com/topics/toronto-neuroface-dataset
type: topic
---

# Toronto NeuroFace Dataset Overview

The Toronto NeuroFace dataset is a publicly available, clinically acquired resource of annotated facial video data from patients with neurological disorders—primarily amyotrophic lateral sclerosis (ALS) and stroke—alongside healthy controls. It is the first open-access, video-based collection of facial landmark annotations sourced from neurological patients with the explicit aim of supporting deep-learning research for quantitative, non-invasive assessment of orofacial motor dysfunction, particularly in relation to dysarthria and related disorders [2309.09038, 2307.12159].

## 1. Dataset Composition and Demographic Overview

The Toronto NeuroFace dataset comprises 36 individuals, distributed across three groups:

| Cohort         | Number of Subjects | Gender (M/F) |
|----------------|-------------------|--------------|
| ALS            | 11                | 4 / 7        |
| Stroke         | 14                | 10 / 4       |
| Healthy Control| 11                | 7 / 4        |

- **Age:** Explicit ages are undisclosed; healthy participants were selected to match the age profile of pathological groups.
- **Totally annotated images:** 3,306 frames; partitioned as 1,015 healthy, 920 ALS, and 1,371 stroke.
- **Overall gender distribution:** 21 males, 15 females.

This cohort composition reflects an emphasis on matching control and disease demographics, but sample sizes, especially for ALS and control cases, are modest [2309.09038].

## 2. Acquisition Protocols and Experimental Tasks

- **Recording device:** Intel® RealSense RGB camera, face distance set at 30–60 cm.
- **Sampling:** 30 frames per second; image resolution of 640×480, recorded under uniform clinical laboratory lighting with minimal background noise.
- **Orofacial tasks:** Standardized, clinically relevant tasks include:
    - Static facial postures (maximal mouth opening, lip protrusion, lip stretching)
    - Repetitive phonation (diadochokinetic sequence “pa-ta-ka”, rapid repetition of /pa/ syllable)
    - Additional tasks in ALS work: “kiss” (lip puckering), “blow” (imitating blowing a candle), lip spread, natural rest, and the “BBP” sentence (“Buy Bobby a puppy”) [2307.12159].
- **Environment:** Data collected in a controlled clinical laboratory to optimize lighting and minimize confounds such as variable head pose or occlusions.

Each video session was manually segmented into repetition-level clips according to the task structure, particularly for tasks requiring multiple iterations such as rapid syllable repetition and sentence articulation [2307.12159].

## 3. Annotation Standards and Metadata

- **Landmark annotation:** 68 two-dimensional facial landmarks per frame, following the dlib-style convention referenced in the literature:
    - 17 jawline, 10 eyebrow, 9 nose, 12 eye, and 20 lip/mouth landmarks.
- **Bounding box:** Manually annotated, tightly enclosing the face in each image.
- **Annotation method:** Fully manual, frame-by-frame expert annotation for all 3,306 facial frames [2309.09038].
- **Metadata per frame/image:** Includes subject identifier, categorical pathology label (ALS, stroke, or control), and task label.
- **Coordinate system:** Ground-truth (x, y) pixel locations on original RGB frames; depth or 3D data are not provided.
- **Task breakdown—ALS identification subset:** 921 patient/session clips comprising the following tasks and instance counts:

| Task    | ALS Repetitions | Control Repetitions |
|---------|-----------------|---------------------|
| SPREAD  | 55              | 59                  |
| KISS    | 59              | 57                  |
| OPEN    | 54              | 55                  |
| BLOW    | 31              | 39                  |
| BBP     | 95              | 111                 |
| PA      | 100             | 110                 |
| PATAKA  | 88              | 108                 |

## 4. Preprocessing, Data Splits, and Model Input Conventions

- **Frame selection:** For cyclic or repetitive movements, three frames per repetition are sampled (onset, peak, and midpoint) to maximize intra-individual kinematic diversity [2309.09038].
- **Data split:** 32 subjects (cohort-balanced) are assigned to the train/validation set, and 4 gender-balanced individuals (2 ALS, 2 stroke) form an independent test set.
- **Augmentation in neural network training:** Random horizontal flipping (probability 0.5) and random adjustment of brightness scaling factor (sampled uniformly from [0.8, 1.2]). No upstream alignment or Procrustes normalization; region proposals and alignment performed dynamically during inference.
- **ALS/control identification preprocessing:** OpenFace 2.0 is used for automatic face detection, head pose estimation, and 200×200 pixel face cropping (grayscale conversion). Landmark extraction is performed with FAN (Facial Alignment Network), after which 26 anatomically relevant landmarks (lips and jawline) are retained for graph construction [2307.12159].

## 5. Evaluation Metrics and Statistical Performance

- **Primary metric:** Normalized Mean Error (NME), quantifying average landmark localization error as a percentage of the bounding-box diagonal; computed as:

  $$
  \mathrm{NME} = \frac{1}{N_{\text{imgs}}} \sum_{k=1}^{N_{\text{imgs}}} \left[ \frac{1}{N_\ell} \sum_{i=1}^{N_\ell} \frac{\sqrt{(x_i^k - \hat{x}_i^k)^2 + (y_i^k - \hat{y}_i^k)^2}}{\mathrm{diag}(B^k)} \right] \times 100\%
  $$

  where $(x_i^k, y_i^k)$ and $(\hat x_i^k, \hat y_i^k)$ are the true and predicted landmark i coordinates for image k, and $\mathrm{diag}(B^k)$ is the annotated bounding box diagonal; $N_\ell=68$.

- **Reported landmark localization results:** Best model (“facial-landmark Mask RCNN”):

| Region     | NME (%) |
|------------|---------|
| All (68)   | 1.79    |
| Jaw (17)   | 2.62    |
| Eyebrows   | 0.02    |
| Nose       | 1.55    |
| Eyes       | 1.03    |
| Mouth      | 1.49    |

Ablation studies (without pretraining or with vanilla Mask RCNN backbone) exhibited higher NMEs (2.7–13.6%). Variance and statistical significance estimates are not provided [2309.09038].

- **ALS identification protocol:** The LOSO framework holds out one subject for testing, with majority-vote decision over all per-repetition frame predictions, and separate splits for per-clip and per-subject evaluation [2307.12159].

## 6. Applications in Automated Dysarthria and ALS Analysis

The Toronto NeuroFace dataset is foundational for:
- Benchmarking facial landmark detection models in neurological cohorts, supporting development of CNN (Mask RCNN-based) systems for telemonitoring of dysarthria [2309.09038].
- Enabling research into computational phenotyping of ALS: Geometry-based “Facial Point Graph” approaches leverage the landmark data to construct Delaunay- and hub-augmented graphs processed by Graph Attention Networks, distinguishing ALS from healthy control based solely on facial motion during clinical tasks [2307.12159].

Applied pipelines extract facial action information—in particular, lip and jaw kinematics—shown to be affected in bulbar-onset ALS, and facilitate both frame-based and subject-wise discrimination. Each frame typically passes through landmark extraction, selection, graph embedding, GAT-based processing, and final majority-vote classification schema.

## 7. Limitations, Biases, and Prospective Extensions

- **Sample size:** The number of ALS and control subjects remains limited, constraining statistical power and generalizability.
- **Acquisition constraints:** All data originate from a single device in a homogeneous, well-lit clinical setting; as such, real-world domain shifts including lighting variability, occlusions, and pose diversity are not represented.
- **Task repertoire:** Only a predefined set of speech and non-speech gestures is included; other orofacial behaviors are absent.
- **Age/exclusion:** Pediatric subjects and elderly individuals (>80 years) are not included; demographic scope is thus truncated.
- **Manual annotation:** While presumed accurate, manual labeling is labor-intensive and exemption from inter-rater variability is not proved.
- **Metadata gaps:** Disease-specific variables such as ALS onset type, duration, and severity gradients are not included in the annotated release [2309.09038, 2307.12159].

A plausible implication is that expansion to broader populations, more variable environmental conditions, and incorporation of diverse task types will be necessary to support robust, home-based, and longitudinal neurofunctional assessment.

Source: https://www.emergentmind.com/topics/toronto-neuroface-dataset