---
title: 'LiteFat: Real-Time Driver Fatigue Detector'
url: https://www.emergentmind.com/topics/litefat
type: topic
---

# LiteFat: Real-Time Driver Fatigue Detector

LiteFat is a lightweight spatio-temporal graph learning model for real-time driver fatigue detection on embedded and resource-constrained robotic devices. It is designed for settings in which conventional vision-based fatigue detectors are considered too slow, memory-intensive, or computationally expensive for rapid in-vehicle inference. The model converts streaming video into a spatio-temporal graph using facial landmark detection, augments landmark geometry with MobileNetV3-Small appearance features, learns the graph structure adaptively rather than imposing a fixed facial topology, and performs classification with a lightweight spatial-temporal graph learning module. On the YawDD benchmark, the paper reports an accuracy of \(0.995\), precision of \(1.00\), recall of \(0.969\), F1-score of \(0.984\), parameter count of \(1.32\times 10^6\), and throughput of \(5.75\) samples/sec [2507.21756].

## 1. Motivation and task formulation

LiteFat is motivated by two linked observations. First, driver fatigue and drowsiness are framed as a major cause of traffic accidents, making rapid fatigue detection a road-safety requirement rather than a purely academic recognition task. Second, many existing vision-based approaches are described as computationally demanding, high-latency, and memory-intensive, which limits their suitability for intelligent vehicles and embedded robotic devices that require real-time inference under constrained resources [2507.21756].

The paper also targets a methodological limitation in prior graph-based fatigue detectors. Many such methods construct facial landmark graphs using predefined facial adjacency structures. The authors argue that this can introduce human bias and may miss latent spatial dependencies between facial landmarks. LiteFat therefore seeks both efficiency and more flexible relational modeling by learning the adjacency structure end-to-end rather than fixing it a priori.

Formally, the problem is posed as classification over a video clip. A frame-level graph is written as \(G=(\mathcal{V},\mathcal{E})\), with adjacency matrix \(A\in\mathbb{R}^{N\times N}\), where \(A_{ij}=1\) if \((v_i,v_j)\in\mathcal{E}\) and \(A_{ij}=0\) otherwise. At time \(t\), the graph feature matrix is \(X^{(t)}\in\mathbb{R}^{N\times D}\). Given the graph and the historical features from the previous \(S\) frames, the objective is to learn a classifier \(f\) that predicts each frame’s label. This formulation places LiteFat in the class of spatio-temporal graph models, but with an explicit emphasis on low-complexity deployment.

## 2. From streaming video to a multimodal graph representation

LiteFat begins with streaming driver video and extracts facial landmarks for each frame. Each face is represented by **68 landmarks**, and each landmark is encoded as \((X,Y,C)\), where \(X\) and \(Y\) are 2D coordinates and \(C\) is a confidence score. One frame is therefore represented by a \(68\times 3\) matrix [2507.21756].

The paper includes a simple robustness rule for failed landmark detection. If detection fails because the driver looks down, turns away, or is partially occluded, then all coordinate values are set to **1**. This handling is meant to prevent missing landmarks from distorting subsequent feature construction. The approach is operationally simple, which is consistent with the broader design goal of maintaining a lightweight pipeline.

Landmarks alone are treated as insufficient because they capture facial geometry but not cues such as skin tone, wrinkles, redness, or broader facial appearance. To supplement them, LiteFat uses **MobileNetV3-Small** to extract a \(D\)-dimensional embedding \(d\) from each frame. The paper then defines a multimodal fusion step,
$$
X = C w d^T
$$
where \(C\) is the landmark coordinate matrix, \(w\) is a learnable multimodal fusion weight vector, and \(d\) is the MobileNet feature embedding. The resulting fused representation serves as the node-feature matrix for the spatio-temporal graph learner.

This combination of landmark geometry and lightweight CNN appearance features is central to LiteFat’s representation strategy. A plausible implication is that the model attempts to preserve the deployment advantages of landmark-based graph reasoning while avoiding the information loss that would arise from using coordinates alone.

## 3. Adaptive graph construction and spatio-temporal learning

LiteFat’s core classifier is a lightweight spatial-temporal graph learning module that combines graph convolution for spatial reasoning with gated temporal convolution for sequence modeling. A central design choice is the use of a self-adaptive adjacency matrix rather than a manually specified facial graph [2507.21756].

To learn graph structure, the model introduces two learnable node-embedding dictionaries,
$$
E_1,E_2\in\mathbb{R}^{N\times c},
$$
and defines the adaptive adjacency as
$$
\tilde{A}_{adp}=\mathrm{softmax}(\mathrm{ReLU}(E_1E_2^T)).
$$
In this construction, \(E_1E_2^T\) models pairwise relation strength between source and target nodes, ReLU suppresses weak or negative connections, and softmax normalizes the result row-wise. The paper’s claim is that this permits the network to discover fatigue-relevant facial dependencies that a fixed topology might miss.

Spatial aggregation is implemented with a graph convolution update of the form
$$
Z = F(X,\tilde{A}_{adp}) = \mathrm{softmax}\left(\tilde{A}_{adp}\,\mathrm{ReLU}(\tilde{A}_{adp}XW^{(0)})W^{(1)}\right),
$$
where \(W^{(0)}\in\mathbb{R}^{D\times H}\) and \(W^{(1)}\in\mathbb{R}^{D\times M}\). The model description interprets this as repeated neighbor aggregation and nonlinear transformation, culminating in an output projection suited to classification.

Temporal dynamics are modeled with a **Temporal Convolutional Network** using dilated causal convolutions:
$$
x \star f(t)=\sum_{s=0}^{k-1} f(s)x(t-d\times s),
$$
where \(k\) is the kernel size and \(d\) is the dilation rate. The temporal block is gated:
$$
h = g(\Theta_1 \star \mathcal{X} + b)\odot \sigma(\Theta_2 \star \mathcal{X} + c),
$$
where \(\mathcal{X}\in\mathbb{R}^{N\times D\times T}\), \(g(\cdot)\) is a tanh activation, \(\sigma(\cdot)\) is a sigmoid gate, and \(\odot\) denotes element-wise multiplication. The paper argues that this structure filters irrelevant temporal variation while emphasizing fatigue-related motion patterns.

Each spatial-temporal layer contains **two GCN layers**, a **gated TCN**, and two parallel temporal convolutions, **TCN-a** and **TCN-b**. The stacked structure is intended to capture short-term motion at lower levels and longer-range dependencies at higher levels. Within the paper’s framing, this is the mechanism by which LiteFat models cues such as yawning onset, sustained mouth opening, distracted head motion, and eye-closure dynamics.

## 4. Supervision, dataset protocol, and implementation setting

LiteFat is trained with cross-entropy loss,
$$
L=-\frac{1}{T}\sum_{i=1}^{T} y_i \log(\hat{y}_i),
$$
where \(y_i\) is the true label and \(\hat{y}_i\) is the predicted probability [2507.21756]. The training setup uses a maximum of **100 epochs**, **early stopping** if the training loss does not improve for **3 consecutive epochs**, the **Adam** optimizer, and an initial learning rate of \(1\times10^{-4}\).

The experiments are conducted on the **YawDD** dataset. Videos are split into clips of up to **5 seconds**, and all frames in a clip are relabeled to share the same label. The paper uses three classes: **normal**, **talking**, and **yawning**. It explicitly treats **talking** as an abnormal state because it may involve head turning, looking away, and distraction. For comparison with prior yawning-detection work, the yawning task is treated as **binary classification** between yawning and normal, whereas the ablation studies use **three-class classification**.

Each training sample is represented by **16 frames**. The hardware and software environment reported in the paper is an **Apple M3 Pro**, with a **12-core CPU** consisting of 6 performance cores and 6 efficiency cores, **18 GB memory**, and macOS **15.0.1**. The reported evaluation metrics include **Accuracy**, **Precision**, **Recall**, and **F1-score** for predictive quality, and **parameter count**, **forward pass time**, **backward pass time**, **throughput**, and **memory usage** for efficiency.

This evaluation protocol matters because LiteFat is positioned as a deployment-oriented method. The inclusion of forward time, backward time, throughput, and memory usage makes the paper’s claims about lightweight inference more concrete than accuracy-only reporting would allow.

## 5. Reported empirical performance

On YawDD yawning detection, LiteFat reports **Accuracy \(=0.995\)**, **Precision \(=1.00\)**, **Recall \(=0.969\)**, and **F1 \(=0.984\)** [2507.21756]. Among the baselines listed in the paper, **2s-IsoSSL-MoCo** reports **Accuracy \(=0.987\)** and **F1 \(=0.983\)**, **JHPFA-Net** reports **Accuracy \(=0.964\)** and **F1 \(=0.939\)**, and **2s-STGCN** reports **Accuracy \(=0.934\)** and **F1 \(=0.895\)**. Within this comparison, LiteFat attains the highest reported accuracy and precision, and its F1-score slightly exceeds that of 2s-IsoSSL-MoCo.

The efficiency comparison is more mixed but remains favorable in the respects emphasized by the paper. LiteFat uses **\(1.32\times10^6\)** parameters, with **1.39 s** forward pass time, **2.16 s** backward pass time, **5.75** samples/sec throughput, and **1905.44 MB** memory usage. **JHPFA-Net** uses **\(2.26\times10^8\)** parameters, **10.35 s** forward, **14.77 s** backward, **0.77** samples/sec, and **2201.03 MB** memory. **2s-IsoSSL-MoCo** uses **\(2.48\times10^7\)** parameters, **1.89 s** forward, **3.11 s** backward, **4.23** samples/sec, and **3433.48 MB** memory. **2s-STGCN** is smaller in parameters at **\(9.97\times10^5\)** and uses less memory at **1367.44 MB**, but LiteFat records the fastest forward pass and the highest throughput among the compared methods. The result is therefore not “smallest in every metric,” but rather a particular balance of high predictive performance and strong inference efficiency.

The ablation studies clarify which parts of the model carry most of the gain. With **STGL = yes** and **MobileNet = no**, the paper reports **AUC \(=0.942\)**, **Accuracy \(=0.826\)**, **Precision \(=0.873\)**, **Recall \(=0.830\)**, and **F1 \(=0.844\)**. With **STGL = no** and **MobileNet = yes**, it reports **AUC \(=0.975\)**, **Accuracy \(=0.920\)**, **Precision \(=0.940\)**, **Recall \(=0.910\)**, and **F1 \(=0.922\)**. With both enabled, the scores rise to **AUC \(=0.998\)**, **Accuracy \(=0.977\)**, **Precision \(=0.974\)**, **Recall \(=0.984\)**, and **F1 \(=0.978\)**. This indicates that MobileNet features alone are already strong, STGL alone is weaker, and the combination is best.

A second ablation isolates spatial and temporal graph components. **w/o TCN** yields **AUC \(=0.981\)**, **Accuracy \(=0.892\)**, **Precision \(=0.917\)**, **Recall \(=0.897\)**, and **F1 \(=0.904\)**. **w/o GCN** yields **AUC \(=0.974\)**, **Accuracy \(=0.864\)**, **Precision \(=0.895\)**, **Recall \(=0.860\)**, and **F1 \(=0.871\)**. The full **STGL** model reaches **AUC \(=0.998\)**, **Accuracy \(=0.977\)**, **Precision \(=0.974\)**, **Recall \(=0.984\)**, and **F1 \(=0.978\)**. The paper interprets this as evidence that both components matter, with spatial modeling contributing especially strongly because removing GCN produces the larger degradation.

## 6. Interpretation, deployment significance, and limitations

LiteFat is explicitly framed as a model for **embedded robotic devices** and intelligent vehicles, not merely as an accuracy-oriented fatigue classifier [2507.21756]. Its deployment-oriented design rests on several choices that are individually modest but collectively coherent: **MobileNetV3-Small** instead of a heavier CNN, adaptive graph learning instead of manually engineered facial topology, and **TCN** rather than recurrent alternatives for temporal modeling. This suggests that the paper’s main contribution is not only the use of graphs for fatigue detection, but the way graph learning is constrained by real-time inference requirements.

The ablations also counter two common simplifications. One is that facial landmark geometry alone should be sufficient for fatigue detection; the reported AUC, accuracy, and F1 values show that adding MobileNet appearance embeddings improves performance substantially. The other is that temporal modeling alone is decisive; the GCN-removal study indicates that spatial facial relations are at least as important, and in the reported setting more important, than TCN dynamics. LiteFat therefore depends on a genuinely multimodal and spatio-temporal formulation rather than on any single component.

The paper is relatively restrained about its limitations, but several are explicit or directly implied. Experiments are mainly on **YawDD**. Final deployment on actual embedded robotic devices is identified as future work. More real-world data are said to be needed for broader validation. Performance under severe occlusion, lighting changes, and more diverse in-cabin conditions is not fully studied. These points matter because the model’s intended use case is safety-critical.

In the broader literature on lightweight learning systems, LiteFat belongs to a family of methods that make efficiency a first-order design variable rather than a post hoc compression step. That family includes hardware-budgeted LiDAR detection in **LiFT** [2501.11159], detector-feature reuse for efficient multi-object tracking in **LITE** [2409.04187], and confidence-adaptive early stopping for Forward-Forward inference in **LightFF** [2404.05241]. LiteFat differs in domain and architecture, but it shares the same general principle: maintain task-relevant structure while reducing computational overhead enough to make real-time deployment plausible on constrained hardware.

Source: https://www.emergentmind.com/topics/litefat