LiteFat: Real-Time Driver Fatigue Detector

• LiteFat is a lightweight spatio-temporal graph model that transforms streaming video into facial landmark graphs for efficient real-time fatigue detection.
• It fuses 68-point facial landmarks with MobileNetV3-Small embeddings and learns adaptive graph structures to capture latent spatial dependencies.
• On the YawDD benchmark, LiteFat achieved 0.995 accuracy, 1.00 precision, 0.969 recall, and 0.984 F1-score, balancing performance with low computational overhead.

LiteFat is a lightweight spatio-temporal graph learning model for real-time driver fatigue detection on embedded and resource-constrained robotic devices. It is designed for settings in which conventional vision-based fatigue detectors are considered too slow, memory-intensive, or computationally expensive for rapid in-vehicle inference. The model converts streaming video into a spatio-temporal graph using facial landmark detection, augments landmark geometry with MobileNetV3-Small appearance features, learns the graph structure adaptively rather than imposing a fixed facial topology, and performs classification with a lightweight spatial-temporal graph learning module. On the YawDD benchmark, the paper reports an accuracy of $0.995$, precision of $1.00$, recall of $0.969$, F1-score of $0.984$, parameter count of $1.32\times 10^6$, and throughput of $5.75$ samples/sec (Ren et al., 29 Jul 2025).

1. Motivation and task formulation

LiteFat is motivated by two linked observations. First, driver fatigue and drowsiness are framed as a major cause of traffic accidents, making rapid fatigue detection a road-safety requirement rather than a purely academic recognition task. Second, many existing vision-based approaches are described as computationally demanding, high-latency, and memory-intensive, which limits their suitability for intelligent vehicles and embedded robotic devices that require real-time inference under constrained resources (Ren et al., 29 Jul 2025).

The paper also targets a methodological limitation in prior graph-based fatigue detectors. Many such methods construct facial landmark graphs using predefined facial adjacency structures. The authors argue that this can introduce human bias and may miss latent spatial dependencies between facial landmarks. LiteFat therefore seeks both efficiency and more flexible relational modeling by learning the adjacency structure end-to-end rather than fixing it a priori.

Formally, the problem is posed as classification over a video clip. A frame-level graph is written as $G=(\mathcal{V},\mathcal{E})$, with adjacency matrix $A\in\mathbb{R}^{N\times N}$, where $A_{ij}=1$ if $(v_i,v_j)\in\mathcal{E}$ and $1.00$0 otherwise. At time $1.00$1, the graph feature matrix is $1.00$2. Given the graph and the historical features from the previous $1.00$3 frames, the objective is to learn a classifier $1.00$4 that predicts each frame’s label. This formulation places LiteFat in the class of spatio-temporal graph models, but with an explicit emphasis on low-complexity deployment.

2. From streaming video to a multimodal graph representation

LiteFat begins with streaming driver video and extracts facial landmarks for each frame. Each face is represented by 68 landmarks, and each landmark is encoded as $1.00$5, where $1.00$6 and $1.00$7 are 2D coordinates and $1.00$8 is a confidence score. One frame is therefore represented by a $1.00$9 matrix (Ren et al., 29 Jul 2025).

The paper includes a simple robustness rule for failed landmark detection. If detection fails because the driver looks down, turns away, or is partially occluded, then all coordinate values are set to 1. This handling is meant to prevent missing landmarks from distorting subsequent feature construction. The approach is operationally simple, which is consistent with the broader design goal of maintaining a lightweight pipeline.

Landmarks alone are treated as insufficient because they capture facial geometry but not cues such as skin tone, wrinkles, redness, or broader facial appearance. To supplement them, LiteFat uses MobileNetV3-Small to extract a $0.969$0-dimensional embedding $0.969$1 from each frame. The paper then defines a multimodal fusion step,

$0.969$2

where $0.969$3 is the landmark coordinate matrix, $0.969$4 is a learnable multimodal fusion weight vector, and $0.969$5 is the MobileNet feature embedding. The resulting fused representation serves as the node-feature matrix for the spatio-temporal graph learner.

This combination of landmark geometry and lightweight CNN appearance features is central to LiteFat’s representation strategy. A plausible implication is that the model attempts to preserve the deployment advantages of landmark-based graph reasoning while avoiding the information loss that would arise from using coordinates alone.

3. Adaptive graph construction and spatio-temporal learning

LiteFat’s core classifier is a lightweight spatial-temporal graph learning module that combines graph convolution for spatial reasoning with gated temporal convolution for sequence modeling. A central design choice is the use of a self-adaptive adjacency matrix rather than a manually specified facial graph (Ren et al., 29 Jul 2025).

To learn graph structure, the model introduces two learnable node-embedding dictionaries,

$0.969$6

and defines the adaptive adjacency as

$0.969$7

In this construction, $0.969$8 models pairwise relation strength between source and target nodes, ReLU suppresses weak or negative connections, and softmax normalizes the result row-wise. The paper’s claim is that this permits the network to discover fatigue-relevant facial dependencies that a fixed topology might miss.

Spatial aggregation is implemented with a graph convolution update of the form

$0.969$9

where $0.984$0 and $0.984$1. The model description interprets this as repeated neighbor aggregation and nonlinear transformation, culminating in an output projection suited to classification.

Temporal dynamics are modeled with a Temporal Convolutional Network using dilated causal convolutions:

$0.984$2

where $0.984$3 is the kernel size and $0.984$4 is the dilation rate. The temporal block is gated:

$0.984$5

where $0.984$6, $0.984$7 is a tanh activation, $0.984$8 is a sigmoid gate, and $0.984$9 denotes element-wise multiplication. The paper argues that this structure filters irrelevant temporal variation while emphasizing fatigue-related motion patterns.

Each spatial-temporal layer contains two GCN layers, a gated TCN, and two parallel temporal convolutions, TCN-a and TCN-b. The stacked structure is intended to capture short-term motion at lower levels and longer-range dependencies at higher levels. Within the paper’s framing, this is the mechanism by which LiteFat models cues such as yawning onset, sustained mouth opening, distracted head motion, and eye-closure dynamics.

4. Supervision, dataset protocol, and implementation setting

LiteFat is trained with cross-entropy loss,

$1.32\times 10^6$0

where $1.32\times 10^6$1 is the true label and $1.32\times 10^6$2 is the predicted probability (Ren et al., 29 Jul 2025). The training setup uses a maximum of 100 epochs, early stopping if the training loss does not improve for 3 consecutive epochs, the Adam optimizer, and an initial learning rate of $1.32\times 10^6$3.

The experiments are conducted on the YawDD dataset. Videos are split into clips of up to 5 seconds, and all frames in a clip are relabeled to share the same label. The paper uses three classes: normal, talking, and yawning. It explicitly treats talking as an abnormal state because it may involve head turning, looking away, and distraction. For comparison with prior yawning-detection work, the yawning task is treated as binary classification between yawning and normal, whereas the ablation studies use three-class classification.

Each training sample is represented by 16 frames. The hardware and software environment reported in the paper is an Apple M3 Pro, with a 12-core CPU consisting of 6 performance cores and 6 efficiency cores, 18 GB memory, and macOS 15.0.1. The reported evaluation metrics include Accuracy, Precision, Recall, and F1-score for predictive quality, and parameter count, forward pass time, backward pass time, throughput, and memory usage for efficiency.

This evaluation protocol matters because LiteFat is positioned as a deployment-oriented method. The inclusion of forward time, backward time, throughput, and memory usage makes the paper’s claims about lightweight inference more concrete than accuracy-only reporting would allow.

5. Reported empirical performance

On YawDD yawning detection, LiteFat reports Accuracy $1.32\times 10^6$4, Precision $1.32\times 10^6$5, Recall $1.32\times 10^6$6, and F1 $1.32\times 10^6$7 (Ren et al., 29 Jul 2025). Among the baselines listed in the paper, 2s-IsoSSL-MoCo reports Accuracy $1.32\times 10^6$8 and F1 $1.32\times 10^6$9, JHPFA-Net reports Accuracy $5.75$0 and F1 $5.75$1, and 2s-STGCN reports Accuracy $5.75$2 and F1 $5.75$3. Within this comparison, LiteFat attains the highest reported accuracy and precision, and its F1-score slightly exceeds that of 2s-IsoSSL-MoCo.

The efficiency comparison is more mixed but remains favorable in the respects emphasized by the paper. LiteFat uses $5.75$4 parameters, with 1.39 s forward pass time, 2.16 s backward pass time, 5.75 samples/sec throughput, and 1905.44 MB memory usage. JHPFA-Net uses $5.75$5 parameters, 10.35 s forward, 14.77 s backward, 0.77 samples/sec, and 2201.03 MB memory. 2s-IsoSSL-MoCo uses $5.75$6 parameters, 1.89 s forward, 3.11 s backward, 4.23 samples/sec, and 3433.48 MB memory. 2s-STGCN is smaller in parameters at $5.75$7 and uses less memory at 1367.44 MB, but LiteFat records the fastest forward pass and the highest throughput among the compared methods. The result is therefore not “smallest in every metric,” but rather a particular balance of high predictive performance and strong inference efficiency.

The ablation studies clarify which parts of the model carry most of the gain. With STGL = yes and MobileNet = no, the paper reports AUC $5.75$8, Accuracy $5.75$9, Precision $G=(\mathcal{V},\mathcal{E})$0, Recall $G=(\mathcal{V},\mathcal{E})$1, and F1 $G=(\mathcal{V},\mathcal{E})$2. With STGL = no and MobileNet = yes, it reports AUC $G=(\mathcal{V},\mathcal{E})$3, Accuracy $G=(\mathcal{V},\mathcal{E})$4, Precision $G=(\mathcal{V},\mathcal{E})$5, Recall $G=(\mathcal{V},\mathcal{E})$6, and F1 $G=(\mathcal{V},\mathcal{E})$7. With both enabled, the scores rise to AUC $G=(\mathcal{V},\mathcal{E})$8, Accuracy $G=(\mathcal{V},\mathcal{E})$9, Precision $A\in\mathbb{R}^{N\times N}$0, Recall $A\in\mathbb{R}^{N\times N}$1, and F1 $A\in\mathbb{R}^{N\times N}$2. This indicates that MobileNet features alone are already strong, STGL alone is weaker, and the combination is best.

A second ablation isolates spatial and temporal graph components. w/o TCN yields AUC $A\in\mathbb{R}^{N\times N}$3, Accuracy $A\in\mathbb{R}^{N\times N}$4, Precision $A\in\mathbb{R}^{N\times N}$5, Recall $A\in\mathbb{R}^{N\times N}$6, and F1 $A\in\mathbb{R}^{N\times N}$7. w/o GCN yields AUC $A\in\mathbb{R}^{N\times N}$8, Accuracy $A\in\mathbb{R}^{N\times N}$9, Precision $A_{ij}=1$0, Recall $A_{ij}=1$1, and F1 $A_{ij}=1$2. The full STGL model reaches AUC $A_{ij}=1$3, Accuracy $A_{ij}=1$4, Precision $A_{ij}=1$5, Recall $A_{ij}=1$6, and F1 $A_{ij}=1$7. The paper interprets this as evidence that both components matter, with spatial modeling contributing especially strongly because removing GCN produces the larger degradation.

6. Interpretation, deployment significance, and limitations

LiteFat is explicitly framed as a model for embedded robotic devices and intelligent vehicles, not merely as an accuracy-oriented fatigue classifier (Ren et al., 29 Jul 2025). Its deployment-oriented design rests on several choices that are individually modest but collectively coherent: MobileNetV3-Small instead of a heavier CNN, adaptive graph learning instead of manually engineered facial topology, and TCN rather than recurrent alternatives for temporal modeling. This suggests that the paper’s main contribution is not only the use of graphs for fatigue detection, but the way graph learning is constrained by real-time inference requirements.

The ablations also counter two common simplifications. One is that facial landmark geometry alone should be sufficient for fatigue detection; the reported AUC, accuracy, and F1 values show that adding MobileNet appearance embeddings improves performance substantially. The other is that temporal modeling alone is decisive; the GCN-removal study indicates that spatial facial relations are at least as important, and in the reported setting more important, than TCN dynamics. LiteFat therefore depends on a genuinely multimodal and spatio-temporal formulation rather than on any single component.

The paper is relatively restrained about its limitations, but several are explicit or directly implied. Experiments are mainly on YawDD. Final deployment on actual embedded robotic devices is identified as future work. More real-world data are said to be needed for broader validation. Performance under severe occlusion, lighting changes, and more diverse in-cabin conditions is not fully studied. These points matter because the model’s intended use case is safety-critical.

In the broader literature on lightweight learning systems, LiteFat belongs to a family of methods that make efficiency a first-order design variable rather than a post hoc compression step. That family includes hardware-budgeted LiDAR detection in LiFT (Lis et al., 19 Jan 2025), detector-feature reuse for efficient multi-object tracking in LITE (Alikhanov et al., 2024), and confidence-adaptive early stopping for Forward-Forward inference in LightFF (Aminifar et al., 2024). LiteFat differs in domain and architecture, but it shares the same general principle: maintain task-relevant structure while reducing computational overhead enough to make real-time deployment plausible on constrained hardware.