BlazeFace Model: Fast Mobile Face Detection

• BlazeFace model is a lightweight neural network optimized for real-time face detection on mobile devices, leveraging 5×5 depthwise convolutions and BlazeBlocks for enhanced speed and accuracy.
• The network achieves sub-millisecond inference with high precision (AP ~98.61%), making it ideal for augmented reality, deepfake detection, and mobile vision pipelines.
• BlazeFace improves video stability through weighted blending post-processing, reducing detection jitter by 30-40% compared to traditional NMS methods.

BlazeFace is a lightweight, high-speed neural network-based face detector optimized for real-time applications, particularly on mobile and resource-constrained devices. Developed by Google Research, BlazeFace is designed to efficiently identify and localize faces and their keypoints within video and image streams, with a particular focus on inference speed and stability for downstream mobile vision tasks.

1. Architectural Features and Design Rationale

BlazeFace departs from conventional mobile CNN architectures by introducing several optimizations aimed at maximizing throughput on mobile GPUs and ensuring robust detection for augmented reality and video analysis.

• Feature Extraction Network: The core architecture is inspired by, but architecturally distinct from, MobileNetV1/V2. Where MobileNets use 3×3 depthwise convolutions, BlazeFace replaces these with 5×5 depthwise convolutions, which increase the receptive field with minimal additional computational cost, particularly relevant in depthwise architectures.
• Building Blocks: The design constructs the network from "BlazeBlocks" and "Double BlazeBlocks." A BlazeBlock consists of an expansion (1×1 convolution), a 5×5 depthwise convolution, and a projection (1×1 convolution) with a residual connection. The Double BlazeBlock further alternates 5×5 depthwise convolutions and projection stages to rapidly extend the receptive field while maintaining network compactness.
• Output Resolution: Unlike classical SSD-based detectors that downsample to low spatial resolutions (e.g., 1×1), BlazeFace maintains high-resolution feature maps (e.g., 8×8), preserving localization precision necessary for small and variably scaled faces.
• Anchor Modifications: The model anchors all predictions at a single, higher-resolution grid (8×8) with six anchors per location, all using a 1:1 aspect ratio. This is tailored for human face geometry and exploits the efficiency of parallel GPU computation.
• Post-processing: To address the temporal instability (jitter) in video caused by non-maximum suppression (NMS), BlazeFace introduces a weighted blending (tie-resolution) strategy for bounding box outputs:

$$\mathbf{b}_{\text{final}} = \frac{\sum_i w_i \mathbf{b}_i}{\sum_i w_i}$$

where $w_i$ is the box confidence, $\mathbf{b}_i$ the parameters, resulting in smoother, more stable localization across frames.

2. Functional Capabilities: Detection, Keypoint Localization, and Video Suitability

BlazeFace predicts not only face bounding boxes but also six key face landmarks (eye centers, ear tragions, mouth center, and nose tip). This facilitates downstream face alignment, pose estimation, and geometric normalization for pipelines requiring robust, aligned facial crops.

• Inference Speed: On flagship mobile devices, BlazeFace achieves sub-millisecond single-frame inference. For example, 0.6 ms (approximately 1667 FPS) on iPhone XS and 1.8 ms on iPhone 7, significantly exceeding typical video frame rate requirements.
• Detection Quality: BlazeFace achieves average precision metrics at or above 98% (AP = 98.61%) on internal face datasets. Keypoint RMSEs and box jitter are lower compared to MobileNetV2-SSD, due to the architectural choices and post-processing modifications.
• Stability in Video: The regression-based post-processing substantially reduces output fluctuation under minor input changes, with a reported 30–40% reduction in jitter compared to NMS-based pipelines.

3. Integration in Practical and Research Pipelines

BlazeFace has been integrated as a preprocessing stage in both academic and industry systems:

• Fake Face Detection in Video: In deepfake detection studies, BlazeFace is utilized as the primary face detector. Frames extracted from video datasets are fed through BlazeFace, and the detected face crops (optionally aligned using the provided keypoints) are then supplied as input to downstream classifiers (e.g., ensemble ResNet/Xception networks) for real-vs-fake discrimination. Pipeline performance is highly contingent on BlazeFace’s ability to provide accurate, low-jitter crops; with this setup, end-to-end k-fold cross-validation accuracy reached up to 91% in detection studies (Shamanth et al., 2022).
• Mobile Device Face Access and AR: BlazeFace is extensively deployed in real-time mobile vision workloads such as access control, driver monitoring, and facial expression analysis. Its output stability, low latency, and support for facial landmarks make it advantageous particularly in use-cases where face detection is a rate-limiting step preceding AR overlays or geometric modeling (Bazarevsky et al., 2019).

4. Comparative Evaluation and Performance Metrics

Multiple studies benchmark BlazeFace alongside established algorithms (MTCNN, Haar cascades, LBP, HOG):

• Inference Latency: At 256×256 input, BlazeFace achieves ~83 ms per frame on Snapdragon 845 (≈12 FPS), which is slower than LBP/HOG in absolute terms but faster or roughly equal to Haar and MTCNN. At 128×128, real-time rates (15.2 FPS) are attainable.
• Detection Fidelity: BlazeFace consistently obtains maximum scores on qualitative benchmarks, with no false positives in no-face scenes and high recall on both single and multi-face imagery. Detection ability is described as “working perfectly” at the designed operating resolutions.
• Video Tracking Suitability: Unlike cascaded detectors (which are fastest on empty scenes), BlazeFace demonstrates its slowest inference in no-face frames. This is an unusual trait attributed to non-cascaded processing. While this may reduce efficiency in surveillance or always-on contexts, it has no impact where face presence is frequent or detection regularity is more important than per-frame resource minimization (Khabarlak, 2022).

Algorithm	Real-time @256×256	Stability in Video	Mobile Suitability
BlazeFace	Borderline	Jitter-free	High
MTCNN	No	No	Good
LBP	Yes	No	Good (fastest)
Haar	No	No	Poor (slow)
HOG	Yes	No	Good (fast)

5. Mobile Design Constraints and Special Considerations

• Input Resolution: BlazeFace is strictly trained and used at two input sizes: 128×128 (front camera) and 256×256 (rear camera). Flexibility in arbitrarily resizing images is limited; applications must adapt input streams to these formats for optimal detection and accuracy.
• Resource Profile: The model is constructed for single-batch, real-time inference on-device, trading off some flexibility (e.g., arbitrary aspect ratios, multi-scale feature pyramids) for very high per-frame throughput and reduced memory footprint, making it suitable for edge runtimes.
• Video Application Trade-offs: The replacement of NMS with weighted blending results in low-jitter, temporally-stable outputs ideal for live video overlays, at the potential cost of higher inference latency on frame sequences containing no faces.
• Suitability in Varied Contexts: For always-on surveillance, cascaded algorithms may demonstrate higher average speed due to superior pruning of non-face regions. However, for interactive, AR, or analysis tasks where prediction regularity and output alignment are critical, BlazeFace is preferential.

6. Impact, Limitations, and Applicability

BlazeFace represents a shift in mobile face detector design, prioritizing regular inference speed, output stability, and easy integration with GPU-optimized AR pipelines. Its introduction provided an immediate uplift in both performance and reliability over contemporaneous models such as MobileNetV2-SSD, both in video stability and raw throughput (Bazarevsky et al., 2019).

• Impact: Enabled higher-level geometric, mesh, or expression estimation models to operate reliably at frame rates constrained by the application, not by face detection latency.
• Limitations: Fixed input size, non-cascaded inference (reducing efficiency in empty-scene scenarios), and the architectural focus on faces (to the exclusion of arbitrary object detection) confine its use to well-defined pipelines with predictable input distributions.
• Academic and Production Adoption: BlazeFace is commonly recommended for mobile and embedded workflows demanding stable face localization, particularly in scenarios where bounding box jitter is detrimental and rapid downstream task execution is required.

7. Summary Table: Salient Features and Trade-offs

Aspect	BlazeFace Characterization
Feature Extractor	5×5 depthwise conv, BlazeBlocks/DoubleBlocks
Anchor Scheme	6 anchors per 8×8 feature map, 1:1 ratio
Keypoints Output	6 facial landmarks per detection
Input Dimensions	128×128 or 256×256 (strict)
Inference Speed	Sub-millisecond (GPU), ~10–15ms (CPU)
Video Stability	Weighted blending, low-jitter outputs
Use-case Optimized	Mobile, AR, live face pipelines
Notable Limitation	Slowest inference on no-face scenes

BlazeFace’s domain-specific architecture, GPU-oriented optimizations, and video-centric post-processing establish it as a principal choice for fast, robust face detection in mobile and real-time vision systems, with particular relevance for applications that require alignment and stability as prerequisites for higher-order analysis (Bazarevsky et al., 2019, Shamanth et al., 2022, Khabarlak, 2022).