--- title: 'BlazeFace: Mobile Face Detection' url: https://www.emergentmind.com/topics/blazeface type: topic --- # BlazeFace: Mobile Face Detection BlazeFace is a neural face detector specifically optimized for sub-millisecond inference on mobile GPUs. Designed to support augmented reality (AR) pipelines demanding highly accurate and temporally stable face localization, BlazeFace introduces a suite of architecture, inference, and deployment innovations distinct from prior lightweight detectors such as MobileNetV1/V2-SSD. Its critical design elements enable throughput of 200–1000+ FPS on flagship devices without quantization or heavy pruning, facilitating real-time downstream tasks including facial keypoint estimation, geometry reconstruction, expression classification, and facial region segmentation [1907.05047]. ## 1. Network Architecture and Innovations BlazeFace’s network architecture comprises a compact, depthwise-separable convolutional backbone tailored for face detection in low-resolution inputs (128×128×3 RGB). The backbone consists of initial 5×5 convolution followed by repeated, customized “BlazeBlock” modules. The key features are: - **Single BlazeBlock**: Composed of a 5×5 depthwise convolution, batch normalization (BN), ReLU, and a 1×1 pointwise convolution with BN. Identity residuals are present if stride is 1. - **Double BlazeBlock**: Diverges from the MobileNetV2 bottleneck by using a Project–DW–Expand order (as opposed to Expand–DW–Project), with residuals operating in the expanded channel space and doubled depthwise convolutions to increase receptive field with fewer blocks. A schematic layer-by-layer breakdown is shown below: | Layer | Input | Kernel / Structure | Output | |-----------------------|-----------------|-------------------------------------------------|--------------| | Convolution | 128×128×3 | 5×5, 3→24, stride 2 | 64×64×24 | | SingleBlazeBlock×2 | 64×64×24 | (5×5 dwise + 1×1 pw) 24→24 | 64×64×24 | | SingleBlazeBlock (s=2)| 64×64×24 | … 24→48 | 32×32×48 | | SingleBlazeBlock×2 | 32×32×48 | … 48→48 | 32×32×48 | | DoubleBlazeBlock (s=2)| 32×32×48 | 1×1:48→96; 5×5 dw; 1×1:96→48 | 16×16×48 | | DoubleBlazeBlock×2 | 16×16×48 | … | 16×16×48 | | DoubleBlazeBlock (s=2)| 16×16×48 | … | 8×8×48 | | DoubleBlazeBlock×2 | 8×8×48 | … | 8×8×48 | The final feature map is of size 8×8×96, to which SSD-style detection heads are attached. The design choice to favor 5×5 (over 3×3) in depthwise convolution is motivated by the large receptive field expansion at marginal additional computational cost per block. Standard formulas for parameter count and multiply-adds (FLOPs) per layer apply: - Parameter count: $P = \sum_l k_l^2\cdot c_{in,l}\cdot c_{out,l}$ - Multiply-adds: $F = \sum_l s_l^2 [k_l^2\cdot c_{in,l} + c_{in,l}\cdot c_{out,l}]$ ## 2. Anchor Scheme and Detection Logic BlazeFace uses an SSD-derived one-stage detection strategy with GPU-optimized anchor assignment. - **Anchor configuration**: Only square (1:1 aspect ratio) anchors are used, reflecting the limited aspect variation in AR face pipelines. Six different anchor scales are mapped exclusively onto the 8×8 feature map (yielding 384 anchors per image). - **Simplified anchor generation**: All anchors are placed at the 8×8 map resolution, eschewing lower-resolution (4×4, 2×2, 1×1) anchor sets characteristic of standard SSD. This approach exploits mobile GPU efficiency for medium-sized tensors and avoids overhead associated with dispatching many small tensors. - **Box regression**: Given anchor $(x_a, y_a, w_a, h_a)$, and predicted offsets $(\Delta x, \Delta y, \Delta w, \Delta h)$, the final box is parameterized as $$ \hat x = x_a + w_a\cdot\Delta x \\ \hat y = y_a + h_a\cdot\Delta y \\ \hat w = w_a\cdot\exp(\Delta w) \\ \hat h = h_a\cdot\exp(\Delta h) $$ - **Anchor–ground-truth matching**: Standard SSD rules are followed—each ground-truth box is matched with the anchor of highest IoU, and all anchors with IoU ≥ 0.5 are assigned that ground-truth. All other anchors are treated as background. This scheme is optimized for mobile GPU throughput and training stability [1907.05047]. ## 3. Tie-Resolution and Temporal Stability Standard SSD-based detectors use Non-Maximum Suppression (NMS) to eliminate redundant detections; however, this can produce visual “jitter” in video streams as different anchors alternate as the selected detection. - **Blending-based NMS alternative**: Rather than keep only the top-scoring box, BlazeFace performs weighted averaging (“blending”) of all boxes overlapping (IoU ≥ $T$, e.g., 0.5) with the highest-scoring detection. The averaged box coordinates take the form: $$ \bar x = \frac{\sum_j (\text{score}_j \cdot x_j)}{\sum_j \text{score}_j} $$ and similarly for $\bar y, \bar w, \bar h$, retaining the maximum confidence as the group score. - **Empirical effect**: This operation increases detection accuracy by ≈10% AP on private AR datasets and decreases output “jitter” by 40% (frontal) and 30% (rear), as measured by RMS output difference across small input translations. Its implementation is computationally lightweight, adding only several FMA operations per group. ## 4. Empirical Evaluation and Benchmarks BlazeFace exhibits a significant improvement in both inference latency and temporal stability compared to MobileNetV2-SSD, with only minor tradeoffs in keypoint regression error. - **Inference speed** (float16, single frame): - iPhone XS: BlazeFace 0.6 ms (AP 98.61%) vs. MobileNetV2-SSD 2.1 ms (AP 97.95%). - Further device benchmarks: | Device | MobileNetV2-SSD (ms) | BlazeFace (ms) | |-----------------|----------------------|---------------| | iPhone 7 | 4.2 | 1.8 | | iPhone XS | 2.1 | 0.6 | | Google Pixel 3 | 7.2 | 3.4 | | Huawei P20 | 21.3 | 5.8 | | Samsung S9+ | 7.2 | 3.7 | - **Regression and jitter accuracy** (median absolute error, normalized by inter-ocular distance, IOD): | Model | Regression error (%) | Jitter (%) | |-------------------|---------------------|------------| | MobileNetV2-SSD | 7.4 | 3.6 | | BlazeFace | 10.4 | 5.3 | - **Ablative findings**: Moving from 3×3 to 5×5 depthwise convolutions halves the number of required bottleneck blocks, reducing parameters and GPU overhead. The single 8×8 anchor map decreases small-tensor dispatch requirements by ~60%. Blended NMS adds ~0.5 AP and reduces temporal jitter by 30–40%. ## 5. Implementation Details and Device-Level Optimizations - **Kernel fusion**: All convolutions are implemented as fused depthwise+pointwise operations (e.g., Metal Performance Shaders on iOS, TFLite GPU on Android). - **Arithmetic and memory**: Float16 arithmetic is employed throughout, minimizing memory footprint and maximizing computational throughput. Weights are arranged per the “channel-major” (NHWC) layout of mobile GPU drivers, circumventing extra transposes. - **Quantization/pruning**: No post-training quantization or pruning is employed; the network’s parameter count (~0.5M) and sub-millisecond latency obviate further reduction steps. ## 6. Application Scope and Practical Significance BlazeFace’s primary utility lies in mobile and embedded contexts requiring efficient, accurate face detection as a precursor for AR tasks, real-time keypoint regression, facial feature analysis, and face region segmentation. Its design demonstrates how task-specific architectural and inference optimizations—specifically, the use of larger depthwise kernels, anchor scheme simplification, and temporally stable box resolution—yield substantial advances in throughput and usability for edge devices [1907.05047]. A plausible implication is that the architectural principles exemplified by BlazeFace, namely GPU-friendliness and jitter mitigation without extensive quantization, are transferable to other dense-prediction models targeting real-time applications on resource-constrained hardware. Source: https://www.emergentmind.com/topics/blazeface