---
title: 'FPI-Det: Face–Phone Interaction Benchmark'
url: https://www.emergentmind.com/topics/fpi-det
type: topic
---

# FPI-Det: Face–Phone Interaction Benchmark

Searching arXiv for the FPI-Det paper to ground the article in the cited preprint.
FPI-Det, short for **Face–Phone Interaction Detection**, is a **specialized face–phone interaction dataset and benchmark** designed for **phone-use detection and understanding** in realistic scenes. It was introduced to address the fact that detecting phone use is not merely an object-recognition problem: reliable inference depends on the relationship between faces and phones, rather than on isolated object presence alone. The benchmark contains **22,879 images** with synchronized annotations for faces and phones, together with **phone-use behavior labels** and additional relation information such as **pixel coordinates** and **pairwise distances** between face and phone instances. It is positioned as a benchmark for **fine-grained human–device interaction detection**, supporting both object detection and **binary phone-use classification** [2509.09111].

## 1. Concept and problem formulation

FPI-Det is centered on **face–phone interaction** as the semantic basis for deciding whether a person is using a phone. The underlying premise is that a phone in an image does not automatically imply phone use, and a face alone does not indicate behavior; only the interaction between the two supports reliable phone-use inference. The benchmark therefore differs from generic object-detection datasets that mainly label isolated objects, because it emphasizes the relationship between a face and a device in behaviorally meaningful scenes [2509.09111].

The dataset was created for environments in which phone misuse or distraction is consequential, specifically **workplaces**, **education**, **transportation**, and **public spaces**. The motivation described for the benchmark is that a person may be holding a phone but not using it, looking at a phone, talking on the phone, using a phone while seated, or using a phone sideways or obliquely. This makes the task dependent on contextual reasoning about relative placement, visibility, and behavioral configuration rather than on category detection alone.

A central distinction in the benchmark is between **object detection** and **phone-use classification**. The former concerns detecting faces and phones; the latter is a binary decision based on co-detection and interaction context. This suggests that FPI-Det occupies an intermediate position between conventional detection benchmarks and behavior-understanding datasets: it uses bounding-box supervision but is explicitly designed to support inference about human–device interaction.

## 2. Dataset composition and annotation design

FPI-Det contains **22,879 images**, **29,279 face annotations**, and **10,255 phone annotations**. The dataset averages **1.28 faces/image** and **0.45 phones/image**, with a maximum density of **up to 37 faces in a single image**. These statistics indicate that the benchmark includes both sparse and crowded scenes, which is relevant to the difficulty of face detection under interaction-heavy conditions [2509.09111].

The image-level composition and split are reported as follows:

| Item | Value |
|---|---:|
| Total images | 22,879 |
| Total face annotations | 29,279 |
| Total phone annotations | 10,255 |
| Double (both face and phone present) | 9,888 |
| Face-only | 12,164 |
| Phone-only | 367 |
| Null | 460 |
| Training | 18,800 |
| Validation | 1,730 |
| Test | 2,349 |

The annotations are described as **synchronized labels** for face bounding boxes, phone bounding boxes, face–phone coexistence, and coarse behavior categories. The benchmark also provides **pixel coordinates** for faces and phones, **absolute pairwise distances**, and a **CSV-formatted annotation file** for evaluating final classification performance on validation and test sets. In addition, it includes fine-grained labels indicating whether the smartphone-use behavior is **genuine**.

The coarse behavior categories listed for the dataset are **Calling**, **Using**, **Seated using**, and **Using obliquely**. These labels define a behavior-aware annotation regime that goes beyond box-level localization. A plausible implication is that the dataset can support future work on structured interaction modeling, even though the baseline evaluation described in the benchmark focuses on detection and binary classification.

## 3. Scenario coverage and benchmark difficulty

The benchmark covers four major scenarios: **Workplace**, **Education**, **Transportation**, and **Public spaces**. These settings were selected because phone use is common in them and often consequential. The intended application areas include **traffic safety and driver distraction monitoring**, **workplace productivity and compliance**, **classroom attention monitoring**, and **human–computer interaction and behavior-aware vision** [2509.09111].

FPI-Det is explicitly described as intentionally difficult. Three challenges are highlighted. The first is **scale variation**: phones can be as small as **20 × 20 pixels**, while faces can exceed **800 × 800 pixels**. The second is **occlusion**, including phones partially covered by hands, faces partially hidden, and frequent side views or partial visibility. The third is **diverse capture conditions**, since images come from **surveillance cameras**, **mobile devices**, and **online videos**, introducing **low light**, **compression artifacts**, **motion blur**, **cluttered scenes**, and **varied viewpoints**.

These properties are important because the dataset is not organized around canonical, high-quality imagery. Instead, it is built around real-world, behavior-critical scenes in which detection and interpretation are entangled. The benchmark’s difficulty profile also explains why face detection may be harder than phone detection: faces occur in dense crowds, with frequent occlusion, pose variation, and lighting changes, whereas phones, although often small, may present a more constrained appearance when visible.

The paper contrasts FPI-Det with several benchmark families. Generic datasets such as **COCO** and **PASCAL VOC** focus on many object categories and typically treat object-level detection as the end goal. Face datasets such as **WIDER FACE** and **FDDB** do not include synchronized phone annotations. Interaction datasets such as **HICO-DET** are broader and do not target the specific safety-critical problem of face–phone interaction. Accordingly, FPI-Det is designed to answer not only “what objects are in the image?” but also “is this person using a phone?”

## 4. Tasks and evaluation protocol

The benchmark defines two formal tasks. The first is object detection of faces and phones. The second is **binary phone-use classification** based on detector outputs and interaction context [2509.09111].

For detection, the reported metrics are **mAP@0.5** and **mAP@0.95**. Performance is additionally analyzed by **object category**, **object size**, **occlusion level**, and **environment**. This evaluation design is aligned with the dataset’s central challenges, because overall mean average precision alone would not reveal the effects of scale, visibility, and scene type.

For phone-use classification, the benchmark defines a **parameter-free binary classifier** with the following rule: **phone-in-use** if a face and a phone are co-detected in the same frame, and **not-in-use** otherwise. The evaluated metrics are **Accuracy**, **Precision**, **Recall**, **F1-score**, and **Specificity**. The class convention is stated explicitly: the positive class is **phone use** (**label 0**), and the negative class is **no phone use** (**label 1**).

This classification setup is deliberately minimal. The binary decision depends mainly on **co-occurrence** rather than highly precise localization. The benchmark therefore exposes an important distinction: precise box regression matters strongly for detection metrics, especially at **IoU 0.95**, whereas the binary interaction decision can remain comparatively stable once both relevant object categories are detected.

## 5. Baseline detectors and benchmark results

The benchmark evaluates representative models from two detector families: **YOLOv8-n**, **YOLOv8-s**, **YOLOv8-x**, **YOLOv11-n**, **YOLOv11-s**, **YOLOv11-m**, **YOLOv11-x**, and the transformer-based **DETR** and **Deformable DETR**. This establishes a comparison between fast one-stage detectors and transformer-based global-reasoning detectors [2509.09111].

The principal detection findings are summarized below.

| Setting | Reported result |
|---|---|
| Best face AP@0.5 | **YOLOv8-x**, **56.5%** |
| Best face AP@0.95 | **YOLOv8-x** and **YOLOv11-x**, **35.8%** |
| Best phone AP@0.5 | **YOLOv8-x**, **92.4%** |
| Best phone AP@0.95 | **YOLOv8-x**, **71.0%** |
| Fastest model | **YOLOv8-n**, **416.7 FPS** on a single NVIDIA Tesla V100 |

The benchmark reports that the **YOLO family performs strongly overall**, and that **phones are generally detected more accurately than faces**. Among the transformer baselines, **DETR** is substantially weaker than both YOLO and **Deformable DETR**, while **Deformable DETR** consistently improves over DETR. The reported improvement from DETR to Deformable DETR is **Face AP@0.5: 39.5 → 45.4**, **Face AP@0.95: 21.2 → 29.7**, **Phone AP@0.5: 57.3 → 68.9**, **Phone AP@0.95: 26.9 → 40.6**, **mAP@0.5: 48.4 → 57.2**, and **FPS: 43.0 → 50.2**. The larger gains at **IoU 0.95** are noted as indicating much tighter localization for Deformable DETR.

The phone-use classification results are comparatively compressed across models. The best overall classifier is **YOLOv11-x**, with **Accuracy: 89.5%**, **Precision: 85.9%**, **Recall: 89.1%**, **F1: 87.5%**, and **Specificity: 89.8%**. Other notable results include **YOLOv8-x** with **89.2% accuracy** and **87.0% F1**, **YOLOv11-s** with the highest **Recall: 92.2%**, and **YOLOv11-m** with the highest **Specificity: 90.6%**. Among the transformer classifiers, **DETR** attains **89.2% accuracy** and **87.0% F1**, whereas **Deformable DETR** attains **88.4% accuracy** and **85.3% F1**.

These results support the benchmark’s observation that classification performance is relatively robust once both objects are detected. A plausible implication is that future gains in binary phone-use classification on this protocol may depend less on sophisticated relational heads than on improving recall for difficult face and phone instances under severe scale variation and occlusion.

## 6. Error analysis, practical uses, and benchmark significance

The benchmark’s qualitative and summary analysis identifies several consistent trends. **Workplace scenes** are reported as the easiest, likely due to more stable lighting and clearer compositions, while **transportation scenes** are the hardest. **Small phones** and **heavily occluded objects** remain the most difficult cases. Common failure modes include **false positives on tablet-like objects**, **missed detections for very small phones**, and **missed detections for heavily occluded phones or faces** [2509.09111].

These findings clarify that FPI-Det is not simply a narrow dataset about two categories. Its significance lies in making visible the operational bottlenecks of interaction-aware detection: failures arise from scale, clutter, and incomplete visibility rather than from a lack of category diversity. This is especially relevant for deployment settings in which missed detections may undermine downstream behavior analysis.

The benchmark is described as useful for **driver distraction detection**, **workplace monitoring**, **classroom attention analysis**, **public safety surveillance**, **edge-deployable phone-use detection systems**, and **human-centered behavior understanding**. It is therefore intended not only as an evaluation set but also as a development resource for models that need to move beyond generic object detection and reason about interaction.

The code and dataset are available at **https://github.com/KvCgRv/FPI-Det**. The images were collected from **https://www.datafountain.cn/competitions/506**, and the faces were **de-identified** to protect privacy. In this respect, FPI-Det combines a behavior-oriented detection task with explicit privacy handling, which is notable for a dataset constructed from realistic scenes.

## 7. Position within interaction-aware vision

FPI-Det is best understood as a **large-scale, interaction-aware benchmark** for detecting phone use through face–phone relationships. Its defining contribution is the combination of **synchronized face/phone annotations**, **diverse real-world scenarios**, **challenging occlusion and scale variation**, and a dual evaluation regime spanning both detection and classification [2509.09111].

A common misconception would be to treat FPI-Det as merely a dataset for finding phones or faces. The benchmark’s formulation rejects that interpretation: it is designed around the premise that phone-use understanding depends on the **spatial and contextual relationship** between a face and a phone. The addition of geometry and relation information, including **pixel coordinates** and **pairwise distances**, reinforces this orientation toward interaction modeling.

Within the broader landscape of computer vision benchmarks, FPI-Det represents a task-specific move from category recognition toward behavior-critical relational reasoning. It retains the operational simplicity of detection-style annotations and evaluation metrics, but directs them toward a higher-level question: whether the observed configuration constitutes genuine smartphone use. In that sense, its significance lies not in category breadth but in the formalization of a narrowly defined, safety-relevant interaction problem.

Source: https://www.emergentmind.com/topics/fpi-det