Error Categories for Pedestrian Detection

• The paper defines the error taxonomy in pedestrian detection by categorizing failures such as false positives and negatives, emphasizing localization, occlusion, and annotation challenges.
• It details methods to quantify errors using segmentation, pose analysis, and specialized metrics like FLAMR and GDPI to assess model performance under varied urban conditions.
• The insights guide model development and benchmarking by highlighting targeted improvements for safety-critical scenarios in autonomous driving systems.

Pedestrian detection—the task of localizing and identifying human figures in urban environments—constitutes a foundational capability for autonomous vehicles and advanced driver-assistance systems. Despite remarkable advances in deep learning, detection failures under complex, realistic visual conditions continue to present major safety challenges. Error categorization is central to benchmarking, diagnosing, and improving pedestrian detection models. Recent literature has established a taxonomy of error types, incorporating occlusion, pose, localization, background confusion, annotation ambiguity, and fairness across subgroups. These categories underpin robust evaluation protocols and inform both methodological choices and safety assurances in automated systems.

1. Fundamental Error Categories in Pedestrian Detection

Comprehensive analyses decompose pedestrian detection errors into discrete, non-overlapping categories, enabling fine-grained attribution of model failures. The principal axes along which errors are organized include:

• False Positives (FP): Model detections not corresponding to any ground-truth (GT) pedestrian instance.
• False Negatives (FN): Missed GT pedestrians for which the model produces no detection above a designated overlap or confidence threshold.

Within these broad outcomes, subcategories have emerged from both manual clusterings and objective, data-driven rules:

False Positives

1. Localization Errors: Detections close in spatial alignment to a true GT, but incorrect in box size or placement (e.g., double detections, body-part detections, oversized boxes).
2. Background Confusion: Detections on scene elements visually similar to pedestrians, such as vertical structures, traffic lights, car parts, and dense foliage.
3. Annotation Errors: Detections caused by GT data issues (e.g., missing annotations or ambiguous/ambivalent labels).
4. Ghost Detections: Spurious detections not spatially correlated to any person; often the most disruptive errors in safety-critical operation.

False Negatives

1. Small Scale: Misses due to a pedestrian’s size falling below a threshold (e.g., height < 50 pixels).
2. Extreme Viewpoint or Pose: Failures for non-standard in-plane rotations or unusual stances (e.g., side-views, cyclists).
3. Heavy Occlusion: More than 35% of the GT region obscured by objects or other people.
4. GT Annotation Omissions: True pedestrians missing or mislabeled in GT.
5. Other/Edge Cases: Rare pose, blur, or extreme cases not captured by standard taxonomy.

In contemporary work, these are further formalized using either rule-based logic or instance/semantic segmentation masks, supporting systematic metrics and fair cross-model benchmarks (Zhang et al., 2016, Feifel et al., 13 Nov 2025).

2. Occlusion-Specific Error Taxonomy and Quantification

Occlusion presents the dominant modality-specific challenge in pedestrian detection and has recently been subjected to rigorous objective quantification based on physical visibility and segmentation priors.

Occlusion Types

• Partial Occlusion: Non-boundary scene objects obstruct part of the pedestrian.
• Self-Occlusion: The pedestrian’s own limbs or body parts occlude one another.
• Truncation: The pedestrian is partially out-of-frame due to image boundaries.
• Inter-Occlusion (Crowd Occlusion): Multiple pedestrians overlap, occluding each other.

Objective Quantification

An objective method leverages 17-point keypoint detection (e.g., OpenPose, Mask R-CNN pose heads) and mesh triangulation:

• Define $A_{\text{full}}$ as the sum of areas of all triangles from the Delaunay triangulation of the canonical skeleton.
• Extract $A_{\text{visible}}$ as the sum of triangles with all vertices in visible keypoints.
• Compute the occlusion ratio:

$$\text{OcclusionRatio} = 1 - \frac{A_{\text{visible}}}{A_{\text{full}}}$$

• Discretize occlusion levels:
  • Unoccluded: $[0.00, 0.05)$
  • Lightly occluded: $[0.05, 0.35)$
  • Heavily occluded: $[0.35, 1.00]$

Edge cases—self-occlusion, truncation at borders, and crowd occlusion—are systematically resolved via rules on keypoint visibility, border mirroring, and mask-based triangle clipping. Experimental validation shows mean absolute error of 0.07 in occlusion ratio and IoU of 0.82 with manual masks (Gilroy et al., 2022).

3. Segmentation-Based Error Decomposition and New Metrics

Instance and semantic segmentation have enabled even finer slices in error analysis, moving beyond aggregate measures toward operationally meaningful categories. A refined set of error categories, leveraging segmentation, includes:

False Negatives

Category	Definition
E (Environmental Occlusion)	Partially hidden by cars, poles, vegetation, or truncated at image borders
C (Crowd Occlusion)	Partially hidden by other pedestrians
A (Ambiguous Occlusion)	Both environmental and crowd occlusion present (threshold-based membership)
F (Foreground)	Fully visible, safety-critical (pixel height $\geq 190$ px)
B (Background)	Fully visible, distant (pixel height $< 190$ px)

False Positives

Category	Definition
S (Scale)	Correctly centered but size/scale mismatched to any GT
L (Localization)	Near a GT but IoU $< 0.5$ (fail to match for mAP, but overlap detected)
G (Ghost)	Not spatially coincident with any GT pedestrian (i.e., “ghost” detection)

Specific metrics (e.g., FLAMR—Filtered Log-Average Miss Rate) are computed for each category and allow direct performance comparison by error source:

$$\text{FLAMR}_P = \exp\left( \frac{1}{|C|} \sum_{c\in C} \log\,MR_P(c) \right)$$

where $MR_P(c)$ is the miss rate for subset $P$ at operating threshold $c$ (Feifel et al., 13 Nov 2025).

Operating-point selection is tailored for practical needs, e.g., choosing a confidence threshold ensuring no missed foreground pedestrian ($MR_F(c^*)=0$), and reporting spurious ghost detections per image ($GDPI(c^*)$) as a measure of nuisance error.

4. Pose- and Joint-Driven Error Categories and Fairness Metrics

Error taxonomies increasingly encompass human pose and per-joint occlusion, supporting the analysis of bias across different pedestrian subpopulations (Khoshkdahan et al., 30 Sep 2025):

Pose-Driven Categories

• Leg Status: Aligned stance (parallel knees, straight legs) vs. non-aligned (flexed knees, walking).
• Elbow Status: Bent (≥90°) vs. straight (<90°).
• Body Orientation: Front, lateral (critical for crossing), or back.

Occlusion-Driven Categories

Per-joint visibility for: lower body (ankle, knee, hip), upper body (wrist, elbow, shoulder), and head (eye, ear, nose).

Fairness Assessment

• Equal Opportunity Difference (EOD): Difference in true positive rates (or inverted miss rates) between subpopulations.

$$EOD^\tau = TPR_{c_{\min}}^\tau - TPR_{c_{\max}}^\tau$$

• Cohen's $h$: Effect size independent of sample size:

$$h^\tau = 2\cdot (\arcsin(\sqrt{MR_{c_{\min}}^\tau}) - \arcsin(\sqrt{MR_{c_{\max}}^\tau}))$$

Lateral views and aligned-leg poses exhibit higher miss rates (e.g., legs aligned EOD = +6.8%, lower body occlusion EOD ≈ 21.1%), while Cascade R-CNN achieves low overall miss rate and minimal pose/occlusion bias.

5. Comparative Analysis and Empirical Findings

Benchmarks reveal substantial differences in detector robustness across error categories:

• For strong models evaluated on CityPersons:
  • Mean mAP drops sharply under increasing occlusion: from 0.72 (unoccluded) to 0.55 (lightly occluded) to 0.34 (heavily occluded) (Gilroy et al., 2022).
  • On EuroCity Persons Dense Pose, lowest overall miss-rate is 5.63% (Cascade R-CNN); the worst miss-rate is 41.87% (MGAN) (Khoshkdahan et al., 30 Sep 2025).
  • Ghost detections (spurious, not explainable by localization or scale) dominate at high recall operating points and are measured explicitly via GDPI.

Backbone comparison (e.g., BGC–HRNet-w32 vs FPN–ResNet-50) highlights that standard metrics like LAMR may obscure safety-critical failures (missed close pedestrians or ghosts). FLAMR$_F^\mathcal{G}$ better captures actual risk trade-offs (Feifel et al., 13 Nov 2025). Moderate correlation between overall LAMR and miss rate on background/distant pedestrians (r ≈ 0.9), but only weak correlation with foreground miss rate (r ≈ 0.5), suggests a misalignment between prevailing benchmarks and application-critical error modes.

6. Implications for Model Development and Benchmarking

The progression from aggregate metrics (e.g., mAP, LAMR) to structure-aware, segmentation-driven, and pose-sensitive error categorizations refines both safety claims and directions for model improvement:

• Leveraging semantic/instance segmentation is essential to partition error sources, especially for occlusion and crowding.
• Error metrics such as FLAMR, EOD, and GDPI should be reported for multiple categories—foreground pedestrians, heavily occluded, crowd occluded, and ghost errors—alongside overall rates.
• Detectors should explicitly address the largest error contributors identified by these categories, for example, improved pose sensitivity to lateral views, and feature design or augmentation for lower-body occlusions.
• Annotation quality, pose diversity, and consistent ground-truth labelling materially affect error distributions and must be controlled for credible evaluation (Zhang et al., 2016).

By adopting a taxonomy-driven approach, future research and deployment can better align system metrics with operational safety, urban context complexity, and fairness across demographic and situational axes.