Pedestrian Attribute Recognition (PAR)

• Pedestrian Attribute Recognition (PAR) is a multi-label task that predicts semantic attributes like gender, age, and clothing from pedestrian images, serving as soft biometrics.
• Techniques range from global CNNs and part-based methods to transformer and vision-language models, addressing challenges such as occlusion, low resolution, and class imbalance.
• Benchmarks like PETA and PA-100K, along with evolving evaluation metrics, highlight PAR’s critical role in surveillance, re-identification, and robust multi-modal analysis.

Pedestrian Attribute Recognition (PAR) is the task of predicting a set of semantic descriptions of a person from an image, such as gender, age group, clothing type and color, carrying objects, hairstyle, and orientation. In the literature, PAR is framed as mining attributes from a given pedestrian image $\mathcal{I}$ with respect to a predefined attribute list $\mathcal{A}=\{a_1,a_2,\dots,a_L\}$, producing a multi-attribute output rather than a single class label. These attributes function as high-level semantic cues and soft biometrics, and PAR is routinely positioned as support for surveillance, pedestrian analysis, person re-identification, tracking, detection, retrieval, and related human-centered tasks (Wang et al., 2019, Jin et al., 2024).

1. Problem setting and technical characteristics

PAR is fundamentally a multi-label classification problem, because one pedestrian image is associated with multiple positive attributes simultaneously; at the same time, it is closely related to multi-task learning, because each attribute can be viewed as a related task with shared visual structure (Wang et al., 2019). A common formalization defines a predefined attribute set $$\mathcal{A}_{set}=\{\mathcal{A}_1,\mathcal{A}_2,\ldots,\mathcal{A}_M\}$$ and aims to predict attributes $\mathcal{A}_i$ from a pedestrian image $\mathcal{I}$, typically through a visual backbone and multi-category classifiers, or through a joint visual-textual model (Jin et al., 2023).

The task is difficult because the evidence for many attributes is spatially localized and often weak. The literature repeatedly emphasizes viewpoint changes, occlusion, severe class imbalance, low resolution, illumination variation, motion blur, background clutter, and long-tailed label distributions as central obstacles. In surveillance imagery, some attributes are rare, some are ambiguous, some are hidden by other pedestrians or objects, and some are affected by low-quality crops or changing viewpoints, so PAR requires both robust global context and fine-grained local reasoning (Wang et al., 2019, Fan et al., 2023).

A recurring design assumption is that many attributes correlate with specific body regions. Head-related attributes are associated with the face or hair region, upper-body attributes with torso regions, lower-body attributes with trousers or skirts, and footwear attributes with the feet. This assumption motivates part-based methods, region-aware prompting, part-aware attention, and local token or query mechanisms. A plausible implication is that PAR is not merely a label-space problem; it is also a localization problem with multi-label supervision (Jha et al., 2023, Wang et al., 2023).

The task boundary has also widened. Beyond closed-set recognition against a fixed label inventory, later work defines open-attribute recognition as an image-text search problem, and more recent unified frameworks explicitly target heterogeneous modalities including RGB images, video sequences, and event streams (Zhang et al., 2023, Xu et al., 5 Mar 2026).

2. Benchmarks, protocols, and evaluation

The classical PAR benchmark ecosystem includes PETA, RAP, RAP-2.0, PA-100K, WIDER, Market-1501 Attribute, DukeMTMC Attribute, PARSE-27K, APiS, HAT, BAP, CRP, CAD, MARS-Attributes, DukeMTMC-VID-Attributes, UAV-Human, and UPAR (Wang et al., 2019). Several datasets have become de facto reference points for model comparison.

Dataset	Scale and labels	Notable property
PETA	19,000 images; 61 binary and 4 multi-class attributes	Evaluation often focuses on 35 common attributes
RAP / RAPv2	41,585 images / 84,928 images; 69 binary attributes	RAP annotates viewpoints, occlusions, and body parts
PA-100K	100,000 images; 26 binary attributes	Outdoor surveillance benchmark
WIDER	13,789 images; 57,524 pedestrians; 14 attributes	Crowded pedestrian scenes
MSP60K	60,122 images; 57 attributes; eight scenarios	Random and cross-domain split protocols

The benchmark design has become a major research topic in its own right. MSP60K was introduced as a large-scale, cross-domain PAR benchmark with 60,122 pedestrian images, over 5,000 unique person IDs, 57 attributes, and eight scenarios. It includes varying camera viewpoints and distances, image resolutions ranging from $30\times80$ to $2005\times3008$, and synthetic degradation: about one-third of the images in each subset are intentionally degraded with blur, occlusion, lighting changes, noise, jpeg compression, and related destructive operations. Its random split uses 30,298 training images, 6,002 validation images, and 23,822 test images, while its cross-domain split trains on five scenarios with 34,128 images and tests on three unseen outdoor scenarios with 24,994 images (Jin et al., 2024).

Evaluation conventions distinguish label-based and example-based criteria. A standard label-based metric is mean accuracy: $$\mathit{mA} = \frac{1}{2L}\sum_{i=1}^{L}\left(\frac{TP_i}{P_i}+\frac{TN_i}{N_i}\right),$$ which averages positive-class and negative-class accuracies across attributes. Example-based metrics evaluate the predicted set per pedestrian image: $$Acc_{exam}=\frac{1}{N}\sum_{i=1}^{N}\frac{|Y_i\cap f(x_i)|}{|Y_i\cup f(x_i)|},$$

$$Prec_{exam}=\frac{1}{N}\sum_{i=1}^{N}\frac{|Y_i\cap f(x_i)|}{|f(x_i)|}, \qquad Rec_{exam}=\frac{1}{N}\sum_{i=1}^{N}\frac{|Y_i\cap f(x_i)|}{|Y_i|},$$

$\mathcal{A}=\{a_1,a_2,\dots,a_L\}$0

Later work continues to report mA, accuracy, precision, recall, and F1, and explicitly emphasizes mA because it is more informative under label imbalance (Wang et al., 2019, Xu et al., 5 Mar 2026).

Cross-domain evaluation has exposed a persistent misconception: high random-split performance does not necessarily indicate deployment robustness. On MSP60K, many prior methods degrade substantially under the cross-domain protocol; for example, HAP drops by 18.22 mA points, 25.53 Acc points, and 19.20 F1 points when moving from random to cross-domain evaluation. Zero-shot MiniGPT-4 also performs poorly on MSP60K, indicating that generic multimodal generation alone is not sufficient for reliable PAR (Jin et al., 2024).

3. Architectural evolution and learning paradigms

Early deep PAR systems were commonly organized as global-image-based models, part-based models, attention-based models, sequential prediction models, graphical-model-based systems, and loss-function-centric methods. Representative families reviewed in the survey include ACN, DeepSAR, DeepMAR, MTCNN, poselet-based pipelines, RAD, PANDA, MLCNN, HydraPlus-Net, VeSPA, DIAA, CNN-RNN, JRL, GRL, JCM, RCRA, A-AOG, and VSGR (Wang et al., 2019). Across these families, the central trade-off was between global context and localized attribute evidence.

Transformer-based PAR strengthened the global-context side of this trade-off. PARFormer is a pure transformer-based multi-task network with four modules: feature extraction, feature processing, viewpoint perception, and attribute recognition. It introduces Batch Random Mask (BRM), Multi-Attribute Center Loss (MACL), and Multi-View Contrastive Loss (MVCL), and uses Asymmetric Loss to mitigate positive-negative imbalance. In its backbone analysis on PA100K, transformer models outperform CNNs, and Swin-B is the strongest among the compared backbones (Fan et al., 2023).

A different line reinterprets PAR as structured generation rather than independent attribute classification. SequencePAR casts PAR as a sequence generation problem, uses a pre-trained CLIP model for image and text encoding, and generates attributes auto-regressively with a Transformer decoder and masked multi-head attention. On PETA it reports 84.92 Accuracy, 90.44 Precision, 90.73 Recall, and 90.46 F1; on PA100K it reports 83.94 / 90.38 / 90.23 / 90.10; and on RAPv2 it reports 70.14 / 81.37 / 81.22 / 81.10. The decoder-depth ablation selects 6 layers as the default, and greedy search performs as well as or better than beam search on PETA (Jin et al., 2023).

Reinforcement-learning-based formulations represent another departure from standard discriminative heads. Rein-PAR formulates PAR as a Markov decision process $\mathcal{A}=\{a_1,a_2,\dots,a_L\}$1, uses state $\mathcal{A}=\{a_1,a_2,\dots,a_L\}$2, binary action space $\mathcal{A}=\{a_1,a_2,\dots,a_L\}$3, Attribute Grouping Strategy (AGS), and Group Optimization Reward (GOR), and trains attribute-group agents with Deep Q-learning. It reports strong mA and recall on PETA, RAP, and PA100K, while also making explicit that reward shaping can favor recall at the cost of precision (Ji et al., 2022).

Video-based PAR introduces temporal aggregation and occlusion handling. “Robust Pedestrian Attribute Recognition Using Group Sparsity for Occlusion Videos” replaces softmax-based temporal attention with a sigmoid-based temporal attention module (STAM) and adds group sparsity over correlated attributes. On occlusion samples from MARS, Chen et al. report 66.39 acc / 55.67 F1, while the proposed method reports 71.94 acc / 61.88 F1; on DukeMTMC-VideoReID occlusion samples, the method improves F1 from 69.03 to 70.21 (Lee et al., 2021).

4. Vision-language, open-vocabulary, and unified multimodal PAR

A major recent shift is the treatment of PAR as a vision-language alignment problem rather than a purely image-only recognition task. PromptPAR expands attribute phrases into sentences, uses CLIP as the backbone for visual and textual feature embedding, fuses them with a multi-modal Transformer, and optimizes the system with region-aware prompt tuning while freezing the pre-trained vision-LLM and the fusion module. It reports that only 0.75% learnable parameters are adjusted compared with the fine-tuning strategy, and with ViT-L/14 prompt tuning uses 3.27M trainable parameters versus 435.93M for the VTB fine-tuning configuration. PromptPAR achieves 88.76 mA / 82.84 Acc / 89.18 F1 on PETA and 87.47 mA / 83.78 Acc / 90.15 F1 on PA100K, and also improves zero-shot PETA-ZS and RAP-ZS performance (Wang et al., 2023).

Open-vocabulary PAR extends this line by removing the fixed classifier head assumption. POAR formulates pedestrian open-attribute recognition as an image-text search problem in a shared embedding space. Its TEMS framework uses a Transformer-based image encoder with attribute tokens, masking to prevent attribute-token shortcut learning, “Masking the Irrelevant Patches” for part-level focus, and a Many-To-Many Contrastive loss with masked tokens. Grouped Knowledge Distillation from CLIP is added to improve unseen-attribute transfer. This reformulation enables unseen attributes to be scored through natural-language descriptions rather than through retraining a closed-set output layer (Zhang et al., 2023).

FOCUS and ViTA-PAR refine attribute-conditioned retrieval and prompting. FOCUS introduces Multi-Granularity Mix Tokens (MGMT), Attribute-guided Visual Feature Extraction (AVFE), and Region-Aware Contrastive Learning (RACL), and explicitly claims stronger open-domain and unseen-attribute generalization than POAR. ViTA-PAR introduces visual attribute prompts, person and attribute context prompting, and visual-textual attribute alignment, but uses only the image branch during inference; on PA100K, ViT-L/14 ViTA-PAR reports 87.82 mA / 90.93 F1 and achieves 50.1853 ms inference on PA-100K, compared with 113.1119 ms for PromptPAR and 80.1845 ms for VTB (An et al., 28 Jun 2025, Park et al., 2 Jun 2025).

LLM-augmented and knowledge-guided frameworks push multimodal interaction further. LLM-PAR combines a ViT backbone, LoRA tuning, a Multi-Embedding Query Transformer (MEQ-Former), Attributes Group Features Aggregate (AGFA), a Q-Former bridge, and a Vicuna-7B language branch used for caption generation, auxiliary attribute prediction, and ensemble aggregation. On MSP60K it reports 80.13 mA / 78.71 Acc / 86.94 F1 under the random split and 66.29 mA / 58.11 Acc / 72.05 F1 under the cross-domain split; on PETA it reports 92.25 mA / 84.59 Acc / 90.39 F1 (Jin et al., 2024). KGPAR instead constructs a multimodal knowledge graph and uses local and global hypergraphs with UniGNN encoders to model attribute co-occurrence, patch-attribute relations, and image-attribute alignment, reporting 88.50 mA / 89.43 F1 on PETA and 79.03 mA / 85.69 F1 on MSP60K random split (Wang et al., 26 Sep 2025).

Recent frameworks also target modality unification, uncertainty, and stronger pre-trained VLM backbones. UniPAR uses a unified data scheduling strategy, a phased fusion encoder with late deep fusion, and dynamic classification heads so that one Transformer can jointly process RGB images, video sequences, and event streams across MSP60K, DukeMTMC, and EventPAR. UAPAR adds Evidential Deep Learning to a CLIP-based architecture, with a Region-Aware Evidence Reasoning module and uncertainty-guided dual-stage curriculum learning, reporting 88.48 mA / 90.46 F1 on PA100K and 90.74 mA / 89.06 F1 on PETA. VLM-PAR uses frozen SigLIP 2 multilingual encoders and a compact attribute-specific cross-attention fusion block, and reports 92.88 mA / 92.32 F1 on PA-100K and 93.52 mA / 92.64 F1 on PETA (Xu et al., 5 Mar 2026, Lou et al., 29 Apr 2026, Sellam et al., 22 Dec 2025).

5. Data-centric optimization, efficiency, and robustness

Class imbalance remains one of the most persistent PAR bottlenecks. “An Empirical Analysis of Optimization Dynamics and Sparsity Boundaries in Large-Scale Pedestrian Attribute Recognition” studies a composite 109,000-image corpus formed by merging PETA and PA-100K and identifies the “majority negative class cheating trap,” in which standard BCE suppresses rare traits. On a ResNet-18 backbone, a calibrated Multi-Label Focal Loss with $\mathcal{A}=\{a_1,a_2,\dots,a_L\}$4 and $\mathcal{A}=\{a_1,a_2,\dots,a_L\}$5 achieves Macro F1-score 62.32%, essentially matching the BCE baseline 62.35%, while yielding better hard-example mining and medium- and rare-frequency behavior. The same work defines the “Sparsity Wall” as a boundary around positive sample fractions below 0.1%, where global loss reweighting becomes ineffective and instance-level intervention is required (Mir, 9 Jun 2026).

A data-centric response is synthetic augmentation. One diffusion-based study uses image-to-image generation with Stable Diffusion v1.4, compares prompt designs and generation settings, and reports that prompt alignment and image properties are critical factors in image generation quality; on RAPzs, the best setting improves mA from 71.98 to 75.23, described as a 4.5% relative improvement (Ayuso-Albizu et al., 2 Sep 2025). Another data-centric study ranks “weak” attributes in RAPv1, RAPv2, and RAPzs, generates prompt-driven synthetic pedestrians with Stable Diffusion 3 and post-generation degradation, and integrates them with a modified Binary Cross Entropy Augmented loss. It reports, for example, hs-BaldHead improvement on RAPv1 from 47.46 to 63.49 F1 and overall dataset-level F1 gains from 79.95 to 80.60 on RAPv1, from 78.68 to 79.77 on RAPv2, and from 77.66 to 78.62 on RAPzs, without architectural changes (Alonso et al., 2 Sep 2025).

Efficiency-oriented work addresses deployability on embedded devices. LWALM compresses the Attribute Localization Model (ALM) by combining CPD-EPC for $\mathcal{A}=\{a_1,a_2,\dots,a_L\}$6 convolutions and truncated SVD for $\mathcal{A}=\{a_1,a_2,\dots,a_L\}$7 convolutions. On PETA, ALM has 17.1M parameters and 1.95 GFLOPs, while LWALM with $\mathcal{A}=\{a_1,a_2,\dots,a_L\}$8 has 7.0M parameters and 0.905 GFLOPs; on PA-100K, ALM has 14.02M parameters and 1.95 GFLOPs, while LWALM with $\mathcal{A}=\{a_1,a_2,\dots,a_L\}$9 has 1.4M parameters and 0.78 GFLOPs. The paper states that the compressed models incur less than 2% accuracy drop overall (Jha et al., 2023).

Robustness now includes adversarial robustness. ASL-PAR is presented as the first adversarial attack and defense framework for PAR, attacking a CLIP-based PromptPAR model through adversarial semantic and label perturbation. Under attack on PETA, PromptPAR drops from 88.76 mA / 82.84 Acc / 89.18 F1 to 48.25 mA / 29.02 Acc / 43.29 F1; on PA100K it drops from 87.47 mA / 83.78 Acc / 90.15 F1 to 50.85 mA / 36.34 Acc / 51.42 F1. A semantic offset defense with input filtering and text-side prompt adjustment partly restores performance, reaching 85.82 mA on PETA in the reported defense ablation (Kong et al., 29 May 2025).

6. Contemporary issues and research directions

Current PAR research converges on several unresolved issues. The first is domain generalization. MSP60K explicitly argues that existing datasets neglect different domains and that recent public PAR benchmarks had approached saturation; UniPAR likewise argues that real deployments face discrepancies in modality, attribute definitions, and environmental scenarios. This suggests that future PAR progress will be evaluated less by single-dataset random-split gains and more by cross-domain, multi-dataset, and multi-modal robustness (Jin et al., 2024, Xu et al., 5 Mar 2026).

The second is the tension between closed-set supervision and open-ended semantics. Closed-set multi-label classifiers remain strong on standard benchmarks, but POAR, FOCUS, and related vision-LLMs show that unseen attributes and cross-dataset transfer demand query-driven or retrieval-based formulations. A plausible implication is that PAR is moving toward hybrid systems in which fixed attribute heads coexist with textual query interfaces and attribute-conditioned feature extraction (Zhang et al., 2023, An et al., 28 Jun 2025).

The third is reliability under imperfect evidence. UAPAR shows that epistemic uncertainty can be estimated at the attribute level and can be used for selective prediction and curriculum learning, while ASL-PAR shows that multimodal PAR systems remain vulnerable to adversarial semantic interference. Together these results make robustness a broader notion encompassing uncertainty estimation, attack resistance, and calibration rather than only raw recognition accuracy (Lou et al., 29 Apr 2026, Kong et al., 29 May 2025).

The fourth is the persistence of data bottlenecks. Two independent data-centric studies argue that performance is constrained by under-represented or weak attributes and by insufficiently challenging data distributions, and the optimization study on the merged 109k corpus shows that some classes remain beyond the reach of global loss reweighting alone below the Sparsity Wall. This suggests that future PAR progress will continue to depend on better benchmark design, synthetic augmentation, attribute-tailored data curation, and selective intervention for extremely sparse attributes (Alonso et al., 2 Sep 2025, Jin et al., 2024, Mir, 9 Jun 2026).

Finally, localization and scene complexity remain open. SequencePAR notes failure cases in crowded scenes with multiple persons, interfering objects, and ambiguous pedestrian localization, and proposes future pedestrian localization modules to reduce background interference. Group-sparse temporal attention, region-aware prompts, hypergraph reasoning, and spatial prior masks all point to the same unresolved fact: accurate attribute prediction still depends heavily on grounding the correct body evidence under clutter, occlusion, and variable viewpoint (Jin et al., 2023, Lee et al., 2021, Lou et al., 29 Apr 2026).