Open-Attribute Recognition for Person Retrieval

• The paper introduces a novel OAPR framework that extends closed-set attribute retrieval to open-set scenarios using text-to-image matching and adaptive contrastive losses.
• It leverages a prompt-augmented CLIP backbone with learnable body tokens to enhance cross-modal alignment between semantic attributes and visual features.
• Empirical evaluations across multiple benchmarks show improved precision and robustness, demonstrating effective generalization to previously unseen attribute classes.

Open-Attribute Recognition for Person Retrieval (OAPR) is a person retrieval setting in which the query is specified by semantic attributes and the retrieval system must remain effective even when some queried attributes were not seen during training. In the 2025 formulation, OAPR is cast as a text-to-image retrieval problem over a universe of attribute classes partitioned into disjoint base and novel subsets, with evaluation on queries that may draw from both subsets (Park et al., 2 Aug 2025). The task generalizes earlier attribute-based person retrieval formulations, which typically assumed a closed-set attribute vocabulary shared between training and inference, and it is closely connected to pedestrian attribute recognition (PAR), cross-modal attribute-image matching, and unified recognition-retrieval benchmarks such as UPAR (Specker et al., 2022).

1. Definition and formal problem statement

In the OAPR formulation, the image pool is denoted by $\mathcal{X}=\{X_1,X_2,\dots,X_N\}$ and the attribute universe by $\mathcal{A}$, partitioned as $$\mathcal{A}=\mathcal{A}_{base}\cup\mathcal{A}_{novel}$$ with $$\mathcal{A}_{base}\cap\mathcal{A}_{novel}=\emptyset$$ (Park et al., 2 Aug 2025). A query $\pi=\{a_1,\dots,a_r\}$ is a subset of attributes drawn from $\mathcal{A}$, and the retrieval objective is to rank gallery images $X$ by a scoring function $s(\pi,X)$. One common choice given in the 2025 definition embeds each attribute $a$ into a text vector $f_{text}(a)$ and each image $\mathcal{A}$0 into a visual vector $\mathcal{A}$1, then scores

$\mathcal{A}$2

The top-$\mathcal{A}$3 images by descending score are returned (Park et al., 2 Aug 2025).

This definition makes the open-set condition explicit. The model is trained using only attributes in $\mathcal{A}$4 and evaluated on queries $\mathcal{A}$5 (Park et al., 2 Aug 2025). The principal departure from conventional attribute-based retrieval is therefore not merely multimodal matching, but generalization to novel attribute classes. The 2025 paper further motivates this by noting that benchmark attributes are often generic and shared across individuals, so closed-set attribute prediction alone can be insufficiently discriminative for retrieving a target person (Park et al., 2 Aug 2025).

Earlier work had already framed attribute-to-image retrieval as a cross-modality person re-identification problem. In "Adversarial Attribute-Image Person Re-identification," the goal is to retrieve person images according to a given attribute description $\mathcal{A}$6 by learning image and attribute mappings,

$\mathcal{A}$7

and ranking with cosine similarity in a shared concept space $\mathcal{A}$8 (Yin et al., 2017). That formulation addresses attribute-image matching directly, but the 2025 OAPR task makes the novel-attribute regime central rather than incidental. This suggests that OAPR is best understood as an extension of attribute-based person retrieval from closed semantic inventories to evolving vocabularies.

2. Antecedents: from PAR and closed-set retrieval to open attributes

A major antecedent is UPAR, the Unified Person Attribute Recognition Dataset, introduced to enable systematic investigation of generalizable pedestrian attribute recognition and attribute-based person retrieval (Specker et al., 2022). UPAR unifies PA100K, PETA, RAPv2, and Market1501 into a joint benchmark of 224,737 images with 40 binary attributes over 12 semantic categories, after harmonizing labels and adding 3.3 million new binary annotations (Specker et al., 2022). The categories are Age, Gender, Hair-Length, Upper-Body Clothing Length, Upper-Body Clothing Color, Lower-Body Clothing Length, Lower-Body Clothing Color, Lower-Body Clothing Type, Accessory-Backpack, Accessory-Bag, Accessory-Glasses, and Accessory-Hat (Specker et al., 2022). By giving each image a fully harmonized 40-attribute vector, UPAR enables cross-domain experiments that had previously been impossible (Specker et al., 2022).

UPAR’s significance for OAPR lies in two points. First, it established a unified attribute space for both recognition and retrieval. Second, it exposed the extent to which domain shift, pose variance, scale variation, and occlusion challenge attribute-centric systems (Specker et al., 2022). The UPAR baseline uses a single fully connected layer over pooled image features to produce 40 independent sigmoid outputs,

$\mathcal{A}$9

and then uses the 40-dimensional confidence vector $$\mathcal{A}=\mathcal{A}_{base}\cup\mathcal{A}_{novel}$$0 or its logit pre-sigmoid as the retrieval descriptor (Specker et al., 2022). No separate metric-learning head is required in that baseline design.

Another line of antecedent work addressed the modality gap explicitly. The 2017 adversarial framework learns image concepts and attribute-derived image-analogous concepts in a common space, combining semantic-ID supervision, adversarial distribution alignment, and semantic consistency regularization (Yin et al., 2017). CLEAR later argued that a sufficiently robust network for person attribute recognition can be adapted to facilitate better performance for retrieval, and it proposed a unified network coupling cross-transformers for recognition with adapter-based retrieval modules and pseudo-descriptions generated by a pre-trained LLM (Bui et al., 2024).

Taken together, these systems define the immediate prehistory of OAPR. UPAR emphasized unified annotation and cross-domain generalization; the adversarial Re-ID formulation emphasized joint concept-space learning; CLEAR emphasized unified recognition and retrieval with language-aware adapters. The 2025 OAPR task reorients the problem around attribute novelty itself (Park et al., 2 Aug 2025).

3. Dataset construction and benchmark design

The OAPR benchmark reconstructs four standard PAR datasets for open-attribute recognition by repartitioning their attribute vocabularies into base and novel subsets (Park et al., 2 Aug 2025). The procedure begins with a standard PAR dataset having attribute set $$\mathcal{A}=\mathcal{A}_{base}\cup\mathcal{A}_{novel}$$1. Its attributes are grouped into $$\mathcal{A}=\mathcal{A}_{base}\cup\mathcal{A}_{novel}$$2 clusters via agglomerative clustering on text embeddings, and within each cluster $$\mathcal{A}=\mathcal{A}_{base}\cup\mathcal{A}_{novel}$$3 a fraction $$\mathcal{A}=\mathcal{A}_{base}\cup\mathcal{A}_{novel}$$4, specified as $$\mathcal{A}=\mathcal{A}_{base}\cup\mathcal{A}_{novel}$$5, is designated as novel:

$$\mathcal{A}=\mathcal{A}_{base}\cup\mathcal{A}_{novel}$$6

Across all clusters, this yields $$\mathcal{A}=\mathcal{A}_{base}\cup\mathcal{A}_{novel}$$7 and $$\mathcal{A}=\mathcal{A}_{base}\cup\mathcal{A}_{novel}$$8 (Park et al., 2 Aug 2025).

Common preprocessing consists of filtering out ambiguous “Other” classes, verbalizing attributes into natural-language phrases, and clustering text embeddings before stratifying 25% novel attributes per cluster (Park et al., 2 Aug 2025). The explicit use of verbalization and clustering is important because it structures novelty semantically rather than assigning it randomly.

Dataset	Attribute partition	Images
PA-100K	26 verbalized attributes; 7 clusters; 18 base, 8 novel	90k train / 10k test
PETA	33 verbalized attributes; 6 clusters; 23 base, 10 novel	11.4k train / 7.6k test
RAPv1	50 verbalized attributes; 8 clusters; 35 base, 16 novel	33,268 train / 8,317 test
RAPv2	50 verbalized attributes; 8 clusters; 38 base, 16 novel	67,943 train / 16,985 test

These reconstructed datasets differ from UPAR’s harmonization procedure. UPAR creates a unified 40-attribute vector across source datasets by manual re-annotation of missing attributes, with each image judged independently and ambiguous colors mapped to “Other” or “Unknown” and discarded (Specker et al., 2022). The OAPR reconstruction instead preserves the original datasets as separate benchmarks but redefines their attribute vocabularies into base and novel partitions (Park et al., 2 Aug 2025). A plausible implication is that UPAR primarily standardizes cross-domain evaluation, whereas OAPR standardizes open-vocabulary evaluation.

The 2025 paper reports that these reconstructions support comprehensive experiments demonstrating the necessity of the OAPR task and the effectiveness of its framework (Park et al., 2 Aug 2025). The benchmark design therefore serves both as an evaluation suite and as a formalization of what counts as novelty in attribute-based person retrieval.

4. Model architectures and representation strategies

The 2025 OAPR framework is built around a lightweight prompt-augmented CLIP backbone intended to learn generalizable body part representations and support open-vocabulary attributes (Park et al., 2 Aug 2025). Its vision encoder is a frozen CLIP ViT-B/16 with two modifications: every Q-K self-attention block in the final VCLS layer is replaced with a V-V self-attention block, and $$\mathcal{A}=\mathcal{A}_{base}\cup\mathcal{A}_{novel}$$9 learnable “body prompt” tokens $$\mathcal{A}_{base}\cap\mathcal{A}_{novel}=\emptyset$$0 are prepended to the image patches and masked from early layers to preserve pre-trained features (Park et al., 2 Aug 2025). The encoder outputs

$$\mathcal{A}_{base}\cap\mathcal{A}_{novel}=\emptyset$$1

where $$\mathcal{A}_{base}\cap\mathcal{A}_{novel}=\emptyset$$2 is the global [CLS] token, $$\mathcal{A}_{base}\cap\mathcal{A}_{novel}=\emptyset$$3 are patch embeddings, and $$\mathcal{A}_{base}\cap\mathcal{A}_{novel}=\emptyset$$4 are prompt-derived body features (Park et al., 2 Aug 2025).

Its text side uses three sets of prompts passed through a frozen CLIP text encoder: body part labels with prompt-ensemble templates, background classes, and attribute names for the $$\mathcal{A}_{base}\cap\mathcal{A}_{novel}=\emptyset$$5 training classes, each prefixed by a small learnable prompt of $$\mathcal{A}_{base}\cap\mathcal{A}_{novel}=\emptyset$$6 tokens (Park et al., 2 Aug 2025). A pseudo body signal is then distilled from raw patch features using those text features as probes. The combined text bank is

$$\mathcal{A}_{base}\cap\mathcal{A}_{novel}=\emptyset$$7

the common direction is

$$\mathcal{A}_{base}\cap\mathcal{A}_{novel}=\emptyset$$8

and the patch-to-text projection is formed as

$$\mathcal{A}_{base}\cap\mathcal{A}_{novel}=\emptyset$$9

The top $\pi=\{a_1,\dots,a_r\}$0 rows of $\pi=\{a_1,\dots,a_r\}$1 are selected to form pseudo body features $\pi=\{a_1,\dots,a_r\}$2, which are distilled into the learnable vision prompt $\pi=\{a_1,\dots,a_r\}$3 via an $\pi=\{a_1,\dots,a_r\}$4 regression loss (Park et al., 2 Aug 2025).

Attribute-related feature selection is then performed by a multi-head cross-attention block:

$\pi=\{a_1,\dots,a_r\}$5

with

$\pi=\{a_1,\dots,a_r\}$6

This architecture operationalizes the claim that different attributes attend to different combinations of body parts (Park et al., 2 Aug 2025).

Earlier systems instantiate related but distinct architectural principles. The 2017 adversarial method uses a ResNet-50 image branch and a four-layer fully connected attribute branch with dimensions $\pi=\{a_1,\dots,a_r\}$7, producing image concepts $\pi=\{a_1,\dots,a_r\}$8 and image-analogous concepts $\pi=\{a_1,\dots,a_r\}$9 in a 128-dimensional concept space (Yin et al., 2017). CLEAR uses a two-branch recognition backbone combining a ViT branch and a Swin-Transformer branch with Channel-Aware Self-Attention, cross-fusion modules SVCF and VSCF, and a frozen recognition model supplemented by three lightweight adapters for retrieval: $\mathcal{A}$0, $\mathcal{A}$1, and $\mathcal{A}$2 (Bui et al., 2024). Its query embedding is the concatenation $\mathcal{A}$3, where $\mathcal{A}$4 encodes hard binary attributes and $\mathcal{A}$5 encodes a pseudo-description derived from a pre-trained GPT encoder (Bui et al., 2024).

These architectures differ in how they address open-ended semantics. The OAPR framework relies on frozen CLIP encoders, body prompts, and attribute-body cross-attention (Park et al., 2 Aug 2025). CLEAR relies on a strong PAR backbone plus language-model-generated pseudo-descriptions (Bui et al., 2024). The adversarial approach relies on a joint concept space shared by image and attribute modalities (Yin et al., 2017). This suggests that open-attribute retrieval is being approached through three distinct representational strategies: prompt-conditioned vision-language priors, recognition-to-retrieval transfer, and explicit modality alignment.

5. Learning objectives and retrieval mechanisms

The 2025 OAPR framework optimizes three losses. The text-to-image contrastive loss is

$\mathcal{A}$6

The Attribute-Body Association loss is

$\mathcal{A}$7

where $\mathcal{A}$8 means attribute $\mathcal{A}$9 belongs to body part $X$0. The overall objective is

$X$1

The paper attributes generalization to novel attributes to the frozen CLIP encoders, semantically aligned body-prompt features, and a cross-modal contrastive loss that does not depend on seeing a particular attribute at train time (Park et al., 2 Aug 2025).

UPAR uses a different training formulation, jointly optimizing attribute classification, triplet discrimination in the attribute embedding, and regularization:

$X$2

Its attribute classification loss is a weighted binary cross-entropy over $X$3 attributes, and its triplet loss is defined over Euclidean distances in $X$4 using anchor, positive, and negative tuples (Specker et al., 2022). Regularization includes weight decay via AdamW, dropout on the pooled feature, label smoothing for multi-label tasks, and data augmentation with Random Erasing, AugMix, and random crop/flip (Specker et al., 2022).

The adversarial attribute-image Re-ID model combines semantic-ID classification on image concepts, adversarial distribution alignment between image and attribute concepts, and semantic consistency for generated attribute concepts. Its total concept-generation loss is

$X$5

The standard GAN objective is written as

$X$6

The paper argues that adversarial alignment reduces modality gap more effectively than vanilla CCA or MMD alignment, while semantic consistency prevents mode collapse and preserves semantic IDs (Yin et al., 2017).

CLEAR separates recognition and retrieval training. It first trains the recognition network with binary cross-entropy,

$X$7

then freezes it and trains only the adapters using an angular-margin retrieval loss,

$X$8

with $X$9 and $s(\pi,X)$0 (Bui et al., 2024). At inference, the gallery is ranked by $s(\pi,X)$1 or $s(\pi,X)$2 (Bui et al., 2024).

Across these formulations, the retrieval mechanism varies substantially. UPAR retrieves by comparing attribute confidence vectors (Specker et al., 2022). The adversarial model retrieves in a shared concept space (Yin et al., 2017). CLEAR retrieves with fused hard and soft query embeddings (Bui et al., 2024). OAPR retrieves by text-image matching mediated by body-aware features and open-vocabulary text encodings (Park et al., 2 Aug 2025). A plausible implication is that the field’s core methodological shift has been from closed fixed-label scoring toward semantically structured cross-modal alignment.

6. Evaluation protocols, empirical findings, and limitations

The OAPR benchmark evaluates retrieval with Label Precision@K (P@K-lbl), defined as the average fraction of query attributes correctly predicted among the top-$s(\pi,X)$3 retrieved images, and Instance Precision@K (P@K-ins), defined as the fraction of queries for which at least one of the top-$s(\pi,X)$4 images matches all query attributes simultaneously (Park et al., 2 Aug 2025). Baselines include Zero-shot CLIP; prompt-tuning methods CoOp, CoCoOp, and MaPLe; re-implemented closed-set PAR models VTB, PromptPAR, and ViTA-PAR; and the existing open-set PAR method POAR (Park et al., 2 Aug 2025). The paper states that across every dataset and both base/novel splits, the proposed OAPR framework achieves the highest P@1-lbl, P@5-lbl, P@1-ins, and P@5-ins. On PA-100K novel attributes, P@1-lbl rises from approximately 53.6% to 55.3%, and P@1-ins from approximately 27.7% to 29.5%; similar relative gains of 1–3% are reported across PETA, RAPv1, and RAPv2 (Park et al., 2 Aug 2025). Ablations on PETA compare four variants—only $s(\pi,X)$5; plus $s(\pi,X)$6; plus $s(\pi,X)$7; and the full model—and report that each additional loss yields systematic gains on both base and novel metrics (Park et al., 2 Aug 2025).

UPAR evaluates attribute recognition by Mean Accuracy (mA) and instance-based F1, and person retrieval by mean Average Precision (mAP) and Cumulative Matching Characteristic Rank-1 accuracy (Specker et al., 2022). It defines two generalization protocols: cross-validation, which trains on exactly one sub-dataset and tests on the other three combined, and leave-one-out, which trains on three datasets and leaves the fourth for test (Specker et al., 2022). The paper reports state-of-the-art performance in cross-domain and specialization settings on PA100K, PETA, RAPv2, Market1501-Attributes, and UPAR, and specifically notes that dropout, AdamW, label smoothing, and AugMix each yield +1–2% gains in mAP and +1–3% in mA, especially in the cross-validation protocol, while Random Erasing helps specialization but can hurt cross-domain generalization if used too aggressively (Specker et al., 2022).

The 2017 adversarial model evaluates retrieval by CMC at Rank-1, Rank-5, Rank-10 and mAP on Market-1501 Attribute, DukeMTMC-Attribute, and PETA (Yin et al., 2017). Its full model reports Market $s(\pi,X)$8, $s(\pi,X)$9; Duke $a$0, $a$1; and PETA $a$2, $a$3, outperforming several alternatives in Rank-1 and, on Market, mAP (Yin et al., 2017). CLEAR reports both attribute recognition and attribute-based retrieval results on PETA, PA100K, Market-1501, RAPv2, and UPAR2024, including PA100K retrieval of 46.6 Rank-1 / 35.9 mAP and Market-1501 retrieval of 56.8 Rank-1 / 43.1 mAP (Bui et al., 2024).

The limitations described in the 2025 OAPR work are specific and technically consequential. Extremely rare or highly fine-grained novel attributes may be under-represented in CLIP pretraining; occlusions or severe viewpoint changes can corrupt pseudo-body features; and the current protocol restricts queries to two attributes, so longer conjunctive phrases may compound errors (Park et al., 2 Aug 2025). Earlier papers identify related constraints: the adversarial model notes that truly unseen attributes remain challenging and suggests zero-shot techniques or attribute word embeddings as possible extensions (Yin et al., 2017), while CLEAR notes that pseudo-description templates are handcrafted per dataset and that true zero-shot synonyms outside the template may still fail (Bui et al., 2024).

These limitations indicate that OAPR is not merely a matter of adding more labels. The open-attribute regime couples vocabulary expansion, part localization, multimodal alignment, and robustness to surveillance-specific degradation. Future directions stated in the OAPR paper include extension to video retrieval with temporal prompts, refinement of pseudo regions for small accessories via lightweight object detectors, movement from discrete attribute sets to free-form natural-language person descriptions, and adaptive hard-negative mining within the contrastive loss (Park et al., 2 Aug 2025). Such directions place OAPR at the intersection of PAR, open-vocabulary vision-language modeling, and person search under operationally evolving semantic requirements.