---
title: Open-Attribute Recognition for Person Retrieval
url: https://www.emergentmind.com/topics/open-attribute-recognition-for-person-retrieval-oapr
type: topic
---

# Open-Attribute Recognition for Person Retrieval

Open-Attribute Recognition for Person Retrieval (OAPR) is a person retrieval setting in which the query is specified by semantic attributes and the retrieval system must remain effective even when some queried attributes were not seen during training. In the 2025 formulation, OAPR is cast as a text-to-image retrieval problem over a universe of attribute classes partitioned into disjoint base and novel subsets, with evaluation on queries that may draw from both subsets [2508.01389]. The task generalizes earlier attribute-based person retrieval formulations, which typically assumed a closed-set attribute vocabulary shared between training and inference, and it is closely connected to pedestrian attribute recognition (PAR), cross-modal attribute-image matching, and unified recognition-retrieval benchmarks such as UPAR [2209.02522].

## 1. Definition and formal problem statement

In the OAPR formulation, the image pool is denoted by $\mathcal{X}=\{X_1,X_2,\dots,X_N\}$ and the attribute universe by $\mathcal{A}$, partitioned as $\mathcal{A}=\mathcal{A}_{base}\cup\mathcal{A}_{novel}$ with $\mathcal{A}_{base}\cap\mathcal{A}_{novel}=\emptyset$ [2508.01389]. A query $\pi=\{a_1,\dots,a_r\}$ is a subset of attributes drawn from $\mathcal{A}$, and the retrieval objective is to rank gallery images $X$ by a scoring function $s(\pi,X)$. One common choice given in the 2025 definition embeds each attribute $a$ into a text vector $f_{text}(a)$ and each image $X$ into a visual vector $f_{img}(X)$, then scores
$$
s(\pi,X)=\sum_{a\in\pi}\cosine(f_{text}(a),f_{vis}(X;\pi)).
$$
The top-$K$ images by descending score are returned [2508.01389].

This definition makes the open-set condition explicit. The model is trained using only attributes in $\mathcal{A}_{base}$ and evaluated on queries $\pi\subseteq(\mathcal{A}_{base}\cup\mathcal{A}_{novel})$ [2508.01389]. The principal departure from conventional attribute-based retrieval is therefore not merely multimodal matching, but generalization to novel attribute classes. The 2025 paper further motivates this by noting that benchmark attributes are often generic and shared across individuals, so closed-set attribute prediction alone can be insufficiently discriminative for retrieving a target person [2508.01389].

Earlier work had already framed attribute-to-image retrieval as a cross-modality person re-identification problem. In "Adversarial Attribute-Image Person Re-identification," the goal is to retrieve person images according to a given attribute description $a\in\{0,1\}^m$ by learning image and attribute mappings,
$$
f_I: I\mapsto z^I\in\mathcal{Z}\subset\mathbb{R}^d,\qquad
f_A: a\mapsto z^A\in\mathcal{Z}\subset\mathbb{R}^d,
$$
and ranking with cosine similarity in a shared concept space $\mathcal{Z}$ [1712.01493]. That formulation addresses attribute-image matching directly, but the 2025 OAPR task makes the novel-attribute regime central rather than incidental. This suggests that OAPR is best understood as an extension of attribute-based person retrieval from closed semantic inventories to evolving vocabularies.

## 2. Antecedents: from PAR and closed-set retrieval to open attributes

A major antecedent is UPAR, the Unified Person Attribute Recognition Dataset, introduced to enable systematic investigation of generalizable pedestrian attribute recognition and attribute-based person retrieval [2209.02522]. UPAR unifies PA100K, PETA, RAPv2, and Market1501 into a joint benchmark of 224,737 images with 40 binary attributes over 12 semantic categories, after harmonizing labels and adding 3.3 million new binary annotations [2209.02522]. The categories are Age, Gender, Hair-Length, Upper-Body Clothing Length, Upper-Body Clothing Color, Lower-Body Clothing Length, Lower-Body Clothing Color, Lower-Body Clothing Type, Accessory-Backpack, Accessory-Bag, Accessory-Glasses, and Accessory-Hat [2209.02522]. By giving each image a fully harmonized 40-attribute vector, UPAR enables cross-domain experiments that had previously been impossible [2209.02522].

UPAR’s significance for OAPR lies in two points. First, it established a unified attribute space for both recognition and retrieval. Second, it exposed the extent to which domain shift, pose variance, scale variation, and occlusion challenge attribute-centric systems [2209.02522]. The UPAR baseline uses a single fully connected layer over pooled image features to produce 40 independent sigmoid outputs,
$$
p_i=\sigma(Wx_i+b),\qquad p_i\in[0,1]^{40},
$$
and then uses the 40-dimensional confidence vector $p_i$ or its logit pre-sigmoid as the retrieval descriptor [2209.02522]. No separate metric-learning head is required in that baseline design.

Another line of antecedent work addressed the modality gap explicitly. The 2017 adversarial framework learns image concepts and attribute-derived image-analogous concepts in a common space, combining semantic-ID supervision, adversarial distribution alignment, and semantic consistency regularization [1712.01493]. CLEAR later argued that a sufficiently robust network for person attribute recognition can be adapted to facilitate better performance for retrieval, and it proposed a unified network coupling cross-transformers for recognition with adapter-based retrieval modules and pseudo-descriptions generated by a pre-trained language model [2403.06119].

Taken together, these systems define the immediate prehistory of OAPR. UPAR emphasized unified annotation and cross-domain generalization; the adversarial Re-ID formulation emphasized joint concept-space learning; CLEAR emphasized unified recognition and retrieval with language-aware adapters. The 2025 OAPR task reorients the problem around attribute novelty itself [2508.01389].

## 3. Dataset construction and benchmark design

The OAPR benchmark reconstructs four standard PAR datasets for open-attribute recognition by repartitioning their attribute vocabularies into base and novel subsets [2508.01389]. The procedure begins with a standard PAR dataset having attribute set $\mathcal{A}_{full}$. Its attributes are grouped into $C$ clusters via agglomerative clustering on text embeddings, and within each cluster $c$ a fraction $\alpha$, specified as $\alpha=0.25$, is designated as novel:
$$
\forall c\in\{1,\dots,C\},\quad |\mathcal{A}_{base}\cap c|=\lfloor(1-\alpha)|c|\rfloor,\quad
|\mathcal{A}_{novel}\cap c|=\lceil\alpha |c|\rceil.
$$
Across all clusters, this yields $\mathcal{A}_{base}$ and $\mathcal{A}_{novel}$ [2508.01389].

Common preprocessing consists of filtering out ambiguous “Other” classes, verbalizing attributes into natural-language phrases, and clustering text embeddings before stratifying 25% novel attributes per cluster [2508.01389]. The explicit use of verbalization and clustering is important because it structures novelty semantically rather than assigning it randomly.

| Dataset | Attribute partition | Images |
|---|---|---|
| PA-100K | 26 verbalized attributes; 7 clusters; 18 base, 8 novel | 90k train / 10k test |
| PETA | 33 verbalized attributes; 6 clusters; 23 base, 10 novel | 11.4k train / 7.6k test |
| RAPv1 | 50 verbalized attributes; 8 clusters; 35 base, 16 novel | 33,268 train / 8,317 test |
| RAPv2 | 50 verbalized attributes; 8 clusters; 38 base, 16 novel | 67,943 train / 16,985 test |

These reconstructed datasets differ from UPAR’s harmonization procedure. UPAR creates a unified 40-attribute vector across source datasets by manual re-annotation of missing attributes, with each image judged independently and ambiguous colors mapped to “Other” or “Unknown” and discarded [2209.02522]. The OAPR reconstruction instead preserves the original datasets as separate benchmarks but redefines their attribute vocabularies into base and novel partitions [2508.01389]. A plausible implication is that UPAR primarily standardizes cross-domain evaluation, whereas OAPR standardizes open-vocabulary evaluation.

The 2025 paper reports that these reconstructions support comprehensive experiments demonstrating the necessity of the OAPR task and the effectiveness of its framework [2508.01389]. The benchmark design therefore serves both as an evaluation suite and as a formalization of what counts as novelty in attribute-based person retrieval.

## 4. Model architectures and representation strategies

The 2025 OAPR framework is built around a lightweight prompt-augmented CLIP backbone intended to learn generalizable body part representations and support open-vocabulary attributes [2508.01389]. Its vision encoder is a frozen CLIP ViT-B/16 with two modifications: every Q-K self-attention block in the final VCLS layer is replaced with a V-V self-attention block, and $N$ learnable “body prompt” tokens $Z=[z_1\dots z_N]$ are prepended to the image patches and masked from early layers to preserve pre-trained features [2508.01389]. The encoder outputs
$$
[f_{cls},f_{img},f_{body}] = VisEnc([X,Z]),
$$
where $f_{cls}\in\mathbb{R}^C$ is the global [CLS] token, $f_{img}\in\mathbb{R}^{L\times C}$ are patch embeddings, and $f_{body}\in\mathbb{R}^{N\times C}$ are prompt-derived body features [2508.01389].

Its text side uses three sets of prompts passed through a frozen CLIP text encoder: body part labels with prompt-ensemble templates, background classes, and attribute names for the $|\mathcal{A}_{base}|$ training classes, each prefixed by a small learnable prompt of $T$ tokens [2508.01389]. A pseudo body signal is then distilled from raw patch features using those text features as probes. The combined text bank is
$$
F_{tb}=[f_{text}^{back};f_{text}^{body}]\in\mathbb{R}^{(N+M)\times C},
$$
the common direction is
$$
f_{text}^{common}=\frac{1}{N+M}\sum_{i=1}^{N+M}F_{tb}[i],
$$
and the patch-to-text projection is formed as
$$
W=f_{img}^{\top}\cdot(F_{tb}-f_{text}^{common})^{\top},\qquad
\hat{Y}=W\cdot f_{img}.
$$
The top $N$ rows of $\hat{Y}$ are selected to form pseudo body features $\hat{Y}_{body}$, which are distilled into the learnable vision prompt $f_{body}$ via an $L_2$ regression loss [2508.01389].

Attribute-related feature selection is then performed by a multi-head cross-attention block:
$$
Q=f_{text}^{att}W^Q,\qquad
K=f_{body}W^K,\qquad
V=f_{body}W^V,\qquad
P=softmax(QK^{\top}/\sqrt{d_k}),
$$
with
$$
f_{att\_img}=PV\in\mathbb{R}^{|\mathcal{A}_{base}|\times C}.
$$
This architecture operationalizes the claim that different attributes attend to different combinations of body parts [2508.01389].

Earlier systems instantiate related but distinct architectural principles. The 2017 adversarial method uses a ResNet-50 image branch and a four-layer fully connected attribute branch with dimensions $m\to128\to256\to512\to128$, producing image concepts $C^I$ and image-analogous concepts $C^A$ in a 128-dimensional concept space [1712.01493]. CLEAR uses a two-branch recognition backbone combining a ViT branch and a Swin-Transformer branch with Channel-Aware Self-Attention, cross-fusion modules SVCF and VSCF, and a frozen recognition model supplemented by three lightweight adapters for retrieval: $f_{ret}^{vis}$, $f_{ret}^{text}$, and $f_{ret}^{attr}$ [2403.06119]. Its query embedding is the concatenation $E^q=[E^h;E^s]$, where $E^h$ encodes hard binary attributes and $E^s$ encodes a pseudo-description derived from a pre-trained GPT encoder [2403.06119].

These architectures differ in how they address open-ended semantics. The OAPR framework relies on frozen CLIP encoders, body prompts, and attribute-body cross-attention [2508.01389]. CLEAR relies on a strong PAR backbone plus language-model-generated pseudo-descriptions [2403.06119]. The adversarial approach relies on a joint concept space shared by image and attribute modalities [1712.01493]. This suggests that open-attribute retrieval is being approached through three distinct representational strategies: prompt-conditioned vision-language priors, recognition-to-retrieval transfer, and explicit modality alignment.

## 5. Learning objectives and retrieval mechanisms

The 2025 OAPR framework optimizes three losses. The text-to-image contrastive loss is
$$
\mathcal{L}_{t2i}
= - \sum_{i=1}^{|\mathcal{A}_{base}|} \log
\frac{\exp(\langle f_{att\_img}^{(i)}, f_{text}^{att\,(i)}\rangle/\tau)}
{\exp(\langle f_{att\_img}^{(i)}, f_{text}^{att\,(i)}\rangle/\tau)+w_{neg}\sum_{j\neq i}\exp(\langle f_{att\_img}^{(i)}, f_{text}^{att\,(j)}\rangle/\tau)}.
$$
The Attribute-Body Association loss is
$$
\mathcal{L}_{aba}=\frac{1}{|\mathcal{A}_{base}|}\sum_{i=1}^{|\mathcal{A}_{base}|}\sum_{j=1}^{N}-y_{i,j}\log P_{i,j},
$$
where $y_{i,j}=1$ means attribute $i$ belongs to body part $j$. The overall objective is
$$
\mathcal{L}_{train}=\mathcal{L}_{t2i}+\lambda_{distill}\mathcal{L}_{distill}+\lambda_{aba}\mathcal{L}_{aba}.
$$
The paper attributes generalization to novel attributes to the frozen CLIP encoders, semantically aligned body-prompt features, and a cross-modal contrastive loss that does not depend on seeing a particular attribute at train time [2508.01389].

UPAR uses a different training formulation, jointly optimizing attribute classification, triplet discrimination in the attribute embedding, and regularization:
$$
L_{total}=L_{attr}+\lambda_{trip}L_{triplet}+L_{reg}.
$$
Its attribute classification loss is a weighted binary cross-entropy over $M=40$ attributes, and its triplet loss is defined over Euclidean distances in $\mathbb{R}^{40}$ using anchor, positive, and negative tuples [2209.02522]. Regularization includes weight decay via AdamW, dropout on the pooled feature, label smoothing for multi-label tasks, and data augmentation with Random Erasing, AugMix, and random crop/flip [2209.02522].

The adversarial attribute-image Re-ID model combines semantic-ID classification on image concepts, adversarial distribution alignment between image and attribute concepts, and semantic consistency for generated attribute concepts. Its total concept-generation loss is
$$
\mathcal{L}_{CG}=\mathcal{L}_{adv}+\mathcal{L}_{sc},\qquad
\mathcal{L}_{concept}=\mathcal{L}_{I}+\mathcal{L}_{CG}.
$$
The standard GAN objective is written as
$$
\min_{\theta_G}\max_{\theta_D}V(D,G)=
\mathbb{E}_{I\sim p_I}[\log D(f_I(I))]
+
\mathbb{E}_{a\sim p_A}[\log(1-D(f_A(a)))].
$$
The paper argues that adversarial alignment reduces modality gap more effectively than vanilla CCA or MMD alignment, while semantic consistency prevents mode collapse and preserves semantic IDs [1712.01493].

CLEAR separates recognition and retrieval training. It first trains the recognition network with binary cross-entropy,
$$
L_{BCE}(\hat{y},y)= -\sum_{i=1}^{N_{attr}}[y_i\log \hat{y}_i+(1-y_i)\log(1-\hat{y}_i)],
$$
then freezes it and trains only the adapters using an angular-margin retrieval loss,
$$
L_{ret}=\beta_1\cdot L_{MA}(E^v,E^h)+\beta_2\cdot L_{MA}(E^v,E^s),
$$
with $\beta_1=0.3$ and $\beta_2=0.7$ [2403.06119]. At inference, the gallery is ranked by $\cos(E^v,E^q)$ or $\varphi(E^v,E^q)$ [2403.06119].

Across these formulations, the retrieval mechanism varies substantially. UPAR retrieves by comparing attribute confidence vectors [2209.02522]. The adversarial model retrieves in a shared concept space [1712.01493]. CLEAR retrieves with fused hard and soft query embeddings [2403.06119]. OAPR retrieves by text-image matching mediated by body-aware features and open-vocabulary text encodings [2508.01389]. A plausible implication is that the field’s core methodological shift has been from closed fixed-label scoring toward semantically structured cross-modal alignment.

## 6. Evaluation protocols, empirical findings, and limitations

The OAPR benchmark evaluates retrieval with Label Precision@K (P@K-lbl), defined as the average fraction of query attributes correctly predicted among the top-$K$ retrieved images, and Instance Precision@K (P@K-ins), defined as the fraction of queries for which at least one of the top-$K$ images matches all query attributes simultaneously [2508.01389]. Baselines include Zero-shot CLIP; prompt-tuning methods CoOp, CoCoOp, and MaPLe; re-implemented closed-set PAR models VTB, PromptPAR, and ViTA-PAR; and the existing open-set PAR method POAR [2508.01389]. The paper states that across every dataset and both base/novel splits, the proposed OAPR framework achieves the highest P@1-lbl, P@5-lbl, P@1-ins, and P@5-ins. On PA-100K novel attributes, P@1-lbl rises from approximately 53.6% to 55.3%, and P@1-ins from approximately 27.7% to 29.5%; similar relative gains of 1–3% are reported across PETA, RAPv1, and RAPv2 [2508.01389]. Ablations on PETA compare four variants—only $\mathcal{L}_{t2i}$; plus $\mathcal{L}_{distill}$; plus $\mathcal{L}_{aba}$; and the full model—and report that each additional loss yields systematic gains on both base and novel metrics [2508.01389].

UPAR evaluates attribute recognition by Mean Accuracy (mA) and instance-based F1, and person retrieval by mean Average Precision (mAP) and Cumulative Matching Characteristic Rank-1 accuracy [2209.02522]. It defines two generalization protocols: cross-validation, which trains on exactly one sub-dataset and tests on the other three combined, and leave-one-out, which trains on three datasets and leaves the fourth for test [2209.02522]. The paper reports state-of-the-art performance in cross-domain and specialization settings on PA100K, PETA, RAPv2, Market1501-Attributes, and UPAR, and specifically notes that dropout, AdamW, label smoothing, and AugMix each yield +1–2% gains in mAP and +1–3% in mA, especially in the cross-validation protocol, while Random Erasing helps specialization but can hurt cross-domain generalization if used too aggressively [2209.02522].

The 2017 adversarial model evaluates retrieval by CMC at Rank-1, Rank-5, Rank-10 and mAP on Market-1501 Attribute, DukeMTMC-Attribute, and PETA [1712.01493]. Its full model reports Market $r1=40.3$, $mAP=20.7$; Duke $r1=46.6$, $mAP=15.7$; and PETA $r1=39.0$, $mAP=27.9$, outperforming several alternatives in Rank-1 and, on Market, mAP [1712.01493]. CLEAR reports both attribute recognition and attribute-based retrieval results on PETA, PA100K, Market-1501, RAPv2, and UPAR2024, including PA100K retrieval of 46.6 Rank-1 / 35.9 mAP and Market-1501 retrieval of 56.8 Rank-1 / 43.1 mAP [2403.06119].

The limitations described in the 2025 OAPR work are specific and technically consequential. Extremely rare or highly fine-grained novel attributes may be under-represented in CLIP pretraining; occlusions or severe viewpoint changes can corrupt pseudo-body features; and the current protocol restricts queries to two attributes, so longer conjunctive phrases may compound errors [2508.01389]. Earlier papers identify related constraints: the adversarial model notes that truly unseen attributes remain challenging and suggests zero-shot techniques or attribute word embeddings as possible extensions [1712.01493], while CLEAR notes that pseudo-description templates are handcrafted per dataset and that true zero-shot synonyms outside the template may still fail [2403.06119].

These limitations indicate that OAPR is not merely a matter of adding more labels. The open-attribute regime couples vocabulary expansion, part localization, multimodal alignment, and robustness to surveillance-specific degradation. Future directions stated in the OAPR paper include extension to video retrieval with temporal prompts, refinement of pseudo regions for small accessories via lightweight object detectors, movement from discrete attribute sets to free-form natural-language person descriptions, and adaptive hard-negative mining within the contrastive loss [2508.01389]. Such directions place OAPR at the intersection of PAR, open-vocabulary vision-language modeling, and person search under operationally evolving semantic requirements.

Source: https://www.emergentmind.com/topics/open-attribute-recognition-for-person-retrieval-oapr