ShopID10K: Dual Benchmark in Retail Vision

• ShopID10K is an overloaded benchmark encompassing a 2021 multimodal product categorization dataset and a 2025 generalizable object re-identification dataset.
• It enables category prediction from image and text tuples, employing standard classification metrics to evaluate deep learning models in retail settings.
• It also supports instance-level retrieval under cross-category generalization, assessing model performance using query-gallery protocols and support pair evaluations.

ShopID10K is an overloaded designation in the arXiv literature on retail vision and e-commerce ML. In one usage, it denotes a retail-product categorization dataset for multimodal classification from image–text tuples, built from Amazon Review Data (2018) and organized around category prediction across 21 classes (Elayanithottathil et al., 2021). In another, it denotes a benchmark for generalizable object re-identification, built from Amazon customer review images and designed to evaluate retrieval of the same object instance in unseen categories using support pairs at test time (Huang et al., 28 Aug 2025). The shared theme is product understanding in realistic shopping environments, but the underlying supervision, task definition, and evaluation protocol differ substantially.

1. Terminological scope and disambiguation

In arXiv usage, “ShopID10K” does not name a single canonical benchmark. The 2021 dataset paper presents ShopID10K as a retail products dataset for predicting category from a product image and description, whereas the 2025 re-identification paper introduces ShopID10K as a 10K-instance benchmark for unseen-category object ReID (Elayanithottathil et al., 2021). A recurrent source of confusion is that both are retail-domain datasets derived from Amazon-related sources, yet one is category-supervised and multimodal, while the other is instance-supervised and retrieval-oriented.

A second source of confusion is the proximity of the name to “Products-10K,” which is a different benchmark for SKU-level product recognition. Products-10K contains 10,000 fine-grained SKU-level products and nearly 150,000 images, includes both in-shop photos and customer images, and emphasizes human verification and low label noise rather than the multimodal categorization or generalizable ReID formulations associated with ShopID10K (Bai et al., 2020). This suggests that the numerical suffix “10K” should not be taken as sufficient evidence of task equivalence.

Usage	Primary task	Key scale
ShopID10K in (Elayanithottathil et al., 2021)	Predict product category from image and description	around 48,000 products, 21 categories
ShopID10K in (Huang et al., 28 Aug 2025)	Retrieve the same instance in a novel category	10K instances, 45K images, 34 categories
Products-10K in (Bai et al., 2020)	Fine-grained SKU-level product recognition	10,000 products, nearly 150,000 images

2. ShopID10K as a multimodal retail-product categorization dataset

The 2021 ShopID10K formulation is explicitly a benchmark for multimodal classification. Each product has a color image resized to 100×100, a product title or description text, and a category label, and the task is to predict the category of a retail product from its image and description (Elayanithottathil et al., 2021). The paper describes the dataset as consisting of roughly 48,000 products spanning 21 categories, with 42,000 training samples and around 6,000 test samples. It also states that the samples are equally distributed among the 21 categories.

The dataset was created to “boost the evaluation of machine learning methods” for prediction from tuples of images and descriptions. In practical terms, the paper positions it for recommendation systems, product search engines, and internal supply logistics, where identifying similar products or assigning the correct category is operationally important. The formulation is therefore not instance identification in the ReID sense; it is supervised category prediction over a fixed taxonomy.

The construction process is described at a relatively high level. Products are sourced from the Amazon Review Data (2018) collection, and the retained fields are organized through a train.csv schema with ImgId, title, description, and categories (Elayanithottathil et al., 2021). The paper does not describe a new manual annotation campaign; the category labels are presented as coming from the Amazon product metadata. The category list has a slight formatting issue in the paper, but the intended taxonomy is stated to consist of 21 classes and includes examples such as Electronics, Beauty, Arts/Crafts/Sewing, Pet Supplies, and Clothing/Shoes/Jewelry.

A notable property of this formulation is that it treats image and language as co-equal modalities. The paper discusses image classification with a ConvNet, text classification with an LSTM and embedding layer, and a combined model formed by concatenating the two streams. This makes ShopID10K, in this usage, a retail multimodal classification benchmark rather than a pure visual-recognition dataset.

3. ShopID10K as a benchmark for generalizable object re-identification

The 2025 ShopID10K formulation belongs to the object ReID literature and is intended to evaluate generalization to unseen object categories without parameter adaptation (Huang et al., 28 Aug 2025). The benchmark contains 10K instances, 45K images, and 34 categories, split into 7 seen categories and 27 unseen categories. The objects are daily-life items from e-commerce platforms, with examples including backpack, bicycle, bag, and shoes.

The dataset is built from Amazon customer review images. Its construction procedure is described in five steps: define 34 daily-life object categories, collect product images by category keyword search, assign instance identity by reviewer-product grouping, detect and filter images, and require at least 3 images per instance (Huang et al., 28 Aug 2025). The identity definition is central: all images uploaded by the same reviewer for a specific product are treated as one instance ID. This produces instance-level supervision without manual identity labeling in the conventional person-ReID sense.

The paper frames the task as follows. During training, a model sees a base dataset in known categories. At test time, for a novel category disjoint from the base categories, the model receives a support set of positive and negative pairs, where $y_{ij}=1$ indicates a same-instance pair and $y_{ij}=0$ a negative pair; the goal is to learn a feature extractor $\phi(\cdot|\mathcal{S})$ that retrieves the correct gallery instance (Huang et al., 28 Aug 2025). The paper also makes a deployment assumption that the category of the test object is known in advance, for example from an upstream detector, so comparisons are only made within the same category.

This version of ShopID10K is explicitly designed to be realistic rather than lab-controlled. The paper emphasizes natural variation in lighting, occlusion, pose or viewpoint, and background. Every instance has at least three images, enabling multi-view evaluation with a query–gallery protocol in which one image is chosen as query and the remaining images act as gallery candidates.

4. Supervision regimes and data semantics

The two ShopID10K usages differ most sharply in what constitutes a label. In the multimodal categorization dataset, the target variable is a category name, and the fields exposed to the model are ImgId, title, description, and category (Elayanithottathil et al., 2021). The core mapping is therefore from a product record to one of 21 product classes. The paper’s implicit objective can be summarized as $$f(\text{image}, \text{description}) \rightarrow \text{category}$$.

In the generalizable ReID dataset, the target is instance identity within category, and the supervision at test time is few-shot and pairwise rather than closed-set categorical (Huang et al., 28 Aug 2025). The benchmark is designed to test whether a method can use a support set of positive and negative pairs to retrieve the same instance in a novel category. This is a fundamentally different evaluation problem: the model is not predicting one of 21 fixed classes, but performing within-category identification under cross-category generalization constraints.

Their provenance also differs in ways that affect interpretation. The categorization dataset is derived from Amazon Review Data (2018) and does not describe manual relabeling, detailed filtering rules, or per-class counts beyond the statement of equal distribution (Elayanithottathil et al., 2021). By contrast, the ReID dataset defines identity through reviewer-product grouping and explicitly enforces a minimum of three images per instance, which is tailored to multi-view retrieval (Huang et al., 28 Aug 2025). A plausible implication is that the ReID benchmark prioritizes identity consistency under appearance variation, while the categorization benchmark prioritizes multimodal semantic separability across a balanced taxonomy.

5. Evaluation protocols and reported performance

The multimodal categorization version of ShopID10K is evaluated with standard classification metrics. The baseline combines a ConvNet for images and an LSTM for descriptions by concatenation, and the reported results are Accuracy 74.57, Validation Accuracy 72.85, Precision 88.54, Recall 75.60, F1 Score (Validation Data) 81.56, and F1 Score (Test Data) 71.56 (Elayanithottathil et al., 2021). The paper also states that the combined text-and-image model achieved around 71.81 F1 score on the test data, while the result table reports 71.56. The paper does not define whether precision, recall, and F1 are macro, micro, or weighted averages, and it does not formally specify the validation split.

The ReID version of ShopID10K is evaluated with standard identification metrics: mAP, Rank-1, and Rank-5 (Huang et al., 28 Aug 2025). The protocol uses query/gallery retrieval within the same category, random selection of base categories for training and novel categories for testing, repetition of the split five times, and reporting of averages. The paper also introduces a cross-domain evaluation in which models trained on PetFace, MVImgNet, or CUTE are tested directly on ShopID10K without domain-specific retraining.

On the main ShopID10K benchmark, the paper reports Triplet at 50.3 mAP and 63.1 Rank-1, Triplet+ at 54.8 mAP, 67.4 Rank-1, and 85.6 Rank-5, and VICP at 58.5 mAP, 68.4 Rank-1, and 87.5 Rank-5 (Huang et al., 28 Aug 2025). On an occluded subset generated using SAM-based processing, the reported numbers are DINOv2 at 25.7 mAP and 36.1 Rank-1, Triplet at 40.5 mAP and 52.4 Rank-1, Triplet+ at 46.8 mAP and 59.2 Rank-1, and VICP at 50.2 mAP and 61.4 Rank-1. The paper also compares against MAML on ShopID10K and reports VICP as best in that comparison as well.

Benchmark usage	Evaluation protocol	Representative reported results
Multimodal categorization	classification on train/test split	Accuracy 74.57; F1 Score (Test Data) 71.56
Generalizable ReID	within-category query/gallery retrieval	VICP: 58.5 mAP, 68.4 Rank-1, 87.5 Rank-5

6. Relation to adjacent retail-vision benchmarks and broader significance

ShopID10K sits within a broader family of retail benchmarks, but it occupies different niches depending on which usage is intended. Products-10K addresses fine-grained SKU-level recognition and emphasizes human-labeled data, in-shop plus customer images, graph-organized labels, and low dataset noise after expert verification (Bai et al., 2020). The Cdiscount large e-retailer dataset, by contrast, operates at a much larger industrial scale with 7,069,896 products, 12,371,293 images, and 5,270 categories at the finest level, but it is a hierarchical category-classification corpus with significant class imbalance, duplicate imagery, and about 10% bad-classification rate (Bellétoile, 2019). Relative to these, the categorization version of ShopID10K is smaller and explicitly multimodal, while the ReID version is centered on unseen-category transfer and identity-sensitive retrieval.

This comparison clarifies a common misconception: ShopID10K should not be assumed to be a SKU-level recognition dataset merely because it is retail-focused and numerically “10K.” SKU-level recognition with subtle local discriminative regions is more directly the problem setting of Products-10K (Bai et al., 2020). Likewise, ShopID10K should not be assumed to be purely visual. The 2021 dataset is defined around image–text tuples, whereas the 2025 dataset is visual but structured around support-pair prompting and within-category retrieval (Elayanithottathil et al., 2021).

Taken together, the two ShopID10K usages illuminate a broader shift in retail ML. One line of work treats product understanding as balanced multimodal categorization over catalog metadata and imagery; the other treats it as generalizable object ReID under realistic viewpoint, occlusion, lighting, and background variation (Huang et al., 28 Aug 2025). This suggests that “ShopID10K” functions less as a singular benchmark identity than as a label attached to two different research programs in e-commerce vision: multimodal category prediction and cross-category instance retrieval.