PartImageNet++: Large-Scale Part Annotations

• The paper introduces PIN++ with 100K images, 406K masks, and 3,308 part categories to enable ImageNet-scale part-based recognition.
• It employs strict, inclusion-aware annotation protocols using manual labeling and quality control to ensure precise part segmentation.
• Multi-scale part-supervised modeling with pseudo-label propagation significantly improves robustness, transferability, and classification accuracy.

PartImageNet++ (PIN++) is a large-scale dataset of object part annotations introduced to enable part-supervised visual modeling at ImageNet-1K scale. It provides detailed, high-quality part segmentation masks for all 1,000 ImageNet-1K categories, with 100 annotated images per category, totaling 100,000 images. The dataset contains 3,308 distinct part categories and 406,364 part masks, and was designed to overcome the scarcity and narrow scope of earlier part datasets while supporting training and evaluation on ImageNet-1K classification, part segmentation, object segmentation, and few-shot learning (Li et al., 4 Jan 2026, Li et al., 2024).

1. Motivation and position within part-based vision

PIN++ was introduced to address a specific bottleneck in part-aware recognition research: the absence of scalable, high-quality part annotations on the standard ImageNet-1K benchmark. The underlying motivation is that part-based perception aligns with human recognition-by-components theory, whereas prior deep recognition systems often relied on shortcut features and lacked a part-based inductive bias. Earlier part-based methods for robustness and transfer were therefore validated mainly on small, non-standard datasets, limiting both comparability and scale (Li et al., 2024).

In scope, PIN++ covers all 1,000 ImageNet-1K categories and is class-balanced, with 100 randomly selected training images per category. It includes creatures, artifacts, rigid objects, and non-rigid objects, and each image contains one foreground category. This full ImageNet-1K coverage distinguishes it from earlier resources that were either narrow in category diversity, small in scale, or designed for different settings.

Dataset	Object categories	Images
PartImageNet (PIN)	158	24K
Pascal-Part	20	19K
ADE20K-Part	80	12.6K
Cityscapes Panoptic-Parts	5	3.5K
PACO	75	76.7K
PIN++	1,000	100K

Relative to PartImageNet, PIN++ reuses 90 high-quality categories from PIN and reannotates 910 remaining ImageNet-1K categories under stricter guidelines, yielding finer masks and more precise part vocabularies. The dataset is described as unique in full ImageNet-1K coverage at consistent density and quality. A plausible implication is that PIN++ makes part-based modeling comparable to mainstream ImageNet-scale recognition in a way earlier datasets could not.

2. Part ontology, annotation schema, and dataset statistics

In PIN++, a “part” is defined as a semantically meaningful component of the object category that both contributes to the whole via composition and can be segmented spatially. The annotation rules require that the set of parts for a category cover the entire object, with the union of parts approximately equal to the object, and that parts be non-overlapping except when an explicit inclusion relation is defined. Overlapping parts are allowed only in these inclusion cases, such as a horn included in a head (Li et al., 4 Jan 2026).

Parts are category-specific, with shared semantics where appropriate. The part vocabulary was seeded from Wikidata, and for categories lacking a clear decomposition, annotators proposed cognitively salient parts. Typical categories have 3–8 parts. Representative examples include quadruped animals with head, body, foot or leg, and tail; ram with horn; camel with hump; car with window, wheel, front side, left side, right side, back side, and top side; hammer with handle, striker, and hammerhead; maraca with a head and a stick subdivided into joint and handle; and acoustic guitar with fingerboard, while strings are omitted because of heavy overlap. For flat or amorphous categories such as flatworm, the foreground object may be treated as a single “part.”

At the representation level, each image has instance-level binary part masks $M_p \in \{0,1\}^{H \times W}$. These may be combined into a composite mask tensor $M \in \{0,1\}^{H \times W \times (K+1)}$ with $K$ part channels plus background. Part categories are specific to object categories but compiled into a global vocabulary of $K = 3308$ distinct part classes. Multiple instances of the foreground category in an image are all annotated, while other categories remain background.

The dataset-wide statistics quantify both scale and annotation density. PIN++ contains 100,000 images, 406,364 masks, and 3,308 part categories. Annotation density is reported as 1–2 masks for 46.27% of images, 3–4 masks for 44.87%, 5–6 masks for 8.66%, and 7+ masks for 0.20%. It provides 317 annotated inclusion relations across 201 object categories. This combination of dense masks, broad coverage, and explicit inclusion semantics makes PIN++ suitable not only for classification-oriented supervision but also for structured dense prediction.

3. Annotation workflow, consistency rules, and quality control

PIN++ uses manual annotation without model assistance. The protocol emphasizes four principles: parts must compose the whole, overlap should be minimal, inclusion relations must be explicitly defined when overlap exists, and annotations should remain consistent across images. Ambiguity is resolved using rules centered on coverage completeness, non-overlap unless inclusion is defined, limited part count for simplicity, and the removal of redundant overlap by retaining only the larger part. Occlusions are labeled as far as visible cues allow, and inclusion relations are used to clarify layered semantics.

Consistency is enforced at the category level. The same annotator labels all 100 images of a category. Inspectors review 10 random images per category, and if at least two fail the annotation principles, the entire category is re-annotated. The annotation effort involved 50 annotators and approximately 8,000 hours, with 10 volunteers helping determine part lists and 5 inspectors overseeing quality control. A total of 37,505 low-quality images were discarded after double review. Formal inter-annotator agreement statistics are not reported; instead, quality control is enforced by strict guideline compliance and iterative re-annotation (Li et al., 4 Jan 2026).

The dataset is split following ImageNet-1K practice. Within the 100K annotated images, PIN++ uses a train/val split per category at a 9:1 ratio. All annotated images are drawn from the ImageNet-1K training set, and the ImageNet-1K validation set remains unannotated. For dense prediction benchmarks, the paper uses 8:1:1 random splits over PIN++. The remaining 1.3M unannotated ImageNet-1K training images are assigned pseudo part labels generated by a part segmentation model trained on PIN++.

Access and usage follow ImageNet conventions. Images inherit ImageNet licensing and are intended for research use, while PIN++ annotations are released for academic research. The dataset is distributed through HuggingFace at https://huggingface.co/datasets/lixiao20/PartImageNetPP, and the accompanying repository is https://github.com/LixiaoTHU/PartImageNetPP.

4. Multi-scale Part-supervised Model and pseudo-label propagation

PIN++ is paired with the Multi-scale Part-supervised recognition Model (MPM), a training framework that injects part supervision into a standard classifier without changing inference-time architecture. MPM augments a conventional recognition backbone with lightweight auxiliary bypass layers attached to the last three scales of the hierarchy. For a ResNet-50-GELU backbone with input resolution $224 \times 224$, the supervised feature resolutions are $28 \times 28$, $14 \times 14$, and $7 \times 7$. Top-down fusion, analogous to FPN-style aggregation, enriches lower-level features with high-level context. During inference, the auxiliary layers are dropped, so the backbone and classification head are unchanged and incur no extra parameters or computation at test time (Li et al., 2024).

The training pipeline has three stages. First, a part segmentation model is trained on PIN++; the reported implementation uses Mask R-CNN with a Swin-B backbone, treating each part category as a class. Second, this model generates pseudo part labels for the remaining unannotated ImageNet-1K training images. During inference on those images, the segmenter outputs candidate masks $M_p$ with class probability vectors $v_p$ over the 3,308 part categories. Third, MPM is trained jointly with both original PIN++ annotations and pseudo labels, using standard classification supervision together with auxiliary part supervision on intermediate features (Li et al., 4 Jan 2026).

A central component of pseudo-label generation is the Category Filter (CF). Given the known object category $M \in \{0,1\}^{H \times W \times (K+1)}$0, CF zeroes out probabilities for parts not associated with that category and then computes the predicted part label by argmax over the filtered distribution. This post-processing substantially improves pseudo-label quality: AP increases from 37.2 to 40.7, and AP50 increases from 58.6 to 64.4. The paper notes that optional refinements such as CRF smoothing, small-component removal via morphology, and a confidence threshold $M \in \{0,1\}^{H \times W \times (K+1)}$1 can be applied conservatively, but CF is the core required step for reproducing the reported numbers (Li et al., 4 Jan 2026).

The modeling formulation uses classification and part-supervision losses. The 2026 paper defines segmentation on annotated images, pseudo-label supervision, and a multi-task objective

$M \in \{0,1\}^{H \times W \times (K+1)}$2

with class-balanced weights $M \in \{0,1\}^{H \times W \times (K+1)}$3 to mitigate part-size imbalance and focal loss as an alternative to cross-entropy for faster convergence. The paper typically sets $M \in \{0,1\}^{H \times W \times (K+1)}$4, averages $M \in \{0,1\}^{H \times W \times (K+1)}$5 across the three supervised scales, and applies EMA of weights and label smoothing $M \in \{0,1\}^{H \times W \times (K+1)}$6. When adversarial training is used, adversarial examples $M \in \{0,1\}^{H \times W \times (K+1)}$7 are generated by PGD with $M \in \{0,1\}^{H \times W \times (K+1)}$8 bound $M \in \{0,1\}^{H \times W \times (K+1)}$9, $K$0 steps, and step size $K$1, maximizing $K$2.

5. Benchmark results across robustness and downstream tasks

On ImageNet-1K classification with adversarial robustness evaluation, PIN++ and MPM yield consistent improvements over strong ResNet-50-based baselines. Under adversarial training with $K$3 $K$4, using a ResNet-50-GELU backbone at $K$5, the reported top-1 accuracies are as follows (Li et al., 2024):

Model	Clean	$K$6	$K$7	$K$8	$K$9
Vanilla robust baseline	67.1	38.1	12.6	5.0	21.6
MPM	67.8	39.1	13.6	6.2	24.3

These gains extend beyond adversarial examples. On ImageNet-C, MPM improves average corruption accuracy from 40.8% to 42.1% without adversarial training and from 34.9% to 36.4% with adversarial training. On out-of-distribution datasets, the average accuracy across ImageNet-A-Plus, ImageNet-Sketch, Stylized ImageNet, and DIN increases from 23.3 to 23.6 without adversarial training and from 24.6 to 25.2 with adversarial training. The work also reports improved alignment with human decisions on distorted images, including error consistency metrics. This suggests that part supervision affects not only worst-case robustness but also the representational bias of the classifier.

PIN++ also establishes dense prediction baselines. For part segmentation, reported highlights include Mask R-CNN with ResNet-50 FPN+Cascade at mask AP 28.7, ResNet-101 FPN+Cascade at 30.3, and ViT-s FPN+Cascade at 32.0; box AP reaches 32.2 and AP50 reaches 53.5. For object segmentation, supervising with object+part labels improves over object-only supervision across capacities. In the ViT-s FPN+Cascade setting, mask AP improves from 55.4 to 58.4, box AP from 58.9 to 61.3, and AP50 reaches 79.0 (Li et al., 4 Jan 2026).

Few-shot learning results show similar gains from part-level supervision. In 5-way 1-shot evaluation, Meta-Baseline improves from 73.8 to 74.7 and DeepEMD from 62.8 to 64.0 when moving from object-only to object+part supervision. In 5-way 5-shot evaluation, Meta-Baseline improves from 90.6 to 91.3 and DeepEMD from 77.8 to 78.7. The few-shot configuration described in the paper adds $K = 3308$0 part CNN branches, each using Conv3×3, ReLU, and MaxPool on the top-3 largest parts by area, and fuses them with normalized weights $K = 3308$1.

Ablation results isolate several causal factors. Training MPM only on real annotations underfits, with clean 65.5, $K = 3308$2 33.0, and $K = 3308$3 9.4; adding pseudo labels with CF raises these to clean 67.8, $K = 3308$4 39.1, and $K = 3308$5 13.6. Multi-scale supervision matters: SV3 with top-down is better than SV1, SV2, and two-stage part-based models. Part supervision also outperforms object-foreground supervision, with average robustness 23.2% versus 22.3% across threats. These results indicate that the reported gains are not attributable merely to auxiliary dense supervision, but specifically to part-structured supervision distributed across multiple scales.

6. Usage patterns, limitations, and interpretive context

The recommended usage pattern begins with training a part segmenter on the 100K annotated PIN++ images, then propagating pseudo part labels to the remaining ImageNet-1K training images using Category Filter, and finally training MPM with three bypass layers and top-down fusion. For classification, the reported recipe uses SGD with learning rate 0.2, momentum 0.9, cosine decay, weight decay $K = 3308$6, batch size 512, 80 epochs, input resolution $K = 3308$7, EMA 0.9998, label smoothing 0.1, and random flip/crop. The bypass layers add approximately 4.5M parameters during training only. For segmentation, the reported backbones include ResNet-50/101 FPN and ViT-det small, with optional Cascade, LSJ augmentation, and federated loss (Li et al., 4 Jan 2026).

The dataset also comes with practical constraints. PIN++ images are single-foreground-category, the ImageNet-1K validation set is unannotated, and pseudo labels are generated only for the training set. Some categories have ambiguous or minimal decompositions, overlapping semantics must be handled through inclusion relations, and some rare parts are underrepresented, motivating class weights and focal loss. Respect for ImageNet licensing is required, and the authors explicitly recommend research use and caution against deployment in safety-critical applications without thorough robustness evaluation.

A recurring misconception in part-aware modeling is that generic segmentation is sufficient to recover meaningful parts. The qualitative analyses reported with PIN++ argue otherwise: VLPart is described as mislabeling unseen object parts, and SAM as category-agnostic and prone to failure when boundaries are not crisp, such as on camel examples (Li et al., 2024). This suggests that the contribution of PIN++ lies not only in mask availability, but in category-specific part semantics, inclusion-aware annotation, and the ability to propagate those semantics into large-scale recognition training.

In that broader context, PIN++ functions as both a dataset and a methodological substrate. As a dataset, it standardizes part supervision across ImageNet-1K. As a modeling resource, it enables classifiers to acquire part-aware intermediate representations without inference-time overhead. Its reported gains in adversarial robustness, common corruptions, out-of-distribution generalization, object segmentation, and few-shot learning position it as a general-purpose foundation for part-aware visual modeling at ImageNet scale.