PerSense-D: Dense Scene Segmentation Benchmark

• PerSense-D is a benchmark for personalized one-shot instance segmentation in dense scenes, defined by severe occlusion, scale variation, and background clutter.
• It uses a fixed support set of 28 images for 28 diverse object categories, with dense query images averaging up to 53 instances per image.
• Evaluation is based on pixel-precise segmentation using metrics like mIoU, stressing methods to separate adjacent merged objects in challenging environments.

PerSense-D is an evaluation benchmark for personalized instance segmentation in dense images, introduced with PerSense as a benchmark “exclusive to personalized instance segmentation in dense images” and later revisited in the PerSense++ follow-up (Siddiqui et al., 2024, Siddiqui et al., 20 Aug 2025). It is organized around a one-shot protocol in which a method receives exactly one support exemplar per object category and must segment all matching instances in dense query images without fine-tuning on the benchmark itself. The benchmark is defined by dense scenes with severe occlusion, scale variation, and background clutter, and it is intended to stress-test methods that must separate many adjacent instances rather than merely localize a few isolated objects.

1. Task definition and benchmark scope

PerSense-D targets personalized instance segmentation under dense-scene conditions. In the benchmark formulation, the support set contains one exemplar per category, and the query set contains crowded scenes in which all instances of the supported category must be segmented individually. The benchmark is therefore neither a generic semantic segmentation dataset nor a conventional few-shot benchmark with episodic train/validation/test partitions; it is a fixed one-shot evaluation protocol (Siddiqui et al., 2024).

The benchmark is explicitly motivated by limitations of existing datasets for dense personalized segmentation. The follow-up description states that COCO, LVIS, and FSS-1000 contain multi-instance scenes but “average only a handful of objects per class,” while no publicly available benchmark focuses exclusively on dense instance segmentation with heavy occlusion and scale variation (Siddiqui et al., 20 Aug 2025). The 2024 introduction similarly characterizes PerSense-D as “exclusively dense,” with an average of 39 instances per image, contrasted with COCO’s approximately 7.7 and LVIS’s approximately 3.3 per class (Siddiqui et al., 2024).

A central misconception is to treat PerSense-D as a training dataset. Both descriptions reject that interpretation. The benchmark has “no traditional train/val/test split,” and methods are evaluated in a one-shot regime with “no fine-tuning on PerSense-D”; the 689 query images are used solely for inference and evaluation, while the 28 support images drive personalization (Siddiqui et al., 2024, Siddiqui et al., 20 Aug 2025).

2. Corpus composition and curation

Across both published descriptions, the benchmark has the same top-level structure: 717 total images, comprising 689 dense query images and 28 support images, with 28 object categories and one support exemplar per category (Siddiqui et al., 2024, Siddiqui et al., 20 Aug 2025). The categories are described as diverse everyday and industrial items, including fruits and vegetables such as apples, mangoes, durians, eggplants, cucumbers, carrots, and potatoes; nuts and seeds such as peanuts and walnuts; and packaged goods and other items such as cookies, bottles, books, and dumbbells (Siddiqui et al., 2024).

The later paper provides additional construction details. It reports a web crawl over Google, Bing, and Yahoo using prompts such as “multiple X,” “lots of X,” and “many X” for each of the 28 classes, producing 2,800 candidate images. Manual filtering then retained 689 query images using three criteria: at least 7 instances per image, sufficient image quality for mask annotation, and evident occlusion and background clutter (Siddiqui et al., 20 Aug 2025). The 2024 description reports closely related quality-control criteria during annotation review: sufficient resolution greater than $800 \times 800$ px, at least 7 objects present, and strong occlusion/background clutter (Siddiqui et al., 2024).

A notable feature is that the support set is fixed rather than randomly sampled. The support images are pre-assigned, with exactly one support image per object category and IDs “00” to “27,” and support masks are supplied to ensure reproducible one-shot personalization (Siddiqui et al., 2024). The 2025 description reiterates that each support image contains a single instance of one category and that the fixed support set is intended to ensure reproducible one-shot experiments (Siddiqui et al., 20 Aug 2025).

Aspect	2024 introduction	2025 follow-up
Total images	717	717
Query / support split	689 / 28	689 / 28
Object categories	28	28
Total annotated instances	28,395	36,837
Average instances per query image	39.6	$\simeq 53$
Instance-count range	7 to 218	7 to 573

The shared headline totals and the differing instance statistics indicate that the benchmark description evolved across publications. This suggests that later reporting may reflect expanded annotation accounting or revised benchmark statistics, although no explicit reconciliation is provided in the published summaries (Siddiqui et al., 2024, Siddiqui et al., 20 Aug 2025).

3. Annotation protocol and data products

PerSense-D uses a semi-automatic annotation pipeline. The annotation procedure begins with “initial model-in-the-loop mask generation via PerSense,” after which trained annotators perform manual refinement using an OpenCV annotation tool and Photoshop’s “quick-selection” and “lasso” tools (Siddiqui et al., 2024). The 2025 description restates this workflow as “PerSense generates an initial mask per instance” followed by human refinement “to pixel-accuracy using OpenCV/Photoshop tools” (Siddiqui et al., 20 Aug 2025).

The annotation outputs include pixel-precise instance masks for every object in each dense query image and one support mask per support image for one-shot exemplars (Siddiqui et al., 2024). The 2025 account adds that each final query image also includes dot annotations at each object center, alongside the full set of instance masks (Siddiqui et al., 20 Aug 2025). Average manual correction time is reported as approximately 15 minutes per dense image in both descriptions (Siddiqui et al., 2024, Siddiqui et al., 20 Aug 2025).

The benchmark emphasizes mask fidelity rather than lightweight labeling. The 2024 paper describes the masks as “pixel-precise,” and the later paper states that no preprocessing is applied beyond standardizing mask formats and storing uniform metadata. It further states that there is “no train-time normalization or augmentation,” because the benchmark is purely for evaluation (Siddiqui et al., 20 Aug 2025). A plausible implication is that the annotation design was intended both to support segmentation benchmarking directly and to enable diagnostics for upstream density-map or point-prompt stages via the provided center dots.

4. Evaluation protocol and metrics

The evaluation protocol is one-shot and fixed-support. The support set contains 28 images and masks, one per class, and all 689 dense query images are used only for inference and scoring (Siddiqui et al., 2024). The follow-up paper makes the same point in procedural terms: each method is run on all 689 query images using the fixed 28 support exemplars, and results are reported overall and per density level (Siddiqui et al., 20 Aug 2025).

The primary reported metric is mean Intersection-over-Union (mIoU). For a predicted mask $P$ and ground-truth mask $G$,

$\mathrm{IoU}(P,G)=\frac{|P\cap G|}{|P\cup G|}.$

The benchmark description gives mean IoU as

$$\mathrm{mIoU}=\frac{1}{|\mathcal{C}|}\sum_{c\in\mathcal{C}}\mathrm{IoU}_c,$$

with the averaging performed over all instances or categories, depending on protocol (Siddiqui et al., 2024). The 2025 follow-up explicitly notes that “all core comparisons use mIoU for segmentation” and that “No Average Precision/AP is reported in the paper” (Siddiqui et al., 20 Aug 2025).

The 2024 benchmark description nonetheless lists standard instance-segmentation metrics for completeness, including AP, AR, and per-instance F1:

$\mathrm{AP}=\int_0^1 p(r)\,\mathrm{d}r,$

$\mathrm{AR}=\frac{1}{K}\sum_{k=1}^{K}R(\tau_k),$

and

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}},\qquad \mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}},\qquad \mathrm{F1}=\frac{2\cdot \mathrm{Precision}\cdot \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}.$$

It also states that most segment-based metrics use $\mathrm{IoU}\ge 0.50$ for a match (Siddiqui et al., 2024).

Because PerSense itself uses density maps, the benchmark descriptions also include density-aware counting diagnostics:

$\simeq 53$0

where $\simeq 53$1 is the estimated object count from a density map and $\simeq 53$2 is the ground-truth count (Siddiqui et al., 2024). The 2025 paper gives the same diagnostics using $\simeq 53$3 and $\simeq 53$4 notation (Siddiqui et al., 20 Aug 2025). This suggests that the benchmark was designed not only to score final masks but also to expose failure modes in density-based prompt generation.

5. Density structure and scene difficulty

The benchmark is designed around dense scenes with broad variation in instance count. In the original 2024 description, the per-image instance-count distribution has mean 39.6 instances per image, median 35, range 7 to 218, and approximate quartiles $\simeq 53$5, $\simeq 53$6, and $\simeq 53$7 (Siddiqui et al., 2024). The same description partitions the 689 query images into four density bins: low density with 7–20 instances (112 images), medium density with 21–50 instances (345 images), high density with 51–100 instances (130 images), and very high density with more than 100 instances (102 images) (Siddiqui et al., 2024).

The 2025 follow-up reports a different density stratification: low density for $\simeq 53$8 with 224 images, medium density for $\simeq 53$9 with 266 images, and high density for $P$0 with 199 images (Siddiqui et al., 20 Aug 2025). It also reports 36,837 total annotated instances, an average of approximately 53 instances per image, a range from 7 to 573 instances, and an average image resolution of $P$1 px (Siddiqui et al., 20 Aug 2025).

Both descriptions converge on the same characterization of scene difficulty. PerSense-D features heavy occlusion, background clutter, and scale variation, including cases with more than 75% mutual occlusion in many scenes, as well as simultaneous presence of small, medium, and large objects (Siddiqui et al., 2024). The follow-up further quantifies intra-class scale variation through the coefficient of variation,

$P$2

of normalized instance areas per category, and states that several classes exhibit very high CV $P$3 (Siddiqui et al., 20 Aug 2025). These properties place the benchmark closer to dense counting and crowded-scene analysis than to conventional sparse instance segmentation.

6. Baselines, empirical behavior, and benchmark significance

The 2024 introduction reports overall mIoU and average inference time for several one-shot or training-free baselines. In that comparison, PerSense with DSALVANet as the density-map generator attains 70.96 mIoU at 2.7 s average inference time, and PerSense with CounTR attains 71.61 mIoU at 2.7 s, outperforming PerSAM, PerSAM-F, Matcher, and Grounded-SAM (Siddiqui et al., 2024).

Method	mIoU (%)	Avg. inference time
PerSAM	24.45	39.8 s
PerSAM-F	29.34	47.8 s
Matcher	62.78	10.2 s
Grounded-SAM	65.92	1.8 s
PerSense (DSALVANet)	70.96	2.7 s
PerSense (CounTR)	71.61	2.7 s

The same paper gives class-wise and density-regime observations. Under low density, defined there as $P$4 objects, most methods plateau around approximately 80–85% mIoU, while PerSense still leads by approximately 5–7 points. At medium density (21–50), PerSense reports 71–74% versus Grounded-SAM at 62–66%, a gap of approximately 8–9 points. In very high density scenes, described there as $P$5, performance drops for all methods, but PerSense retains approximately 65% mIoU versus approximately 55% for other baselines. The paper also notes a specific failure mode: Grounded-SAM can outperform PerSense on “flat” instances with zero inter-object gaps, such as tightly stacked books, where density maps blur (Siddiqui et al., 2024).

The 2025 follow-up broadens the comparison to both end-to-end trained or fine-tuned methods and training-free methods, again using PerSense-D. It reports the following density-stratified mIoU values:

Method	Low / Med / High mIoU (%)	Overall mIoU (%)
C3Det	52.70 / 46.64 / 39.11	48.60
SegGPT	59.81 / 53.34 / 52.05	55.50
PerSAM-F	38.18 / 34.84 / 26.73	29.30
PseCo	53.99 / 65.23 / 68.55	61.83
GeCo	63.92 / 63.40 / 74.49	65.95
PerSAM	32.27 / 28.75 / 20.25	24.45
TFOC	62.78 / 65.38 / 65.69	62.63
Matcher	58.62 / 58.30 / 68.00	62.80
GroundedSAM	58.36 / 66.24 / 64.97	65.92
PerSense (DMG1)	66.36 / 67.27 / 74.78	70.96
PerSense (DMG2)	59.84 / 73.51 / 77.57	71.61

These results are significant less because they establish a universal ranking than because they demonstrate what PerSense-D is constructed to penalize: merged instances, missed small objects, and unstable prompting under clutter and occlusion. The benchmark motivation in the later paper explicitly connects this to industrial automation, cargo monitoring, and biomedical imaging, and to technical failure modes of box-prompt methods under dense overlap and NMS suppression (Siddiqui et al., 20 Aug 2025). In that sense, PerSense-D functions as a stress benchmark for one-shot, dense-scene, instance-level segmentation rather than as a broad proxy for generic segmentation performance.