---
title: 'PerSense-D: Dense Scene Segmentation Benchmark'
url: https://www.emergentmind.com/topics/persense-d
type: topic
---

# PerSense-D: Dense Scene Segmentation Benchmark

PerSense-D is an evaluation benchmark for personalized instance segmentation in dense images, introduced with PerSense as a benchmark “exclusive to personalized instance segmentation in dense images” and later revisited in the PerSense++ follow-up [2405.13518] [2508.14660]. It is organized around a one-shot protocol in which a method receives exactly one support exemplar per object category and must segment all matching instances in dense query images without fine-tuning on the benchmark itself. The benchmark is defined by dense scenes with severe occlusion, scale variation, and background clutter, and it is intended to stress-test methods that must separate many adjacent instances rather than merely localize a few isolated objects.

## 1. Task definition and benchmark scope

PerSense-D targets personalized instance segmentation under dense-scene conditions. In the benchmark formulation, the support set contains one exemplar per category, and the query set contains crowded scenes in which all instances of the supported category must be segmented individually. The benchmark is therefore neither a generic semantic segmentation dataset nor a conventional few-shot benchmark with episodic train/validation/test partitions; it is a fixed one-shot evaluation protocol [2405.13518].

The benchmark is explicitly motivated by limitations of existing datasets for dense personalized segmentation. The follow-up description states that COCO, LVIS, and FSS-1000 contain multi-instance scenes but “average only a handful of objects per class,” while no publicly available benchmark focuses exclusively on dense instance segmentation with heavy occlusion and scale variation [2508.14660]. The 2024 introduction similarly characterizes PerSense-D as “exclusively dense,” with an average of 39 instances per image, contrasted with COCO’s approximately 7.7 and LVIS’s approximately 3.3 per class [2405.13518].

A central misconception is to treat PerSense-D as a training dataset. Both descriptions reject that interpretation. The benchmark has “no traditional train/val/test split,” and methods are evaluated in a one-shot regime with “no fine-tuning on PerSense-D”; the 689 query images are used solely for inference and evaluation, while the 28 support images drive personalization [2405.13518] [2508.14660].

## 2. Corpus composition and curation

Across both published descriptions, the benchmark has the same top-level structure: 717 total images, comprising 689 dense query images and 28 support images, with 28 object categories and one support exemplar per category [2405.13518] [2508.14660]. The categories are described as diverse everyday and industrial items, including fruits and vegetables such as apples, mangoes, durians, eggplants, cucumbers, carrots, and potatoes; nuts and seeds such as peanuts and walnuts; and packaged goods and other items such as cookies, bottles, books, and dumbbells [2405.13518].

The later paper provides additional construction details. It reports a web crawl over Google, Bing, and Yahoo using prompts such as “multiple X,” “lots of X,” and “many X” for each of the 28 classes, producing 2,800 candidate images. Manual filtering then retained 689 query images using three criteria: at least 7 instances per image, sufficient image quality for mask annotation, and evident occlusion and background clutter [2508.14660]. The 2024 description reports closely related quality-control criteria during annotation review: sufficient resolution greater than \(800 \times 800\) px, at least 7 objects present, and strong occlusion/background clutter [2405.13518].

A notable feature is that the support set is fixed rather than randomly sampled. The support images are pre-assigned, with exactly one support image per object category and IDs “00” to “27,” and support masks are supplied to ensure reproducible one-shot personalization [2405.13518]. The 2025 description reiterates that each support image contains a single instance of one category and that the fixed support set is intended to ensure reproducible one-shot experiments [2508.14660].

| Aspect | 2024 introduction | 2025 follow-up |
|---|---:|---:|
| Total images | 717 | 717 |
| Query / support split | 689 / 28 | 689 / 28 |
| Object categories | 28 | 28 |
| Total annotated instances | 28,395 | 36,837 |
| Average instances per query image | 39.6 | \(\simeq 53\) |
| Instance-count range | 7 to 218 | 7 to 573 |

The shared headline totals and the differing instance statistics indicate that the benchmark description evolved across publications. This suggests that later reporting may reflect expanded annotation accounting or revised benchmark statistics, although no explicit reconciliation is provided in the published summaries [2405.13518] [2508.14660].

## 3. Annotation protocol and data products

PerSense-D uses a semi-automatic annotation pipeline. The annotation procedure begins with “initial model-in-the-loop mask generation via PerSense,” after which trained annotators perform manual refinement using an OpenCV annotation tool and Photoshop’s “quick-selection” and “lasso” tools [2405.13518]. The 2025 description restates this workflow as “PerSense generates an initial mask per instance” followed by human refinement “to pixel-accuracy using OpenCV/Photoshop tools” [2508.14660].

The annotation outputs include pixel-precise instance masks for every object in each dense query image and one support mask per support image for one-shot exemplars [2405.13518]. The 2025 account adds that each final query image also includes dot annotations at each object center, alongside the full set of instance masks [2508.14660]. Average manual correction time is reported as approximately 15 minutes per dense image in both descriptions [2405.13518] [2508.14660].

The benchmark emphasizes mask fidelity rather than lightweight labeling. The 2024 paper describes the masks as “pixel-precise,” and the later paper states that no preprocessing is applied beyond standardizing mask formats and storing uniform metadata. It further states that there is “no train-time normalization or augmentation,” because the benchmark is purely for evaluation [2508.14660]. A plausible implication is that the annotation design was intended both to support segmentation benchmarking directly and to enable diagnostics for upstream density-map or point-prompt stages via the provided center dots.

## 4. Evaluation protocol and metrics

The evaluation protocol is one-shot and fixed-support. The support set contains 28 images and masks, one per class, and all 689 dense query images are used only for inference and scoring [2405.13518]. The follow-up paper makes the same point in procedural terms: each method is run on all 689 query images using the fixed 28 support exemplars, and results are reported overall and per density level [2508.14660].

The primary reported metric is mean Intersection-over-Union (mIoU). For a predicted mask \(P\) and ground-truth mask \(G\),
$$
\mathrm{IoU}(P,G)=\frac{|P\cap G|}{|P\cup G|}.
$$
The benchmark description gives mean IoU as
$$
\mathrm{mIoU}=\frac{1}{|\mathcal{C}|}\sum_{c\in\mathcal{C}}\mathrm{IoU}_c,
$$
with the averaging performed over all instances or categories, depending on protocol [2405.13518]. The 2025 follow-up explicitly notes that “all core comparisons use mIoU for segmentation” and that “No Average Precision/AP is reported in the paper” [2508.14660].

The 2024 benchmark description nonetheless lists standard instance-segmentation metrics for completeness, including AP, AR, and per-instance F1:
$$
\mathrm{AP}=\int_0^1 p(r)\,\mathrm{d}r,
$$
$$
\mathrm{AR}=\frac{1}{K}\sum_{k=1}^{K}R(\tau_k),
$$
and
$$
\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}},\qquad
\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}},\qquad
\mathrm{F1}=\frac{2\cdot \mathrm{Precision}\cdot \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}.
$$
It also states that most segment-based metrics use \(\mathrm{IoU}\ge 0.50\) for a match [2405.13518].

Because PerSense itself uses density maps, the benchmark descriptions also include density-aware counting diagnostics:
$$
\mathrm{MAE}=\frac{1}{N}\sum_{i=1}^{N}\left|\hat n_i-n_i\right|,
\qquad
\mathrm{RMSE}=\sqrt{\frac{1}{N}\sum_{i=1}^{N}(\hat n_i-n_i)^2},
$$
where \(\hat n_i\) is the estimated object count from a density map and \(n_i\) is the ground-truth count [2405.13518]. The 2025 paper gives the same diagnostics using \(c_i\) and \(\hat c_i\) notation [2508.14660]. This suggests that the benchmark was designed not only to score final masks but also to expose failure modes in density-based prompt generation.

## 5. Density structure and scene difficulty

The benchmark is designed around dense scenes with broad variation in instance count. In the original 2024 description, the per-image instance-count distribution has mean 39.6 instances per image, median 35, range 7 to 218, and approximate quartiles \(Q_1 \approx 20\), \(Q_2 \approx 35\), and \(Q_3 \approx 53\) [2405.13518]. The same description partitions the 689 query images into four density bins: low density with 7–20 instances (112 images), medium density with 21–50 instances (345 images), high density with 51–100 instances (130 images), and very high density with more than 100 instances (102 images) [2405.13518].

The 2025 follow-up reports a different density stratification: low density for \(C_I \le 30\) with 224 images, medium density for \(30 < C_I \le 60\) with 266 images, and high density for \(C_I > 60\) with 199 images [2508.14660]. It also reports 36,837 total annotated instances, an average of approximately 53 instances per image, a range from 7 to 573 instances, and an average image resolution of \(839 \times 967\) px [2508.14660].

Both descriptions converge on the same characterization of scene difficulty. PerSense-D features heavy occlusion, background clutter, and scale variation, including cases with more than 75% mutual occlusion in many scenes, as well as simultaneous presence of small, medium, and large objects [2405.13518]. The follow-up further quantifies intra-class scale variation through the coefficient of variation,
$$
\mathrm{CV}=\frac{\sigma_{\mathrm{area}}}{\mu_{\mathrm{area}}},
$$
of normalized instance areas per category, and states that several classes exhibit very high CV \(>0.8\) [2508.14660]. These properties place the benchmark closer to dense counting and crowded-scene analysis than to conventional sparse instance segmentation.

## 6. Baselines, empirical behavior, and benchmark significance

The 2024 introduction reports overall mIoU and average inference time for several one-shot or training-free baselines. In that comparison, PerSense with DSALVANet as the density-map generator attains 70.96 mIoU at 2.7 s average inference time, and PerSense with CounTR attains 71.61 mIoU at 2.7 s, outperforming PerSAM, PerSAM-F, Matcher, and Grounded-SAM [2405.13518].

| Method | mIoU (%) | Avg. inference time |
|---|---:|---:|
| PerSAM | 24.45 | 39.8 s |
| PerSAM-F | 29.34 | 47.8 s |
| Matcher | 62.78 | 10.2 s |
| Grounded-SAM | 65.92 | 1.8 s |
| PerSense (DSALVANet) | 70.96 | 2.7 s |
| PerSense (CounTR) | 71.61 | 2.7 s |

The same paper gives class-wise and density-regime observations. Under low density, defined there as \(\le 20\) objects, most methods plateau around approximately 80–85% mIoU, while PerSense still leads by approximately 5–7 points. At medium density (21–50), PerSense reports 71–74% versus Grounded-SAM at 62–66%, a gap of approximately 8–9 points. In very high density scenes, described there as \(>50\), performance drops for all methods, but PerSense retains approximately 65% mIoU versus approximately 55% for other baselines. The paper also notes a specific failure mode: Grounded-SAM can outperform PerSense on “flat” instances with zero inter-object gaps, such as tightly stacked books, where density maps blur [2405.13518].

The 2025 follow-up broadens the comparison to both end-to-end trained or fine-tuned methods and training-free methods, again using PerSense-D. It reports the following density-stratified mIoU values:

| Method | Low / Med / High mIoU (%) | Overall mIoU (%) |
|---|---|---:|
| C3Det | 52.70 / 46.64 / 39.11 | 48.60 |
| SegGPT | 59.81 / 53.34 / 52.05 | 55.50 |
| PerSAM-F | 38.18 / 34.84 / 26.73 | 29.30 |
| PseCo | 53.99 / 65.23 / 68.55 | 61.83 |
| GeCo | 63.92 / 63.40 / 74.49 | 65.95 |
| PerSAM | 32.27 / 28.75 / 20.25 | 24.45 |
| TFOC | 62.78 / 65.38 / 65.69 | 62.63 |
| Matcher | 58.62 / 58.30 / 68.00 | 62.80 |
| GroundedSAM | 58.36 / 66.24 / 64.97 | 65.92 |
| PerSense (DMG1) | 66.36 / 67.27 / 74.78 | 70.96 |
| PerSense (DMG2) | 59.84 / 73.51 / 77.57 | 71.61 |

These results are significant less because they establish a universal ranking than because they demonstrate what PerSense-D is constructed to penalize: merged instances, missed small objects, and unstable prompting under clutter and occlusion. The benchmark motivation in the later paper explicitly connects this to industrial automation, cargo monitoring, and biomedical imaging, and to technical failure modes of box-prompt methods under dense overlap and NMS suppression [2508.14660]. In that sense, PerSense-D functions as a stress benchmark for one-shot, dense-scene, instance-level segmentation rather than as a broad proxy for generic segmentation performance.

Source: https://www.emergentmind.com/topics/persense-d