PS-RPN: Pairwise-Similarity for Small Objects

• The paper introduces a novel PS metric that jointly models position and shape similarity to improve label assignment in dense, small object detection.
• It employs an auxiliary proposal branch with standard RPN classification and regression losses to generate high-quality positive queries without altering DETR inference.
• Empirical results on benchmarks like VisDrone show substantial gains, with AP increases up to +2.6, validating PS-RPN's effectiveness in challenging detection scenarios.

Pairwise-Similarity Region Proposal Network (PS-RPN) is a label-assignment and proposal-training module introduced as part of the Noise-Resilient Query Optimization (NRQO) paradigm for DETR-based small object detection. Within NRQO, NT-FPN performs noise-tolerant feature fusion, whereas PS-RPN operates on the supervision side: it matches anchors to ground-truth boxes using a closed-form Pairwise-Similarity (PS) metric that jointly models position and shape similarity, trains an auxiliary proposal branch with classification/objectness and box regression losses, and supplies a sufficient number of high-quality positive queries during training without changing standard DETR inference (Fang et al., 25 Jul 2025).

1. Problem setting and motivation

PS-RPN is motivated by diminished query quality in existing label assignment strategies for small object detection. In this setting, common assignment metrics typically rely either on overlap, as in IoU-like criteria, or on center distance alone, as in DotD and NWD. The reported failure mode for IoU is that it is brittle for small boxes: tiny shifts or scale changes drastically alter IoU, producing unstable or sparse positives. Distance-only metrics alleviate localization sensitivity but ignore box shape, so they become ambiguous when multiple small objects are spatially nearby yet differ in width, height, or aspect ratio (Fang et al., 25 Jul 2025).

The need for PS-RPN is framed specifically in the DETR family. Although DETR-based detectors learn object queries end-to-end, they still depend on reliable supervision and sufficiently informative positives during training. When assignment favors position while ignoring shape, small objects tend to yield few high-quality positives, which degrades query learning. PS-RPN addresses this by jointly modeling position and shape similarity between anchors and ground truth, with the explicit aim of producing more and better-quality positives, especially in dense small-object scenarios where many instances are close in space but differ in shape.

Small objects intensify both classes of assignment error. Their limited spatial footprint makes overlap-based criteria highly sensitive, and densely clustered instances cause position-only strategies to mix labels. The intended effect of PS-RPN is therefore not merely to increase positive counts, but to reduce confusion in supervision by incorporating a second discriminative cue—shape—alongside position. This suggests that the method targets a supervision bottleneck rather than a decoder redesign.

2. Pairwise-Similarity metric

The core of PS-RPN is the Pairwise-Similarity metric between an anchor and a ground-truth box. It is defined as an exponential of the negative sum of a position term and a shape term:

$$\mathcal{PS}=\exp\!\left(-\left(\mathcal{PS}_{\text{position}}+\mathcal{PS}_{\text{shape}}\right)\right).$$

The paper states that $\mathcal{PS}\in(0,1]$, that larger values correspond to closer agreement in both position and shape, and that exact match yields similarity approaching $1$ (Fang et al., 25 Jul 2025).

The position term measures normalized center distance in $x$ and $y$ using anisotropy factors $m$ and $n$:

$$\mathcal{PS}_{\text{position}}= \sqrt{ \left(\frac{m\cdot(x_g-x)}{w_g+w}\right)^2+ \left(\frac{n\cdot(y_g-y)}{h_g+h}\right)^2 }.$$

Here $(x_g,y_g)$ and $(x,y)$ denote the ground-truth and anchor centers, and $\mathcal{PS}\in(0,1]$0 denote the corresponding widths and heights. The normalization by $\mathcal{PS}\in(0,1]$1 and $\mathcal{PS}\in(0,1]$2 is described as damping sensitivity to absolute scale, which is especially relevant when both boxes are small. The Euclidean combination in the normalized space is reported to provide rotation invariance in that normalized coordinate frame, and the formulation is symmetric under swapping ground truth and anchor because the denominators use sums.

The shape term measures normalized differences in width and height:

$\mathcal{PS}\in(0,1]$3

This term is intended to provide a scale-relative comparison of box geometry. Equal widths and heights lead to small shape distance, whereas different scale or aspect ratio increases it.

A notable design choice is that PS-RPN introduces no additional weighting coefficients in the metric itself. Instead, the balancing of horizontal and vertical contributions is handled through $\mathcal{PS}\in(0,1]$4 and $\mathcal{PS}\in(0,1]$5, which are computed from dataset statistics rather than tuned by the user. The paper defines them as averages of normalized center distances along the $\mathcal{PS}\in(0,1]$6 and $\mathcal{PS}\in(0,1]$7 axes over the training set, with $\mathcal{PS}\in(0,1]$8 training images, $\mathcal{PS}\in(0,1]$9 ground-truth boxes in image $1$0, and $1$1 anchors in image $1$2. In that sense, the claim of operating “without the need for additional hyperparameters” refers to the PS metric itself: it uses no learned weights or tuned balancing coefficients, and $1$3 are derived automatically from data rather than manually selected (Fang et al., 25 Jul 2025).

3. Assignment mechanism and positive query generation

PS-RPN adopts a MaxIoUAssigner-style workflow but replaces IoU with the Pairwise-Similarity metric in all assignment decisions. The thresholds are positive $1$4, negative $1$5, and minimum positive $1$6. These thresholds are inherited from a standard MaxIoUAssigner and are not presented as PS-specific design parameters (Fang et al., 25 Jul 2025).

Under this procedure, any anchor with $1$7 to any ground-truth box is assigned as a positive for that ground truth. Anchors with $1$8 to all ground truths are assigned as negatives. In addition, for each ground-truth box, the highest-similarity anchor is force-marked positive if its similarity is at least $1$9, ensuring that each ground truth has at least one positive even when no anchor reaches the positive threshold. When multiple ground truths compete for the same anchor, the anchor is assigned to the ground truth with the highest $x$0, following standard MaxIoUAssigner behavior.

This assignment rule is central to the method’s notion of positive query generation. PS-RPN is reported to generate a greater count of reliable positives per ground truth than position-only strategies, particularly in dense small-object scenes. The paper’s interpretation is that this provides “a sufficient number of high-quality positive queries,” which then improves query learning for the DETR training pipeline.

PS-RPN does not replace DETR’s query decoder. Rather, it augments training by improving the positive query set derived from anchors through PS-based assignment and by training an auxiliary proposal branch. The relationship to Hungarian matching is therefore complementary: the paper does not state that Hungarian matching is removed, and standard DETR losses remain part of the training stack. A common misconception is to treat PS-RPN as an alternative decoder query mechanism; the formulation in NRQO instead places it on the supervision side of the architecture (Fang et al., 25 Jul 2025).

4. Training objective, architectural role, and inference behavior

Architecturally, PS-RPN is presented primarily as a label-assignment and training objective built on anchors and ground truth rather than as a fully elaborated standalone head design. The paper states that it introduces a novel Pairwise-Similarity metric and adopts a MaxIoUAssigner-based strategy in which IoU is replaced by PS. It refers to classification/objectness and regression losses typical of region proposal networks, implying the existence of an auxiliary proposal branch, but it does not detail additional heads, convolutional layers, or proposal encoders (Fang et al., 25 Jul 2025).

The PS-RPN loss is written as

$x$1

The exact forms of these losses are not specified. The paper only states that they are standard RPN-style classification/objectness and box regression losses applied to positives and negatives assigned by the PS metric.

At the level of the full NRQO objective, PS-RPN is combined with the NT-FPN contrastive losses:

$x$2

Here $x$3 and $x$4 are the NT-FPN spatial and semantic contrastive losses, and $x$5 balances those losses. The paper states that PS-RPN is added without extra scaling.

The inputs to PS-RPN are multi-scale features from NT-FPN, anchors at each scale, and ground-truth boxes. Its outputs are assigned positives and negatives together with proposal objectness and box-regression supervision that improve the quality of positives available to the transformer decoder. The computational overhead is described as minimal because PS computation consists of simple arithmetic on anchor–ground-truth pairs and reuses MaxIoUAssigner logic. Its use is training-only: no changes to inference are reported, and runtime at inference remains that of the underlying DETR baseline (Fang et al., 25 Jul 2025).

5. Empirical behavior and reproducibility

The principal ablation reported for PS-RPN appears in Table 3 on VisDrone 2019. The baseline without NT-FPN and without PS-RPN attains AP $x$6 and AP$x$7 $x$8. Adding PS-RPN alone raises performance to AP $x$9 and AP$y$0 $y$1, corresponding to gains of $y$2 AP and $y$3 AP$y$4. The same row also reports AP$y$5 $y$6, AP$y$7 $y$8, AP$y$9 $m$0, and AP$m$1 $m$2. NT-FPN alone yields AP $m$3 and AP$m$4 $m$5, while the full NRQO configuration with both modules reaches AP $m$6 and AP$m$7 $m$8 (Fang et al., 25 Jul 2025).

Setting	AP	AP$m$9
Baseline (w/o NT-FPN, w/o PS-RPN)	28.5	20.5
PS-RPN only	30.9	23.1
NT-FPN only	31.6	23.7
Full NRQO	32.1	24.5

These results are interpreted in the paper as evidence that PS-RPN alone yields substantial improvements, especially for small objects, and that its gains compound when combined with NT-FPN. The article also states that overall evaluations are conducted on VisDrone 2019, SODA-D, and COCO 2017, with AP as the primary metric and AP$n$0 and AP$n$1 also reported; for SODA-D, the small-object subsets are denoted eS, rS, and gS.

The reported reproduction details use a ResNet-50 backbone across baselines, a single RTX 3090 GPU, a 12-epoch schedule, AdamW, batch size $n$2, initial learning rate $n$3, and weight decay $n$4. The global NRQO coefficient is reported as $n$5, with ablation indicating low sensitivity. The paper does not specify the number of queries or anchors, and no repository link is provided (Fang et al., 25 Jul 2025).

6. Relation to adjacent methods, misconceptions, and limitations

PS-RPN is best understood by contrast with three nearby families of methods. Relative to Faster R-CNN’s RPN, it preserves the assigner logic—positive, negative, and force-positive decisions—but replaces IoU with the PS metric. The stated benefit is greater robustness for small objects, because PS is less sensitive to tiny localization changes and also discriminates box shape in crowded scenes. Relative to DETR’s Hungarian matching, PS-RPN is a local anchor-based assignment mechanism used to improve positive supervision and query quality; it is not presented as a replacement for the global bipartite matching used by the transformer detector. Relative to OTA- and SimOTA-style assignment, PS-RPN is described as simpler: it uses a closed-form similarity calibrated by dataset-derived $n$6 and $n$7, does not introduce extra coefficients, and reuses fixed thresholds inherited from MaxIoUAssigner (Fang et al., 25 Jul 2025).

A separate but related line of work is SRPN for nuclei and cell detection in histology images. SRPN inserts an embedding layer into an RPN and uses contrastive or triplet losses so that anchors with the same label are close in embedding space and different-label anchors are separated. It is therefore a similarity-based RPN in the metric-learning sense, but it does not compute explicit pairwise similarity scores at inference; classification still uses a standard per-anchor Softmax or focal head (Sun et al., 2021). This distinction matters because PS-RPN, despite its name, is not an inference-time similarity scorer either. Its novelty lies in replacing IoU with a joint position-and-shape similarity inside label assignment and auxiliary proposal supervision for DETR-based small object detection, rather than introducing an embedding-space classifier.

The limitations reported for PS-RPN are correspondingly specific. Its shape term compares width and height independently, so extreme aspect-ratio differences or rotated objects may not be fully captured without orientation-aware penalties or rotated boxes. Because $n$8 and $n$9 are computed from training-set statistics, large domain shifts in camera geometry or object scale may require recomputation. In crowded scenes where objects have very similar centers and shapes, PS may still struggle to differentiate instances, although the paper states that it remains stronger than position-only metrics. The paper also does not report sensitivity to the fixed thresholds, so their behavior in extremely low-contrast settings is left open. This suggests that PS-RPN is most reliable when shape remains a discriminative cue and when dataset statistics used for calibration remain representative (Fang et al., 25 Jul 2025).