PF-RPN: Prompt-Free Universal Region Proposals

• The paper introduces a novel PF-RPN architecture that eliminates external prompts by leveraging a Sparse Image-Aware Adapter, Cascade Self-Prompt, and Centerness-Guided Query Selection.
• PF-RPN is designed for zero-shot cross-domain detection, enabling high-quality, class-agnostic proposals transferable across 19 diverse datasets.
• Empirical results demonstrate significant improvements in average recall, latency, and VRAM efficiency compared to traditional RPNs and prompt-based methods.

Prompt-Free Universal Region Proposal Network (PF-RPN) denotes a class-agnostic proposal mechanism that identifies potential objects from image features without exemplar-, category-, or text prompts. In the most specific usage, it refers to the 2026 PF-RPN architecture, which introduces a Sparse Image-Aware Adapter (SIA), a Cascade Self-Prompt (CSP), and Centerness-Guided Query Selection (CG-QS) for zero-shot cross-domain proposal generation across 19 datasets (Tang et al., 18 Mar 2026). In a broader conceptual usage, the term also captures properties already present in the original Region Proposal Network (RPN) of Faster R-CNN: a fully convolutional, class-agnostic proposal head that shares features with the detector and generates nearly cost-free proposals without external proposal generators (Ren et al., 2015).

1. Antecedents in classical Region Proposal Networks

The original Faster R-CNN RPN established the core proposal-generation template from which later prompt-free formulations can be interpreted. It is a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position, shares full-image convolutional features with the detection network, and thereby makes proposals nearly cost-free relative to external proposal algorithms such as Selective Search or EdgeBoxes (Ren et al., 2015). In that framework, “prompt-free” is a conceptual characterization: no external proposal “prompts” are required, because the proposal mechanism operates directly on shared convolutional features and is trained end-to-end.

The canonical RPN architecture uses a deep CNN such as VGG-16 with 13 shareable convolutional layers and feature stride approximately 16 pixels on the last convolutional map for single-scale inputs rescaled to a shorter side $s=600$ pixels. A $3\times 3$ convolution produces a 512-D feature vector for VGG-16, followed by ReLU, and two sibling $1\times 1$ convolutions emit $2k$ classification scores and $4k$ regression values. With the typical choice $k=9$, anchors are formed from the Cartesian product of three scales, $\{128^2, 256^2, 512^2\}$, and three aspect ratios, $\{1:1, 1:2, 2:1\}$. The anchors and proposal functions are translation-invariant, and the outputs are a two-class softmax for objectness and parameterized bounding-box deltas for each anchor.

Its box parameterization is relative to an anchor with center $(x_a,y_a)$, width $w_a$, and height $3\times 3$0: $3\times 3$1 The inverse transform is

$3\times 3$2

Training is IoU-driven: positives have IoU $3\times 3$3 with any ground-truth box or are the highest-IoU anchor for each ground-truth box, negatives have IoU $3\times 3$4 with all ground-truth boxes, and others are ignored. Mini-batches are image-centric, with 256 anchors per image and at most 128 positives. Proposal generation applies the predicted deltas to anchors, clips boxes to image boundaries, ranks them by objectness, applies NMS with IoU threshold $3\times 3$5, and commonly retains the top 300 proposals at test time. With VGG-16 and shared features, total detection time is approximately 198 ms/image on a K40 GPU, yielding about 5 fps including proposals and detection, while the RPN’s marginal cost is about 10 ms/image when features are shared.

These properties explain why the original RPN is often described, retrospectively, as both prompt-free and universal: prompt-free because it removes external proposal inputs, and universal because it predicts class-agnostic objectness and box refinements over shared, translation-invariant features.

2. Problem formulation and motivation of PF-RPN

The 2026 PF-RPN addresses a different failure mode from the original Faster R-CNN bottleneck. Its starting point is that existing localization methods typically rely on exemplar images, predefined categories, or textual descriptions, and that prompt-free open-vocabulary detection often invokes large captioners to auto-generate text, creating heavy memory, latency, and potential bias (Tang et al., 18 Mar 2026). Conventional RPNs are also described as struggling on unseen domains when trained on specific detection datasets.

The explicit goal is to propose high-quality, class-agnostic region proposals from an input image $3\times 3$6, suitable as the front-end of downstream detection pipelines, without any external prompts. In this formulation, “prompt-free” no longer means merely removing Selective Search or EdgeBoxes. It means removing any dependence on image prompts, exemplar prompts, category lists, or textual prompts, while still identifying potential objects. “Universal” refers to zero-shot cross-domain applicability: the model is jointly fine-tuned on 5% COCO and 5% ImageNet with pseudo boxes, then directly applied without fine-tuning to underwater object detection, industrial defect detection, and remote sensing image object detection.

This design targets the object-localization stage itself rather than downstream classification semantics. A plausible implication is that PF-RPN treats proposal generation as a transferable front-end problem: the region generator should remain useful even when category vocabularies, imaging conditions, and object appearance statistics change. The reported emphasis on latency and memory further distinguishes it from captioner-based prompt-free pipelines.

3. Architecture: SIA, CSP, and CG-QS

PF-RPN adopts Grounding DINO as the base detector, with either a Swin-B or ResNet-50 backbone, and uses multi-level image features $3\times 3$7 for $3\times 3$8, where deeper levels capture semantics and shallower ones encode fine structure (Tang et al., 18 Mar 2026). A single learnable visual embedding $3\times 3$9 is initialized, updated by SIA, and then iteratively refined by CSP. The flattened multi-level features form transformer memory $1\times 1$0, and PF-RPN replaces Grounding DINO’s language-guided query selection with CG-QS.

The SIA module is a mixture-of-experts adapter. It computes compact global features $1\times 1$1 by GAP, feeds them to a router MLP, selects the top-$1\times 1$2 feature levels with $1\times 1$3 and default $1\times 1$4, normalizes their weights by softmax, and updates $1\times 1$5 by cross-attention over concatenated global and local features: $1\times 1$6 To avoid over-activation of a few levels, SIA adds a router load-balancing loss

$1\times 1$7

The CSP module refines the embedding in a deep-to-shallow cascade. Starting from $1\times 1$8, it forms a similarity mask at level $1\times 1$9 by thresholding cosine similarity: $2k$0 and updates the embedding by masked average pooling: $2k$1 The default cascade depth is 3 iterations. The stated purpose is to progressively purify the embedding by aggregating object-internal cues while suppressing background.

The CG-QS module predicts a centerness score $2k$2 for each query using a lightweight MLP. Its supervision follows the FCOS-style centerness target

$2k$3

where $2k$4 are distances from the query to the matched ground-truth box edges. The centerness loss is

$2k$5

Final candidate queries are ranked using a combination of centerness and classification scores derived from dot products between the refined embedding and queries.

A central architectural distinction from classical anchor-based RPNs is that PF-RPN decodes proposals as in DETR-like architectures, without NMS, whereas the Faster R-CNN RPN uses anchors, box-delta regression relative to anchors, and NMS-based top-$2k$6 proposal selection.

4. Objective functions, training regime, and inference

PF-RPN uses a compound objective composed of regression, classification, router regularization, and centerness terms (Tang et al., 18 Mar 2026). Regression follows DETR-like configurations: $2k$7 with $2k$8 in DETR-style parameterization. GIoU is defined through

$2k$9

where $4k$0 is the smallest enclosing box of $4k$1 and $4k$2. Classification is a contrastive loss between queries and the refined embedding $4k$3, following Grounding DINO’s configuration. The total loss is

$4k$4

with an ablated best $4k$5.

Training uses joint fine-tuning on 5% COCO, comprising 80 detection classes, and 5% ImageNet, comprising 1000 classification classes, with pseudo bounding boxes for ImageNet images. The stated reason is to reduce the distribution gap between image classification pretraining and detection fine-tuning. An empirical study reports strong gains from 1% to 5% and diminishing returns beyond 5%. Hardware consists of four NVIDIA RTX 4090 GPUs. The key hyperparameters are $4k$6 feature levels in SIA, $4k$7, cascade iterations $4k$8, and centerness weight $4k$9. Optimizer, batch size, schedule, and augmentations follow the Grounding DINO setup and are not altered by PF-RPN.

At inference time, queries are decoded as in DETR-like architectures, without NMS, and ranked with the combined classification-centerness score. Performance is reported at top-100, top-300, and top-900 proposals using AR@100/300/900. The query count $k=9$0 follows the base model’s default. This inference procedure places PF-RPN closer to transformer-based dense decoding than to the classical Faster R-CNN proposal pipeline, even though both operate as class-agnostic proposal generators.

5. Empirical performance, ablations, and detector integration

Evaluation is conducted on 19 datasets in zero-shot cross-domain settings: ODinW13, which includes 13 varied domains such as wildlife, household, and aerial imagery, and CD-FSOD, which includes ArTaxOr, Clipart1k, DIOR, DeepFish, NEU-DET, and UODD (Tang et al., 18 Mar 2026). Metrics are AR@100/300/900 and AR$k=9$1/AR$k=9$2/AR$k=9$3; AP is reported when PF-RPN is integrated into detectors.

On CD-FSOD, PF-RPN reports 60.7/65.3/68.2 AR and 38.5/61.9/80.3 for small, medium, and large objects. The gains are $k=9$4 AR over Grounding DINO with prompt-free “object,” $k=9$5 AR over Grounding DINO with true class names, $k=9$6 AR over YOLOE-v8-L, and $k=9$7 AR over Cascade RPN, whose corresponding scores are 45.8/52.0/56.9. On ODinW13, PF-RPN reports 76.5/78.6/79.8 AR and 45.4/71.9/85.8 for small, medium, and large objects, with gains of $k=9$8 AR over prompt-free Grounding DINO, $k=9$9 AR over Grounding DINO with true class names, and $\{128^2, 256^2, 512^2\}$0 AR over Cascade RPN.

The efficiency profile is a major part of the reported contribution. CSP iterations add marginal overhead: increasing from 1 to 3 iterations changes latency from 214.3 to 218.9 ms/image, an increase of approximately 4.6 ms/image. PF-RPN with a Grounding DINO backbone reaches 4.6 FPS and about 0.5G VRAM, while integration with YOLO-World reaches about 25.1 FPS. Relative to GenerateU, PF-RPN reports $\{128^2, 256^2, 512^2\}$1 AR@100 on CD-FSOD together with about $\{128^2, 256^2, 512^2\}$2 faster inference, 4.6 versus 0.22 FPS, and about 95% less VRAM, 0.5G versus 12.2G.

Ablations attribute gains to all three modules. On CD-FSOD AR@100, the baseline Grounding DINO score is 52.9; adding SIA yields 57.8, CSP yields 58.1, CG-QS yields 54.4, SIA+CSP yields 60.2, SIA+CG-QS yields 60.2, CSP+CG-QS yields 59.6, and all three yield 60.7. Removing the MoE from SIA lowers performance from 60.7/65.3/68.2 to 58.6/61.2/63.5 on CD-FSOD, and from 76.5/78.6/79.8 to 68.7/71.7/73.8 on ODinW13. Swin-B gives a $\{128^2, 256^2, 512^2\}$3 AR@100 gain over the base Grounding DINO, while ResNet-50 gives $\{128^2, 256^2, 512^2\}$4. The centerness weight is best at $\{128^2, 256^2, 512^2\}$5; too small weakens centerness learning, and too large hurts regression.

PF-RPN is also evaluated as a component inside downstream detectors. Replacing RPN in DE-ViT raises AP from 33.0 to 36.7, a gain of $\{128^2, 256^2, 512^2\}$6. Integration into CD-ViTO raises AP from 29.6 to 35.1 on CD-FSOD, a gain of $\{128^2, 256^2, 512^2\}$7. These results are consistent with the intended use of PF-RPN as a front-end proposal generator rather than a standalone recognition model.

6. Prompt-free universality through self-supervised RPN pre-training

A distinct but related instantiation of the prompt-free universal idea appears in RPN pre-training. “Label-Efficient Object Detection via Region Proposal Network Pre-Training” introduces Aligned Detection Pre-Training (ADePT), which pre-trains the RPN using only self-supervised signals and maps naturally onto a PF-RPN interpretation: no task prompts, no human annotations, and transfer across datasets, tasks, and detector variants (Dong et al., 2022).

In ADePT, the RPN is trained to regress and score pseudo labels generated by Selective Search. During pre-training, a random subset of $\{128^2, 256^2, 512^2\}$8 boxes is taken as pseudo ground-truth, anchors are matched by IoU-based assignment, and the standard RPN loss is applied: $\{128^2, 256^2, 512^2\}$9 Here, $\{1:1, 1:2, 2:1\}$0 is binary cross-entropy and $\{1:1, 1:2, 2:1\}$1 is Smooth L1, active only for positives. The pretext is paired with a BYOL-style detector similarity loss over three augmented views $\{1:1, 1:2, 2:1\}$2, and the total pre-training objective is

$\{1:1, 1:2, 2:1\}$3

In joint training, gradients from $\{1:1, 1:2, 2:1\}$4 flow to the backbone, while RPN parameters are updated by $\{1:1, 1:2, 2:1\}$5 only. For stability, the detector loss uses Selective Search proposals rather than evolving RPN proposals.

The architecture is an anchor-based RPN on top of a ResNet-50 backbone with FPN. During pre-training it uses FPN levels $\{1:1, 1:2, 2:1\}$6, base anchor areas $\{1:1, 1:2, 2:1\}$7, aspect ratios $\{1:1, 1:2, 2:1\}$8, top-64 proposals before NMS, and $\{1:1, 1:2, 2:1\}$9 proposals per image for the detector loss. Unlabeled ImageNet is used as pre-training data. Selective Search boxes are pre-computed with default parameters $(x_a,y_a)$0, $(x_a,y_a)$1, and $(x_a,y_a)$2, then filtered by aspect ratio and normalized scale. Pre-training uses LARS with learning rate 0.1 and cosine decay for 100 epochs on $(x_a,y_a)$3 V100 GPUs.

The downstream evidence supports the claim that proposal pre-training improves label efficiency. On full COCO, ADePT (separate) reports 41.9 AP$(x_a,y_a)$4 / 37.6 AP$(x_a,y_a)$5, compared with SoCo at 41.8 / 37.3. On COCO 10%, the corresponding scores are 28.6 / 25.9 versus 28.2 / 25.5; on COCO 1%, 15.3 / 14.1 versus 15.0 / 13.9. On SODA10M, ADePT (joint) reports 43.6 AP$(x_a,y_a)$6, compared with 43.3 for SoCo. On VOC few-shot detection, the paper reports large margins under very small $(x_a,y_a)$7, including Split 2 novel classes at $(x_a,y_a)$8 shots, where ADePT (joint) reaches 5.55 versus 2.29 for SoCo, and Split 3 at $(x_a,y_a)$9, where ADePT (joint) reaches 5.72 versus 3.42. TIDE error stratification on COCO reports lower localization error for ADePT than for SoCo, with Loc error 6.28 versus 6.45.

This line of work broadens the meaning of PF-RPN. In the 2026 architecture, prompt-freeness is realized through image-only query refinement and centerness-guided selection. In ADePT, prompt-freeness is realized through self-supervised pre-training without human labels or prompts. The common theme is that proposal generation itself, rather than only the backbone, is treated as a transferable objectness and localization module.

7. Limitations, ambiguities, and future directions

The 2026 PF-RPN identifies several failure modes and sensitivities (Tang et al., 18 Mar 2026). Without CG-QS, queries near object edges can produce inaccurate boxes; centerness alleviates but does not eliminate all such errors. Small and occluded objects remain difficult even with SIA+CSP, leading to misses or lower-quality proposals. The model is sensitive to $w_a$0 and $w_a$1: too many feature levels add noise, and too large a centerness weight over-emphasizes centerness and hurts regression. Performance also depends on the base detector, with stronger backbones and transformer heads setting a higher ceiling.

The self-supervised pre-training route has a different set of limitations (Dong et al., 2022). It depends on Selective Search pseudo labels, which can be noisy in low light or extreme blur; it assumes an anchor-based RPN with specific scales and aspect ratios during pre-training; and its benefits are most pronounced in label-scarce regimes, while gains are smaller in fully labeled large-scale settings. These observations suggest that “universal” does not imply invariance to all forms of domain shift or architectural choice.

Across the cited works, the term PF-RPN therefore spans three related meanings: a conceptual reading of classical RPN as a prompt-free and universal proposal generator, a specific Grounding DINO-based architecture with SIA, CSP, and CG-QS, and a transferable pre-trained proposal module that can be learned without human labels. Future steps proposed in the literature include anchor-free variants, IoU-aware objectives, better pseudo labels from unsupervised segmentation or masking, cross-domain pre-training corpora, multi-task pre-training with auxiliary geometric signals, and compatibility with modern detectors. This suggests that PF-RPN is best understood not as a single fixed design, but as a research program centered on class-agnostic, transferable proposal generation without external prompts.