Point2RBox-v3: Efficient Point-Supervised OOD Method

• The paper presents a self-bootstrapping approach that refines pseudo labels via Progressive Label Assignment and Prior-Guided Dynamic Mask Loss, improving detection under weak supervision.
• Point2RBox-v3 leverages watershed segmentation and SAM-derived cues to produce dynamic pseudo boxes for scale-aware FPN label assignment in remote-sensing imagery.
• Empirical results show that combining improved pseudo-label quality with refined label utilization enhances performance while reducing annotation costs compared to previous methods.

Searching arXiv for the core paper and closely related predecessors/competitors. Point2RBox-v3 is a point-supervised oriented object detection (OOD) method that learns oriented bounding boxes from one annotated point and a class label per instance. It was introduced as a self-bootstrapping extension of the Point2RBox line, with two stated targets: improving pseudo-label quality and improving pseudo-label utilization. Its defining additions are Progressive Label Assignment (PLA), which uses stage-dependent pseudo boxes for standard FPN label assignment, and Prior-Guided Dynamic Mask Loss (PGDM-Loss), which combines watershed and SAM-derived mask supervision according to scene sparsity. In the reported formulation, Point2RBox-v3 retains the Point2RBox-v2 detector backbone and loss structure while coupling pseudo-label refinement and detector training more tightly than earlier variants (Zhang et al., 30 Sep 2025).

1. Position within point-supervised oriented detection

Point-supervised OOD assumes that training annotations provide only a point per object, typically a center-like click, together with the class label, while inference still requires full oriented boxes. This supervision regime is materially weaker than horizontal-box or rotated-box supervision because width, height, and orientation are all latent. The motivation is annotation economy: Point2RBox-v3 cites Point2RBox as reporting that annotating an RBox is 36.5% more expensive than an HBox and 104.8% more expensive than a point (Zhang et al., 30 Sep 2025).

The Point2RBox family evolved through distinct mechanisms for recovering missing geometry. The original "Point2RBox: Combine Knowledge from Synthetic Visual Patterns for End-to-end Oriented Object Detection with Single Point Supervision" introduced synthetic pattern knowledge combination, transform self-supervision, and a classification-score-based assignment strategy on an FPN-free YOLOF design (Yu et al., 2023). "Point2RBox-v2: Rethinking Point-supervised Oriented Object Detection with Spatial Layout Among Instances" replaced external sketch-like priors with spatial-layout reasoning, notably Gaussian overlap loss, Voronoi watershed loss, consistency loss, edge loss, and copy-paste augmentation, while assigning all points to P3 because object size was unavailable (Yu et al., 6 Feb 2025). Point2RBox-v3 explicitly keeps the strong Point2RBox-v2 baseline and adds PLA and PGDM-Loss as training-time mechanisms for scale-aware assignment and scene-adaptive mask supervision (Zhang et al., 30 Sep 2025).

A recurring source of confusion is the existence of nearby but distinct method families. "P2RBox: Point Prompt Oriented Object Detection with SAM" is a SAM-based pseudo-label generation pipeline that re-ranks point-prompted masks with centrality and semantic/boundary guidance before training a conventional rotated detector (Cao et al., 2023). "PointOBB-v3: Expanding Performance Boundaries of Single Point-Supervised Oriented Object Detection" is another neighboring single-point OOD framework, built around multi-view MIL, symmetry-based angle learning, and an end-to-end detector branch without additional priors (Zhang et al., 23 Jan 2025). Point2RBox-v3 is not a renaming of either method; it is a separate continuation of the Point2RBox-v2 line.

2. Task formulation and integrated self-bootstrapping

The method targets OOD under point supervision in remote-sensing imagery. During training, only a point set and class labels are given. The detector must still predict oriented rectangles, effectively parameterized by center $(x_c,y_c)$, width $w$, height $h$, and angle $\theta$. Point2RBox-v3 argues that earlier point-supervised methods suffer from two coupled deficiencies: pseudo labels are often poor, and coarse geometric information already present in pseudo labels is not fully exploited by the detector, especially in FPN label assignment (Zhang et al., 30 Sep 2025).

The full pipeline is explicitly progressive. In the early stage, pseudo masks and pseudo boxes are produced from watershed segmentation seeded by annotated points. In sparse scenes, PGDM-Loss may instead route the image to a SAM branch to obtain stronger mask supervision. Those mask- or watershed-derived signals are used not only for width/height supervision, but also for FPN label assignment through PLA. After a switch epoch, the detector’s own predictions near each point become dynamic pseudo labels, so the model moves from static pseudo labels to online pseudo labels generated by its current forward pass. The paper characterizes this as self-bootstrapping rather than teacher-student learning, because there is no EMA teacher and no separate teacher network (Zhang et al., 30 Sep 2025).

This integrated design is the method’s central conceptual move. Pseudo labels are not treated merely as weak regression targets; they are also used to decide where each object is learned in the pyramid. A plausible implication is that Point2RBox-v3 addresses a limitation already visible in Point2RBox-v2, where all points were assigned to P3 and scale information therefore remained underused (Yu et al., 6 Feb 2025).

3. Progressive Label Assignment

PLA is the mechanism that restores standard multi-level assignment under point supervision. In conventional anchor-based or anchor-free detectors, FPN assignment depends on object scale. Under point supervision, no width or height is given, so earlier systems often avoided classical assignment or collapsed supervision onto a single level. PLA uses pseudo boxes as approximate size carriers and does so progressively, beginning with watershed outputs and later replacing them with dynamic detector predictions (Zhang et al., 30 Sep 2025).

In the early stage, pseudo labels are constructed by a geometric-image-processing chain:

$$V = \text{Voronoi}(X), \qquad S = \text{Watershed}(I, X, V), \qquad PL = \text{minAreaRect}(S),$$

where $X$ is the set of annotated points, $V$ is the Voronoi partition, $S$ is the watershed segmentation, and $PL$ is the pseudo-label set. These coarse oriented boxes are then fed to a standard label assignment procedure.

After the switch epoch $E_s$, PLA replaces static watershed boxes with dynamic pseudo labels from the detector itself. For each ground-truth point $w$0, one candidate prediction is selected from each FPN level by choosing the box whose associated anchor point is closest to $w$1. The final pseudo label is the highest-scoring candidate:

$w$2

Those $w$3 boxes are then used for standard FPN label assignment. The novelty claim is specific: Point2RBox-v3 states that it is the first end-to-end point-supervision model to employ dynamic pseudo labels for label assignment (Zhang et al., 30 Sep 2025).

The stage transition is a genuine curriculum. The ablation varies the switch epoch over $w$4. The best performance is reported at 3 or 6, and the adopted default is 6. Setting $w$5 means always using network predictions; setting $w$6 means always using watershed. Intermediate switching performs better than either extreme, which the paper interprets as evidence that predictions are unreliable too early, while static watershed labels become limiting too late (Zhang et al., 30 Sep 2025).

PLA is described as detector-agnostic with respect to standard FPN assignment because both anchor-based and anchor-free detectors rely on scale information. In the concrete implementation, it plugs into the FPN-based Point2RBox-v2 detector rather than replacing the detector architecture itself.

4. Prior-Guided Dynamic Mask Loss

PGDM-Loss is the method’s mask-supervision module. Its premise is that watershed and SAM have complementary failure modes: watershed works well in dense scenes where neighboring instances provide useful spatial separation cues, but performs poorly in sparse scenes; SAM is stronger in sparse scenes because it relies on visual semantics and boundaries, but degrades in dense scenes through over-segmentation or merging (Zhang et al., 30 Sep 2025).

The routing rule is based on scene sparsity. If the number of instances in an image is at most $w$7, the image is sent to the SAM branch; otherwise it is processed by the watershed branch. The best threshold reported in the ablation is

$w$8

This is the dynamic component of PGDM. It is not an EMA-style temporal update of masks across epochs; rather, it is per-image branch selection according to the number of instances (Zhang et al., 30 Sep 2025).

For a SAM-routed instance $w$9 of class $h$0, candidate masks are denoted

$h$1

The chosen mask is

$h$2

where $h$3 are mask-quality indicators and $h$4 are class-specific prior weights. The five indicators listed for this scoring are center alignment, color consistency, rectangularity, circularity, and aspect ratio reliability. The class-specific weights may be positive, zero, or negative depending on whether a feature is characteristic, irrelevant, or misleading for that category. The example given is that circularity for basketball court is assigned a negative weight to avoid selecting only the center circle (Zhang et al., 30 Sep 2025).

The center-alignment term is explicitly defined as

$h$5

when the prompt point lies inside the mask’s minimum-area bounding rectangle, with a heavy penalty otherwise. Color consistency is

$h$6

Rectangularity and circularity are

$h$7

and aspect ratio reliability uses $h$8, expecting $h$9 for remote-sensing categories that are not extremely elongated (Zhang et al., 30 Sep 2025).

After mask selection, PGDM supervises detector scale rather than semantic segmentation. Given a selected mask $\theta$0, width and height targets are derived in the predicted box frame:

$\theta$1

where $\theta$2 is the predicted center and $\theta$3 is the rotation matrix of the current prediction. The resulting mask loss is a Gaussian Wasserstein Distance loss:

$\theta$4

The total PGDM loss averages this instance-level loss over objects in the image. Thus PGDM contributes primarily to width/height learning, which in turn improves later pseudo boxes and PLA behavior (Zhang et al., 30 Sep 2025).

5. Architecture, implementation, and empirical behavior

Point2RBox-v3 retains the detector components of Point2RBox-v2: ResNet-50, FPN, and the PSC angle coder, together with inherited losses grouped as $\theta$5, including Gaussian overlap loss, edge loss, copy-paste based box regression loss, and self-supervised consistency loss (Zhang et al., 30 Sep 2025). The implementation uses PyTorch 1.13.1 and MMRotate 1.0.0, with AdamW, initial learning rate $\theta$6, 500 warmup iterations, and a 12-epoch schedule. Random flip is used for all datasets. DOTA and STAR are split into $\theta$7 patches with overlap 200, DIOR is resized to $\theta$8, and RSAR is resized to $\theta$9. No multi-scale testing is used (Zhang et al., 30 Sep 2025).

A practical design choice is that MobileSAM is used only during training. It is not used at inference time, so test-time speed is unaffected. On DOTA-v1.0, MobileSAM is reported to achieve the same AP$$V = \text{Voronoi}(X), \qquad S = \text{Watershed}(I, X, V), \qquad PL = \text{minAreaRect}(S),$$0 as the basic SAM model in their setup while reducing training time by 10 hours. The routing threshold also materially affects cost: the reported total training times are 13.6h for $$V = \text{Voronoi}(X), \qquad S = \text{Watershed}(I, X, V), \qquad PL = \text{minAreaRect}(S),$$1, 19.5h for $$V = \text{Voronoi}(X), \qquad S = \text{Watershed}(I, X, V), \qquad PL = \text{minAreaRect}(S),$$2, 23.5h for $$V = \text{Voronoi}(X), \qquad S = \text{Watershed}(I, X, V), \qquad PL = \text{minAreaRect}(S),$$3, and 79.0h for $$V = \text{Voronoi}(X), \qquad S = \text{Watershed}(I, X, V), \qquad PL = \text{minAreaRect}(S),$$4, so sending all images to SAM is both slower and less accurate (Zhang et al., 30 Sep 2025).

The main reported results are summarized below.

Dataset	End-to-end AP$$V = \text{Voronoi}(X), \qquad S = \text{Watershed}(I, X, V), \qquad PL = \text{minAreaRect}(S),$$5	Two-stage / FCOS AP$$V = \text{Voronoi}(X), \qquad S = \text{Watershed}(I, X, V), \qquad PL = \text{minAreaRect}(S),$$6
DOTA-v1.0	59.61	66.09
DOTA-v1.5	48.34	56.86
DOTA-v2.0	34.47	41.28
DIOR	41.50	46.40
STAR	14.60	19.60
RSAR	40.80	45.96

On DOTA-v1.0, the end-to-end result is 59.61, compared with 51.00 for Point2RBox-v2, 59.04 for P2RBox, 46.00 for PMS-SAM-RSD, and 41.20 for PointOBB-v3. In the two-stage FCOS track, Point2RBox-v3 reaches 66.09, compared with 62.61 for Point2RBox-v2, 59.04 for P2RBox, and 49.24 for PointOBB-v3. The paper also emphasizes strong category-level behavior on Bridge (BR), Roundabout (RA), and Soccer-ball field (SBF) on DOTA-v1.0, while remaining strong on dense categories such as SH, SV, LV, PL, ST, and TC (Zhang et al., 30 Sep 2025).

The module ablations indicate that PLA contributes more than PGDM when added individually, but that the gains are complementary. Starting from Point2RBox-v2 on DOTA-v1.0, the baseline is 51.0 end-to-end and 62.6 FCOS; adding PLA only gives 56.6/64.6; adding PGDM only gives 54.2/63.9; adding both yields 59.6/66.1. This suggests that the method’s central claim is not merely better pseudo-mask quality but the combination of better pseudo-label quality with better pseudo-label utilization (Zhang et al., 30 Sep 2025).

6. Interpretation, neighboring methods, and limitations

Point2RBox-v3 is best understood as a method that recasts point-supervised OOD as a coupled problem of pseudo-label refinement and label-assignment control. Relative to the original Point2RBox, which learned regression from synthetic visual patterns and transform consistency on an FPN-free detector, Point2RBox-v3 moves toward a more conventional FPN detector by restoring multi-level assignment through dynamically estimated pseudo scales (Yu et al., 2023). Relative to Point2RBox-v2, which relied on spatial layout, Voronoi watershed, and all-P3 assignment, Point2RBox-v3 keeps the same backbone detector but uses pseudo labels directly for assignment and uses scene-adaptive mask selection to compensate for the density dependence of watershed (Yu et al., 6 Feb 2025).

The relation to other point-supervised OOD lines is instructive. The SAM-based P2RBox uses point-prompted SAM masks, a Constrainer Module, an Inspector Module, and symmetry-axis estimation to generate pseudo rotated boxes before training standard detectors (Cao et al., 2023). PointOBB-v3 instead uses multi-view MIL, Scale-Sensitive Consistency, Scale-Sensitive Feature Fusion, symmetry-based Self-Supervised Angle loss, Dense-to-Sparse matching, and an end-to-end detector branch with Instance-Aware Weighting (Zhang et al., 23 Jan 2025). Point2RBox-v3 differs from both: it neither centers its design on MIL-based pseudo-box emergence nor on synthetic-pattern priors, but on self-bootstrapped pseudo-label assignment and scene-dependent mask supervision (Zhang et al., 30 Sep 2025).

The paper explicitly identifies several limitations. PLA gain may be smaller when object scales are uniform, because its advantage is improved scale-aware assignment. Performance in sparse scenes remains partly dependent on SAM characteristics; the authors note that SAM is more sensitive to color than to texture or edges, so it can fail when sparse objects blend with their surroundings or have unclear boundaries. Some categories, including basketball court and ground track field, remain difficult because of weak boundaries or ambiguous segmentation. The method also introduces additional engineering complexity through the progressive assignment schedule, MobileSAM integration, prior-guided scoring, and scene-routing logic (Zhang et al., 30 Sep 2025).

The future directions stated for the method are correspondingly practical rather than purely architectural: richer prompts for SAM, edge-aware algorithms to generate rough mask prompts, more holistic architectural upgrades, and more inventive loss designs to extract stronger supervision from extremely sparse annotations. This suggests that Point2RBox-v3 should be read less as the end of the Point2RBox line than as a reorganization of weak supervision around a single thesis: pseudo labels matter not only as regression targets, but as dynamic control signals for how the detector allocates responsibility across scale.