SFUOD: Source-Free Unknown Object Detection

• SFUOD is an open-world extension of source-free object detection that adapts a source-trained detector to identify both known and unknown objects without accessing source images.
• Collaborative Tuning fuses source-dependent and target-dependent features, leading to a significant gain in known-class detection (e.g., +7.66 mAP improvement).
• PAUL employs principal axis projection and cosine similarity to generate reliable pseudo-labels for unknown instances, boosting unknown recall under domain shifts.

Source-Free Unknown Object Detection (SFUOD) is an open-world extension of source-free object detection in which a detector trained on a labeled source domain is adapted to an unlabeled target domain without ever accessing source images, while the target domain contains both source-defined classes and previously unseen classes (Park et al., 23 Jul 2025). In this setting, the detector is required not only to retain detection capability on known classes but also to detect instances of novel categories as a special unknown class. The formulation therefore relaxes the closed-set assumption of source-free object detection and reframes adaptation under simultaneous domain shift and category-set expansion.

1. Formal setting and task definition

SFUOD extends the source-free object detection setting from a closed-set regime to an open-world regime. The source dataset is defined as $D_s=\{(x_s^i,y_s^i)\}_{i=1}^{N_s}$, where each label is $y_s^i=\{(b_j,c_j)\}_{j=1}^{J_i}$ with $c_j\in\mathcal{Y}_s$. The target dataset is unlabeled and given by $D_t=\{x_t^i\}_{i=1}^{N_t}$. Known classes are $\mathcal{Y}_s=\{1,\ldots,K\}$, and unknown classes in the target domain are $\mathcal{Y}_t=\{K+1,\ldots,K+K'\}$ (Park et al., 23 Jul 2025).

The detection objective is explicitly twofold. First, the adapted detector should produce accurate detections for base-class instances, with predicted labels $\hat c\in\mathcal{Y}_s$. Second, it should assign the label “unknown” to instances belonging to $\mathcal{Y}_t$. In the reported formulation, these two goals are operationalized as maximizing known mean average precision and maximizing unknown recall.

This definition places SFUOD at the intersection of domain adaptation and open-world detection. A plausible implication is that SFUOD is stricter than conventional open-world object detection because the source data are unavailable during adaptation, and stricter than conventional source-free object detection because the target label space is no longer assumed to be identical to the source label space.

2. Central difficulties in the source-free open-world regime

Two difficulties are identified as central when moving from closed-set SFOD to SFUOD (Park et al., 23 Jul 2025). The first is knowledge confusion. Because the source-trained detector only knows $\mathcal{Y}_s$, it tends during adaptation to misclassify unknown objects as one of the base classes, and conversely may confuse base-class instances under domain shift.

The second is ineffective pseudo-labeling. Standard SFOD relies on a teacher–student Mean-Teacher framework, but the MT teacher has no notion of unknown classes and therefore cannot generate reliable pseudo-labels for them. This limitation is structural rather than incidental: pseudo-labeling mechanisms inherited from closed-set adaptation assume that high-confidence predictions over known classes are sufficient supervision.

These difficulties clarify why simply reusing source-free object detection machinery is inadequate in the open-world case. This suggests that SFUOD requires both improved feature adaptation and an explicit mechanism for assigning supervisory signals to unknown instances.

3. CollaPAUL: collaborative tuning and principal axis–based unknown labeling

To address these difficulties, the reported method introduces CollaPAUL, a framework that integrates Collaborative Tuning and Principal Axis-based Unknown Labeling (PAUL) into a Mean-Teacher pipeline (Park et al., 23 Jul 2025).

In Collaborative Tuning, an auxiliary target encoder is added alongside the source-trained student DETR encoder. The auxiliary branch extracts backbone features $f\in\mathbb{R}^{C\times h\times w}$, computes the per-pixel activation map $y_s^i=\{(b_j,c_j)\}_{j=1}^{J_i}$0, selects the top-$y_s^i=\{(b_j,c_j)\}_{j=1}^{J_i}$1 activations $y_s^i=\{(b_j,c_j)\}_{j=1}^{J_i}$2, and applies singular value decomposition,

$y_s^i=\{(b_j,c_j)\}_{j=1}^{J_i}$3

followed by truncation to the top-$y_s^i=\{(b_j,c_j)\}_{j=1}^{J_i}$4 components to produce $y_s^i=\{(b_j,c_j)\}_{j=1}^{J_i}$5. Cross-attention with object queries $y_s^i=\{(b_j,c_j)\}_{j=1}^{J_i}$6 and a small MLP then yields target-dependent features $y_s^i=\{(b_j,c_j)\}_{j=1}^{J_i}$7. In parallel, the student DETR encoder produces source-dependent features $y_s^i=\{(b_j,c_j)\}_{j=1}^{J_i}$8.

Feature fusion occurs between decoder layers $y_s^i=\{(b_j,c_j)\}_{j=1}^{J_i}$9 and $c_j\in\mathcal{Y}_s$0 through cross-domain attention: $c_j\in\mathcal{Y}_s$1

$c_j\in\mathcal{Y}_s$2

An MLP refines $c_j\in\mathcal{Y}_s$3, and the resulting feature $c_j\in\mathcal{Y}_s$4 is fed into decoder layer $c_j\in\mathcal{Y}_s$5 together with the usual decoder input. Repeating this for $c_j\in\mathcal{Y}_s$6 collaborative layers, with the empirical best reported at $c_j\in\mathcal{Y}_s$7, is intended to preserve source knowledge while enriching target-dependent representation.

PAUL handles unknown pseudo-label generation. After the Mean-Teacher teacher produces $c_j\in\mathcal{Y}_s$8 proposal features and class confidences, the method first selects $c_j\in\mathcal{Y}_s$9 known proposals whose maximum base-class confidence exceeds a threshold $D_t=\{x_t^i\}_{i=1}^{N_t}$0. Their feature vectors $D_t=\{x_t^i\}_{i=1}^{N_t}$1 are used to compute principal axes $D_t=\{x_t^i\}_{i=1}^{N_t}$2 via PCA. Known and remaining proposals are projected as

$D_t=\{x_t^i\}_{i=1}^{N_t}$3

Objectness is then estimated by intra-class cosine similarity: $D_t=\{x_t^i\}_{i=1}^{N_t}$4 With

$D_t=\{x_t^i\}_{i=1}^{N_t}$5

the method defines

$D_t=\{x_t^i\}_{i=1}^{N_t}$6

and combines them as

$D_t=\{x_t^i\}_{i=1}^{N_t}$7

All proposals satisfying $D_t=\{x_t^i\}_{i=1}^{N_t}$8 are assigned the pseudo-label $D_t=\{x_t^i\}_{i=1}^{N_t}$9.

The architecture is therefore explicitly bifurcated: Collaborative Tuning addresses knowledge adaptation, while PAUL addresses unknown-label construction. This suggests that the framework treats representation transfer and unknown discovery as separable but coupled subproblems.

4. Optimization procedure and supervision structure

The reported training algorithm operates on target-domain inputs $\mathcal{Y}_s=\{1,\ldots,K\}$0 using weak and strong augmentations. The teacher network $\mathcal{Y}_s=\{1,\ldots,K\}$1, without gradient updates, processes the weakly augmented image $\mathcal{Y}_s=\{1,\ldots,K\}$2 and outputs proposals $\mathcal{Y}_s=\{1,\ldots,K\}$3. Known pseudo-labels are obtained by selecting proposals whose maximum confidence over $\mathcal{Y}_s=\{1,\ldots,K\}$4 exceeds $\mathcal{Y}_s=\{1,\ldots,K\}$5, and PAUL is then applied to the remaining proposals to generate unknown pseudo-labels (Park et al., 23 Jul 2025).

The student network $\mathcal{Y}_s=\{1,\ldots,K\}$6 is applied to the strongly augmented image $\mathcal{Y}_s=\{1,\ldots,K\}$7. The detection loss is

$\mathcal{Y}_s=\{1,\ldots,K\}$8

The student parameters are updated by backpropagation with respect to $\mathcal{Y}_s=\{1,\ldots,K\}$9, and the teacher is updated by exponential moving average: $\mathcal{Y}_t=\{K+1,\ldots,K+K'\}$0

The reported implementation uses AdamW, a batch size of 8 with 2 per GPU on four 3090s, $\mathcal{Y}_t=\{K+1,\ldots,K+K'\}$1, and thresholds $\mathcal{Y}_t=\{K+1,\ldots,K+K'\}$2. Collaborative layers inside $\mathcal{Y}_t=\{K+1,\ldots,K+K'\}$3 fuse $\mathcal{Y}_t=\{K+1,\ldots,K+K'\}$4 and $\mathcal{Y}_t=\{K+1,\ldots,K+K'\}$5 during this optimization process.

A common misconception is that unknown detection in this setting is achieved solely by confidence thresholding. The described optimization contradicts that view: PAUL combines a confidence criterion with PCA-derived objectness, and the final unknown mask is the logical disjunction $\mathcal{Y}_t=\{K+1,\ldots,K+K'\}$6. The method is therefore not purely a score-thresholding procedure.

5. Benchmarks, metrics, and reported performance

The experimental setup uses two benchmarks: weather adaptation from Cityscapes to Foggy Cityscapes with fog density 0.02, and cross-scene adaptation from Cityscapes to BDD100K in daytime conditions (Park et al., 23 Jul 2025). Known classes are Car, Truck, and Bus. Unknown classes are Person, Rider, Motorcycle, Bicycle, and Train.

Three evaluation metrics are reported. Known mAP is the mean AP over base classes at 0.5 IoU. U-Recall is the fraction of novel-class instances detected as “unknown.” H-Score is the harmonic mean of Known mAP and U-Recall.

On Cityscapes→Foggy Cityscapes, CollaPAUL reports Known mAP 32.32, U-Recall 10.59, and H-Score 15.95. The best SFOD baseline, Mean Teacher, reports Known mAP 26.56, U-Recall 6.02, and H-Score 8.88. On Cityscapes→BDD100K, CollaPAUL reports Known mAP 28.21, U-Recall 8.57, and H-Score 13.15, while the best SFOD baseline reports Known mAP 26.68, U-Recall 6.86, and H-Score 9.89.

These results indicate that the proposed setting is not evaluated solely in terms of base-class retention. The inclusion of U-Recall and H-Score makes explicit that SFUOD performance is defined by a trade-off between adaptation on known categories and sensitivity to novel categories.

6. Ablation findings, qualitative behavior, and stated limitations

The ablation results isolate the contribution of the two principal modules (Park et al., 23 Jul 2025). Collaborative Tuning, relative to no collaborative module, yields a gain of $\mathcal{Y}_t=\{K+1,\ldots,K+K'\}$7 points in known mAP. PAUL, relative to confidence-only unknown labeling, yields a gain of $\mathcal{Y}_t=\{K+1,\ldots,K+K'\}$8 points in U-Recall. Combining both gives Known mAP 32.32, U-Recall 10.59, and H-Score 15.95. Varying the number of collaborative layers shows the best result at $\mathcal{Y}_t=\{K+1,\ldots,K+K'\}$9.

Qualitatively, on Foggy Cityscape, CollaPAUL is reported to correctly mark pedestrians and bicycles as unknown using red boxes, whereas SFOD baselines confuse them or miss them entirely. This qualitative pattern is consistent with the stated motivation of alleviating knowledge confusion.

The stated limitations are also specific. Although the framework substantially alleviates knowledge confusion and improves unknown recall under the strict source-free constraint, it still relies on heuristic thresholds $\hat c\in\mathcal{Y}_s$0 and PCA-based objectness, which may falter under extreme domain shifts. The future directions proposed in the source include learned objectness priors, threshold calibration via self-supervision, and the incorporation of a few seed unknown examples to tighten the open-world detection loop.

A plausible implication is that SFUOD remains sensitive to the quality of pseudo-label construction. Because both known and unknown supervision are synthesized from target-only predictions, threshold design and feature geometry remain central to robustness.