---
title: 'CollaPAUL: Source-Free Unknown Object Detection'
url: https://www.emergentmind.com/topics/collapaul
type: topic
---

# CollaPAUL: Source-Free Unknown Object Detection

Searching arXiv for the specific paper and closely related SFUOD context.
CollaPAUL is a framework for **Source-Free Unknown Object Detection (SFUOD)** that extends source-free object detection beyond the standard closed-set assumption by adapting a detector to an unlabeled target domain **without access to source data** while also detecting target-domain objects that are outside the source label space as **unknown objects** [2507.17373]. The name denotes **Collaborative tuning and Principal Axis-based Unknown Labeling**, and the method is formulated around two coupled requirements: transfer of source-trained knowledge for known classes and discovery of undefined target objects that cannot be assigned valid source labels. In the formulation presented in the literature, CollaPAUL is not a generic collaborative alignment paradigm, nor is it related to collagen repair despite the lexical similarity of names appearing in other domains; rather, it is a detector adaptation framework built on a Mean Teacher pipeline, using **Deformable DETR** with a **ResNet-50** backbone, with collaborative representation adaptation and a pseudo-labeling mechanism for unknowns as its central technical contributions [2507.17373].

## 1. Problem setting and conceptual scope

CollaPAUL is defined in the context of **Source-Free Unknown Object Detection**, a setting introduced to relax the restrictive premise of conventional source-free object detection. In the standard source-free formulation, a detector is adapted from a labeled source domain to an unlabeled target domain **without requiring access to labeled source data**, but the target is still assumed to contain only the same categories available during source training. The paper identifies this as a **closed set** assumption and argues that it prevents the detector from detecting undefined objects [2507.17373].

The formal input consists of labeled source data
$$
D_s = \{(x_s^i, y_s^i)\}_{i=1}^{|D_s|}
$$
and unlabeled target data
$$
D_t = \{x_t^i\}_{i=1}^{|D_t|}.
$$
Each source annotation is written as
$$
y_s^i = \{(b^j, c^j)\}_{j=1}^{J_i},
$$
with labels drawn from the known class set
$$
\mathcal{Y}_s = \{1,2,\dots,K\}.
$$
The target domain may contain both objects whose classes belong to $\mathcal{Y}_s$ and **novel/undefined objects** not present in source training, which must be detected as **unknown** [2507.17373].

The motivation is operational rather than purely taxonomic. A source-trained detector transferred without source data may be adequate for known categories, but it tends either to ignore novel target objects or to force them into one of the known source classes. The paper characterizes this difficulty as **knowledge confusion**: source-dependent features remain useful for known categories, yet can mislead the detector when unknown objects are present in the target domain [2507.17373]. This framing places CollaPAUL within open-world domain adaptation for detection, but under the additional source-free constraint.

## 2. Framework composition

CollaPAUL stands for **Collaborative tuning and Principal Axis-based Unknown Labeling**. The framework is built on a **Mean Teacher** style source-free adaptation pipeline in which student and teacher share the same detector architecture and initialization. The student is trained from pseudo-labels generated on target data, and the teacher is updated by exponential moving average:
$$
\Bar{\theta}_t \leftarrow \alpha \theta_t + (1-\alpha)\theta_s.
$$
The baseline detector is **Deformable DETR** with a **ResNet-50** backbone [2507.17373].

The architecture adds two main components on top of this base.

First, **collaborative tuning** addresses representation adaptation. It combines **source-dependent knowledge** from the source-trained detector with **target-dependent knowledge** extracted by an auxiliary target encoder. The stated aim is to reduce knowledge confusion by preventing the detector from relying solely on features that are informative for source classes but potentially misleading for unknown target objects [2507.17373].

Second, **Principal Axis-based Unknown Labeling (PAUL)** addresses pseudo-label generation for undefined target objects. Because unknown categories have no source supervision, direct confidence-based pseudo-labeling is considered unreliable. PAUL instead estimates **objectness** by projecting proposal features onto the principal axes of known-object features, then combines this geometric signal with teacher prediction confidence to identify proposals that should be labeled as unknown [2507.17373].

This division of labor is explicit in the reported ablation narrative. Collaborative tuning mainly improves **known mAP**, whereas PAUL mainly improves **unknown recall**; the full framework produces the best balanced performance under **H-Score** [2507.17373]. A plausible implication is that CollaPAUL is structured as a dual mechanism: one module targets representation quality under source-free adaptation, and the other targets label-space openness.

## 3. Collaborative tuning

Collaborative tuning is designed around a distinction between **source-dependent knowledge** and **target-dependent knowledge**. Source-dependent knowledge comes from the detector pre-trained on source classes and remains useful for known categories. Target-dependent knowledge is extracted from target data by an auxiliary encoder intended to capture target-domain structure without being overly constrained by source labels [2507.17373].

The auxiliary target encoder comprises four operations:

1. **top-$k$ feature selection**
2. **truncated SVD reconstruction**
3. **cross-attention with object queries**
4. **MLP projection**

Starting from backbone feature maps $f \in \mathbb{R}^{C \times h \times w}$, the method computes an activation map by channel averaging, selects the top-$k$ activated features $f_a \in \mathbb{R}^{k \times C}$, and applies SVD:
$$
f_a = U \Sigma V^T.
$$
A truncated reconstruction using the top-$r$ components is then formed as
$$
f'_a = U' \Sigma' {V'}^T.
$$
This reconstruction is intended to isolate latent target-domain information. The resulting feature is processed with cross-attention using object queries $q_{obj}$ and then an MLP, yielding target-domain features $f_t$ [2507.17373].

Fusion with source-derived representations is performed through **cross-domain attention**. If the source detector produces features $f_s$, the attention uses object-query-based projections
$$
q = f_s \cdot W_q^T,\quad k=[f_s;f_t]\cdot W_k^T,\quad v=[f_s;f_t]\cdot W_v^T,
$$
where $q \in \mathbb{R}^{N_q \times D}$ and $k,v \in \mathbb{R}^{2N_q \times D}$. The attention weights are defined as
$$
attn = \sigma\left(\begin{Bmatrix} q\cdot k_s^T \ \ q\cdot k_t^T \end{Bmatrix}\right) \in \mathbb{R}^{N_q \times 2 \times N_q},
$$
and the fused output is
$$
f_{cd} = attn \cdot \{v_s; v_t\} \in \mathbb{R}^{N_q\times D}.
$$
These fused features are inserted through special **collaborative layers** between decoder layers, with the paper stating that the layers are inserted into the first $L$ decoder layers and that **$L=3$** is used in the main experiments [2507.17373].

The layerwise recurrence is written as
$$
f_t^l = \theta^{clb_l}(f_s^l, f_t^{l-1}), \qquad f_s^{l+1} = \theta_s^{dec_{l+1}([f_s^l; f_t^l]) ].
$$
The decoder self-attention also uses **prefix tuning** to incorporate target features. The paper further reports that cross-domain attention outperforms prefix tuning because it explicitly aligns source-dependent and target-dependent features through identical object queries rather than mixing all features indiscriminately [2507.17373].

## 4. Principal Axis-based Unknown Labeling

PAUL is the pseudo-labeling component responsible for unknown-object supervision in the unlabeled target domain. The teacher model processes a target image $x_t$ and produces $N_q$ proposals. Known proposals are first pseudo-labeled using a confidence threshold; the reported threshold is **0.3**. If $N_k$ proposals are accepted as known, the remaining proposals are $N_{re}=N_q-N_k$ [2507.17373].

The underlying assumption is that known and unknown objects share **objectness**, whereas background proposals do not. PAUL therefore derives a geometric criterion from known proposal features. Let $f_{kn}$ denote known proposal features, $f_{re}$ the remaining proposal features, and $P$ the principal axes. The projection step is
$$
\Bar{f}_{kn}=f_{kn}\cdot P^T, \qquad \Bar{f}_{re}=f_{re}\cdot P^T.
$$
For the $i$-th known proposal, the objectness score is estimated via cosine similarity among projected known proposals:
$$
s_{kn}^{i}=\frac{1}{N_k-1}\sum_{j=1, j\neq i}^{N_k} d(\Bar{f}_{kn}^i,\Bar{f}_{kn}^j),
$$
where $d(\cdot,\cdot)$ is cosine similarity. The threshold is then set as the average known objectness score,
$$
\delta = \text{avg}(S_{kn}), \qquad S_{kn}=\{s_{kn}^1,\dots,s_{kn}^{N_k}\}.
$$
An **objectness mask** is defined as
$$
M_{obj}= \{m_{obj}^i \mid m_{obj}^i=\mathbb{I}(s_{un}^i\geq \delta)\}_{i=1}^{N_{re} ].
$$
In parallel, teacher class predictions provide a **confidence mask**
$$
M_{conf}= \{m_{obj}^j \mid m_{obj}^j=\mathbb{I}(c_{un}^j\geq \epsilon)\}_{j=1}^{N_{re} ],
$$
where $c_{un}^j$ is the unknown confidence score and $\epsilon$ is a confidence threshold. The final unknown mask is obtained by logical union:
$$
M_{unk}= M_{obj} \vee M_{conf}.
$$
Selected proposals are then assigned pseudo-labels, including class and box labels, from teacher predictions [2507.17373].

Within the paper’s interpretation, PAUL is more reliable than raw confidence alone because it retains proposals that behave like real objects in the feature geometry induced by known objects. This suggests that PAUL operationalizes a weak notion of open-world objectness without requiring source data or explicit unknown-category supervision.

## 5. Training pipeline and optimization

CollaPAUL uses a Mean Teacher training procedure rather than introducing a fundamentally new global optimization objective. The base detector is trained with standard detection losses,
$$
\mathcal{L}_{det}=\mathcal{L}_{cls}(\hat{c}_{s},\hat{c}_{t}) + \mathcal{L}_{\mathrm{L}_1}(\hat{b}_{s},\hat{b}_{t}) + \mathcal{L}_{giou}(\hat{b}_{s},\hat{b}_{t}),
$$
where $\mathcal{L}_{cls}$ is focal classification loss, $\mathcal{L}_{L_1}$ is box regression loss, and $\mathcal{L}_{giou}$ is generalized IoU loss [2507.17373].

The operational procedure is described as follows:

1. the teacher predicts proposals on weakly augmented target images;
2. known objects are pseudo-labeled by confidence threshold;
3. PAUL identifies likely unknown proposals and assigns unknown labels;
4. the student is trained on strongly augmented images using the resulting pseudo-labels;
5. the teacher is updated via EMA from the student;
6. collaborative tuning injects target-dependent information into the detector during decoder processing [2507.17373].

This arrangement is significant because CollaPAUL’s novelty lies less in loss design than in pseudo-label construction and representation fusion. The detector continues to optimize standard detection criteria, but those criteria are applied to a pseudo-labeled target distribution that now includes both known and unknown instances.

## 6. Benchmarks, metrics, and empirical results

The empirical evaluation is conducted on two SFUOD benchmarks derived from source-free object detection settings:

- **Weather adaptation**: source **Cityscapes**, target **Foggy Cityscapes (fog density 0.02)**
- **Cross-scene adaptation**: source **Cityscapes**, target **BDD100K daytime subset**

Known classes are described as vehicle categories such as **Car**, **Truck**, and **Bus**. Unknown classes include other road objects such as **Person**, **Rider**, **Motorcycle**, **Train**, and **Bicycle** [2507.17373].

The reported metrics are **Known mAP**, **U-Recall**, and **H-Score**, with H-Score defined as the harmonic mean of known mAP and unknown recall. These metrics jointly evaluate adaptation to known classes and the capacity to discover undefined objects [2507.17373].

| Benchmark | Method | Known mAP / U-Recall / H-Score |
|---|---|---|
| Cityscapes $\rightarrow$ Foggy Cityscapes | Source only | 26.56 / 0.00 / 0.00 |
| Cityscapes $\rightarrow$ Foggy Cityscapes | Mean Teacher | 16.91 / 6.02 / 8.88 |
| Cityscapes $\rightarrow$ Foggy Cityscapes | DRU | 22.97 / 3.60 / 6.22 |
| Cityscapes $\rightarrow$ Foggy Cityscapes | **CollaPAUL** | **32.32 / 10.59 / 15.95** |
| Cityscapes $\rightarrow$ BDD100K | Source only | 24.41 / 0.00 / 0.00 |
| Cityscapes $\rightarrow$ BDD100K | Mean Teacher | 26.42 / 6.08 / 9.89 |
| Cityscapes $\rightarrow$ BDD100K | DRU | 17.24 / 6.86 / 9.82 |
| Cityscapes $\rightarrow$ BDD100K | **CollaPAUL** | **28.21 / 8.57 / 13.15** |

On both benchmarks, CollaPAUL surpasses the listed baselines in known-class detection, unknown-object recall, and the harmonic balance between them [2507.17373]. The paper’s quantitative pattern is especially notable because gains are not confined to unknown recall; collaborative tuning also improves known-class transfer, indicating that open-set adaptation is not treated as a separate post hoc module but as part of the feature adaptation process.

The ablation study reinforces this interpretation. On Foggy Cityscapes, the **DRU baseline** yields **22.97 / 3.60 / 6.22**; adding **collaborative tuning only** gives **30.63 / 3.56 / 6.38**; adding **PAUL only** gives **25.40 / 6.46 / 10.30**; and the **full CollaPAUL** gives **32.32 / 10.59 / 15.95** [2507.17373]. This decomposition supports the claim that the two components are complementary rather than redundant.

## 7. Interpretation, limitations, and relation to other usages of the name

CollaPAUL is best understood as a **source-free open-world detector for unknown-object-aware domain adaptation** [2507.17373]. Its practical assumptions are explicit: a labeled source detector is available initially, the target domain is unlabeled, the source class set is fixed and predefined, and unknown classes may appear in the target and should be detected as unknown rather than being forced into known labels [2507.17373].

Several technical points delimit its scope. The best tradeoff in the reported experiments occurs at **$L=3$** collaborative layers; too few layers underuse the integration mechanism, while too many degrade performance. For PAUL, **$\epsilon=0.3$** is reported as a strong setting, and larger thresholds improve known mAP but reduce U-Recall, reflecting a standard precision–recall tradeoff [2507.17373]. The objectness mask substantially improves unknown recall, and combining objectness and confidence masks yields the best H-Score [2507.17373].

A common potential misconception concerns the meaning of “unknown.” In this framework, “unknown” does not denote a set of semantically identified new categories. It denotes target-domain objects that are **not part of the source label space** and therefore must be localized and assigned the label **unknown** rather than mapped to a known class [2507.17373]. Another possible misunderstanding arises from the name itself. The literature cited here contains no indication that CollaPAUL is related to cultural alignment frameworks or biomaterials research that use superficially similar vocabulary; the explicit mention is confined to the SFUOD detector framework, where the acronym expands to **Collaborative tuning and Principal Axis-based Unknown Labeling** [2507.17373].

The broader significance of CollaPAUL lies in the claim that source-free object detection need not remain bound to a closed-set assumption. By combining target-dependent representation extraction with principal-axis-based unknown pseudo-labeling, the method demonstrates that a detector trained on source classes can be adapted to a new domain while also discovering previously undefined objects as unknowns [2507.17373]. This suggests a transition from conventional source-free adaptation toward a more open-world source-free detection regime, though the formulation remains anchored to the specific detection benchmarks and architectural choices reported in the original work.

Source: https://www.emergentmind.com/topics/collapaul