FALCON-SFOD: Robust Source-Free Object Detection

• The paper introduces FALCON-SFOD, a domain adaptation framework that integrates foundation model priors with imbalance-aware noise robust pseudo-labeling to combat degraded object focus and noisy labels.
• It employs Spatial Prior-Aware Regularization (SPAR) to align feature maps using OV-SAM masks and Imbalance-aware Noise-Robust Pseudo-Labeling (IRPL) to sharpen pseudo-label distributions.
• Empirical results across diverse benchmarks demonstrate significant mAP improvements over prior methods, validating its effectiveness under severe domain shifts.

FALCON-SFOD (Foundation-Aligned Learning with Clutter Suppression and Noise Robustness) is a domain adaptation framework for Source-Free Object Detection (SFOD) that augments Mean-Teacher self-labeling with foundation model priors and imbalance-aware noise-robust pseudo-labeling. FALCON-SFOD is designed to address the challenges of degraded object focus and noisy pseudo-labels when adapting an object detector pre-trained on a labeled source domain to an unlabeled target domain, particularly under significant domain shift, without direct access to source data during adaptation (VCR et al., 19 Dec 2025).

1. Source-Free Object Detection and Problem Motivation

The SFOD problem entails adapting a detector $h_\mathrm{pre}$, originally trained on source dataset $\mathcal{D}_S$, to a target dataset $\mathcal{D}_T$ that is entirely unlabeled, with no $\mathcal{D}_S$ access during adaptation. The prevalent solution—the Mean-Teacher paradigm—employs a student $h^{st}$, updated through pseudo-labels generated by a teacher $h^{te}$ (its EMA copy). Domain shift renders pseudo-labels unreliable due to two core issues:

• Diffuse feature activations in the backbone $g(x)$, generating strong background clutter responses.
• Resultant degradation in both localization (bounding boxes drifting to clutter) and classification (erroneous labels).

FALCON-SFOD addresses these with two complementary mechanism:

• SPAR (Spatial Prior-Aware Regularization): Imposes foundation model-driven, class-agnostic spatial priors to enforce foreground structure in feature maps.
• IRPL (Imbalance-aware Noise-Robust Pseudo-Labeling): Modifies the pseudo-labeling loss for robustness against noisy teacher predictions and foreground-background imbalance.

2. SPAR: Spatial Prior-Aware Regularization

SPAR enhances the discriminativity of the detector’s feature space by leveraging offline, class-agnostic binary masks generated by a frozen open-vocabulary segmenter (OV-SAM). For a target image $x^t$, segmentation produces mask $A_G(x^t)\in\{0,1\}^{H'\times W'}$ where $A_G[j,k]=1$ indicates foreground. The detector’s feature map $a = g^{st}(x^t)\in\mathbb{R}^{H\times W\times C}$ is channel-averaged (yielding $A_S$), then resized and aligned to $A_G$.

The SPAR loss is:

$$\mathcal{L}_\mathrm{SPAR}(x^t) = \frac{\lambda_1}{H'W'} \sum_{j,k} \lvert A_S[j,k] - A_G[j,k] \rvert \; + \; \lambda_2\left(1 - \frac{2\sum_{j,k}A_S[j,k]A_G[j,k]}{\sum_{j,k}A_S[j,k] + \sum_{j,k}A_G[j,k] + \varepsilon}\right)$$

with fixed hyperparameters $\lambda_1=1$, $\lambda_2=2$, and $\varepsilon=10^{-6}$. The first (ℓ₁) term enforces pixelwise alignment, while the Dice loss fosters overlap and shape correspondence with the prior mask. OV-SAM masks are computed once and cached, yielding zero online inference overhead. This prior-driven constraint directly sharpens object-focus in the learned feature space.

3. IRPL: Imbalance-Aware Noise-Robust Pseudo-Labeling

IRPL addresses pseudo-label noise and the inherent foreground-background class imbalance. For each pseudo-box $(\hat b, \hat c)$ (teacher-generated), the student produces class probabilities $\mathbf{p}$. The peak-adjust transform:

$$t = \arg\max_i p_i,\quad p'_i = \begin{cases} \frac{p_i + m}{1 + m}, & i = t \ \frac{p_i}{1 + m}, & i \neq t \end{cases}$$

($m=10^4$) sharpens the distribution, accentuating the student’s most confident class.

The IRPL loss aggregates per-box contributions:

$$\mathcal{L}_{\mathrm{IRPL}} = \sum_{(\hat b, \hat c)\in\hat{\mathcal{Y}}^t} w_{\hat c} \big[\alpha(-\log p'_{\hat c}) + \beta (1-p_{\hat c})\big] + \gamma D_{\mathrm{KL}}(\bar{\mathbf p} \| \mathcal{U})$$

where $w_{fg}=2$ for foreground and $w_{bg}=1$ for background, $\alpha=0.1$, $\beta=2$, $\gamma=0.01$. The KL-divergence term $D_{\mathrm{KL}}(\bar{\mathbf p} \| \mathcal{U})$ penalizes class distribution collapse. If the student agrees with teacher, the gradient on $-\log p'_{\hat c}$ is diminished (“early stop”); if not, the full gradient is backpropagated, enabling loss correction.

Key hyperparameters (fixed for all reported experiments) are:

• Margin $m=10^4$,
• Foreground/background weighting $(w_{fg}, w_{bg}) = (2,1)$,
• Confidence threshold for pseudo-labels $=0.8$.

4. Theoretical Risk Bound Analysis

The detection risk decomposes as $$R^\mathrm{det}(f) = R^\mathrm{cls} + R^\mathrm{reg}$$, where $R^\mathrm{cls}$ is classification risk and $R^\mathrm{reg}$ is localization (regression) risk. Under Mean-Teacher with noisy pseudo-labels, Lemma 1 implies

$$R^{cls}_{clean}(f_c)\leq \frac{1}{\lambda}R^{cls}_{noisy}(f_c)$$

for transition matrix minimum diagonal $\lambda = \min_j T_{jj}>0$. Lemma 2 yields

$$R^\mathrm{reg}_\mathrm{clean}(f_r) \leq R^\mathrm{reg}_\mathrm{noisy}(f_r) + \eta_{reg} + 2\zeta$$

where $\eta_{reg}$ quantifies bounding box misalignment and $\zeta$ the expected rate of missing true boxes.

Theorem 1 aggregates both, showing the excess risk is multiplicative in $1/\lambda$ for standard Mean-Teacher. However, Theorem 2 demonstrates that IRPL, via the peak-adjust loss, replaces this with an additive $O(\delta)$ term (with $\delta\rightarrow 0$ as $m\rightarrow \infty$), thereby strictly tightening the risk bound when $\lambda<1$. SPAR’s prior-driven regularization directly decreases the localization error terms.

5. Implementation Protocols and Hyperparameter Settings

FALCON-SFOD uses Faster-R-CNN + FPN as the base detector (identical to Simple-SFOD), trained and adapted on a single NVIDIA RTX A6000. All hyperparameters remain fixed throughout experiments:

• SPAR: $\lambda_1=1$, $\lambda_2=2$, $\varepsilon=10^{-6}$
• IRPL: $m=10^4$, $\alpha=0.1$, $\beta=2$, $\gamma=0.01$, $w_{fg}=2$, $w_{bg}=1$
• Optimization: SGD with momentum $0.9$ and weight decay $10^{-4}$; learning rates—source: $0.04$, adaptation: $0.0025$; batch size $4$
• Teacher EMA decay $\delta=0.9996$

OV-SAM masks are generated once per image and stored. There is no requirement for source data or online segmentation during target adaptation.

6. Empirical Results and Comparative Analysis

FALCON-SFOD surpasses prior state-of-the-art methods (Simple-SFOD, PETS, DRU, among others) on representative SFOD benchmarks across diverse domain shifts. Salient results:

• Cityscapes $\rightarrow$ Foggy Cityscapes: mAP $=$ 46.9% (Simple-SFOD: 45.0%; DRU: 43.7%)
• Sim10k $\rightarrow$ Cityscapes (car-only): AP$_{\text{car}}$ $=$ 58.8% (Simple-SFOD: 55.4%)
• KITTI $\rightarrow$ Cityscapes (car-only): AP $=$ 50.1% (DRU: 45.1%; PETS: 47.0%)
• Cityscapes $\rightarrow$ BDD100k: mAP $=$ 36.9% (Simple-SFOD: 34.3%)
• PascalVOC $\rightarrow$ Clipart: 35.5% (vs. 33.6% baseline)
• FLIR Visible $\rightarrow$ Infrared: 58.5%; FLIR Infrared $\rightarrow$ COCO: 20.9%

Ablation studies confirm both SPAR and IRPL contribute independently and additively to performance. For instance, adding SPAR to the baseline (C$\rightarrow$F) increases mAP to 46.1%; IRPL alone yields 45.8%; both together, 46.9%. Tail class improvements are marked, with significant AP gains for train (+4.1%), truck (+4.0%), bus (+2.9%), and motorcycle (+2.4%); head classes benefit less, evidencing IRPL’s skew-balancing effect.

7. Ablation, Limitations, and Potential Extensions

Critical ablation results are structured as follows:

Component Abla.	C→F mAP	S→C AP_car
Baseline	45.0	55.4
+SPAR	46.1	57.5
+IRPL	45.8	56.8
Full	46.9	58.8

The effectiveness of OV-SAM masks in SPAR surpasses alternatives such as GSAM or ESC-Net. IRPL ablation shows incremental boosts from the peak-adjust transform, fg/bg weighting, and entropy regularization.

Limitations include reliance on the quality of offline segmentation masks (where severe errors degrade performance) and fixed hyperparameters, which may require sympathetic retuning for extreme shifts. Precomputing masks for large datasets is parallelizable but can require up to 20 minutes.

Prospective research avenues proposed include developing online or self-supervised mask refinement, extending SPAR to transformer or multi-scale architectures, integrating class-aware priors (e.g., Open-Vocabulary DETR), and combining IRPL with alternative noise-robust or curriculum learning approaches (VCR et al., 19 Dec 2025).