OSDA: Open Set Domain Adaptation

• OSDA is an unsupervised domain adaptation variant where the target domain includes both source-shared and target-private unknown classes.
• OSDA methods mitigate negative transfer by aligning shared features and rejecting unknowns using adversarial, pseudo-labeling, and teacher-student strategies.
• Source-free OSDA further challenges adaptation by restricting access to source samples, necessitating robust, threshold-free unknown detection techniques.

Searching arXiv for recent and foundational OSDA papers to ground the article. Open Set Domain Adaptation (OSDA) is the variant of unsupervised domain adaptation in which a labeled source domain and an unlabeled target domain do not share the same label space: the source contains only the known classes, whereas the target contains both shared classes and target-private unknown classes. The task is therefore dual: transfer recognition of shared categories across domain shift, and reject target samples that do not belong to any source class. This makes OSDA structurally different from closed-set UDA, because indiscriminate alignment of all target samples with the source induces negative transfer by pulling unknown-class target features into known-class source clusters (Saito et al., 2018, Jang et al., 2022).

1. Formal problem setting

A standard formulation introduces a labeled source domain

$$\mathcal{D}^s = \{(\mathbf{x}^s_i,y^s_i)\}_{i=1}^{n_s}$$

with labels in a source class set $\mathcal{Y}^s$, and an unlabeled target domain

$\mathcal{D}^t = \{\mathbf{x}^t_j\}_{j=1}^{n_t}$

whose full label space satisfies

$$\mathcal{Y}^t = \mathcal{Y}^s \cup \mathcal{Y}^{unk},$$

with $\mathcal{Y}^{unk}\neq\emptyset$. The resulting classifier is expected to map target inputs into either one of the shared classes or a single reject label,

$$f:\mathbb{R}^{d_0}\to\mathcal{Y}^s\cup\{\text{unk}\}.$$

This formulation appears, with minor notational variation, across backpropagation-based OSDA, graph-based methods, and semantic-recovery variants (Wang et al., 2021, Saito et al., 2018, Jing et al., 2021).

Several papers also quantify the degree of label-space mismatch by the openness

$O = 1-\frac{|C_s|}{|C_t|},$

which increases as the proportion of target-private classes grows (Bucci et al., 2020, Liu et al., 2023). In practice, OSDA must resolve two intertwined uncertainties: whether a target sample belongs to the shared label space at all, and, if it does, how to align it with the source under covariate shift.

A stricter variant is source-free OSDA (SF-OSDA), which removes access to source-domain samples during adaptation and assumes that only a pretrained source model and unlabeled target samples are available. This setting is motivated by privacy, confidentiality, or transmission constraints, and makes OSDA harder because adaptation must proceed without simultaneous source-target access (Jahan et al., 2023, Nejjar et al., 2024).

2. Core difficulties and evaluation criteria

The defining failure mode of OSDA is negative transfer. If a method matches source and target distributions as though the label spaces were identical, target samples from unknown classes are forced toward known-class regions, degrading both shared-class recognition and unknown rejection (Saito et al., 2018, Jang et al., 2022). This is why OSDA methods typically attempt some form of known/unknown separation before, during, or jointly with alignment.

The literature repeatedly identifies three technical difficulties. The first is covariate shift: $p(x)\neq q(x)$, so a domain-invariant representation is required. The second is category shift: target-private classes exist but have no source counterparts. The third, emphasized in imbalanced OSDA, is label shift: $p(y)\neq q(y)$, sometimes combined with intra-domain class imbalance, so a single global unknown-detection threshold can be brittle (Ru et al., 2023). Source-free settings add a fourth difficulty: error accumulation from imperfect pseudo-labeling or imperfect known/unknown splitting, because no source samples remain available to stabilize training (Jahan et al., 2023).

Evaluation in classification OSDA commonly uses the known-class average accuracy $OS^*$, unknown accuracy $\mathcal{Y}^s$0, and a harmonic mean. One standard form is

$\mathcal{Y}^s$1

while some papers write the same quantity as H-score (Wang et al., 2021, Silva et al., 2024). A recurrent criticism of the older $\mathcal{Y}^s$2 metric is that it can under-weight unknown rejection when the number of known classes is large; ROS therefore argues for the harmonic mean because it penalizes imbalance between shared-class accuracy and unknown detection (Bucci et al., 2020). In semantic segmentation, the analogous H-score is computed from $\mathcal{Y}^s$3 and $\mathcal{Y}^s$4 (Choe et al., 2024, Ren et al., 4 Jan 2026).

3. Main methodological families

OSDA methods can be grouped by how they separate unknowns from knowns and how they restrict alignment to the shared subspace.

Family	Mechanism	Representative papers
Adversarial boundary learning	Classifier forms an unknown boundary; generator is driven either to align target samples with source-known regions or to reject them as unknown	(Saito et al., 2018, Chang et al., 2020)
Unknown-aware adversarial alignment	Source and target-known are aligned while target-unknown is explicitly segregated	(Jang et al., 2022)
Progressive pseudo-labeling and rejection	Confident target-known samples are added gradually; low-confidence or rejected samples are treated separately	(Wang et al., 2021, Luo et al., 2022, Liu et al., 2023)
Teacher-student SF-OSDA	EMA teacher guides known/unknown splitting and stabilizes pseudo-labels	(Jahan et al., 2023)
Unknown discovery or exploitation	Unknowns are clustered, synthesized, or used as hard negatives to reshape decision boundaries	(Zhuang et al., 2022, Nejjar et al., 2024, Silva et al., 2024)
Dual-space or multimodal separation	Unknowns are detected through routing-space inconsistency or prompt-gradient behavior	(Du et al., 2023, Chen, 16 May 2025)

The earliest adversarial formulation, “Open Set Domain Adaptation by Backpropagation,” trains a classifier to place target samples on a boundary with target unknown probability $\mathcal{Y}^s$5, while the feature generator is adversarially encouraged to push each target either toward source-known regions or away as unknown (Saito et al., 2018). “Mind the Gap” generalizes this line by decoupling separation and alignment into a Sample Separation Network and a Distribution Matching Network, with soft weights $\mathcal{Y}^s$6 used to exclude probable unknowns from alignment and with a mutual-learning module that transfers information between the two networks (Chang et al., 2020).

Unknown-aware adversarial learning further sharpens this distinction. UADAL introduces a three-way discriminator over source, target-known, and target-unknown, with an adversarial objective that aligns $\mathcal{Y}^s$7 and $\mathcal{Y}^s$8 while driving $\mathcal{Y}^s$9 away. Its theoretical analysis derives the optimal discriminator and rewrites the generator objective in terms of KL divergences, thereby formalizing the simultaneous alignment-and-segregation objective (Jang et al., 2022).

A second major family replaces global alignment with progressive or curriculum-style pseudo-labeling. OSLPP learns a common subspace by graph embedding, progressively selecting confident target-known samples and rejecting probable unknowns; selected and rejected samples then reshape the graph used in the next subspace update (Wang et al., 2021). PGL decomposes the target hypothesis space into shared and unknown subspaces and progressively pseudo-labels the most confident known samples, while SF-PGL adapts this idea to the source-free case through balanced pseudo-labeling that selects confident samples from each category at a fixed ratio and uses class-specific confidence thresholds as uncertainty weights (Luo et al., 2022). SPLOS similarly adopts self-paced learning, replacing manual threshold tuning with a self-tuned instructive threshold and a cross-domain mixup curriculum that moves from easier to harder target-known samples (Liu et al., 2023).

Recent source-free methods emphasize stable unknown separation. USD uses the Jensen–Shannon distance between a hard pseudo-label and a temporally ensembled teacher prediction, then fits a two-component Gaussian Mixture Model to separate target samples into known and unknown subsets. Its co-training objective combines instance-weighted pseudo-cross-entropy, information maximization, triplet regularization, temporal consistency, and curriculum guidance (Jahan et al., 2023). RRDA adopts a different two-stage design: first recall unknowns by generating synthetic target-domain features near the source classifier’s decision boundary, then refine the model with an extended classifier and any off-the-shelf SFDA objective such as SHOT or AaD (Nejjar et al., 2024).

A third trend rejects the assumption that all unknowns should remain a single amorphous reject class. SCDA explicitly argues that indiscriminately treating implicit target categories as one “unknown” can ignore decision boundaries among them; it therefore alternates between discovering target-private classes by clustering and restructuring the classifier to include newly discovered classes (Zhuang et al., 2022). Closely related unknown-exploitation approaches mine high-confidence unknown samples and use them as hard negatives to tighten one-vs-all decision boundaries, either directly, after augmentation, or through GAN-generated synthetic negatives (Silva et al., 2024).

4. Benchmarks, protocols, and empirical patterns

Most classification papers evaluate on Office-31, Office-Home, and VisDA-C/VisDA. Office-31 commonly appears with 10 known and 21 unknown classes (Saito et al., 2018, Wang et al., 2021), while Office-Home appears under more than one protocol: 25 known and 40 unknown in OSLPP, USD, and RRDA-style settings (Wang et al., 2021, Jahan et al., 2023, Nejjar et al., 2024), but 20 known and 45 unknown in the OVANet-based unknown-exploitation study (Silva et al., 2023). This suggests that direct cross-paper comparison requires attention to protocol, not only headline H-score.

Large domain gaps materially change behavior. “Mind the Gap” repurposes PACS for OSDA and reports that existing OSDA methods suffer a larger relative drop when the gap grows, with heavy overlap between known and unknown target features under large shifts (Chang et al., 2020). Reproducibility is itself a documented issue: ROS reports that re-running public code for OSBP, STA, and UAN yields OS accuracies 1.3–4.9 percentage points lower than originally reported on the same splits (Bucci et al., 2020).

Within a fixed protocol, benchmark progress has been substantial. OSLPP reports average HOS values of 87.4% on Office-31 and 67.0% on Office-Home (Wang et al., 2021). UADAL reports approximately 88.1% and 68.7% HOS on the same datasets, with theoretical guarantees for unknown-aware alignment (Jang et al., 2022). DSD, a threshold-free Mixture-of-Experts approach using inconsistencies between image-feature and routing-feature pseudo-labels, reports average HOS values of 94.2% on Office-31 and 75.5% on VisDA (Du et al., 2023). In source-free settings, USD reports 83.3% HOS on Office-31, 62.6% on Office-Home, and 69.4% on VisDA-C, while RRDA reports large improvements over naive open-set extensions of SHOT and AaD, including 93.8 HOS for SHOT+RRDA on Office-31 and 78.7 on VisDA (Jahan et al., 2023, Nejjar et al., 2024).

Alternative evaluation protocols extend beyond plain rejection. SR-OSDA introduces D2AwA and I2AwA to assess not only open-set recognition but also semantic attribute recovery for unseen categories, measuring seen accuracy $\mathcal{D}^t = \{\mathbf{x}^t_j\}_{j=1}^{n_t}$0, unseen accuracy $\mathcal{D}^t = \{\mathbf{x}^t_j\}_{j=1}^{n_t}$1, and harmonic mean $\mathcal{D}^t = \{\mathbf{x}^t_j\}_{j=1}^{n_t}$2 in a generalized zero-shot style protocol (Jing et al., 2021).

5. Extensions and generalizations

Several lines of work generalize classical OSDA rather than merely improving it.

Source-free OSDA removes source samples from adaptation. SF-PGL, USD, and RRDA all target this regime, but they differ in how they stabilize unknown separation: balanced pseudo-labeling in SF-PGL, JSD-plus-GMM teacher-student splitting in USD, and synthetic feature inversion with classifier expansion in RRDA (Luo et al., 2022, Jahan et al., 2023, Nejjar et al., 2024). A recurring theme is that source-free methods must explicitly learn a target-unknown feature space rather than merely exclude uncertain samples.

Imbalanced OSDA (IOSDA) augments the problem with intra-domain class imbalance and inter-domain label shift. OMEGA addresses this through moving-threshold estimation, unknown-aware target clustering, and gradual alignment via neighborhood clustering, reporting gains on Office-Home, VisDA-C, and DomainNet under imbalanced protocols (Ru et al., 2023). This broadens OSDA from a covariate-shift problem with unknowns to a joint covariate-shift, label-shift, and category-mismatch problem.

Semantic enrichment is another extension. SCDA discovers multiple implicit target-private classes rather than treating all unknowns as a single output (Zhuang et al., 2022). SR-OSDA goes further by attempting to recover semantic attributes for unseen target classes through structure-preserving partial alignment and attribute propagation over a visual graph (Jing et al., 2021). A plausible implication is that OSDA can serve not only as a rejection mechanism but also as a discovery mechanism.

Open-set domain adaptation has also moved into dense prediction. BUS introduces OSDA for semantic segmentation (OSDA-SS), identifies boundary prediction and unknown-shape prediction as the central failures of closed-set UDA baselines, and addresses them with a dilation-erosion contrastive loss and OpenReMix augmentation (Choe et al., 2024). SATS then argues explicitly for a two-stage framework—known/unknown separation followed by unknown-aware domain adaptation—reporting H-Score improvements of +3.85% on GTA5-to-Cityscapes and +18.64% on SYNTHIA-to-Cityscapes (Ren et al., 4 Jan 2026).

Multimodal and prompt-based variants have also appeared. A CLIP-based method introduces prompt-driven cross-domain alignment and gradient-aware open-set separation, using the $\mathcal{D}^t = \{\mathbf{x}^t_j\}_{j=1}^{n_t}$3-norm of prompt gradients as a known/unknown detector on Office-Home (Chen, 16 May 2025). Separately, the acronym OSDA has been reused in remote sensing for “Open-Set Discovery and Automatic Interpretation of Land-cover in Remote Sensing Imagery,” a three-stage framework combining SAM-style mask discovery, multimodal LLMs, and LLM-as-judge evaluation (Chen et al., 23 Sep 2025).

6. Conceptual debates and open problems

One persistent misconception is that all unknowns can be collapsed into a single homogeneous reject class without loss. SCDA explicitly challenges this assumption, arguing that the decision boundaries between implicit categories are ignored when all unknowns are treated as one class (Zhuang et al., 2022). DSD, RRDA, and semantic-recovery methods likewise introduce clustered or multi-subclass representations for unknowns, suggesting that “unknown” is often an operational label rather than a semantically coherent category (Du et al., 2023, Nejjar et al., 2024, Jing et al., 2021).

A second unresolved issue is thresholding. Many OSDA pipelines still rely on confidence or entropy thresholds, whether fixed or heuristic. The literature contains several alternatives: self-tuned thresholds in SPLOS, Beta-mixture estimation in UADAL, two-component GMMs over JSD in USD, per-class moving thresholds in OMEGA, and a threshold-free dual-space mismatch rule in DSD (Liu et al., 2023, Jang et al., 2022, Jahan et al., 2023, Ru et al., 2023, Du et al., 2023). This suggests that unknown detection remains as much a calibration problem as a representation problem.

Calibration and drift are recurrent concerns, especially in source-free settings. SF-PGL reports that balanced pseudo-labeling improves calibration and makes the trained model less prone to over-confident or under-confident predictions on target data (Luo et al., 2022). USD attributes major gains to teacher-student co-training because student self-splitting leads to runaway error accumulation (Jahan et al., 2023). RRDA similarly emphasizes stable two-stage refinement after synthetic unknown recall (Nejjar et al., 2024).

Finally, the literature shows an active debate over one-stage versus two-stage training. Unified formulations attempt to discover unknowns and align knowns simultaneously, whereas SATS explicitly defends a separating-then-adapting strategy for semantic segmentation on the grounds that annotation imbalance between known and unknown classes induces negative transfer and underfitting in one-stage systems (Ren et al., 4 Jan 2026). More broadly, open problems noted across the literature include automatic estimation of unknown-class proportion, scalability to many unknown classes, robustness under large domain gaps, extension beyond a single collapsed unknown head, and the move from mere rejection toward discovery, semantic attribution, and continual open-world adaptation (Wang et al., 2021, Chang et al., 2020, Choe et al., 2024).