LEAD: Learning Decomposition for SF-UniDA

• LEAD is a source-free universal domain adaptation method that decomposes target features into source-known and unknown components using SVD of classifier weights.
• It constructs adaptive, instance-level decision boundaries by combining cosine distances from target prototypes and source anchors for reliable pseudo-label assignment.
• The approach shows improved H-scores and significant runtime reductions on benchmarks like Office-31, Office-Home, VisDA, and DomainNet.

LEAD, short for Learning Decomposition, is a method for source-free universal domain adaptation that addresses the central identification problem in Universal Domain Adaptation under simultaneous covariate shift and label shift: determining whether an unlabeled target sample belongs to shared source-known categories or to target-private unknown categories when source data are unavailable during adaptation (Qu et al., 2024). Its defining idea is to decompose target features into source-known and source-unknown components using the source classifier’s weight geometry, then replace global thresholding or iterative clustering with adaptive, instance-level decision boundaries for pseudo-labeling (Qu et al., 2024).

1. Problem setting and adaptation objective

Universal Domain Adaptation relaxes the closed-set assumption by allowing both covariate shift and label shift. In the formulation used by LEAD, the source label set is $\mathcal{Y}_s$ and the target label set is $\mathcal{Y}_t$, with shared labels $\mathcal{Y}_k=\mathcal{Y}_s\cap\mathcal{Y}_t$ and target-private unknown labels $\mathcal{Y}_u=\mathcal{Y}_t\setminus\mathcal{Y}_s$ (Qu et al., 2024). The distribution shift has two components: $P_s(x)\neq P_t(x)$ and $\mathcal{Y}_s\neq \mathcal{Y}_t$, with the target label discrepancy unknown beforehand in the UniDA setting (Qu et al., 2024).

Source-free UniDA adds an additional deployment constraint: no source data are accessible during adaptation. LEAD assumes only a source-trained model $f^s_\theta=h^s_\theta\circ g^s_\theta$ is provided, together with unlabeled target samples $\{x_i^t\}_{i=1}^{N_t}$ (Qu et al., 2024). As in prior source-free adaptation, the classifier is frozen, $h^t_\theta=h^s_\theta$, while the target feature extractor $g^t_\theta$ is updated on the target domain (Qu et al., 2024).

Within this setting, LEAD pursues two coupled objectives. The first is unknown detection: identifying target samples in $\mathcal{Y}_t$0 and avoiding assignment of known-class labels. The second is target pseudo-labeling: assigning robust pseudo-labels to target samples likely to belong to $\mathcal{Y}_t$1, so that those labels can supervise adaptation of the target encoder (Qu et al., 2024). The method is motivated by the observation that the key bottleneck in SF-UniDA is not merely classification under shift, but discrimination between covariate-shifted known samples and genuinely target-private unknown samples.

2. Orthogonal feature decomposition and unknownness estimation

LEAD constructs an orthogonal decomposition of each normalized target feature into components aligned with source-known and source-unknown subspaces (Qu et al., 2024). Let $\mathcal{Y}_t$2 denote a unit-normalized target feature and let $\mathcal{Y}_t$3 be a learned source-known subspace. The corresponding decomposition is

$\mathcal{Y}_t$4

where $\mathcal{Y}_t$5 for an orthonormal basis $\mathcal{Y}_t$6 of $\mathcal{Y}_t$7 (Qu et al., 2024). The intended interpretation is that known target instances should remain more aligned with the source-known subspace, whereas target-private unknown instances should exhibit larger projections onto the orthogonal complement.

In LEAD, the source-known subspace is derived from the source classifier weights $\mathcal{Y}_t$8. Because those class weight vectors are not necessarily orthonormal, the method performs singular value decomposition,

$\mathcal{Y}_t$9

and uses the first $\mathcal{Y}_k=\mathcal{Y}_s\cap\mathcal{Y}_t$0 columns of $\mathcal{Y}_k=\mathcal{Y}_s\cap\mathcal{Y}_t$1, $\mathcal{Y}_k=\mathcal{Y}_s\cap\mathcal{Y}_t$2, as the known basis, while the remaining columns $\mathcal{Y}_k=\mathcal{Y}_s\cap\mathcal{Y}_t$3 define the unknown basis (Qu et al., 2024). For a normalized target feature

$\mathcal{Y}_k=\mathcal{Y}_s\cap\mathcal{Y}_t$4

LEAD writes

$\mathcal{Y}_k=\mathcal{Y}_s\cap\mathcal{Y}_t$5

with $\mathcal{Y}_k=\mathcal{Y}_s\cap\mathcal{Y}_t$6, $\mathcal{Y}_k=\mathcal{Y}_s\cap\mathcal{Y}_t$7, and

$\mathcal{Y}_k=\mathcal{Y}_s\cap\mathcal{Y}_t$8

(Qu et al., 2024).

The unknownness score is defined as

$\mathcal{Y}_k=\mathcal{Y}_s\cap\mathcal{Y}_t$9

LEAD models the distribution of $\mathcal{Y}_u=\mathcal{Y}_t\setminus\mathcal{Y}_s$0 with a two-component Gaussian Mixture Model, exploiting a bimodal pattern with a low-mean common component and a high-mean private component, denoted by $\mathcal{Y}_u=\mathcal{Y}_t\setminus\mathcal{Y}_s$1 (Qu et al., 2024). This gives the method a one-dimensional, decomposition-based indicator of unknownness rather than a confidence- or entropy-based heuristic. The reported ablations further show that using entropy as the decomposition indicator under the same framework is inferior to $\mathcal{Y}_u=\mathcal{Y}_t\setminus\mathcal{Y}_s$2 (Qu et al., 2024).

3. Instance-level decision boundaries and pseudo-label assignment

A central claim of LEAD is that global thresholds are sub-optimal under UniDA because covariate shift is heterogeneous across classes and samples (Qu et al., 2024). To address this, the method constructs instance-level decision boundaries that combine distances to target prototypes and source anchors.

Target prototypes $\mathcal{Y}_u=\mathcal{Y}_t\setminus\mathcal{Y}_s$3 are estimated from top-$\mathcal{Y}_u=\mathcal{Y}_t\setminus\mathcal{Y}_s$4 high-confidence target instances for each class, while source anchors $\mathcal{Y}_u=\mathcal{Y}_t\setminus\mathcal{Y}_s$5 are taken directly from $\mathcal{Y}_u=\mathcal{Y}_t\setminus\mathcal{Y}_s$6 (Qu et al., 2024). Using cosine distance

$\mathcal{Y}_u=\mathcal{Y}_t\setminus\mathcal{Y}_s$7

LEAD defines two commonness scores for sample $\mathcal{Y}_u=\mathcal{Y}_t\setminus\mathcal{Y}_s$8 and class $\mathcal{Y}_u=\mathcal{Y}_t\setminus\mathcal{Y}_s$9: $P_s(x)\neq P_t(x)$0

$P_s(x)\neq P_t(x)$1

clips both to $P_s(x)\neq P_t(x)$2, and fuses them through the geometric mean

$P_s(x)\neq P_t(x)$3

(Qu et al., 2024). Under equal distances, $P_s(x)\neq P_t(x)$4, which encodes a bias toward source-anchor proximity as evidence of knownness and treats target-prototype proximity alone as less decisive because target-private contamination is possible (Qu et al., 2024).

LEAD then estimates a class-wise expectation $P_s(x)\neq P_t(x)$5 of $P_s(x)\neq P_t(x)$6 using the same top-$P_s(x)\neq P_t(x)$7 sampling employed for prototype computation. With $P_s(x)\neq P_t(x)$8 taken from the GMM private component, it defines the per-instance boundary

$P_s(x)\neq P_t(x)$9

(Qu et al., 2024). This boundary varies across both classes and samples, rather than being fixed globally.

Pseudo-label assignment proceeds by first choosing

$\mathcal{Y}_s\neq \mathcal{Y}_t$0

If

$\mathcal{Y}_s\neq \mathcal{Y}_t$1

the sample is classified as unknown; otherwise it is assigned the known pseudo-label

$\mathcal{Y}_s\neq \mathcal{Y}_t$2

(Qu et al., 2024). This direct computation of $\mathcal{Y}_s\neq \mathcal{Y}_t$3 removes the need for iterative clustering to derive pseudo-labeling thresholds.

A plausible implication is that LEAD turns unknown detection from a global calibration problem into a local compatibility test between decomposition magnitude and class-conditioned commonness. That interpretation follows from the structure of $\mathcal{Y}_s\neq \mathcal{Y}_t$4, but the concrete mechanism remains the boundary definition above.

4. Training objective and source-free optimization procedure

LEAD optimizes the target encoder with three losses: pseudo-label learning, feature decomposition regularization, and feature consensus regularization (Qu et al., 2024). The pseudo-label learning term uses certainty weighting based on the sample’s distance from the adaptive boundary. With predicted softmax probabilities $\mathcal{Y}_s\neq \mathcal{Y}_t$5, the certainty weight is defined through a Student’s $\mathcal{Y}_s\neq \mathcal{Y}_t$6-based form,

$\mathcal{Y}_s\neq \mathcal{Y}_t$7

with $\mathcal{Y}_s\neq \mathcal{Y}_t$8 (Qu et al., 2024). The corresponding cross-entropy term is

$\mathcal{Y}_s\neq \mathcal{Y}_t$9

Unknown-labeled instances are not mapped to an extra $f^s_\theta=h^s_\theta\circ g^s_\theta$0-th class; instead they are assigned a uniform target distribution to avoid biasing known-class decision boundaries (Qu et al., 2024).

The feature decomposition regularizer explicitly encourages separation between common and private samples. LEAD defines

$f^s_\theta=h^s_\theta\circ g^s_\theta$1

and then uses

$f^s_\theta=h^s_\theta\circ g^s_\theta$2

where $f^s_\theta=h^s_\theta\circ g^s_\theta$3 indicates private versus common pseudo-label assignment (Qu et al., 2024). This term pushes private samples toward larger unknown projections and common samples toward larger known projections.

The feature consensus regularizer promotes neighborhood consistency in feature space. If $f^s_\theta=h^s_\theta\circ g^s_\theta$4 denotes the nearest-neighbor set for sample $f^s_\theta=h^s_\theta\circ g^s_\theta$5 constructed by cosine similarity, and empirically $f^s_\theta=h^s_\theta\circ g^s_\theta$6, then

$f^s_\theta=h^s_\theta\circ g^s_\theta$7

and

$f^s_\theta=h^s_\theta\circ g^s_\theta$8

(Qu et al., 2024). The full objective is

$f^s_\theta=h^s_\theta\circ g^s_\theta$9

with $\{x_i^t\}_{i=1}^{N_t}$0 controlling the trade-off (Qu et al., 2024).

The adaptation procedure begins with $\{x_i^t\}_{i=1}^{N_t}$1 and frozen $\{x_i^t\}_{i=1}^{N_t}$2. It then computes the SVD of $\{x_i^t\}_{i=1}^{N_t}$3, decomposes each target feature into known and unknown components, fits the two-component GMM to $\{x_i^t\}_{i=1}^{N_t}$4, constructs target prototypes with top-$\{x_i^t\}_{i=1}^{N_t}$5 sampling where $\{x_i^t\}_{i=1}^{N_t}$6 and $\{x_i^t\}_{i=1}^{N_t}$7 is estimated via Silhouette-based selection, computes $\{x_i^t\}_{i=1}^{N_t}$8 and $\{x_i^t\}_{i=1}^{N_t}$9, assigns pseudo-labels, and updates $h^t_\theta=h^s_\theta$0 by SGD (Qu et al., 2024). As features evolve, decomposition-based quantities and boundaries are periodically refreshed.

The method’s stabilization strategy consists of certainty weighting $h^t_\theta=h^s_\theta$1, consensus regularization, and temperature-free softmax supervision; no teacher EMA or temperature scaling is required (Qu et al., 2024).

5. Empirical behavior, benchmarks, and computational profile

LEAD is evaluated on OPDA, OSDA, and PDA settings across Office-31, Office-Home, VisDA, and DomainNet, with H-score used for OSDA and OPDA: $h^t_\theta=h^s_\theta$2 (Qu et al., 2024). Implementation uses a single RTX-3090, SGD with momentum $h^t_\theta=h^s_\theta$3, batch size $h^t_\theta=h^s_\theta$4, learning rates $h^t_\theta=h^s_\theta$5 for Office-31 and Office-Home and $h^t_\theta=h^s_\theta$6 for VisDA and DomainNet, with $h^t_\theta=h^s_\theta$7 set to $h^t_\theta=h^s_\theta$8 on Office-31, $h^t_\theta=h^s_\theta$9 on VisDA, and $g^t_\theta$0 on Office-Home and DomainNet. Inference uses normalized entropy threshold $g^t_\theta$1 (Qu et al., 2024).

The reported headline OPDA results are as follows.

Scenario	Result
VisDA OPDA	LEAD achieves $g^t_\theta$2, surpassing GLC $g^t_\theta$3 by $g^t_\theta$4
Office-Home OPDA	LEAD $g^t_\theta$5 vs UMAD $g^t_\theta$6; LEAD+UMAD $g^t_\theta$7
Office-31 OPDA	LEAD $g^t_\theta$8 average H-score; LEAD+UMAD $g^t_\theta$9
DomainNet OPDA	LEAD $\mathcal{Y}_t$00 average H-score

The OSDA results are Office-Home $\mathcal{Y}_t$01, Office-31 $\mathcal{Y}_t$02, and VisDA $\mathcal{Y}_t$03, with gains over UMAD of $\mathcal{Y}_t$04, $\mathcal{Y}_t$05, and $\mathcal{Y}_t$06, respectively (Qu et al., 2024). In PDA, the reported accuracies are Office-Home $\mathcal{Y}_t$07, Office-31 $\mathcal{Y}_t$08, and VisDA $\mathcal{Y}_t$09, with gains over UMAD of $\mathcal{Y}_t$10, $\mathcal{Y}_t$11, and $\mathcal{Y}_t$12 (Qu et al., 2024).

The paper attributes these results to three properties: decomposition reveals structure unavailable to entropy or confidence alone, instance-level boundaries adapt to per-class and per-sample variability, and the method avoids high-dimensional clustering instability while achieving large runtime savings (Qu et al., 2024). The efficiency claims are concrete. Projection and distance computations are $\mathcal{Y}_t$13 per instance, while the SVD is $\mathcal{Y}_t$14 once. LEAD avoids iterative $\mathcal{Y}_t$15-means inner loops for boundary derivation, relying instead on a one-dimensional GMM over $\mathcal{Y}_t$16 (Qu et al., 2024).

On VisDA, boundary derivation time is reduced from $\mathcal{Y}_t$17 to $\mathcal{Y}_t$18, approximately a $\mathcal{Y}_t$19 reduction. On DomainNet, the reported comparison is $\mathcal{Y}_t$20 to $\mathcal{Y}_t$21 (Qu et al., 2024). The abstract also highlights that, in the VisDA OPDA scenario, LEAD outperforms GLC by $\mathcal{Y}_t$22 overall H-score and reduces $\mathcal{Y}_t$23 time to derive pseudo-labeling decision boundaries (Qu et al., 2024). The slight difference between “approximately $\mathcal{Y}_t$24” and “$\mathcal{Y}_t$25” reflects the two reported phrasings in the source material.

6. Ablations, robustness, complementarity, and limitations

The ablation results indicate that all three loss components are complementary. In OPDA on VisDA, performance rises from $\mathcal{Y}_t$26 with $\mathcal{Y}_t$27 alone to $\mathcal{Y}_t$28 with $\mathcal{Y}_t$29, and to $\mathcal{Y}_t$30 with all three losses (Qu et al., 2024). Instance-level boundaries also clearly outperform global thresholds: on Office-Home OPDA, the comparison is $\mathcal{Y}_t$31 versus $\mathcal{Y}_t$32 (Qu et al., 2024). The method is reported to be stable around the chosen defaults for $\mathcal{Y}_t$33, stable near $\mathcal{Y}_t$34 for nearest-neighbor size, and moderately robust to the inference threshold $\mathcal{Y}_t$35 (Qu et al., 2024). It also maintains stable H-scores under varying numbers of unknown classes (Qu et al., 2024).

LEAD is described as complementary to most existing methods. The paper gives a blending form

$\mathcal{Y}_t$36

with $\mathcal{Y}_t$37 (Qu et al., 2024). Gains are substantial with UMAD and moderate with GLC, though occasional small trade-offs are noted, such as PDA on VisDA at $\mathcal{Y}_t$38, which the paper suggests may reflect gradient conflicts; joint tuning of $\mathcal{Y}_t$39 and $\mathcal{Y}_t$40 is suggested as mitigation (Qu et al., 2024).

The limitations are stated in terms of assumptions and failure modes. The source-known subspace must capture discriminative directions of shared classes; if $\mathcal{Y}_t$41 is poorly trained or the domain shift is extreme, the projections may become unreliable (Qu et al., 2024). If the bimodality of $\mathcal{Y}_t$42 weakens, for example under severe noise or minimal overlap in $\mathcal{Y}_t$43 across domains, GMM separation may be ambiguous (Qu et al., 2024). Target prototypes estimated from top-$\mathcal{Y}_t$44 target samples can also be contaminated when the classifier is highly uncertain at early stages, although periodic refresh and certainty weighting are reported to mitigate this issue (Qu et al., 2024).

The practical guidance follows directly from these observations. The recommended hyperparameter range for $\mathcal{Y}_t$45 is dataset-dependent within $\mathcal{Y}_t$46, $\mathcal{Y}_t$47 is described as robust for certainty weighting, $\mathcal{Y}_t$48 is used for consensus, and $\mathcal{Y}_t$49 gives good H-scores, with slight adjustment in the range $\mathcal{Y}_t$50–$\mathcal{Y}_t$51 possible per dataset (Qu et al., 2024). The method is positioned for settings such as safety-critical recognition under privacy constraints, large-scale deployments where source data retention is infeasible, and scenarios requiring efficient adaptation without iterative clustering (Qu et al., 2024).

Taken together, LEAD defines SF-UniDA adaptation as a decomposition-and-boundary problem: it extracts an unknown-aware feature component from the source classifier geometry and uses that component in a per-instance decision rule coupled to pseudo-label supervision (Qu et al., 2024). This suggests a broader methodological pattern in source-free adaptation: when access to source data is removed, the classifier itself becomes a structural prior from which usable target-side uncertainty geometry can be recovered.