DAN-LPE: Label-Proportion-Aware Domain Adaptation

• The paper introduces DAN-LPE, which augments DANN with on-line label-proportion estimation to correct label shift and enhance domain adaptation results.
• It combines adversarial representation learning with continuous moment-matching of target label distributions using the source confusion matrix for precise correction.
• Empirical evaluations on text and visual benchmarks show improved target performance and robustness under significant class prior mismatches.

Label-proportion-aware Domain-Adversarial Neural Network (DAN-LPE) encompasses a family of unsupervised domain adaptation methods designed to address scenarios where the label distributions (class priors) differ between the source and target domains—so-called label shift. Standard Domain-Adversarial Neural Networks (DANN) align feature distributions between domains but presume similar class priors, failing under severe label shift. DAN-LPE augments DANN with explicit label proportion estimation and correction, combining adversarial representation learning with continuous moment-matching of target label proportions using the source confusion matrix. This strategy enforces alignment on the target label distribution, mitigating negative-transfer effects observed in conventional DANN models under label shift (Chen et al., 2020, Tachet et al., 2020).

1. Theoretical Foundations and Motivation

Standard unsupervised domain adaptation assumes that $p_S(y) \approx p_T(y)$ or that domain shifts arise mainly from conditional feature differences, not from class-prior discrepancies. However, empirical studies reveal that mismatch between source and target label distributions severely degrades target accuracy in adversarial adaptation setups. DAN-LPE explicitly models this scenario as a generalized label shift (GLS) setting, which postulates the existence of a feature map $g: X \to Z$ such that $D_S(Z \mid Y=y) = D_T(Z \mid Y=y)$ for all classes $y$ (Tachet et al., 2020).

An actionable implication of GLS is that target feature marginals can be matched by importance-weighting source samples with $w_y = p_T(y)/p_S(y)$:

$D_T(Z) = \sum_y w_y D_S(Z, Y=y) = D^w_S(Z),$

enabling direct correction for prior mismatch through class-proportion-aware reweighting (Tachet et al., 2020).

2. DAN-LPE Architectures and Key Components

DAN-LPE augments the standard DANN architecture with on-line label-proportion estimation and weighted adversarial losses. The complete pipeline comprises the following neural components (Chen et al., 2020):

• Feature Extractor ($F$): Maps input $x$ to deep features $F(x)$.
• Label Predictor ($C$): Outputs logits or probabilities $g: X \to Z$0.
• Domain Discriminator ($g: X \to Z$1): Receives $g: X \to Z$2 through a gradient-reversal layer and predicts domain assignment.
• Label-Proportion Estimator ($g: X \to Z$3 or $g: X \to Z$4): Maintains a soft estimate of target priors (class-proportion vector), updated to match moment statistics derived from source confusion and target predictions.

In advanced variants, both adversarial and classifier losses are reweighted by the label-proportion estimate, and multi-phase update regimes are used to stabilize training (Chen et al., 2020, Tachet et al., 2020).

3. Label Proportion Estimation via Confusion Matrix

The empirical source confusion matrix $g: X \to Z$5, estimated from held-out source data, encodes the conditional distribution of predicted labels given true labels. The target prediction histogram $g: X \to Z$6 is observed on the unlabeled target set.

Under label shift:

$g: X \to Z$7

where $g: X \to Z$8 denotes the (unknown) target prior vector. The standard black-box shift correction is recovered by inverting the confusion system:

$g: X \to Z$9

DAN-LPE enforces this relationship by minimizing a moment-matching loss:

$D_S(Z \mid Y=y) = D_T(Z \mid Y=y)$0

where $D_S(Z \mid Y=y) = D_T(Z \mid Y=y)$1 is projected onto the probability simplex ($D_S(Z \mid Y=y) = D_T(Z \mid Y=y)$2, $D_S(Z \mid Y=y) = D_T(Z \mid Y=y)$3) (Chen et al., 2020, Tachet et al., 2020). This mechanism can equivalently be posed as a constrained least-squares optimization or regularized quadratic program.

4. Joint Optimization and Training Algorithms

DAN-LPE employs a joint min-max optimization encompassing three main losses (Chen et al., 2020):

1. Task Loss ($D_S(Z \mid Y=y) = D_T(Z \mid Y=y)$4): Source-domain classification loss, often cross-entropy.
2. Domain-Adversarial Loss ($D_S(Z \mid Y=y) = D_T(Z \mid Y=y)$5): Discriminator loss, aligned via a gradient reversal layer. For importance-weighted DANN, source inputs are weighted by $D_S(Z \mid Y=y) = D_T(Z \mid Y=y)$6.
3. Proportion-Matching Loss ($D_S(Z \mid Y=y) = D_T(Z \mid Y=y)$7): Quadratic loss enforcing agreement between target pseudo-label histogram and source confusion-propagated class proportions.

The min-max objective is:

$D_S(Z \mid Y=y) = D_T(Z \mid Y=y)$8

Training is performed in two phases (Chen et al., 2020):

• Phase I: Alternating updates of feature extractor, label classifier, domain discriminator, and periodic projected-gradient descent on $D_S(Z \mid Y=y) = D_T(Z \mid Y=y)$9.
• Phase II: With $y$0 fixed, standard DANN training with continued source example reweighting.

For stability, exponential moving averages and constraints on $y$1 are employed. In some implementations, an epoch-level update of $y$2 is used via solving a quadratic program (Tachet et al., 2020).

5. Theoretical Guarantees

Under the GLS assumption, DAN-LPE provides error-decomposition bounds:

$y$3

where $y$4 is the class-balanced source error rate and $y$5 quantifies the discrepancy of confusion patterns between source and target (Tachet et al., 2020). When conditionals are perfectly matched ($y$6), the domain gap is governed primarily by the class prior mismatch and the balanced error rate.

Additionally, if source and target errors are small and $y$7-weighted marginals align, conditional distributions are nearly matched, supporting the soundness of the label-proportion correction approach.

6. Empirical Evaluation and Observed Impact

DAN-LPE has been empirically validated on text and vision domain adaptation benchmarks with synthetic and natural label shift (Chen et al., 2020, Tachet et al., 2020). Key findings include:

• On Yelp sentiment transfer between business categories with divergent class priors, DAN-LPE improved target accuracy up to 3% and halved label-proportion estimation error relative to standard DANN.
• On visual domain adaptation (MNIST↔USPS, VisDA, Office-31, Office-Home), average gains of up to 9.3 percentage points in classification accuracy were observed as the Jensen–Shannon divergence between source and target priors increased.
• In psychotherapy behavioral coding, DAN-LPE achieved best F1 scores in 5 of 6 transfer scenarios and consistently improved label-distribution estimation.

These results substantiate the necessity of label-proportion correction under severe label shift and demonstrate that DAN-LPE preserves or improves accuracy even when source and target priors are similar.

7. Practical Considerations, Robustness, and Limitations

Implementation best practices include moderate learning rates (e.g., $y$8), batch sizes around 64, frequent updates of the label-proportion estimate, and use of projected gradient descent to maintain valid probability vectors (Chen et al., 2020).

Robustness to hyper-parameters is reported, but several limitations are noted (Tachet et al., 2020):

• Accuracy of $y$9 or $w_y = p_T(y)/p_S(y)$0 estimation depends on the calibration of pseudo-labels and invertibility of the confusion matrix—under small sample or degenerate class settings, instability can occur.
• DAN-LPE assumes the existence of a representation space satisfying GLS; if such a representation cannot be realized, performance may be suboptimal even under perfect label shift correction.
• Exponential moving averages of $w_y = p_T(y)/p_S(y)$1 mitigate some instability but do not guarantee global convergence.

A plausible implication is that the successful operationalization of DAN-LPE depends critically on quality of feature representations and classifier calibration. In summary, DAN-LPE extends DANN with principled solutions for label shift, theoretically justified under GLS, and empirically validated to mitigate negative adaptation under strong prior mismatch (Chen et al., 2020, Tachet et al., 2020).