USCAL: Unsupervised SCAL Framework

• The paper introduces USCAL, an unsupervised framework that unifies adversarial regularization with structure- and contrastive-conditioned objectives to boost domain adaptation in both computer vision and NLP.
• It employs iterative clustering with source-seeded centers and surrogate cluster-classifiers, significantly improving intra-class compactness and aligning semantic representations.
• In NLP, dual-view transformer encoders and adversarial contrastive loss are applied to enhance semantic textual similarity and robustness across diverse language tasks.

Unsupervised SCAL (USCAL) designates two distinct but methodologically related frameworks for unsupervised representation learning and domain adaptation, unified by the principle of adversarially-regularized, structure- or contrastive-conditioned objectives. In both computer vision and NLP, USCAL leverages adversarial learning mechanisms not only for standard distribution alignment but also for the preservation or exploitation of structured or challenging data relationships in the absence of target (or downstream) supervision (Wang et al., 2021, Miao et al., 2021).

1. Structural Conditioning and Local Structure Exploitation

The USCAL paradigm in unsupervised domain adaptation (UDA) formally exploits local structure in the target data through iterative clustering. Given an unlabeled target set $\mathcal{D}^t = \{x_i\}_{i=1}^{n_t}$, pretrained features $z_i = G(x_i)$ are clustered into $K$ groups—presumed to match the semantic classes—by spherical $K$-means with cosine distance: $$\min_{\{C_k^t, \mu_k^t\}}\sum_{k=1}^K\sum_{x_i\in C_k^t}\mathcal{L}_{\text{dist}}(z_i, \mu_k^t),\qquad \mathcal{L}_{\text{dist}}(u,v) = \tfrac{1}{2}(1-\tfrac{\langle u,v\rangle}{\|u\|\|v\|})$$ Clusters are seeded from source domain class centers $\mu^s_k$ to maximize semantic alignment. Clusters are alternately reassigned by nearest-center selection and re-centered by normalized means, effectively establishing a pseudo-label structure that persists through adversarial adaptation (Wang et al., 2021).

2. Architecture and Adversarial Training Pipelines

USCAL pipelines in vision (UDA) consist of:

• Feature extractor $G$: ResNet-50 backbone truncated after the average pooling layer, outputting 2048-dimensional features.
• Source classifier $F$: One fully connected (FC) layer (2048 to $K$) + softmax.
• Surrogate cluster-classifier $F_S$: FC+softmax network mimicking the discrete output of $K$-means clusters on target data to yield a differentiable approximation.
• Domain discriminator $D$: Two-layer MLP acting on a “structure-conditioned” feature $S(x)$, defined as the outer product $G(x)\otimes F_S(G(x))$ (dimension $dK$), passed through ReLU and sigmoid.
• Gradient Reversal Layer (GRL): Implements the minimax update by reversing gradients from $D$ to $G$.

In the NLP setting, USCAL leverages:

• Transformer backbone $f_\theta$: BERT$_{base}$ or RoBERTa$_{base}$ (12 layers, 768-dim hidden states).
• Projector: Two-layer MLP (“hidden” $\approx$2048 $\rightarrow$ 256). Adversarial perturbations are generated in feature space using a single-step Fast Gradient Method (FGM) directly on the token-embedding input.

3. Optimization, Losses, and Algorithmic Flow

USCAL for Vision:

The joint objective is

$$\min_{G,F}\max_D\ \mathcal{L}_{\text{cls}} - \lambda\mathcal{L}_{\text{adv}}$$

with: $$\mathcal{L}_{\text{cls}} = \mathbb{E}_{(x,y)\sim\mathcal{D}^s}[\ell_{\text{ce}}(F(G(x)), y)],$$

$$\mathcal{L}_{\text{adv}} = -\mathbb{E}_{x\sim\mathcal{D}^s}\log D(S(x)) - \mathbb{E}_{x\sim\mathcal{D}^t}\log(1-D(S(x))).$$

Alternating optimization cycles update cluster assignments, fit $F_S$ by cross-entropy to (hard) cluster pseudo-labels, and train the adversarial pipeline (sample minibatches, update $D$, $G$, $F$, $F_S$) using SGD with momentum. No separate structural regularization is required; intra-class compactness preservation is induced by the cluster conditions within $\mathcal{L}_{\text{adv}}$.

USCAL for NLP:

For each input $x_i$, two "views" $x_i^{emb1}$, $x_i^{emb2}$ are drawn by dropout; respective hidden representations $$h_i^{(1)}, h_i^{(2)}=f_\theta(x_i^{emb1}), f_\theta(x_i^{emb2})$$. An adversarial perturbation $\delta_i$ is computed by maximizing contrastive loss in the embedding space under an $\ell_2$ norm constraint: $$\delta_i = \varepsilon\frac{\nabla_{x_i^{emb1}}\mathcal{L}_{CL}(x_i^{emb1}, x_i^{emb2})}{\|\nabla_{x_i^{emb1}}\mathcal{L}_{CL}(x_i^{emb1}, x_i^{emb2})\|_2}$$ The final batch objective combines clean and adversarial contrastive terms: $$\mathcal{L}_{USCAL} = \frac{1}{N}\sum_{i=1}^N[\mathcal{L}_{CL}^{(\text{clean})}(i) + \alpha\mathcal{L}_{CL}^{(\text{adv})}(i)]$$ where $\alpha$ controls the adversarial emphasis and InfoNCE loss is adopted on projected features (Miao et al., 2021).

USCAL (Vision) High-level Pseudocode

Input : Labeled source D^s, Unlabeled target D^t, #classes K, max_epochs E, iterations per epoch M Output : G, F, F_S parameters Initialize G from pretrained, F, F_S, D randomly for epoch = 1 to E: 1. Extract G(x) on D^s, compute class centers μ^s_k 2. Initialize μ^t_k ← μ^s_k (k=1..K) 3. Spherical K-means on G(x) for D^t to assign ŷ_i 4. Build pseudo-labeled set Ŝ^t = {(x_i, ŷ_i)} for iter = 1 to M: a) Sample B_s ⊂ D^s, B_t ⊂ D^t b) Update F_S: min_F_S E_{(x,ŷ)∈B_t}[ℓ_ce(F_S(G(x)), ŷ)] c) Forward G, F, F_S, build S(x) d) Update D to maximize 𝓛_adv e) Update G, F to minimize 𝓛_cls – λ𝓛_adv (via GRL)

(Wang et al., 2021)

USCAL (NLP) High-level Pseudocode

Given: Corpus X, encoder f_θ, projector, batch size N, temperature τ, adv-weight α, perturbation radius ε Repeat until convergence: Sample {x₁,…,x_N} For i=1…N: xᵉ¹_i ← embed+dropout(x_i) xᵉ²_i ← embed+dropout(x_i) hᵉ¹_i ← f_θ(xᵉ¹_i); zᵉ¹_i ← Proj(hᵉ¹_i) hᵉ²_i ← f_θ(xᵉ²_i); zᵉ²_i ← Proj(hᵉ²_i) Compute L_clean via InfoNCE for each i For i=1…N: g_i ← ∇_{xᵉ¹_i}L_clean(i) δ_i ← ε·g_i / ||g_i||_2 x^{adv}_i ← xᵉ¹_i + δ_i h^{adv}_i ← f_θ(x^{adv}_i); z^{adv}_i ← Proj(h^{adv}_i) Compute L_adv via InfoNCE for (zᵉ¹_i, z^{adv}_i) Combine: L ← (1/N)∑_i [L_clean(i) + α·L_adv(i)] θ ← θ − η·∇_θ L

(Miao et al., 2021)

4. Implementation Parameters and Architectural Details

Vision (UDA):

Component	Architecture Details	Hyper-parameters
Feature Extractor	ResNet-50 up to avgpool (2048-dim)	lr $\eta_0$=0.001 (G); 10× for others
Classifiers	FC (2048 → K) + softmax (F, F_S)	batch size 32/domain; momentum 0.9
Discriminator	FC($dK\rightarrow$1024)→ReLU→FC(1024→1)→sigm	$\lambda=1$; schedule: $\eta_p = \eta_0(1+\alpha p)^{-\beta}$
Clustering	Spherical $K$-means, source center init	$K$ = #classes

Learning rates are adapted per training progress, with typical values $\alpha=10, \beta=0.75$ (Office-31/Home) or $\alpha=5, \beta=2.25$ (VisDA-2017) (Wang et al., 2021).

NLP (Unsupervised Sentence Embedding):

Component	Architectural Details	Hyper-parameters
Encoder	BERT$_{base}$/RoBERTa$_{base}$	AdamW, lr 3e-5, weight decay 0.01
Projector	MLP (2048$\rightarrow$256)	batch size 64, max seq=32, $\tau$=0.05
Adversarial	$\varepsilon$ in $\{0.1,...,0.5\}$	$\alpha=1.0$

Optimization uses linear warmup, evaluation every 250 steps for model selection (Miao et al., 2021).

5. Empirical Results and Comparative Analysis

Vision Benchmarks

On Office-31 (31 classes, 6 tasks):

• USCAL: 88.6% accuracy (ResNet-50). Hybrid USCAL+SPL: 90.5%.
• DANN, CDAN+E, and RSDA-MSTN: 82.2%, 87.7%, and 91.1% respectively, underscoring USCAL’s improved intra-class alignment.

On Office-Home (65 classes, 12 tasks):

• USCAL: 68.3%. USCAL+SPL: 72.0%.
• Prior SOTA (SRDC): 71.3% (Wang et al., 2021).

On VisDA-2017 (Synthetic→Real, ResNet-101):

• USCAL: 80.1% average, outperformed BSP+CDAN and DMP.

Ablation results confirm that

• Structure conditioning substantially improves adaptation over “no condition” (82.0%→93.5%).
• Differentiable surrogate classifier $F_S$ further improves results vs. non-differentiable cluster assignment (88.6% vs. 85.8%).

NLP: Semantic Textual Similarity and Robustness

On SentEval STS tasks (with BERT$_{base}$):

• USCAL: 77.29% average Spearman’s $\rho$, surpassing SimCSE (75.54%).
• Breakdown: STS12: 70.61, STS13: 82.73, STS14: 76.21, STS15: 82.61, STS16: 77.85, STS-B: 78.56, SICK-R: 72.48.

The USCAL adversarial-contrastive approach also substantially improves robustness in NLI tasks when applied in the supervised SCAL variant, achieving up to 58.6% accuracy on ANLI (Miao et al., 2021).

6. Theoretical Considerations

Theoretically, USCAL’s vision variant bounds domain discrepancy by conditioning the domain discriminator: $$d_{\mathcal{H}_D}(\mathcal{D}^s(S), \mathcal{D}^t(S)) \geq d_{\mathcal{H}}(\mathcal{D}^s(G), \mathcal{D}^t(G))$$ This ensures that, compared to conventional adversarial UDA, intra-class compactness is preserved while minimizing the conditioned $\mathcal{H}$-distance. The target error is thereby upper-bounded by the sum of source error and the conditioned domain distance: $$\mathcal{E}_T(h)\leq \mathcal{E}_S(h)+\tfrac{1}{2}d_\mathcal{H}+\mathcal{E}_T(h^*)$$ This analytic perspective underlines why local structure conditioning benefits both transferability and representation geometry, as demonstrated empirically (Wang et al., 2021).

7. Significance and Prospective Directions

USCAL advances unsupervised domain adaptation and sentence representation learning by embedding adversarial mechanisms in ways that are sensitive to local structure or challenging data relationships. In both modalities, these mechanisms result in meaningful improvements for end-task accuracy and representation robustness, demonstrated across vision and NLP tasks. A plausible implication is that extensions of USCAL using more flexible or domain-specific structure induction, or extending adversarial perturbation to richer latent subspaces, may further enhance generalization and robustness. The unified adversarial-structural perspective pioneered here motivates ongoing inquiry at the interface of representation structure and adversarial transfer (Wang et al., 2021, Miao et al., 2021).