GradVac: Gradient Alignment in HSI Transfer

• GradVac is a gradient alignment technique that mitigates conflicting gradients in multi-task and cross-domain HSI transfer using an EMA-adaptive cosine similarity threshold.
• It adjusts source gradients by injecting a fraction of the target gradient when alignment falls below a preset threshold, ensuring cooperative parameter updates.
• Empirical results demonstrate significant overall accuracy improvements in high-conflict scenarios, establishing GradVac as a crucial component in the ADGKT framework.

GradVac is a gradient alignment technique designed to mitigate optimization conflicts in multi-task and cross-domain learning, specifically within the context of cross-scene hyperspectral imaging (HSI) transfer. Incorporated into the Agreement–Disagreement Guided Knowledge Transfer (ADGKT) framework, GradVac systematically aligns source and target gradients in the shared encoder, reducing destructive interference and enabling more effective joint training across heterogeneous data distributions (Huo et al., 8 Dec 2025).

1. Formalization of the GradVac Gradient Alignment Objective

Given source and target gradients, $g_s$ and $g_t$, of shared encoder parameters respective to source loss $\mathcal{L}_s$ and target loss $\mathcal{L}_t$, GradVac operates by assessing their cosine similarity:

$$\varphi = \cos \theta = \frac{g_s \cdot g_t}{\|g_s\|\|g_t\|}$$

A running threshold $\alpha$ is maintained via exponential moving average (EMA):

$$\alpha^{(t)} = (1 - \beta)\alpha^{(t-1)} + \beta\varphi^{(t-1)} \quad\text{with}\quad \alpha^{(0)} = 0$$

When $\varphi < \alpha$, indicating unacceptable gradient conflict, GradVac perturbs the source gradient by injecting a fraction $\eta$ of the target gradient, calculated so the angle between $g_s'$ and $g_t$0 after adjustment reaches a pre-set target cosine similarity $g_t$1:

$g_t$2

where

$g_t$3

This is equivalent to solving for the closest adjustment $g_t$4 to $g_t$5 under the constraint $g_t$6 and then substituting $g_t$7. This closed-form adjustment targets minimization of destructive interference in shared parameter updates.

2. Intuitive Rationale and Role in Optimization Dynamics

The underlying intuition is that conflicting gradients (i.e., low $g_t$8, nearly orthogonal or opposed directions) can degrade learning, as parameter updates compete rather than cooperatively advance generalization across tasks. GradVac detects these conditions using an adaptive, scenario-sensitive threshold $g_t$9 via EMA, then reorients the source gradient toward the target direction just enough to achieve a configurable minimal alignment ($\mathcal{L}_s$0). This gradient “vaccination” modulates learning stochastically across optimization trajectories, rather than enforcing constant coupling.

In practical terms, standard joint updates for shared parameters $\mathcal{L}_s$1 follow:

$\mathcal{L}_s$2

With GradVac, the update replaces $\mathcal{L}_s$3 by $\mathcal{L}_s$4 when $\mathcal{L}_s$5, actively suppressing destructive interactions. The EMA-driven $\mathcal{L}_s$6 reflects how much gradient divergence is natural for the current task/domain pair, adapting the intervention frequency.

3. Procedure and Algorithmic Integration

The GradVac mechanism is implemented per training step according to the following procedure:

Step	Description	Key Operations
1	Compute Gradients	$\mathcal{L}_s$7, $\mathcal{L}_s$8
2	Calculate Cosine Similarity	$\mathcal{L}_s$9
3	Update Threshold	$\mathcal{L}_t$0
4	Check and Adjust	If $\mathcal{L}_t$1: compute $\mathcal{L}_t$2, set $\mathcal{L}_t$3; else, $\mathcal{L}_t$4
5	Update Parameters	$\mathcal{L}_t$5

Editor's term: “EMA-adaptive gradient thresholding” denotes the adaptive management of $\mathcal{L}_t$6 via stepwise exponential averaging.

4. Hyperparameter Choices and Ablation Outcomes

GradVac’s performance is sensitive to its hyperparameters:

• $\mathcal{L}_t$7: EMA momentum for threshold $\mathcal{L}_t$8 (e.g., $\mathcal{L}_t$9 for I→P; $$\varphi = \cos \theta = \frac{g_s \cdot g_t}{\|g_s\|\|g_t\|}$$0 for H→P)
• $$\varphi = \cos \theta = \frac{g_s \cdot g_t}{\|g_s\|\|g_t\|}$$1: Post-adjustment cosine similarity, set to a high value (e.g., $$\varphi = \cos \theta = \frac{g_s \cdot g_t}{\|g_s\|\|g_t\|}$$2), not extensively varied in experiments.

Empirical ablation (see Table 6 from (Huo et al., 8 Dec 2025)) demonstrates:

Scenario	OA Change	OA (Before → After)
I→P	$$\varphi = \cos \theta = \frac{g_s \cdot g_t}{\|g_s\|\|g_t\|}$$3	$$\varphi = \cos \theta = \frac{g_s \cdot g_t}{\|g_s\|\|g_t\|}$$4
H→P	$$\varphi = \cos \theta = \frac{g_s \cdot g_t}{\|g_s\|\|g_t\|}$$5	$$\varphi = \cos \theta = \frac{g_s \cdot g_t}{\|g_s\|\|g_t\|}$$6
P→H	$$\varphi = \cos \theta = \frac{g_s \cdot g_t}{\|g_s\|\|g_t\|}$$7	$$\varphi = \cos \theta = \frac{g_s \cdot g_t}{\|g_s\|\|g_t\|}$$8
I→H	$$\varphi = \cos \theta = \frac{g_s \cdot g_t}{\|g_s\|\|g_t\|}$$9	$\alpha$0

These results indicate that GradVac yields substantial improvement where cross-scene gradients are highly discordant, but may degrade performance if domain shifts are moderate or structured differently. This highlights the necessity of robust $\alpha$1 updating and complementary mechanisms.

5. Empirical Impact and Interaction With Companion Methods

Within ADGKT, GradVac constitutes one part of the “agreement” block, and is coupled immediately with LogitNorm. LogitNorm normalizes pre-softmax logits, reducing the risk of magnitude-based domination by either the source or target branch. Ablation reveals:

• GradVac alone (✓ – – –): Immediate OA gains in high-conflict scenarios.
• GradVac + LogitNorm (✓ ✓ – –): Further improvement (e.g., $\alpha$2 OA on I→P, $\alpha$3 on H→P).
• Full Agreement–Disagreement Block (✓ ✓ ✓ ✓): Optimal overall accuracy ($\alpha$4 OA on I→P).

Without agreement mechanisms, naïve joint training often underfits small-target domains or results in poor compromise solutions. GradVac produces immediate OA improvements when severe gradient conflicts exist. LogitNorm prevents the source network (typically with more abundant data) from overwhelming optimization via large logits and commensurately larger gradient magnitudes. Together, these mechanisms form the prerequisite for the “disagreement” block’s ensemble feature extraction, which targets diversity in target scene representation.

6. Contextual Role within Cross-Scene HSI Transfer

GradVac was developed to address a core limitation of prior cross-domain HSI transfer methods—specifically, the prevalence of gradient conflicts between source and target tasks in the optimization of shared encoder parameters. By enabling adaptive gradient realignment, GradVac facilitates balanced knowledge transfer, allowing more complete exploitation of scene diversity in joint learning setups. Its empirical efficacy is scene-pair dependent, excelling in large domain shift situations, but its corrective actions can become counterproductive in low-conflict or tightly coupled domains. Thus, GradVac is best deployed in concert with dynamic thresholding and complementary mechanisms such as LogitNorm for robust performance across varied cross-scene transfer settings (Huo et al., 8 Dec 2025).