GOLD: Online Low-rank Directional Adaptation

• The paper demonstrates that updating only the classifier’s golden subspace via AGOP achieves efficient and robust adaptation under varying test conditions.
• GOLD employs a lightweight adapter with a frozen backbone and prototype-based regularization to maintain source semantics and mitigate prediction drift.
• Empirical evaluations on CIFAR and ImageNet benchmarks show that GOLD outperforms state-of-the-art CTTA methods while reducing computational overhead.

Guided Online Low-rank Directional Adaptation (GOLD) is a source-free continual test-time adaptation method designed for online adaptation to unlabeled data streams under distribution shift while preserving online inference efficiency and generalization. It is introduced in "The Golden Subspace: Where Efficiency Meets Generalization in Continual Test-Time Adaptation" (Lai et al., 23 Mar 2026). The method is organized around the claim that effective adaptation should occur not across the full parameter space, but within a small task-relevant feature subspace that is sufficient to change predictions with minimal perturbation to the pretrained representation. The paper terms this privileged subspace the golden subspace, proves its existence in a single-step adaptation setting, identifies it with the row space of the pretrained classifier, and proposes an online mechanism based on sample-wise Average Gradient Outer Product (AGOP) to maintain this subspace during continual adaptation. The implementation couples a frozen backbone and classifier with a lightweight low-rank adapter, a compact scaling vector, EMA-based self-training, and prototype-based regularization; the authors also release code at the stated repository (Lai et al., 23 Mar 2026).

1. Continual test-time adaptation setting and the efficiency–generalization trade-off

The method is formulated in the setting of Continual Test-Time Adaptation (CTTA), where a model pretrained on labeled source data is deployed on an unlabeled target stream

$$\mathcal{D}_T = \{\mathcal{X}_1, \mathcal{X}_2, \ldots, \mathcal{X}_T\},$$

with batches $\mathcal{X}_t$ arriving sequentially from a distribution that changes over time, $p_t(x)\neq p_{t+1}(x)$. At each step, the model predicts on the current batch and updates itself using only the current unlabeled test data, without access to source samples and without offline retraining (Lai et al., 23 Mar 2026).

Within this setting, the central difficulty is the efficiency–generalization trade-off. Updating more parameters or invoking heavier test-time mechanisms can improve short-term fitting to a new domain, but the cost is higher FLOPs, higher memory use, and greater susceptibility to error accumulation, pseudo-label noise amplification, and parameter drift. The paper characterizes this as a structural tension in CTTA: aggressive adaptation may improve immediate fit while degrading long-horizon stability as the stream evolves.

GOLD addresses this by restricting adaptation to what the paper describes as a minimal feature subspace sufficient to produce the desired output correction. In the paper’s interpretation, the golden subspace is the minimal-update subspace that yields the needed output change while preserving as much of the pretrained representation as possible. This suggests that the method is not merely a low-rank parameterization for efficiency, but a geometric constraint intended to reconcile online plasticity with representation preservation.

2. Golden subspace theory and the classifier row space

The theoretical analysis is given for a single-step adaptation setting with a linear classifier

$z = Wf,$

where $f\in\mathbb{R}^L$ is the feature vector and $W\in\mathbb{R}^{C\times L}$ is the classifier weight matrix (Lai et al., 23 Mar 2026).

Given a desired logit correction $\Delta y\in\mathbb{R}^C$, the paper studies the minimum-norm feature perturbation

$$\min_{\Delta f\in\mathbb{R}^L}\ \tfrac{1}{2}\|\Delta f\|_2^2 \quad\text{s.t.}\quad W\Delta f=\Delta y.$$

Its unique minimum-norm solution is

$\Delta f^\star = W^{+}\Delta y,$

where $W^{+}$ is the Moore–Penrose pseudoinverse. For a batch version,

$\mathcal{X}_t$0

has solution

$\mathcal{X}_t$1

The paper also writes

$\mathcal{X}_t$2

and notes the rank bound

$\mathcal{X}_t$3

The key linear-algebraic conclusion is that the minimum-norm solution lies in the row space of $\mathcal{X}_t$4, equivalently the column space of $\mathcal{X}_t$5. In the appendix argument, stationarity of the Lagrangian gives $\mathcal{X}_t$6, hence $\mathcal{X}_t$7. Under an SVD $\mathcal{X}_t$8,

$\mathcal{X}_t$9

The paper therefore identifies the golden subspace exactly with the subspace spanned by the principal right singular vectors, or eigen-directions, of the classifier and states that it “coincides with the row space of the pretrained classifier” (Lai et al., 23 Mar 2026).

This characterization rules out an arbitrary notion of low rank. The relevant subspace is not any compressed feature basis, but the particular set of directions already encoded by the pretrained classifier as effective for changing logits. A plausible implication is that GOLD’s low-rank structure is tied directly to predictive geometry rather than to generic dimensionality reduction.

3. AGOP as an online estimator of the subspace

Because directly updating $p_t(x)\neq p_{t+1}(x)$0 online is described as expensive and unstable, GOLD uses sample-wise Average Gradient Outer Product (AGOP) as a proxy for the classifier’s geometry (Lai et al., 23 Mar 2026).

For a sample $p_t(x)\neq p_{t+1}(x)$1, the gradient surrogate is

$p_t(x)\neq p_{t+1}(x)$2

The mini-batch AGOP is

$p_t(x)\neq p_{t+1}(x)$3

where

$p_t(x)\neq p_{t+1}(x)$4

is the set of high-confidence samples in batch $p_t(x)\neq p_{t+1}(x)$5, $p_t(x)\neq p_{t+1}(x)$6 is the maximum softmax probability, and $p_t(x)\neq p_{t+1}(x)$7 is a confidence threshold. The online estimate is updated via exponential moving average: $p_t(x)\neq p_{t+1}(x)$8 Every $p_t(x)\neq p_{t+1}(x)$9 batches, GOLD performs eigendecomposition

$z = Wf,$0

and selects the top-$z = Wf,$1 eigenvectors

$z = Wf,$2

The paper motivates this construction through prior work on AGOP representations, stating an empirical relation of the form

$z = Wf,$3

with $z = Wf,$4 often near $z = Wf,$5. In the CTTA setting, where labels are unavailable, GOLD combines confidence-filtered pseudo-labeling and AGOP over test samples to track the same subspace online.

The paper further argues that AGOP-derived subspaces quickly align with the true golden subspace and that the AGOP spectrum is strongly low-rank: the top 64–128 eigenvectors capture over 99% of the energy in the reported experiments. This supports the operational claim that the classifier-relevant geometry can be maintained online without retraining the classifier itself.

4. Adapter design, projection mechanism, and learning objective

GOLD is implemented as a lightweight adapter attached to a frozen backbone $z = Wf,$6 and classifier $z = Wf,$7 (Lai et al., 23 Mar 2026). The pretrained backbone remains frozen, and the method computes source-derived class prototypes

$z = Wf,$8

which serve as semantic anchors during test-time adaptation.

For a feature $z = Wf,$9, the current golden-subspace basis $f\in\mathbb{R}^L$0 defines a projection

$f\in\mathbb{R}^L$1

An elementwise scaling vector $f\in\mathbb{R}^L$2 is then applied: $f\in\mathbb{R}^L$3 The adapted feature is produced in residual form,

$f\in\mathbb{R}^L$4

For a batch, the paper writes

$f\in\mathbb{R}^L$5

This parameterization has three properties emphasized in the paper: only a low-dimensional subspace is modified; the residual form guarantees $f\in\mathbb{R}^L$6 when $f\in\mathbb{R}^L$7; and updates are concentrated in a compact and interpretable set of directions. The trainable state is correspondingly small: the method performs a single gradient step on the scaling vector $f\in\mathbb{R}^L$8 and a small set of normalization parameters.

Optimization uses an EMA teacher and two losses. The self-training consistency loss is

$f\in\mathbb{R}^L$9

where $W\in\mathbb{R}^{C\times L}$0 are student logits on the current batch, $W\in\mathbb{R}^{C\times L}$1 are logits on an augmented view, and $W\in\mathbb{R}^{C\times L}$2 are teacher logits. The prototype-based contrastive loss selects the nearest source prototype $W\in\mathbb{R}^{C\times L}$3 for each sample by cosine similarity and uses

$W\in\mathbb{R}^{C\times L}$4

with

$W\in\mathbb{R}^{C\times L}$5

and temperature $W\in\mathbb{R}^{C\times L}$6. The total objective is

$W\in\mathbb{R}^{C\times L}$7

The golden subspace itself is updated online by initializing $W\in\mathbb{R}^{C\times L}$8, updating $W\in\mathbb{R}^{C\times L}$9 with AGOP from high-confidence samples, and periodically recomputing $\Delta y\in\mathbb{R}^C$0 via eigendecomposition. This yields a subspace that begins from classifier geometry and gradually incorporates target-domain semantics without classifier retraining.

5. Empirical evaluation across classification and segmentation

The paper evaluates GOLD on standard CTTA corruption benchmarks—CIFAR10-C, CIFAR100-C, and ImageNet-C—each containing 15 corruption types at severity level 5, with online one-pass adaptation over the stream. Source backbones include WideResNet-28, ResNeXt-29, and ResNet-50 (Lai et al., 23 Mar 2026).

On these classification benchmarks, GOLD reports the best average error on all three datasets:

• CIFAR10-C: 14.1%
• CIFAR100-C: 28.6%
• ImageNet-C: 59.3%

The paper states that these results outperform baselines including TENT, Ada, CoTTA, RMT, DSS, EATA, SAR, BeCoTTA, SANTA, and OBAO, with especially notable gains on harder corruptions and on the more challenging CIFAR100-C and ImageNet-C settings.

For segmentation, the evaluation uses CarlaTTA, a gradual semantic segmentation benchmark based on CARLA, with five sequences: day2night, clear2fog, clear2rain, dynamic, and highway. GOLD achieves the best mIoU on three of the five sequences and is described as especially strong on the difficult highway setting, where both covariate shift and label distribution shift appear.

The paper also emphasizes long-term stability. It reports that GOLD remains well-aligned with the true golden subspace as adaptation progresses, maintains performance under repeated domain exposure, and achieves the best error rate in a 10-round long-term CIFAR10-C experiment:

• 10-round long-term CIFAR10-C: 14.15%

Relative to CoTTA, RMT, BeCoTTA, SANTA, EATA, SAR, and ViDA in that experiment, the method is presented as combining short-horizon accuracy with more stable long-horizon generalization.

6. Efficiency profile, ablations, and stated limitations

A defining feature of GOLD is its lightweight adaptation profile. The reported efficiency figures are:

• trainable parameter ratio: 0.373%
• FLOPs: 1425.14G
• peak GPU memory: 5.37 GB

The paper further reports runtime around 0.25 seconds per batch and states that AGOP plus periodic eigendecomposition add only modest overhead (Lai et al., 23 Mar 2026). It positions this efficiency as close to, or only slightly above, other adapter-based methods such as SANTA, while remaining much cheaper than full-model update methods such as CoTTA.

Ablation results support the individual components of the method. The paper states that initializing with $\Delta y\in\mathbb{R}^C$1 improves over having no subspace projection; AGOP-based online updates further improve or stabilize long-term adaptation; the prototype contrastive loss helps preserve source semantics and reduce drift; and the full method combining $\Delta y\in\mathbb{R}^C$2 initialization, AGOP, and $\Delta y\in\mathbb{R}^C$3 performs best.

The practical interpretation advanced by the paper is that GOLD improves both efficiency and generalization because it updates only the directions that matter most for changing predictions. Its rationale is distributed across several elements: the subspace is theoretically justified by the classifier’s row space; AGOP allows that subspace to track target-domain evolution without retraining the classifier; the adapter is low-rank and residual, which minimizes drift; and prototype anchoring together with EMA consistency reduces noisy pseudo-label effects.

The authors also identify several assumptions and possible limitations. The theoretical argument treats the classifier as linear in the final feature space. The golden subspace is estimated from high-confidence pseudo-labels, so extremely noisy confidence estimates could degrade AGOP quality. The method assumes that enough confident samples arrive over time to estimate AGOP reliably. Periodic eigendecomposition, while efficient in the reported setting, still introduces overhead. The paper also notes that its strongest justification is for CTTA with a frozen backbone and lightweight adaptation; if the domain shift is so large that the pretrained feature space is severely misaligned, a small subspace adapter may be insufficient.

Taken together, these points define GOLD as a structured, low-rank feature adaptation framework for CTTA in which adaptation is constrained to a classifier-governed subspace and updated online through AGOP. A plausible implication is that the method reframes the usual CTTA design question—from how much of the model to update to which feature directions are minimally sufficient for prediction-preserving adaptation.