CELA: Confidence-Aware LayerNorm Adapter

• The paper introduces CELA, a test-time adaptation component that updates only the LayerNorm affine parameters to adjust HSI classifiers for degradation-induced distribution shifts.
• CELA employs confidence-aware entropy minimization on selected high-confidence target samples, ensuring lightweight and stable adaptation at inference without source data or labels.
• Empirical evaluations, particularly on the WHLK dataset, show that using CELA leads to significant performance gains under various degradations including noise, blur, and fog.

Searching arXiv for the cited HyperTTA and LayerNorm-tuning papers, and a canonical test-time adaptation reference for context. Confidence-aware Entropy-minimized LayerNorm Adapter (CELA) is the test-time adaptation component of the HyperTTA framework for hyperspectral image (HSI) classification under degradation-induced distribution shift. It is designed to make a Transformer-based HSI classifier adaptable at inference time without access to source data or target annotations, and it does so by updating only the affine parameters of Layer Normalization layers while minimizing prediction entropy on selected high-confidence unlabeled target samples. Within HyperTTA, CELA functions as the mechanism that enables source-free, label-free, lightweight adaptation to previously unseen degradations such as noise, blur, compression, fog, stripes, and sensor failures (Yue et al., 10 Sep 2025).

1. Role within HyperTTA

HyperTTA is organized around three components: MDS, SSTC, and CELA. MDS creates multi-degradation target inputs, SSTC trains a robust spectral-spatial Transformer classifier on clean labeled data, and CELA adapts SSTC at test time using high-confidence entropy minimization on LayerNorm affine parameters. In this formulation, SSTC provides the base classifier and CELA provides the online adaptation mechanism (Yue et al., 10 Sep 2025).

The motivation for CELA arises from a central empirical observation: HSI classifiers trained on clean source data are highly vulnerable to distribution shifts caused by real-world degradations. Under such shifts, a fixed classifier may become overconfident on wrong predictions or uncertain on difficult samples. CELA is introduced specifically to let the model adjust to target degradation statistics during inference while avoiding the instability associated with updating the full network on noisy pseudo-labels or unreliable predictions.

The paper explicitly frames CELA as a method to adapt dynamically to unseen target distributions, minimize prediction uncertainty on unlabeled target samples, and prevent unreliable gradient updates from noisy or low-confidence predictions. In that sense, CELA is not a generic adapter learned during source training; it is a test-time procedure applied to an already trained classifier.

2. Parameterization and optimization target

CELA updates only the affine parameters of Layer Normalization layers, namely the scale and shift parameters $\gamma$ and $\beta$. All other model parameters remain frozen during test-time adaptation, including convolutional weights, Transformer weights, and classifier-head weights. The paper states that “we update only the affine parameters—namely, the scale $\gamma$ and shift $\beta$—of the Layer Normalization (LayerNorm) layers in the Transformer, while keeping all other model parameters frozen” (Yue et al., 10 Sep 2025).

This design makes the parameter partition explicit. The updated parameter set is $\{\gamma, \beta\}$, and the frozen parameter set comprises the rest of the classifier. The intended effect is a compact adaptation interface: $\gamma$ rescales normalized activations and $\beta$ shifts them. The paper gives four reasons for this choice. It is lightweight, it aligns naturally with Transformer architectures in which LayerNorm is standard, it reduces the risk of overfitting or catastrophic drift relative to full-model updating, and it provides a compact mechanism for adjusting to new target-domain activation statistics.

A common misconception is to read “adapter” here as implying an inserted bottleneck module. In CELA, the adaptation site is the pre-existing LayerNorm affine transformation rather than an additional learned subnetwork. The method is therefore adapter-like in function but not in the sense of adding extra MHA- or FFN-side modules.

3. Confidence-aware entropy minimization

CELA adapts by minimizing prediction entropy, but only on a selected subset of high-confidence target samples. For an unlabeled target sample $\mathbf{x}_j^t$, let the model output be $\hat{\mathbf{p}}_j \in \mathbb{R}^K$, where $\hat{p}_j^{(k)}$ is the predicted probability for class $\beta$0. Confidence is defined as the maximum predicted class probability:

$\beta$1

Given a threshold $\beta$2, the threshold-selected set is

$\beta$3

To avoid selecting too few samples, the method introduces a fallback based on mini-batch ranking. Let $\beta$4 denote the indices of the top $\beta$5 most confident samples in the mini-batch. The final selected set is

$\beta$6

Here $\beta$7 is the batch size. In implementation, the threshold is set to $\beta$8 (Yue et al., 10 Sep 2025).

The entropy of the prediction for $\beta$9 is

$\gamma$0

The adaptation loss over the selected set is

$\gamma$1

This loss is minimized only with respect to the LayerNorm affine parameters. The update rule is

$\gamma$2

where $\gamma$3 is the test-time learning rate.

The confidence-aware restriction is the central distinction from plain entropy minimization on all target samples. Low-confidence predictions are excluded from the loss, which reduces the chance that noisy degradations or ambiguous samples will corrupt the LayerNorm update. A plausible implication is that the hybrid threshold-plus-top-ranking rule trades off selection precision against the need for a minimally sized adaptation set in each mini-batch.

4. Episodic adaptation protocol and implementation

CELA is episodic rather than cumulative. For each test sample or mini-batch, the LayerNorm parameters are reset to their original pre-trained values $\gamma$4, a small number of adaptation steps are performed, and the adapted values are not carried over to the next episode. This prevents cumulative drift across batches and localizes adaptation to the current test episode (Yue et al., 10 Sep 2025).

The implied test-time loop is specified in the framework description. An unlabeled degraded target mini-batch $\gamma$5 is received, the current model computes $\gamma$6 for each sample, confidence scores $\gamma$7 are formed, the subset $\gamma$8 is chosen by the hybrid confidence rule, entropy loss is computed on $\gamma$9, gradients are backpropagated only through LayerNorm $\beta$0, those parameters are updated using SGD, and the LayerNorm parameters are then reset for the next episode.

Aspect	Specification
Updated parameters	LayerNorm affine parameters $\beta$1
Frozen parameters	All other classifier/backbone parameters
Optimizer at test time	SGD
Test-time learning rate	$\beta$2
Test-time batch size	$\beta$3
Adaptation duration	One epoch
Confidence threshold	$\beta$4
Episode policy	Reset LayerNorm parameters to pre-trained values for each test episode

CELA itself is not trained separately. The main model is trained on clean labeled source data with the SSTC backbone and label smoothing. The implementation settings given for this source training are Adam with learning rate $\beta$5, epochs of $\beta$6 on PU and $\beta$7 on WHLK, label smoothing factor $\beta$8, patch sizes $\beta$9 for PU and $\{\gamma, \beta\}$0 for WHLK, and batch sizes $\{\gamma, \beta\}$1 for PU and $\{\gamma, \beta\}$2 for WHLK. These settings define the pre-trained model state to which the LayerNorm parameters are episodically reset at test time.

5. Empirical behavior and ablation evidence

The direct evidence for CELA is provided by an ablation study on the WHLK dataset in the table titled “Ablation Study on MRF, LSL, and CELA Modules on WHLK Dataset.” In that study, the full model (“Ours”) is compared with a variant denoted “w/o CELA.” Removing CELA reduces performance across several degradation types. On Blur, OA drops from $\{\gamma, \beta\}$3 to $\{\gamma, \beta\}$4; on Fog, from $\{\gamma, \beta\}$5 to $\{\gamma, \beta\}$6; on JPEG, from $\{\gamma, \beta\}$7 to $\{\gamma, \beta\}$8; on Additive noise, from $\{\gamma, \beta\}$9 to $\gamma$0; on Zero-mean Gaussian, from $\gamma$1 to $\gamma$2; and on Poisson, from $\gamma$3 to $\gamma$4 (Yue et al., 10 Sep 2025).

These results indicate that CELA contributes materially to robustness, with particularly large gains under several severe corruptions. The paper interprets the broader HyperTTA results as validating both the SSTC backbone and the proposed TTA scheme. At the system level, HyperTTA is compared against Tent, CoTTA, BNAdapt, CFA, SAR, and SoTTA. On the PU dataset, HyperTTA achieves OA $\gamma$5, AA $\gamma$6, and Kappa $\gamma$7. On the WHLK dataset, it achieves OA $\gamma$8, AA $\gamma$9, and Kappa $\beta$0, which are reported as the best overall results among the compared methods.

The qualitative evidence reported in the paper is consistent with the quantitative findings. The classification maps show cleaner spatially coherent predictions, fewer artifacts, better preservation of class boundaries, and greater stability under severe noise and blur. Since these are system-level observations on HyperTTA rather than isolated measurements of CELA alone, they are most appropriately read as evidence for the integrated SSTC-plus-CELA design.

6. Conceptual context and relation to LayerNorm-based parameter-efficient tuning

The choice of LayerNorm as the adaptation site has independent support from LayerNorm-based parameter-efficient tuning in Transformers. The paper "Parameter-Efficient Tuning on Layer Normalization for Pre-trained LLMs" shows that simply tuning the gain and bias terms of LayerNorm while freezing the rest of the model can be effective and highly lightweight, using only $\beta$1 trainable parameters in BERT-large and $\beta$2 in BERT-base (Qi et al., 2022). That work is not about hyperspectral image classification, confidence-aware filtering, or entropy minimization, but it establishes the broader point that LayerNorm gain and bias are meaningful adaptation knobs.

In that earlier formulation, LayerNorm is written as

$\beta$3

where $\beta$4 and $\beta$5 are the learnable gain and bias. This contextualizes CELA’s reliance on $\beta$6 and $\beta$7: the HyperTTA paper uses LayerNorm’s affine terms as a compact adaptation interface at test time, whereas the PLM work studies them as a parameter-efficient tuning target during downstream adaptation.

Several misconceptions are clarified by this comparison. CELA is not a generic PEFT method for downstream supervised fine-tuning; it is a source-free, label-free test-time adaptation procedure. It does not introduce explicit confidence estimation beyond maximum softmax probability, and it does not perform entropy minimization over all target samples. Conversely, the LayerNorm-tuning work does not discuss confidence-aware learning or entropy minimization. The connection between the two is architectural rather than objective-level: both treat LayerNorm affine parameters as sufficient levers for nontrivial adaptation, but they do so in different domains and with different optimization signals.

Taken together, these facts position CELA as a specialized LayerNorm-based TTA mechanism for HSI degradation robustness: SSTC supplies the stable spectral-spatial backbone, while CELA supplies selective, episodic, entropy-minimizing adjustment to the current target distribution.