Unsupervised Adaptive Normalization (UAN)

• UAN is a family of unsupervised, data-dependent normalization strategies that adapt normalization rules dynamically to match shifting or multimodal feature distributions.
• Techniques include learned interpolation (e.g., AdaLIN), online updating of statistics (as in MixNorm and DUA), and Gaussian mixture-based methods that integrate clustering with normalization.
• Empirical studies in tasks like image translation, domain adaptation, and re-identification show that UAN methods improve performance by balancing global stability with local adaptation.

Searching arXiv for the cited UAN-related papers to ground the article in the literature. Unsupervised Adaptive Normalization (UAN) denotes, in the literature represented here, a family of normalization strategies in which the normalization mechanism is adjusted without label supervision to match the structure, shift, or multimodality of activations or features. Across these works, adaptation occurs in several distinct ways: by learning an interpolation between normalization operators inside a generator, by updating running normalization statistics online at test time, by combining domain-specific and domain-shared branches, by embedding Gaussian-mixture clustering into normalization, or by scaling activations according to unsupervised code-length criteria. The common thread is that normalization is treated not as a fixed preprocessing rule but as a task- and distribution-dependent component of the model itself (Kim et al., 2019, Faye et al., 2024).

1. Conceptual scope and problem setting

A recurring motivation for UAN is that fixed normalization assumptions are often mismatched to the actual data regime. Batch Normalization depends on sufficiently large mini-batches and presumes a uniform distribution of samples, while LayerNorm, InstanceNorm, and GroupNorm still impose a single set of statistics per layer, group, or instance (Faye et al., 2024). Several of the methods considered here start from the observation that activation or feature distributions may be multimodal, domain-specific, or rapidly shifting, so a single normalization rule can be inadequate (Hu et al., 2021, Mirza et al., 2021).

In unsupervised image-to-image translation, the difficulty is not merely covariate shift but the coexistence of tasks requiring “style-only” change and tasks requiring large geometric change. U-GAT-IT therefore introduces AdaLIN, which adaptively blends Instance Normalization and Layer Normalization so that the model can control shape and texture changes by learned parameters depending on datasets (Kim et al., 2019). In unsupervised domain generalization for person re-identification, the challenge is that none of the source domains carries identity labels, so normalization must simultaneously support pseudo-label generation and cross-domain generalization; the proposed framework accordingly combines domain-specific BN with shared IN (Qi et al., 2021).

In test-time and source-free adaptation, the emphasis shifts toward the failure of stored BN statistics under distribution shift. MixNorm states that training-time BN moments become biased on shifted test streams, especially when batch size is small or the stream mixes multiple distributions, and addresses this by interpolating global running estimates with per-sample local statistics derived from augmentations (Hu et al., 2021). DUA and UBNA pursue related goals by continuously adapting BN running statistics with decaying momentum, thereby mixing source and target statistics without back-propagating through network weights (Mirza et al., 2021, Klingner et al., 2020). AdaFlow applies domain-adaptive normalization inside a normalizing flow and adapts to a new distribution by just conducting forward propagation once per sample (Yamaguchi et al., 2018).

A plausible implication is that “UAN” is best understood not as a single standardized layer but as a design pattern for unsupervised, data-dependent control of normalization.

2. Learned interpolation between normalization operators

One influential realization of UAN is AdaLIN in U-GAT-IT. The layer is defined as

$$\mathrm{AdaLIN}(a,\gamma,\beta)=\gamma\cdot\bigl(\rho\cdot \hat a_I+(1-\rho)\cdot \hat a_L\bigr)+\beta,$$

with

$$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$

and

$$\rho \leftarrow \mathrm{clip}_{[0,1]}(\rho-\tau\,\Delta\rho).$$

Here $a\in\mathbb R^{C\times H\times W}$ is the input activation map, $\mu_I,\sigma_I$ are the per-channel InstanceNorm statistics, $\mu_L,\sigma_L$ are the per-layer LayerNorm statistics, $\rho\in[0,1]$ is a learnable gate, and $\gamma,\beta\in\mathbb R^C$ are affine parameters that in U-GAT-IT are dynamically generated per sample from the attention feature map via a small MLP (Kim et al., 2019).

The interpretation is explicit. If $\rho=1$, AdaLIN reduces to affine IN and preserves per-channel detailed statistics; if $\rho=0$, it reduces to affine LN and normalizes across all channels. For intermediate $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$0, the output is a convex combination

$$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$1

In the U-GAT-IT architecture, AdaLIN appears only in the decoder of each generator, specifically in four AdaResBlocks, while the up-sampling blocks use a simplified LIN that also blends IN and LN but without dynamic $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$2 (Kim et al., 2019).

A related but structurally different interpolation appears in the person ReID framework of domain-specific adaptive normalization. There, each residual-block feature map $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$3 is split into two equal channel slices, one routed to an IN branch and one to a domain-specific BN branch. The combined output is

$$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$4

The paper also gives a more generic formulation,

$$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$5

with per-domain, per-channel vectors $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$6, though the implemented design fixes a half-and-half split rather than learning $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$7 (Qi et al., 2021).

These two examples encode different notions of adaptation. AdaLIN learns a scalar gate per layer in the decoder and produces per-sample affine parameters from attention features (Kim et al., 2019). DSAN fixes the split ratio but makes the BN statistics and affine terms domain-specific while keeping IN shared (Qi et al., 2021). This suggests that UAN can operate either through explicit learnable mixing coefficients or through structured branch factorization.

3. Test-time and source-free adaptation through normalization statistics

A second major UAN lineage treats normalization statistics as online state variables for adaptation under shift. In MixNorm, at each normalization layer one observes the current test-sample feature map $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$8 and an augmented view $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$9, while maintaining exponential-moving global estimates initialized from training-time BN statistics, $$\rho \leftarrow \mathrm{clip}_{[0,1]}(\rho-\tau\,\Delta\rho).$$0 and $$\rho \leftarrow \mathrm{clip}_{[0,1]}(\rho-\tau\,\Delta\rho).$$1 (Hu et al., 2021). For sample-local statistics,

$$\rho \leftarrow \mathrm{clip}_{[0,1]}(\rho-\tau\,\Delta\rho).$$2

the running estimates are updated as

$$\rho \leftarrow \mathrm{clip}_{[0,1]}(\rho-\tau\,\Delta\rho).$$3

Using the pair $$\rho \leftarrow \mathrm{clip}_{[0,1]}(\rho-\tau\,\Delta\rho).$$4 as a local batch of size two, MixNorm then computes local moments and interpolates them with the global estimates:

$$\rho \leftarrow \mathrm{clip}_{[0,1]}(\rho-\tau\,\Delta\rho).$$5

followed by standard BN-style normalization

$$\rho \leftarrow \mathrm{clip}_{[0,1]}(\rho-\tau\,\Delta\rho).$$6

The stated purpose is stable adaptation even when $$\rho \leftarrow \mathrm{clip}_{[0,1]}(\rho-\tau\,\Delta\rho).$$7 and test samples come from arbitrary, mixed domains (Hu et al., 2021).

DUA uses a simpler online update rule. If $$\rho \leftarrow \mathrm{clip}_{[0,1]}(\rho-\tau\,\Delta\rho).$$8 are the current BN running statistics and $$\rho \leftarrow \mathrm{clip}_{[0,1]}(\rho-\tau\,\Delta\rho).$$9 are the moments of a small target batch formed by augmentations of the current incoming image, then with a decaying momentum schedule $a\in\mathbb R^{C\times H\times W}$0,

$a\in\mathbb R^{C\times H\times W}$1

The momentum decays as $a\in\mathbb R^{C\times H\times W}$2 with lower bound $a\in\mathbb R^{C\times H\times W}$3, and the updated BN moments are used immediately for future inputs (Mirza et al., 2021).

UBNA follows the same general principle but formulates it as a batch-dependent BN learning rate. With $a\in\mathbb R^{C\times H\times W}$4 and $a\in\mathbb R^{C\times H\times W}$5, the adapted BN moments at layer $a\in\mathbb R^{C\times H\times W}$6 are

$a\in\mathbb R^{C\times H\times W}$7

and UBNA$a\in\mathbb R^{C\times H\times W}$8 introduces an additional per-layer decay $a\in\mathbb R^{C\times H\times W}$9 (Klingner et al., 2020). All network weights and BN affine parameters remain frozen.

AdaFlow represents a source-free variant inside a normalizing flow. Its Adaptive Batch-Normalization replaces batch statistics by per-domain statistics:

$\mu_I,\sigma_I$0

where $\mu_I,\sigma_I$1 are the means and variances of the current domain and $\mu_I,\sigma_I$2 are shared across domains. For a new domain, $\mu_I,\sigma_I$3 and $\mu_I,\sigma_I$4 are computed unsupervisedly by a forward pass over the new-domain samples, with no back-propagation (Yamaguchi et al., 2018).

Across these methods, the factual commonality is that adaptation is achieved by changing normalization statistics rather than retraining the main weights (Hu et al., 2021, Mirza et al., 2021, Klingner et al., 2020, Yamaguchi et al., 2018).

4. Mixture-based and information-theoretic formulations

The 2024 paper explicitly titled “Unsupervised Adaptive Normalization” frames UAN as a one-stage clustering-and-normalization layer based on a Gaussian mixture model (Faye et al., 2024). It assumes activations $\mu_I,\sigma_I$5 follow

$\mu_I,\sigma_I$6

with $\mu_I,\sigma_I$7 and $\mu_I,\sigma_I$8. Responsibilities are

$\mu_I,\sigma_I$9

Instead of a separate EM loop, the mixture parameters $\mu_L,\sigma_L$0 are treated as learnable parameters and updated by backpropagation or by a moving-average scheme that mimics one EM iteration per batch (Faye et al., 2024).

Given these parameters, the normalized contribution from component $\mu_L,\sigma_L$1 is

$\mu_L,\sigma_L$2

weighted by

$\mu_L,\sigma_L$3

so that the UAN-normalized activation is

$\mu_L,\sigma_L$4

The paper’s central claim is that clustering and normalization are merged inside the backpropagation loop, with GMM parameters aligned to the target task during training (Faye et al., 2024).

Regularity Normalization offers a different unsupervised mechanism. Rather than using moments or mixture components, it computes a normalization factor from universal code length under the Minimum Description Length principle. For activations $\mu_L,\sigma_L$5 at layer $\mu_L,\sigma_L$6 and batch index $\mu_L,\sigma_L$7, it defines an incremental universal code

$\mu_L,\sigma_L$8

with code length

$\mu_L,\sigma_L$9

In the layerwise algorithm, each activation is scaled as

$\rho\in[0,1]$0

Under a Gaussian specialization, the single-sample code length is

$\rho\in[0,1]$1

up to the stated additive reformulation (Lin, 2019).

These formulations widen the meaning of UAN. One branch models multimodality explicitly through Gaussian mixtures and learns the cluster parameters jointly with the network (Faye et al., 2024). Another branch treats normalization as unsupervised attention driven by statistical regularity rather than by first- and second-order moments alone (Lin, 2019). A common misconception is that UAN must be a BN variant; the literature here includes BN-based, IN/LN-interpolative, mixture-based, and MDL-based realizations.

5. Architectural patterns and empirical behavior

The empirical evidence reported in these papers spans translation, classification, domain adaptation, re-identification, semantic segmentation, anomaly detection, reinforcement learning, and object detection.

In U-GAT-IT, the generator uses an encoder with standard IN residual blocks, an attention CAM, and a decoder containing four AdaResBlocks equipped with AdaLIN plus two up-sampling blocks. The discriminator uses two-scale PatchGAN discriminators with spectral normalization and CAM, but does not use AdaLIN (Kim et al., 2019). In the selfie2anime ablation, KID$\rho\in[0,1]$2 for U-GAT-IT with AdaLIN is reported as $\rho\in[0,1]$3, compared with $\rho\in[0,1]$4 for IN, $\rho\in[0,1]$5 for LN, $\rho\in[0,1]$6 for AdaIN, and $\rho\in[0,1]$7 for GN. The same paper reports that across 135 participants, U-GAT-IT was preferred $\rho\in[0,1]$8 on selfie2anime, $\rho\in[0,1]$9 on horse2zebra, $\gamma,\beta\in\mathbb R^C$0 on cat2dog, $\gamma,\beta\in\mathbb R^C$1 on photo2portrait, and $\gamma,\beta\in\mathbb R^C$2 on photo2vangogh (Kim et al., 2019).

In the unsupervised DG-ReID setting, the baseline on Ma + D$\gamma,\beta\in\mathbb R^C$3Ms achieves mAP $\gamma,\beta\in\mathbb R^C$4, Rank-1 $\gamma,\beta\in\mathbb R^C$5, IBN-Net50 gives mAP $\gamma,\beta\in\mathbb R^C$6, Rank-1 $\gamma,\beta\in\mathbb R^C$7, and UAN/DSAN raises this to mAP $\gamma,\beta\in\mathbb R^C$8, Rank-1 $\gamma,\beta\in\mathbb R^C$9. Ablations report only DSBN at mAP $\rho=1$0, Rank-1 $\rho=1$1, DSBN + IN at mAP $\rho=1$2, Rank-1 $\rho=1$3, and the 50/50 split as best relative to 25/75 or 75/25. Clustering quality, measured by Adjusted Mutual Information, rises from approximately $\rho=1$4 to approximately $\rho=1$5 (Qi et al., 2021).

For test-time adaptation, MixNorm reports that under the standard protocol on CIFAR-10C with $\rho=1$6, TENT error is approximately $\rho=1$7 and MixNormBN approximately $\rho=1$8, while under small-batch or $\rho=1$9 mixed corruptions, TENT error jumps to approximately $\rho=0$0 on CIFAR-10C and MixNorm remains around approximately $\rho=0$1 at $\rho=0$2. In source-free UDA, SVHN$\rho=0$3MNIST changes from $\rho=0$4 to $\rho=0$5 error, SVHN$\rho=0$6USPS from $\rho=0$7 to $\rho=0$8, and SVHN$\rho=0$9MNIST-M from $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$00 to $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$01. For zero-shot classification with a CLIP RN50 backbone, replacing BN with MixNorm at inference changes CIFAR-100 from $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$02 to $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$03 accuracy and CIFAR-10 from $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$04 to $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$05, while Food-101 and STL-10 remain on-par (Hu et al., 2021).

DUA reports, at severity 5 on CIFAR-10C with ResNet-26, source error $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$06 using 80 samples; on CIFAR-100C, $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$07; and on ImageNet-C with ResNet-18, $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$08 with 100 samples. In object detection on KITTI$$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$09KITTI-Fog at 30 m visibility, mAP@50 for Car, Ped, and Cyc changes from $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$10 to $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$11 with 25 images (Mirza et al., 2021). UBNA, on semantic segmentation with VGG-16, batch size $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$12, and $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$13 steps, reports GTA-5$$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$14Cityscapes mIoU(19) of $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$15 for no adaptation, $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$16 for AdaBN, $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$17 for UBNA, and $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$18 for UBNA$$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$19; on SYNTHIA$$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$20Cityscapes mIoU(16), the corresponding values are $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$21, $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$22, $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$23, and $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$24 (Klingner et al., 2020).

The mixture-based 2024 UAN paper reports, for a shallow CNN at learning rate $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$25, test error of $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$26 for BN, $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$27 for MixtureNorm with $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$28, and $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$29 for UAN with $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$30; at learning rate $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$31, BN diverges, MixtureNorm plateaus, and UAN retains stable training with test error around $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$32. On DenseNet-40 and DenseNet-100 for CIFAR-100, the reported errors are $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$33 for BN, $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$34 for MixtureNorm, and $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$35 for UAN. In AdaMatch + UAN domain adaptation, source-domain accuracy changes from $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$36 to $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$37 and target-domain accuracy from $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$38 to $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$39 (Faye et al., 2024).

Regularity Normalization reports that on imbalanced MNIST with $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$40, test error is $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$41 for no norm, $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$42 for BN, $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$43 for LN, $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$44 for RN, and $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$45 for LN+RN. In reinforcement learning, DQN+RN reaches $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$46 on LunarLander compared with $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$47 for the baseline, and DQN+RN+LN reaches $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$48 on CartPole compared with $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$49 for the baseline (Lin, 2019).

6. Interpretive themes, limitations, and extensions

Several interpretive themes recur across the literature. First, many UAN methods explicitly balance preservation and adaptation. AdaLIN is motivated by the contrast between IN, which is described as sufficient for pure texture transfers but limited in global context, and LN, which better supports large style or shape shifts but can destroy fine content structure (Kim et al., 2019). DSAN similarly combines shared IN, which removes style-specific mean and variance, with domain-specific BN, which preserves enough content-discriminative variance for clustering (Qi et al., 2021). MixNorm formalizes the same trade-off as one between a slow-moving global anchor and rapid local correction (Hu et al., 2021).

Second, many UAN methods are deliberately lightweight. DUA states that it requires no additional network parameters, no domain discriminator, and no self-supervised head, with overhead approximately $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$50–$$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$51 runtime because only forward passes and channelwise moment computations are added (Mirza et al., 2021). AdaFlow states that adaptation involves only a single forward pass per sample and reports adaptation time of $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$52 for AdaFlow versus $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$53 for full fine-tuning in its experiments (Yamaguchi et al., 2018). UBNA likewise freezes all network weights and adapts only BN running statistics with a decaying schedule (Klingner et al., 2020).

Third, the literature also states clear limitations. DUA notes that moment matching only adjusts first two moments and may not fully correct higher-order or label-conditional shifts; it also warns that rapidly mixed test distributions may require re-tuning of the decaying-momentum scheme to avoid catastrophic forgetting (Mirza et al., 2021). AdaFlow notes that only second-order moments are aligned explicitly and that robustness to outlier contamination in adaptation batches remains to be tested (Yamaguchi et al., 2018). The 2024 UAN paper identifies the choice of the number of clusters $$\hat a_I=\frac{a-\mu_I}{\sqrt{\sigma_I^2+\epsilon}},\qquad \hat a_L=\frac{a-\mu_L}{\sqrt{\sigma_L^2+\epsilon}},$$54 as dataset-sensitive and notes a small memory overhead for very wide layers (Faye et al., 2024).

Possible extensions are stated directly in several papers. U-GAT-IT proposes that the AdaLIN idea can be ported to multi-domain style transfer GANs, video-to-video translation or temporal GANs, unsupervised domain adaptation networks, variational autoencoders or diffusion models for style-conditional generation, and self-supervised representation learning (Kim et al., 2019). DUA suggests extending the same decayed-momentum update concept to other normalization schemes that maintain running statistics, including GroupNorm, InstanceNorm, and LayerNorm (Mirza et al., 2021). The 2024 UAN paper proposes integration into generative adversarial networks, automatic selection or pruning of clusters during training, and extension to hierarchical or spatially-varying mixtures (Faye et al., 2024).

A plausible synthesis is that UAN methods differ chiefly in what is adapted: interpolation weights, running moments, domain paths, mixture parameters, or code-length scalings. The literature collectively indicates that unsupervised adaptation of normalization can function as a primary adaptation mechanism rather than merely as an auxiliary training detail.