Adaptive Context Normalization (ACN)

• Adaptive Context Normalization (ACN) is a context-conditioned normalization principle that replaces fixed parameters with sample-dependent ones based on auxiliary structures.
• It adapts affine parameters using signals like inter-channel similarity, discrete context labels, or learned attention weights, making it applicable to time-series, image, and point-set learning.
• Empirical studies show ACN improves optimization and representation quality across multiple networks while addressing limitations of standard normalization methods.

Adaptive Context Normalization (ACN) denotes a family of context-conditioned normalization schemes in which the normalization applied to a sample depends on auxiliary structure rather than only on fixed batch-, layer-, or instance-level statistics. The term is not standardized across the literature. It directly names at least three distinct lines of work: Adaptive CN for channel-identifiable multivariate time-series forecasting (Lee et al., 31 May 2025), “Adaptative Context Normalization” for supervised context-conditioned image normalization (Faye et al., 2024), and Attentive Context Normalization for robust permutation-equivariant point-set learning (Sun et al., 2019). Closely related precursors and analogues include Context Normalization (Faye et al., 2023), unsupervised cluster-based normalization mechanisms that dynamically form mixture components (Faye et al., 2024), and several adjacent adaptive-normalization methods that are not ACN proper (Zhang et al., 2023, Cho et al., 2019, Rad et al., 2020, Bhattacharyya et al., 12 Jul 2026).

1. Terminology and scope

The abbreviation ACN is polysemous. In time-series forecasting, it names an extension of Channel Normalization in which affine parameters are dynamically adjusted from the current sample’s inter-channel similarity structure, primarily to improve channel identifiability (CID) (Lee et al., 31 May 2025). In image processing, it denotes a supervised normalization layer in which each sample is assigned a predefined context $r_i$, and the sample is normalized with parameters $\theta_{r_i}=\{\mu_{r_i},\sigma_{r_i}\}$ learned for that context (Faye et al., 2024). In permutation-equivariant learning on point sets, ACN expands to Attentive Context Normalization, where normalization statistics are computed with learned attention weights so that outliers are downweighted during contextual aggregation (Sun et al., 2019).

A useful way to organize the literature is by the source of “context.” In one formulation, context is the channel identity and channel-similarity structure within the current multivariate time series (Lee et al., 31 May 2025). In another, context is a provided discrete label such as superclass, domain, or dataset identity (Faye et al., 2024, Faye et al., 2023). In a third, context is the point-set structure itself, summarized by attention-weighted global statistics over an unordered set (Sun et al., 2019).

Formulation	Conditioning signal	Representative paper
Adaptive CN	Inter-channel similarity in the current sample	(Lee et al., 31 May 2025)
Adaptative Context Normalization	Predefined context label $r$	(Faye et al., 2024)
Attentive Context Normalization	Learned set attention weights $w$	(Sun et al., 2019)

This terminological variability is a recurrent source of confusion. Several normalization methods are thematically adjacent but not ACN in the literal sense, including Affine Collaborative Normalization (Zhang et al., 2023), Adaptive Convolution-based Normalization (Cho et al., 2019), Adaptive Local Contrast Normalization (Rad et al., 2020), and layer-wise normalization routing in Transformers (Bhattacharyya et al., 12 Jul 2026).

2. Core mathematical pattern

Across its variants, ACN replaces a fixed normalization rule with one conditioned on context. In the time-series formulation built on LayerNorm-style statistics, the normalized representation is written as

$$\operatorname{Norm}\left(z_{b,c,d}\right)=\frac{z_{b,c,d}-\mu_{b,c}}{\sigma_{b,c}},$$

and the conventional shared affine map is replaced by channel-specific or sample-conditioned parameters, yielding

$$\hat{z}_{b,c,d}=\alpha_{b,c,d}\cdot \operatorname{Norm}(z_{b,c,d})+\beta_{b,c,d}.$$

ACN then makes $\alpha_{b,c,d}$ and $\beta_{b,c,d}$ depend on the current sample through a channel similarity matrix $\hat S$, so the affine parameters become dynamic rather than static (Lee et al., 31 May 2025).

In the supervised image formulation, the central transformation is context-indexed rather than sample-statistic-indexed: $$\hat{x}_i \gets \frac{x_i-\mu_{r_i}}{\sqrt{\sigma_{r_i}^2+\epsilon}}.$$ Here $\theta_{r_i}=\{\mu_{r_i},\sigma_{r_i}\}$0 is a context label supplied with the sample, and the trainable parameter set is augmented to include $\theta_{r_i}=\{\mu_{r_i},\sigma_{r_i}\}$1. A related precursor, Context Normalization, uses a learned mapping from a one-hot context ID to an embedding $\theta_{r_i}=\{\mu_{r_i},\sigma_{r_i}\}$2, then to $\theta_{r_i}=\{\mu_{r_i},\sigma_{r_i}\}$3 and $\theta_{r_i}=\{\mu_{r_i},\sigma_{r_i}\}$4 through affine maps, so that normalization parameters are generated from context rather than estimated from an arbitrary mini-batch (Faye et al., 2024, Faye et al., 2023).

In Attentive Context Normalization for point sets, the normalized feature map is

$\theta_{r_i}=\{\mu_{r_i},\sigma_{r_i}\}$5

where $\theta_{r_i}=\{\mu_{r_i},\sigma_{r_i}\}$6 and the weights

$\theta_{r_i}=\{\mu_{r_i},\sigma_{r_i}\}$7

are learned within the network. The defining change is that the mean and variance are no longer uniform first- and second-order moments across all points; they are weighted statistics that suppress outliers when computing the contextual signal shared across the set (Sun et al., 2019).

These formulations differ in implementation detail, but they share a common structural idea: normalization parameters or normalization statistics are conditioned on task-relevant structure that ordinary BN, LN, IN, or GN would ignore.

3. Principal formulations

The channel-identifiability formulation treats ACN as an extension of Channel Normalization for multivariate time-series forecasting. Its motivation is that many models are non-CID: if two channels have identical local inputs, they can produce identical outputs even when the correct forecasts differ. Channel Normalization replaces shared affine parameters with channel-specific ones, and ACN further introduces global parameters $\theta_{r_i}=\{\mu_{r_i},\sigma_{r_i}\}$8 and local parameters $\theta_{r_i}=\{\mu_{r_i},\sigma_{r_i}\}$9, mixing the local parameters through a similarity matrix

$r$0

The final dynamic affine parameters are

$r$1

which preserve channel-specific identity while adapting to the current sample’s channel context (Lee et al., 31 May 2025).

The supervised context-conditioned formulation treats context as an externally given partition of the data. Contexts can be superclasses, domains, dataset identities, or partitions produced by another algorithm. Samples sharing the same context are normalized with the same parameters, and those parameters are learned jointly with the rest of the network by backpropagation. In this usage, ACN is best understood as a supervised replacement for the mixture-component estimation of Mixture Normalization: instead of running EM to infer Gaussian components, it uses predefined contexts as proxies for components and learns context-indexed normalization parameters end to end (Faye et al., 2024).

The attentive permutation-equivariant formulation begins from Context Normalization as a set-level contextual operator for PointNet-style networks. Ordinary Context Normalization computes moments uniformly over all points, which is sensitive to outliers. Attentive Context Normalization replaces equal weighting with learned local and global attention. Local attention is

$r$2

global attention is

$r$3

and their product provides the weighting used in the normalization statistics. This preserves permutation-equivariance while making context aggregation robust to high outlier ratios (Sun et al., 2019).

A broader lineage also includes context-conditioned but non-canonical ACN analogues. Context Normalization conditions normalization on discrete contexts through learned embeddings (Faye et al., 2023). An unsupervised cluster-based analogue, UCB-Norm, dynamically forms Gaussian-mixture-like components over activations when prior contexts are unavailable (Faye et al., 2024). In time-series forecasting, AdaMamba’s Adaptive Normalization Block applies context-conditioned detrending and reversible standardization at the input level; it is ACN-like in the broad sense that normalization depends on the current sequence’s temporal context, but it is not an ACN layer in the strict affine-modulation sense (Jeon, 7 Dec 2025).

4. Architectural integration and application domains

In time-series forecasting, ACN is inserted within the encoder $r$4. If the backbone already uses LayerNorm, ACN replaces LN; if the backbone has no normalization, ACN is added. The paper evaluates this strategy on iTransformer, RMLP, S-Mamba, and TSMixer, with $r$5 and channel-indexed parameters in $r$6. This integration is explicitly orthogonal to input/output normalization methods such as RevIN, because ACN operates inside the encoder rather than as reversible preprocessing (Lee et al., 31 May 2025).

In image models, context-conditioned normalization has been inserted both as a replacement for existing BN layers and as an initial layer. In the ConvNet experiments, the third BN layer is replaced by ACN or ACN-base. In the ViT experiments, an initial BN layer is replaced by ACN. In AdaMatch domain adaptation, ACN is used as an initial layer so that source and target domains can be normalized with domain-specific parameters. The earlier Context Normalization work similarly uses CN-Channels as the first ConvNet layer, CN-Patches in ViT, and CN-Channels as the first layer of AdaMatch (Faye et al., 2024, Faye et al., 2023).

In permutation-equivariant point-set learning, ACN is embedded in Attentive Residual Blocks. Shared perceptrons process each point independently, while ACN provides the only cross-point interaction by computing weighted set statistics. A standard block uses two substructures of the form

$r$7

and stacks of such blocks are used for line fitting, wide-baseline stereo, and point-cloud classification. The architecture uses 3 residual layers for 2D point cloud classification, 6 for robust line fitting, and 12 for stereo; in 3D point cloud classification, ACN is inserted into PointNet (Sun et al., 2019).

The conditioning signal therefore changes with domain. In forecasting it is latent inter-channel similarity; in supervised image models it is a provided context identifier; in permutation-equivariant learning it is learned attention over the set. This suggests that “context” in ACN is a methodological role rather than a single object.

5. Empirical behavior

For multivariate time-series forecasting, Adaptive CN reports consistent gains across 12 datasets and four backbones. Average MSE improves from $r$8 to $r$9 for iTransformer, from $w$0 to $w$1 for RMLP, from $w$2 to $w$3 for S-Mamba, and from $w$4 to $w$5 for TSMixer. The main ablation on iTransformer gives $w$6 for no normalization change, $w$7 for adaptive-only local parameters, $w$8 for CN-only global parameters, and $w$9 for full ACN. The same paper also quantifies overhead on iTransformer/PEMS08: the base model has 3.2M parameters, 7.7 sec/epoch, and 2.0 ms inference, whereas $$\operatorname{Norm}\left(z_{b,c,d}\right)=\frac{z_{b,c,d}-\mu_{b,c}}{\sigma_{b,c}},$$0ACN adds 1.4M parameters and yields 10.8 sec/epoch and 2.5 ms inference (Lee et al., 31 May 2025).

For image processing, supervised context-conditioned ACN reports sizable gains when meaningful contexts are available. On CIFAR-100 with ViT, the reported accuracies are 55.63 for ViT+BN, 65.87 for ViT+ACN-base, and 67.38 for ViT+ACN. In domain adaptation from MNIST to SVHN with AdaMatch, target-domain accuracy improves from 25.08 for AdaMatch to 43.10 for AdaMatch+ACN-base and 54.70 for AdaMatch+ACN. The earlier Context Normalization lineage reports similarly large context-aware gains: on CIFAR-100 with ViT, ViT gives 52.37% accuracy, ViT+BN 53.35%, ViT+CN-Patches 63.80%, and ViT+CN-Channels 62.48%; on a blended CIFAR-100+MNIST dataset, the baseline ViT and ViT+BN collapse, while ViT+CN-Patches reaches 77.09% accuracy and ViT+CN-Channels 74.92% (Faye et al., 2024, Faye et al., 2023).

For robust permutation-equivariant learning, Attentive Context Normalization yields its clearest gains in high-outlier regimes. In line fitting, at 80% outliers the reported error is $$\operatorname{Norm}\left(z_{b,c,d}\right)=\frac{z_{b,c,d}-\mu_{b,c}}{\sigma_{b,c}},$$1 for ACNe and $$\operatorname{Norm}\left(z_{b,c,d}\right)=\frac{z_{b,c,d}-\mu_{b,c}}{\sigma_{b,c}},$$2 for CNe; at 90% outliers the values are $$\operatorname{Norm}\left(z_{b,c,d}\right)=\frac{z_{b,c,d}-\mu_{b,c}}{\sigma_{b,c}},$$3 and $$\operatorname{Norm}\left(z_{b,c,d}\right)=\frac{z_{b,c,d}-\mu_{b,c}}{\sigma_{b,c}},$$4. In 2D point cloud classification on MNIST with 60% outliers, ACNe reaches 93.7, compared with 87.2 for CNe and 54.8 for PointNet. In wide-baseline stereo with weighted 8-point estimation, outdoor mAP at $$\operatorname{Norm}\left(z_{b,c,d}\right)=\frac{z_{b,c,d}-\mu_{b,c}}{\sigma_{b,c}},$$5 is $$\operatorname{Norm}\left(z_{b,c,d}\right)=\frac{z_{b,c,d}-\mu_{b,c}}{\sigma_{b,c}},$$6 for ACNe, compared with $$\operatorname{Norm}\left(z_{b,c,d}\right)=\frac{z_{b,c,d}-\mu_{b,c}}{\sigma_{b,c}},$$7 for OANet and $$\operatorname{Norm}\left(z_{b,c,d}\right)=\frac{z_{b,c,d}-\mu_{b,c}}{\sigma_{b,c}},$$8 for CNe. The ablation on attention structure reports $$\operatorname{Norm}\left(z_{b,c,d}\right)=\frac{z_{b,c,d}-\mu_{b,c}}{\sigma_{b,c}},$$9 for local attention only, $$\hat{z}_{b,c,d}=\alpha_{b,c,d}\cdot \operatorname{Norm}(z_{b,c,d})+\beta_{b,c,d}.$$0 for global attention only, $$\hat{z}_{b,c,d}=\alpha_{b,c,d}\cdot \operatorname{Norm}(z_{b,c,d})+\beta_{b,c,d}.$$1 for local+global attention, and $$\hat{z}_{b,c,d}=\alpha_{b,c,d}\cdot \operatorname{Norm}(z_{b,c,d})+\beta_{b,c,d}.$$2 for local+global with supervision (Sun et al., 2019).

These results do not support a single universal claim about ACN’s magnitude of benefit. They do support a narrower conclusion: when the relevant context signal is well matched to the task—channels for CID, domain/superclass labels for heterogeneous image data, or learned inlierness for point sets—context-conditioned normalization can materially improve both optimization and representation quality.

6. Limitations, ambiguities, and related methods

The limitations are formulation-specific. Adaptive CN for time series requires a predefined fixed channel count $$\hat{z}_{b,c,d}=\alpha_{b,c,d}\cdot \operatorname{Norm}(z_{b,c,d})+\beta_{b,c,d}.$$3, because its parameters are channel-indexed; it is therefore suited to fixed-channel single-task settings rather than variable-channel or foundation-model settings, for which the same paper proposes PCN instead. Its gains are also smaller on already CID and already data-dependent backbones, and its formal theory is carried mainly by the CN analysis rather than by a separate theorem for ACN (Lee et al., 31 May 2025). Supervised image ACN depends on available and meaningful context labels and is not unsupervised; the paper explicitly identifies dynamic context acquisition as future work. It also leaves some implementation details under-specified, including the exact dimensionality of $$\hat{z}_{b,c,d}=\alpha_{b,c,d}\cdot \operatorname{Norm}(z_{b,c,d})+\beta_{b,c,d}.$$4 and the operational details of the mixture-style inference variant (Faye et al., 2024). Attentive Context Normalization is specialized to PointNet-like permutation-equivariant architectures and still degrades under extreme outlier ratios, although more gracefully than its non-attentive predecessor (Sun et al., 2019).

A second limitation is semantic: the acronym ACN does not identify a single canonical layer across arXiv. This is evident from the coexistence of Attentive Context Normalization (Sun et al., 2019), Adaptative Context Normalization (Faye et al., 2024), and Adaptive CN (Lee et al., 31 May 2025). The ambiguity extends to neighboring methods that are often retrieved in the same searches but are not identical. Affine Collaborative Normalization addresses channel misalignment in medical transfer learning and is explicitly not ACN (Zhang et al., 2023). Adaptive Convolution-based Normalization uses style-conditioned convolution in image translation rather than context-conditioned affine normalization (Cho et al., 2019). Adaptive Local Contrast Normalization is an image-space illumination-normalization front-end based on locally predicted Gaussian-mixture filter weights (Rad et al., 2020). AutoNorm performs layer-wise learned selection between LN and DyT via differentiable gating and studies optimization noise in adaptive normalization routing, but it is not a context-normalization layer in the stronger sense (Bhattacharyya et al., 12 Jul 2026). Unsupervised cluster-based normalization (Faye et al., 2024), adaptive time-series detrending and reversible normalization (Jeon, 7 Dec 2025), and continual-learning adaptive normalization with EMA-estimated feature scales (Marasco et al., 18 Mar 2026) are best read as ACN-like mechanisms rather than as instances of a single ACN standard.

Taken together, the literature suggests that ACN is best understood as a context-conditioned normalization principle rather than a uniquely defined module. The stable common denominator is the replacement of fixed normalization statistics or fixed affine parameters with quantities conditioned on task-relevant structure. What varies is the nature of that structure: channel identity, discrete context labels, mixture components, local visual neighborhoods, set attention weights, or temporal distribution history.