Instance-wise Patch Normalization (IPN)

• IPN is a normalization mechanism that combines global instance and local patch statistics to maintain both overall distribution alignment and patch-level details.
• It addresses non-stationarity and temporal distribution shifts by mitigating oversmoothing while preserving local peaks and regime changes.
• A learnable scalar α enables a controlled interpolation between instance-level and patch-level normalization, enhancing forecasting stability in DeCoP.

Searching arXiv for the specified papers to ground the article in current records. Instance-wise Patch Normalization (IPN) is a normalization mechanism for patch-based time-series pre-training introduced in DeCoP, a dependency-controlled self-supervised framework for non-stationary time series. In DeCoP, IPN is an input-level procedure that normalizes each temporal patch using a learnable mixture of instance-level and patch-level statistics, with the stated goal of mitigating distributional shifts while preserving the unique characteristics of each patch. It is positioned as a response to limitations of sequence-level instance normalization in patch-based temporal modeling, particularly under temporal distribution shifts, multi-scale patterns, and susceptibility to spurious correlations (Wu et al., 18 Sep 2025).

1. Definition and problem setting

IPN is defined on a univariate time series $x \in \mathbb{R}^L$ that is divided into patches. Unlike standard instance normalization, which uses one mean and variance for the whole sequence, IPN combines two sources of statistics: the global mean and variance of the full time series, and the local mean and variance of each individual patch. The combination is controlled by a learnable scalar $\alpha$, so that normalization can interpolate between instance-dominated and patch-dominated behavior (Wu et al., 18 Sep 2025).

The motivation for IPN is tied to three difficulties identified for time-series pre-training. First, temporal distribution shifts and non-stationarity cause the distribution of values and dependency patterns to change over time, which can make sequence-level normalization unstable or misaligned across train and test settings. Second, time series contain both short-term fluctuations and long-term trends, and pure instance-level normalization tends to oversmooth local variations and weaken patch-level semantic structure. Third, coarse-grained normalization across all patches of an instance can blur local differences and encourage reliance on spurious correlations, degrading generalization under distribution shift (Wu et al., 18 Sep 2025).

Within that framing, IPN is intended to serve three simultaneous functions: align the overall distribution across time and between train and test through instance statistics, preserve localized patch semantics through patch statistics, and provide a controllable trade-off between the two via $\alpha$. The paper contrasts this behavior with instance normalization, which is described as good for removing global shifts but prone to suppressing peaks and transitions, and with patch-based normalization schemes that do not mix patch and instance statistics in a controlled way (Wu et al., 18 Sep 2025).

2. Mathematical formulation

Given a univariate time series $x \in \mathbb{R}^L$, DeCoP divides it into overlapping or non-overlapping patches of length $P$ with stride $S$. The number of patches is

$N = \frac{L - P}{S} + 1. \tag{1}$

The patch sequence is denoted

$X_p = \{ I_1, I_2, \dots, I_N \},$

where each patch $I_n \in \mathbb{R}^P$ is a contiguous temporal segment (Wu et al., 18 Sep 2025).

For each patch $I_n$, IPN computes the patch-wise mean and variance:

$\alpha$0

$\alpha$1

On the full time series, it computes the instance-level statistics

$\alpha$2

These are mixed using a learnable scalar parameter $\alpha$3:

$\alpha$4

$\alpha$5

Each patch is then normalized as

$\alpha$6

where $\alpha$7 is a small constant for numerical stability. The normalized patch sequence is

$\alpha$8

The limiting cases are explicitly identified. If $\alpha$9, the behavior is instance-level and IN-like; if $\alpha$0, normalization is dominated by patch-level statistics; intermediate values blend global and local information. During pretraining, DeCoP operates on the normalized patches, whereas during finetuning the original scale is recovered by denormalization using the stored mean, variance, and $\alpha$1 (Wu et al., 18 Sep 2025).

This formulation makes the intended mechanism explicit. The instance-level terms anchor each patch to the global distribution of the sequence, while the patch-specific terms retain local level and spread, thereby preserving peaks, regime changes, and short-term variations. A plausible implication is that IPN is best understood not as a replacement for either instance normalization or patch normalization, but as a learned interpolation between them.

3. Position within DeCoP

IPN is applied at the input level of DeCoP, before patch embedding and before the downstream temporal learner. The processing order is described as raw time series $\alpha$2, then patching, then IPN, producing normalized patches $\alpha$3, which are subsequently used by the masked time-series modeling component, the hierarchical Dependency Controlled Learning (DCL) module, and the Instance-level Contrastive Module (ICM) (Wu et al., 18 Sep 2025).

The interaction with DCL is described through the latent representation $\alpha$4 derived from the normalized patches:

$\alpha$5

Because IPN stabilizes the input distribution while preserving intra-patch semantics, DCL’s window-based multi-scale encoding is described as operating on cleaner and more stable patch distributions, reducing susceptibility to noisy local shifts. The intended consequence is that inter-patch dependencies learned by DCL more closely reflect genuine temporal relationships than artifacts introduced by drift or overly coarse normalization (Wu et al., 18 Sep 2025).

IPN also interacts with ICM, which constructs denoised positive pairs $\alpha$6 via frequency-domain filtering and applies contrastive learning on their representations. Two roles are stated. First, both views are normalized with IPN, so contrastive gradients are less dominated by global scale differences or non-stationary offsets. Second, IPN and ICM are complementary: IPN handles input-side distribution shift and local semantics, whereas ICM removes time-variant low-amplitude noise in the frequency domain to generate robust global positive pairs. In the architectural description, IPN appears directly after patching as “Instance-wise Patch Norm,” with DCL and ICM following as latent-level modules (Wu et al., 18 Sep 2025).

This positioning matters because IPN is not presented as an isolated normalization trick. Rather, it is the input-side mechanism that prepares patch representations for multi-scale dependency modeling and instance-level contrastive learning. This suggests that its role is primarily infrastructural: it conditions the statistical geometry of the patch sequence before the more expressive latent modules operate.

4. Comparison with other normalization strategies

The main explicit comparison is to instance normalization. The DeCoP paper reports a qualitative comparison between original data, IN-normalized data, and IPN-normalized data. The original train and test distributions differ due to non-stationarity; IN aligns train and test distributions better, but local peaks and semantics are smoothed out; IPN aligns the distributions similarly to IN while keeping local semantics such as peaks visible. The authors summarize this by stating that IN tends to oversmooth patch-level variations and weaken semantic expressiveness, whereas IPN adaptively normalizes both fine-grained patch-level statistics and coarse-grained instance-level distributions, retaining informative intra-patch variations more effectively (Wu et al., 18 Sep 2025).

The paper also situates IPN against other broad normalization families. Patch-based norms such as PatchTST-style normalization are described as using fixed statistics and not mixing patch and instance information in a controlled way. Batch normalization and layer normalization are characterized as operating over batch or feature dimensions and as not being directly tailored to temporal patch statistics and cross-time distribution shifts (Wu et al., 18 Sep 2025).

A related but distinct line of work is Patch-aware Batch Normalization (PBN), proposed for cross-domain robustness in computer vision. PBN normalizes spatial patches of feature maps using statistics computed across the batch and blends those patch statistics with globally accumulated batch-normalization statistics. Its formulation is explicitly batch-wise rather than instance-wise, and it is designed to preserve BN-style semantic robustness while exploiting patch-level diversity (Qi et al., 2023). The distinction is central: in the taxonomy implied by these two papers, IPN is an instance-wise patch normalization for time-series inputs, whereas PBN is a batch-wise patch normalization for visual feature maps.

The contrast can be summarized as follows.

Method	Statistics source	Domain
IN	One mean/variance per sequence or instance	Time series / general
IPN	Mixture of instance-wise and patch-wise statistics	Time series
PBN	Patch-wise batch statistics blended with global BN statistics	Computer vision

PBN is relevant chiefly as a conceptual analogue. It shows that patch-structured normalization can be combined with global statistics to improve robustness under domain shift, but it does not implement per-instance per-patch statistics. A plausible implication is that IPN occupies a more localized normalization regime than PBN, since its statistics are tied to each time-series instance rather than aggregated across the batch (Qi et al., 2023).

5. Empirical role in DeCoP

DeCoP reports ablation evidence for IPN in forecasting. In in-domain forecasting, removing IPN (“W/o IPN”) increases MSE by about $\alpha$7 relative to the full model. In cross-domain forecasting, removal of IPN also increases MSE, with the paper stating an increase of $\alpha$8 under in-domain and cross-domain settings. The text characterizes these gains as modest relative to DCL and ICM but consistent across settings, supporting the claim that IPN contributes to better generalization by stabilizing the inputs (Wu et al., 18 Sep 2025).

The broader performance context in DeCoP includes state-of-the-art results on ten datasets with lower computing resources. The abstract states that DeCoP improves MSE by $\alpha$9 on ETTh1 over PatchTST using only $x \in \mathbb{R}^L$0 of the FLOPs. In the detailed results, for ETTh1, DeCoPLinear achieves MSE $x \in \mathbb{R}^L$1, lower than PatchTST’s $x \in \mathbb{R}^L$2. In cross-domain forecasting, the authors highlight a $x \in \mathbb{R}^L$3 MSE improvement over PatchTST in ETTm1 $x \in \mathbb{R}^L$4 ETTh1 transfer. In classification, DeCoPMLP reaches $x \in \mathbb{R}^L$5 on Epilepsy $x \in \mathbb{R}^L$6 Epilepsy, surpassing SimMTM and PatchTST (Wu et al., 18 Sep 2025).

The computational profile is also reported. On ETTh1, DeCoP uses approximately $x \in \mathbb{R}^L$7M FLOPs in pretraining, versus $x \in \mathbb{R}^L$8M for PatchTST and $x \in \mathbb{R}^L$9M for SimMTM, while achieving lower MSE than both PatchTST and SimMTM. IPN itself is described as adding only a pass to compute instance mean and variance over $P$0, patch means and variances for each patch, and a single scalar $P$1, yielding essentially $P$2 overhead. The convergence analysis further indicates that DeCoP converges faster and exhibits smaller training-validation loss gaps than PatchTST, and IPN is described as part of the controllable normalization underlying this stability (Wu et al., 18 Sep 2025).

These results do not isolate IPN as the dominant source of performance gain, and the paper does not claim that it is. Rather, IPN appears as one of three key contributions, with its specific empirical effect concentrated on stabilizing forecasting under distribution shift. This suggests that its main importance is systemic rather than standalone: it prepares the signal in a way that allows DCL and ICM to operate more effectively.

6. Interpretation, practical considerations, and limitations

IPN is presented as a mechanism for handling both distribution shift and multi-scale temporal dependencies. At short scales, patch-wise statistics retain high-frequency fluctuations and local temporal patterns; at long scales, instance-level statistics provide a coherent global backdrop so that patches from different times occupy a consistent range. This is described as important for DCL, whose windows $P$3 grow across encoder blocks, with example configurations such as $P$4 or $P$5. If inputs are unstable or overly smoothed, window-based multi-scale encoding may either focus on noise or lose meaningful local structure (Wu et al., 18 Sep 2025).

Several implementation details are explicit. In forecasting, patch size and stride are both $P$6; in classification, patch size and stride are $P$7. In the univariate formulation, normalization is applied per patch along the temporal dimension. The paper primarily presents the univariate case, though it notes that the implementation can be extended channel-wise for multivariate series. Means and variances are computed along time indices: over $P$8 steps for instance statistics and over $P$9 steps for each patch (Wu et al., 18 Sep 2025).

The limitations discussed for IPN are largely implicit rather than experimentally developed. The method uses a single learnable scalar $S$0, which may be too coarse for highly heterogeneous series in which different patches would require different balances between global and local normalization. The formulation is univariate, and the paper does not investigate structured multivariate normalization or cross-channel covariance. IPN is also described in an offline pre-training setting, so online or streaming deployment would require evolving instance statistics. Finally, the framework pairs IPN with linear or MLP temporal learners in DCL, leaving open the question of how it behaves with more complex or attention-based encoders (Wu et al., 18 Sep 2025).

These limitations define the natural scope of interpretation. IPN is not a universal normalization doctrine; it is a specific hybrid normalization designed for patch-based self-supervised time-series learning under non-stationarity. A plausible implication is that future variants would likely focus on finer-grained control of $S$1, multivariate generalization, and adaptation to streaming or more complex encoder settings, but those extensions are not developed in the reported work (Wu et al., 18 Sep 2025).