Extended Whitening Filters

• Extended whitening filters are transformations that not only whiten data but also impose a useful secondary structure on related matrices or processes.
• They leverage the unitary non-uniqueness of standard whitening to enforce objectives like diagonalization or triangularization, thereby enhancing algorithmic efficiency.
• Applications span from sensor networks and multirate filter banks to adaptive neural circuits and optimization preconditioners, reducing latency and improving accuracy.

Extended whitening filters are whitening transformations selected not only to produce white outputs, but also to impose useful secondary structure on other vectors, matrices, processes, or downstream computations. In the strict formulation introduced as linear extended whitening filters, the central mechanism is the non-uniqueness of standard whitening: once a primary covariance is mapped to the identity, a residual unitary degree of freedom can be exploited to diagonalize, triangularize, or otherwise regularize a secondary object without sacrificing second-order whitening. Later work has broadened the same design principle to adjacency-constrained graph filters, multirate filter banks, convolutional approximations, conditional normalization layers, adaptive neural circuits, optimizer preconditioners, and low-latency scientific pipelines, while retaining whitening as the organizing operation (Krishnamoorthy, 2013).

1. Algebraic foundation and non-uniqueness

Let $\mathbf{v}\in\mathbb{C}^M$ be a zero-mean random vector with positive-definite Hermitian covariance $\mathbf{\Sigma}=\mathrm{Cov}(\mathbf{v})$. A linear standard whitening filter (SWF) is any full-rank matrix $\mathbf{F}\in\mathbb{C}^{M\times M}$ satisfying

$$\mathbf{F}\mathbf{\Sigma}\mathbf{F}^H=\mathbf{I}_M.$$

If $\mathbf{F}$ is an SWF and $\mathbf{Q}$ is unitary, then $\mathbf{W}=\mathbf{Q}\mathbf{F}$ is also an SWF, because unitary post-transforms preserve the identity covariance. This yields an infinite family of whitening filters that are equivalent with respect to the first two central moments of the whitened vector. A canonical representative is the Cholesky-based SWF: if $\mathbf{\Sigma}=\mathbf{L}\mathbf{L}^H$, then $\mathbf{F}_c=\mathbf{L}^{-1}$. An eigenvalue-based SWF is $\mathbf{F}_e=\mathbf{A}^{-1/2}\mathbf{Q}_\Sigma^H$ when $\mathbf{\Sigma}=\mathrm{Cov}(\mathbf{v})$0. Every SWF can be written as $\mathbf{\Sigma}=\mathrm{Cov}(\mathbf{v})$1 for some unitary $\mathbf{\Sigma}=\mathrm{Cov}(\mathbf{v})$2 (Krishnamoorthy, 2013).

This non-uniqueness is the decisive algebraic fact behind extended whitening. Standard whitening is often presented as a single covariance-normalizing map, but the theorem that every SWF differs from the Cholesky-based filter only by a unitary factor shows that whitening is a structured equivalence class rather than a unique operator. The secondary design freedom is therefore not an approximation or heuristic; it is intrinsic to exact second-order whitening. A common misconception is that all exact whitening transforms are interchangeable in every practical sense. They are interchangeable only at second order. Their side-effects on other matrices, numerical workflows, and downstream algorithms may differ substantially.

2. Linear extended whitening in the strict sense

A linear extended whitening filter (EWF) is an SWF that, in addition, has desirable secondary properties when applied to another random vector, matrix, or process. The general construction is simple. One first chooses a baseline SWF $\mathbf{\Sigma}=\mathrm{Cov}(\mathbf{v})$3 for the primary vector $\mathbf{\Sigma}=\mathrm{Cov}(\mathbf{v})$4. One then transforms a secondary object by $\mathbf{\Sigma}=\mathrm{Cov}(\mathbf{v})$5, computes a decomposition that exposes a useful unitary factor $\mathbf{\Sigma}=\mathrm{Cov}(\mathbf{v})$6, and finally defines

$\mathbf{\Sigma}=\mathrm{Cov}(\mathbf{v})$7

Because $\mathbf{\Sigma}=\mathrm{Cov}(\mathbf{v})$8 is unitary, $\mathbf{\Sigma}=\mathrm{Cov}(\mathbf{v})$9 still holds. The difference from ordinary whitening is that the unitary factor is no longer arbitrary; it is chosen to enforce structure on the secondary object (Krishnamoorthy, 2013).

The original paper gives three explicit constructions. If $\mathbf{F}\in\mathbb{C}^{M\times M}$0 is a second random vector with covariance $\mathbf{F}\in\mathbb{C}^{M\times M}$1, and $\mathbf{F}\in\mathbb{C}^{M\times M}$2 is an eigendecomposition, then $\mathbf{F}\in\mathbb{C}^{M\times M}$3 whitens $\mathbf{F}\in\mathbb{C}^{M\times M}$4 and makes $\mathbf{F}\in\mathbb{C}^{M\times M}$5 diagonal. If $\mathbf{F}\in\mathbb{C}^{M\times M}$6 is rectangular and $\mathbf{F}\in\mathbb{C}^{M\times M}$7 is a QR decomposition, then $\mathbf{F}\in\mathbb{C}^{M\times M}$8 is upper triangular. If $\mathbf{F}\in\mathbb{C}^{M\times M}$9 is square and $$\mathbf{F}\mathbf{\Sigma}\mathbf{F}^H=\mathbf{I}_M.$$0 is a polar decomposition, then $$\mathbf{F}\mathbf{\Sigma}\mathbf{F}^H=\mathbf{I}_M.$$1 is Hermitian positive semidefinite. These examples define the archetypal EWF pattern: exact whitening of a primary covariance together with a designed side-effect on a secondary operator (Krishnamoorthy, 2013).

The associated trade-offs are also explicit. Enforcing secondary structure reduces the effective degrees of freedom in the whitening transform, but does not affect second-order whitening performance. The original treatment notes that higher-order effects are largely neglected, so EWFs are primarily second-order constructions. This suggests that the main gain of an EWF is architectural: it co-designs whitening with subsequent linear algebra, replacing a formally arbitrary unitary factor by one that simplifies the rest of the pipeline.

3. Constrained, graph-structured, and multirate realizations

A major extension of the EWF idea appears when the whitening transform itself is structurally constrained. In distributed estimation over sensor networks, Kar, Varshney, and Chen formulate an adjacency-based spatial whitening scheme in which each sensor can combine only its neighbors’ observations. The admissible transforms are

$$\mathbf{F}\mathbf{\Sigma}\mathbf{F}^H=\mathbf{I}_M.$$2

with the mean-preserving constraint $$\mathbf{F}\mathbf{\Sigma}\mathbf{F}^H=\mathbf{I}_M.$$3. Whitening is no longer exact in the global sense; instead, the transformed covariance $$\mathbf{F}\mathbf{\Sigma}\mathbf{F}^H=\mathbf{I}_M.$$4 is made as diagonal as possible by minimizing the log-determinant divergence to a diagonal matrix. After reparameterization by $$\mathbf{F}\mathbf{\Sigma}\mathbf{F}^H=\mathbf{I}_M.$$5, the problem becomes

$$\mathbf{F}\mathbf{\Sigma}\mathbf{F}^H=\mathbf{I}_M.$$6

and a block-coordinate descent algorithm yields a closed-form row update

$$\mathbf{F}\mathbf{\Sigma}\mathbf{F}^H=\mathbf{I}_M.$$7

The resulting algorithm converges to a stationary point of a non-convex problem. In the reported 50-node random geometric graph experiments with exponentially correlated noise, spatial whitening significantly lowers MSE compared to the unwhitened strategy, and performance approaches PCA whitening as the adjacency radius grows (Kar et al., 2012).

A different structured realization is the signal matched multirate whitening filter bank. There whitening is extended from a single-rate predictor to an $$\mathbf{F}\mathbf{\Sigma}\mathbf{F}^H=\mathbf{I}_M.$$8-channel multirate analysis bank whose outputs are whitened within each channel and mutually whitened across channels by solving a constrained projection problem. The method yields a fast exact least-squares algorithm with a lattice-like structure and an order-recursive recovery of the analysis filter-bank coefficients from lattice parameters. Simulations show that the design can whiten Gaussian and non-Gaussian processes with minimum as well as non-minimum phase. In the coding-gain comparison for AR(2) inputs, Lu’s biorthogonal filter bank gives 6.64 dB and 4.92 dB for two reported pole configurations, whereas the signal matched multirate whitening filter bank gives 14.56 dB and 10.60 dB, respectively (Fatimah et al., 2014).

These two cases clarify an important point. Exact global whitening is not always the appropriate design target. Under communication topology constraints or multirate architectural constraints, the meaningful extension is often a constrained or approximate whitening transform whose utility lies in implementability and secondary structure rather than in unconstrained covariance annihilation.

4. Convolutional, adaptive, and learned embodiments

In spiking neural networks, whitening has been extended from a dense global linear map to local convolutional filters suitable for neuromorphic hardware. The visual-feature-learning work based on STDP uses ZCA whitening as pre-processing and then derives per-channel convolution kernels from patch-level whitening matrices by impulse responses. This approximation is local, hardware-friendly, and stable across patch sizes and datasets. On CIFAR-10, with 64 filters, standard whitening gives 57.66% and whitening kernels 57.07%, compared with 48.27% for on-center/off-center color coding; with 1024 filters, standard whitening gives 63.37% and whitening kernels 63.64%, compared with 56.93% for on-center/off-center color coding. Cross-dataset experiments on CIFAR-10 and STL-10 further show that the whitening kernels are fairly stable across datasets (Falez et al., 2020).

In generative modeling, whitening becomes a trainable normalization module. The whitening-and-coloring batch transform for GANs computes a batch covariance, applies Cholesky whitening with $$\mathbf{F}\mathbf{\Sigma}\mathbf{F}^H=\mathbf{I}_M.$$9, and then applies a learnable coloring matrix $\mathbf{F}$0 and bias $\mathbf{F}$1, or class-conditional $\mathbf{F}$2 in conditional settings. This generalizes BatchNorm and conditional BatchNorm from diagonal rescaling to full-feature whitening plus full multivariate coloring. The reported Cholesky whitening is more than $\mathbf{F}$3 faster than ZCA whitening in the whitening-only pipeline, and the full whitening-coloring pipeline is more than $\mathbf{F}$4 faster. On conditioned CIFAR-10, the cBN baseline reports IS 8.62, whereas cWC SN + Projection discriminator with tuned learning rate reports $\mathbf{F}$5 (Siarohin et al., 2018).

Adaptive neural-circuit models and optimizer designs push the same idea further. In adaptive whitening with gain-modulating interneurons, whitening is formulated through unit marginal-variance constraints along an overcomplete frame, and the effective whitening operator is realized as $\mathbf{F}$6 with fixed synaptic weights and adaptive gains. A non-negative-gain variant implements an upper-bound form of whitening that avoids amplifying low-variance directions and improves robustness to ill-conditioned inputs (Duong et al., 2023). In large-scale optimization, Zeta introduces a dual whitening pipeline in which coordinate whitening

$\mathbf{F}$7

precedes spectral whitening via Newton–Schulz orthogonalization. The paper states that the order is not tunable: coordinate whitening establishes the statistical isotropy that spectral whitening requires. Reported speedups include 1.67$\mathbf{F}$8 over AdamW for Qwen3-1.7B and 1.25$\mathbf{F}$9 over AdamW for Qwen3-8B, with downstream gains in both language and vision tasks (Chen et al., 12 Jun 2026).

Taken together, these systems show that extended whitening is not limited to a post hoc preprocessing stage. It can be embedded into feature learning, normalization, adaptive circuitry, or preconditioning, provided that whitening remains the operation that regularizes geometry and the extension lies in how that geometry is exploited.

5. Whitening in inference, language, and activation space

In Bayesian synthetic likelihood, whitening is used to reduce the cost of covariance estimation for high-dimensional summary statistics. Whitening BSL fixes an approximate whitening matrix $\mathbf{Q}$0 at a parameter value $\mathbf{Q}$1 in a high posterior density region, transforms the summaries, and then applies shrinkage covariance estimation in the whitened space. Under Gaussian assumptions, the variance of the log synthetic likelihood scales as

$\mathbf{Q}$2

for full-covariance BSL, but as

$\mathbf{Q}$3

when the summaries are uncorrelated. Empirically, the paper reports that wBSL can reduce the number of model simulations required to implement BSL by more than an order of magnitude, without much loss of accuracy (Priddle et al., 2019).

In distributional semantics, whitening has been redefined relative to a non-uniform base measure. Zipfian whitening computes the mean and covariance of word embeddings under empirical word frequencies $\mathbf{Q}$4, rather than under the implicit uniform measure over types. The weighted mean is

$\mathbf{Q}$5

and the weighted covariance is obtained from the row-scaled matrix with rows $\mathbf{Q}$6. The paper argues that both classical and Zipfian approaches are instances of exponential families with different base measures, and reports that simply performing PCA whitening weighted by the empirical word frequency that follows Zipf’s law significantly improves task performance, surpassing established baselines (Yokoi et al., 2024).

In LLMs, activation-space whitening has been used as a training-free detector for policy violations. For each policy category and each layer, the method estimates a mean and covariance from in-policy final-token activations, performs PCA whitening, and uses the Euclidean norm in whitened space as a compliance score; the squared norm is equal to the Mahalanobis distance in the original activation space. The default choice is $\mathbf{Q}$7 principal components. On the reported benchmark, 100 samples per category already yield F1 = 74.3%, and 750 samples per category yield F1 = 77.7%. The whitening-based detector reaches F1 = 74.3% with Llama-3.1-8B and F1 = 82.2% with Qwen-2.5-7B, compared with 73.1% for DynaGuard-8B (CoT) and 70.1% for GPT-4o-mini as judge; white-box latency is reported as 0.03–0.05 s per sample (Rachmil et al., 3 Dec 2025).

These examples broaden the operative meaning of an extended whitening filter. Whitening need not act on raw sensor vectors alone. It can act on summaries, embeddings, or hidden activations, and the “extension” may be a change of base measure, a shrinkage model, or a detection statistic defined in the whitened space. This suggests that extended whitening is best understood as a design pattern: decorrelate first, then exploit the transformed geometry for an additional objective.

6. Communications, astronomy, imaging, and conceptual scope

The canonical application in the original linear EWF paper is quasi-static MIMO maximum-likelihood detection. Starting from a standard whitening filter $\mathbf{Q}$8 for the interference-plus-noise covariance, one computes $\mathbf{Q}$9 and sets $\mathbf{W}=\mathbf{Q}\mathbf{F}$0. The received model becomes

$\mathbf{W}=\mathbf{Q}\mathbf{F}$1

with $\mathbf{W}=\mathbf{Q}\mathbf{F}$2 upper triangular and $\mathbf{W}=\mathbf{Q}\mathbf{F}$3 white, so the ML metric reduces to $\mathbf{W}=\mathbf{Q}\mathbf{F}$4. The reported simplification is that there is one less matrix-vector multiplication per detection instance, because the extra $\mathbf{W}=\mathbf{Q}\mathbf{F}$5 multiplication normally used after QR decomposition is absorbed into the whitening filter itself (Krishnamoorthy, 2013).

In gravitational-wave searches, whitening has been extended toward zero-latency causal operation. A 2017 study replaces the original high-latency, phase-preserving whitening with a zero-latency minimum-phase FIR filter and shows that it reproduces signal-to-noise ratio sufficiently consistent with the original filter for both noise and artificial GW signals, with excellent agreement in $\mathbf{W}=\mathbf{Q}\mathbf{F}$6; the reported gain is a 16 s latency reduction in the whitening stage (Tsukada et al., 2017). A later gauge-theoretic formulation treats whitening as parallel transport on a manifold of spectra. There the correction kernel

$\mathbf{W}=\mathbf{Q}\mathbf{F}$7

updates a minimum-phase whitening filter as the PSD drifts. The paper reports that parallel transport along this connection strictly preserves the minimum-phase property while exactly conserving the matched-filter SNR, numerically certifies flatness through holonomy at the level of machine epsilon, and validates the method on 15,347 binary black hole injections across the LIGO-Virgo network. Implemented in the production sgnl pipeline, the framework reduces whitening latency by 1.0 s (33%) at a 4-second noise estimation cadence, and stride-reduction experiments show that up to 91% of baseline trigger latency can be eliminated with sub-second pipeline cadence (Kennington et al., 28 Apr 2026).

In image enhancement, wavelet-optimized whitening treats local multiscale variance equalization itself as the filter objective. The method equalizes, or whitens, the power in the $\mathbf{W}=\mathbf{Q}\mathbf{F}$8 wavelet spectrum of the input image at all scales and locations, and a bilateral weighting by the local variance in the wavelet planes suppresses the halos produced by discontinuities. The standard version is about two times faster than multiscale Gaussian normalization, while the bilateral version is slower but provides significantly better results in the presence of spikes or edges (Auchère et al., 2022).

Across these domains, the strict original statement remains valid: all EWFs are SWFs, but not all SWFs are EWFs (Krishnamoorthy, 2013). Later literature, however, often uses the same underlying idea in a broader sense that includes approximate, constrained, adaptive, or geometry-aware whiteners. This broader usage suggests a family resemblance rather than a single canonical formalism. What unifies the family is not a fixed implementation, but the deliberate use of whitening degrees of freedom to satisfy a second design objective—structural simplification, locality, robustness, conditionality, latency reduction, or improved statistical efficiency.