Independent Quaternion Batch Normalization

• IQBN is a component-wise normalization scheme that treats each quaternion element (r, i, j, k) independently, avoiding full covariance computation.
• It applies standard batch normalization per component, using individual means and variances to boost training stability, especially in low-batch settings.
• Empirical evaluations indicate that IQBN improves training efficiency and convergence in quaternion-valued networks compared to full Quaternion BN.

Searching arXiv for the cited quaternion and related normalization papers to ground the article. Independent Quaternion Batch Normalization (IQBN) is a normalization scheme for quaternion-valued neural networks that treats the four components of a quaternion activation independently, normalizing each one with its own mean and variance while retaining quaternion grouping for parameterization and implementation efficiency. In this sense, IQBN is the diagonal counterpart to full Quaternion Batch Normalization (QBN), which centers a quaternion feature and whitens its full $4$-dimensional component vector through a covariance transform. The formulation appears as a practical per-component variant in "Deep Quaternion Networks" (Gaudet et al., 2017) and is later used explicitly in QUAN as a normalization mechanism introduced for training stability (Grant et al., 5 Sep 2025).

1. Conceptual definition and historical position

A quaternion is a four-dimensional hyper-complex number of the form $q = r + x\,i + y\,j + z\,k$, with real part $r$ and imaginary parts along the basis $\{i,j,k\}$. In quaternion networks, a feature at a given channel and spatial location is therefore naturally represented as a $4$-vector, typically written as $\mathbf{q} = [r,x,y,z]^\top \in \mathbb{R}^4$ or, equivalently, $q = a + b\mathbf{i} + c\mathbf{j} + d\mathbf{k}$ depending on notation (Gaudet et al., 2017).

Within this setting, IQBN denotes independent normalization across the four quaternion components. “Independent” refers to per-component normalization: the real part and the three imaginary parts are each normalized using their own statistics, and no cross-component covariance is estimated or used. The normalization is therefore diagonal in the quaternion component basis. This distinguishes IQBN from QBN, whose defining operation is whitening of the full $4 \times 4$ covariance of the quaternion component vector (Grant et al., 5 Sep 2025).

The conceptual role of IQBN is analogous to ordinary real-valued batch normalization applied componentwise. "Deep Quaternion Networks" states this relation explicitly: real BN normalizes each scalar channel independently, and IQBN is the quaternion analog, treating $r,x,y,z$ independently and thereby becoming equivalent to running real BN on each component. A plausible implication is that IQBN is best understood not as abandoning quaternion structure, but as restricting normalization to a diagonal approximation while leaving quaternion mixing to other layers, especially quaternion convolutions (Gaudet et al., 2017).

2. Mathematical formulation

In the QUAN formulation, a quaternion-valued feature map is represented as a tensor $X$ with shape $q = r + x\,i + y\,j + z\,k$0, where $q = r + x\,i + y\,j + z\,k$1 is batch size, $q = r + x\,i + y\,j + z\,k$2 is the number of quaternion channels, $q = r + x\,i + y\,j + z\,k$3 are spatial dimensions, and $q = r + x\,i + y\,j + z\,k$4 indexes the components $q = r + x\,i + y\,j + z\,k$5. The per-component tensors are denoted $q = r + x\,i + y\,j + z\,k$6 (Grant et al., 5 Sep 2025).

For each component $q = r + x\,i + y\,j + z\,k$7 and quaternion channel $q = r + x\,i + y\,j + z\,k$8, IQBN computes the batch statistics

$q = r + x\,i + y\,j + z\,k$9

where the expectation is taken over batch and spatial dimensions, typically $r$0. The normalized components are then

$r$1

The affine stage is likewise diagonal:

$r$2

with $r$3 broadcast across $r$4, or equivalently parameterized as shape $r$5 per component. In tensor form,

$r$6

where $r$7 and $r$8 denotes element-wise multiplication with broadcasting (Grant et al., 5 Sep 2025).

The earlier quaternion-network formulation gives the same operation in diagonal-covariance form. There the per-component variances are

$r$9

with diagonal covariance approximation

$\{i,j,k\}$0

and normalization

$\{i,j,k\}$1

Its affine transform remains diagonal, with per-component scales and a $\{i,j,k\}$2-vector bias (Gaudet et al., 2017).

3. Relation to full Quaternion Batch Normalization

QBN generalizes batch normalization to quaternion-valued activations by centering, whitening the $\{i,j,k\}$3D component vector, and then applying a learnable affine reparameterization. For a mini-batch of quaternion component vectors, it computes a batch mean $\{i,j,k\}$4 and full covariance $\{i,j,k\}$5, then applies the whitening transform

$\{i,j,k\}$6

followed by

$\{i,j,k\}$7

In "Deep Quaternion Networks", $\{i,j,k\}$8 is symmetric, with $\{i,j,k\}$9 independent parameters due to symmetry; the diagonal is initialized to $4$0 and off-diagonals to $4$1 (Gaudet et al., 2017).

By contrast, IQBN replaces the full covariance with a diagonal approximation and removes the whitening of inter-component correlations. This yields a simpler parameterization and a different inductive bias: IQBN controls componentwise scale and shift but does not explicitly decorrelate the quaternion components.

Scheme	Statistics and normalization	Affine stage
QBN	Full $4$2 covariance, whitening via $4$3	Symmetric $4$4 and $4$5
IQBN	Per-component means and variances only	Diagonal scales $4$6 and $4$7

The computational and numerical trade-off is central. For QBN, one forms a $4$8 covariance per quaternion channel and factors an SPD matrix; the factorization cost per channel is $4$9 with a small constant, and numerical sensitivity can arise when the batch is small or components are nearly collinear. IQBN computes four means and variances per channel, normalizes in $\mathbf{q} = [r,x,y,z]^\top \in \mathbb{R}^4$0 per sample, avoids matrix inversion or factorization, and is described as much cheaper and typically more numerically robust. The price is that IQBN sacrifices explicit modeling of inter-component correlations (Gaudet et al., 2017).

This contrast is also central in QUAN. There, QBN whitening is described as expensive, nontrivial to implement efficiently, and unstable for small batches, whereas IQBN is presented as diagonal-only normalization that avoids the matrix square-root entirely. This suggests that the practical distinction between QBN and IQBN is not merely algebraic; it also concerns trainability, kernel efficiency, and robustness under typical minibatch regimes (Grant et al., 5 Sep 2025).

4. Training-time, inference-time, and implementation mechanics

At training time, IQBN computes per-component batch means and variances, normalizes each component independently, applies a per-component affine transform, and updates running statistics. In the quaternion-network formulation, the training-time steps for one quaternion channel are: compute per-component means $\mathbf{q} = [r,x,y,z]^\top \in \mathbb{R}^4$1, center $\mathbf{q} = [r,x,y,z]^\top \in \mathbb{R}^4$2, compute per-component variances, normalize each component by its own standard deviation plus $\mathbf{q} = [r,x,y,z]^\top \in \mathbb{R}^4$3, apply the diagonal affine transform, and update running means and running variances with momentum $\mathbf{q} = [r,x,y,z]^\top \in \mathbb{R}^4$4 (Gaudet et al., 2017).

Inference uses stored running statistics rather than batch statistics. For IQBN, the inference-time transform is the componentwise normalization

$\mathbf{q} = [r,x,y,z]^\top \in \mathbb{R}^4$5

followed by the learned affine stage (Gaudet et al., 2017).

Two momentum conventions appear in the literature summarized here. One source uses the update

$\mathbf{q} = [r,x,y,z]^\top \in \mathbb{R}^4$6

with typical BN momenta $\mathbf{q} = [r,x,y,z]^\top \in \mathbb{R}^4$7 in $\mathbf{q} = [r,x,y,z]^\top \in \mathbb{R}^4$8. QUAN presents exponential moving averages as

$\mathbf{q} = [r,x,y,z]^\top \in \mathbb{R}^4$9

with momentum $q = a + b\mathbf{i} + c\mathbf{j} + d\mathbf{k}$0, for example $q = a + b\mathbf{i} + c\mathbf{j} + d\mathbf{k}$1 (Gaudet et al., 2017, Grant et al., 5 Sep 2025).

Implementation in QUAN is explicitly real-valued: each quaternion is represented as four real-valued channels grouped together as $q = a + b\mathbf{i} + c\mathbf{j} + d\mathbf{k}$2, and IQBN is implemented as a component-wise BN over these grouped channels, with per-component affine parameters and running statistics. This design allows the use of standard tensor operations and custom CUDA kernels for quaternion convolution while keeping normalization simple and efficient. IQBN is placed after quaternion convolution and before the nonlinearity, mirroring standard CNN practice; SiLU was found best in the reported ablations. The method is compatible with AMP, and a common practice is to compute batch statistics in float32 for numerical stability and cast outputs back to the current precision (Grant et al., 5 Sep 2025).

Backward propagation through IQBN follows the standard BN derivative formulas independently per quaternion component. In the broader quaternion-network treatment, all normalization steps are described as differentiable, and autograd frameworks can propagate through the relevant reductions and transforms as long as the numerical conditions required by the chosen normalization are satisfied (Gaudet et al., 2017).

5. Empirical usage and reported results

The empirical record summarized in these sources separates clearly into two strands. In "Deep Quaternion Networks", the reported experiments use quaternion networks with QBN rather than IQBN. On CIFAR-10/100, the shallow ResNet-like models report test errors of Real $q = a + b\mathbf{i} + c\mathbf{j} + d\mathbf{k}$3 with $q = a + b\mathbf{i} + c\mathbf{j} + d\mathbf{k}$4 parameters, Complex $q = a + b\mathbf{i} + c\mathbf{j} + d\mathbf{k}$5 with $q = a + b\mathbf{i} + c\mathbf{j} + d\mathbf{k}$6 parameters, and Quaternion $q = a + b\mathbf{i} + c\mathbf{j} + d\mathbf{k}$7 with $q = a + b\mathbf{i} + c\mathbf{j} + d\mathbf{k}$8 parameters; the deep $q = a + b\mathbf{i} + c\mathbf{j} + d\mathbf{k}$9-layer models report Real $4 \times 4$0 with $4 \times 4$1 parameters, Complex $4 \times 4$2 with $4 \times 4$3 parameters, and Quaternion $4 \times 4$4 with $4 \times 4$5 parameters. On KITTI Road Segmentation, IoU is reported as Real $4 \times 4$6 with $4 \times 4$7 parameters, Complex $4 \times 4$8 with $4 \times 4$9 parameters, and Quaternion $r,x,y,z$0 with $r,x,y,z$1 parameters. These gains are obtained with QBN, and the paper explicitly states that IQBN-specific metrics are not reported separately (Gaudet et al., 2017).

In QUAN, IQBN is part of the final architecture and is introduced explicitly for training stability. The paper reports that “traditional quaternion batch normalization showed on average $r,x,y,z$2 less accuracy than IQBN across the CIFAR-10 experiments,” and therefore QBN was not included in the final models. For classification, QUAN-WRN with $r,x,y,z$3K parameters reaches $r,x,y,z$4 on CIFAR-10 and $r,x,y,z$5 on CIFAR-100. The broader evaluation spans CIFAR-10/100, ImageNet, COCO, DOTA, and robotic perception tasks, and the paper states that QUAN achieves higher accuracy with fewer parameters and faster convergence compared to existing convolution and quaternion-based models, while establishing the SOTA for quaternion CNNs in oriented object detection downstream tasks (Grant et al., 5 Sep 2025).

These results support a narrow but important empirical conclusion. The 2017 paper demonstrates the viability and competitiveness of quaternion normalization in the full-whitening form, whereas the 2025 QUAN results indicate that, within that architecture, the diagonal approximation embodied by IQBN can be more effective than traditional quaternion whitening. A plausible implication is that the practical value of IQBN is architecture-dependent but strongest where computational efficiency and stable optimization dominate the benefits of full covariance modeling.

6. Practical interpretation, limitations, and common misconceptions

The main practical decision is whether normalization should model cross-component correlations explicitly. When meaningful correlations among quaternion components are expected or desired, full QBN can decorrelate the components and then allow learned re-correlation through $r,x,y,z$6. When batch sizes are small, covariance is ill-conditioned, or speed and stability are priorities, IQBN is the preferred simplification because it is much cheaper and often more numerically robust (Gaudet et al., 2017).

A common misconception is that IQBN itself is the source of rotation equivariance in quaternion architectures. QUAN states the opposite: IQBN does not enforce rotational equivariance; rotation-aware behavior arises primarily from the quaternion convolution and architectural design. IQBN’s role is to keep each component’s scale controlled without disrupting the Hamilton-based mixing that expresses the relevant cross-component structure (Grant et al., 5 Sep 2025).

Another misconception is that IQBN is simply “ordinary BN on four unrelated channels.” That description is incomplete. Although its normalization is componentwise, the grouping of channels in fours is preserved throughout the network, and the affine parameters and running statistics are stored per component per quaternion channel group. In QUAN this grouped design is important for integration with QSPPF, QC3k2, and QC2PSA modules, as well as with the real-valued implementation of Hamilton-based operations (Grant et al., 5 Sep 2025).

The limitations are equally clear. IQBN does not model cross-component covariance explicitly. If an application strictly requires quaternion whitening or rotation-invariant normalization at the component level, full QBN may be more principled, but this comes at the cost of computation and possible instability. The QUAN paper also notes that full ablations against GroupNorm, LayerNorm, or standard component-wise BN were not included, and that some impairment on large datasets such as COCO and DOTA relative to non-quaternion baselines is likely due to smaller model sizes rather than IQBN specifically (Grant et al., 5 Sep 2025).

Taken together, the literature presents IQBN as a diagonal, per-component normalization rule for quaternion networks that preserves quaternion grouping while discarding covariance whitening. Its significance lies less in quaternion algebra per se than in the balance it strikes between structured representation and practical optimization: it is faster and more stable than full quaternion whitening, yet remains compatible with quaternion convolutions, grouped channel organization, and quaternion-specific architectural blocks (Gaudet et al., 2017, Grant et al., 5 Sep 2025).