Establish the causal role of magnitude removal in additive-noise robustness

Establish whether removing activation magnitude information through canonicalization causes the reduced additive-Gaussian-noise robustness observed for the Gauge-Canonical Transformer on CIFAR-100.

Background

In the CIFAR-100 vision experiment, GCT achieves strong clean accuracy and robustness to contrast, brightness, and input-scale changes, but its averaged Gaussian-noise accuracy is lower than that of the best baseline. The paper proposes that canonicalization may remove magnitude information that helps distinguish signal from additive noise.

The reported comparison is correlational: it does not isolate magnitude removal from the other architectural differences between GCT and the baselines. Controlled interventions or alternative representations would be needed to determine whether magnitude removal is the causal explanation.

References

One possible explanation is that canonicalization removes magnitude information that can help distinguish signal from additive noise. The CIFAR-100 results are consistent with this explanation, although the causal role of magnitude removal remains to be tested.

— Neural Dimensional Analysis: Foundations of Dimensionless and Scale-Consistent Computation  (2610.17969 - Kim et al., 13 Oct 2026) in Appendix, Section 'Full results', subsection 'Vision: robustness to photometric and input-unit shifts', paragraph 'Additive-noise robustness' (App.\ref{sec:exp-vit})