Empirical equivalence of layerwise and end-to-end equivariance in trained CNNs

Determine whether end-to-end equivariance is empirically realized through layerwise equivariance in trained convolutional neural networks, despite layerwise equivariance not being the only theoretical way to realize equivariant functions.

Background

The paper discusses the relationship between architectural, layerwise equivariance and equivariance of the overall network function. Although layerwise equivariance is a standard construction, the cited theoretical work argues that it is not generally the only mechanism capable of producing an equivariant function. The unresolved issue is whether trained convolutional neural networks nevertheless exhibit this layerwise realization empirically. The question is relevant because the paper treats the equivariant parameter subspace as a meaningful object for analyzing training dynamics and stochastic weight averaging.

References

From this parameter restriction arises the structural question: is layerwise equivariance the only way to realize equivariance? \citet{NEURIPS2023_2c74f005} first tackled this question in theory, arguing that this is not the general scenario despite the existence of supporting examples; they nevertheless conjectured that it holds empirically for trained CNNs.

— Boosting Data Augmentation with Stochastic Weight Averaging  (2608.14373 - Huang et al., 14 Aug 2026) in Section 2, Related Work, paragraph “Equivariance and data augmentation”