Normalization Across Other Symmetry Groups and Parameter-Sharing Structures

Determine whether normalization analogous to the block normalization introduced for SO(3)-equivariant layers is useful for other symmetry groups and other forms of parameter sharing, including convolutional networks.

Background

The paper introduces block normalization to address learning-rate mismatches caused by irrep-block structure in SO(3)-equivariant layers. The method rescales each block update independently so that its spectral step matches the learning rate, while preserving the update direction within the block and leaving Adam's moment estimates unchanged.

The authors evaluate the method only in SO(3)-equivariant networks, using a controlled toy model and an e3nn interatomic-potential model on three molecular datasets. They explicitly leave unresolved whether the same normalization principle transfers to other symmetry groups or to different parameter-sharing mechanisms, such as those used in convolutional networks.