Evaluation beyond convolutional image-classification covariance heads

Investigate the decomposition-free polynomial matrix-logarithm normalizer in second-order transformer heads, partial-correlation representations, higher-order spectral iterations, and dense-prediction settings, where covariance dimensions and spectral distributions may differ from those evaluated for convolutional global covariance pooling.

Background

The paper evaluates the proposed decomposition-free logarithmic normalizers only for image classification using a convolutional global covariance pooling head with covariance dimension d=256. It identifies second-order transformer heads, partial-correlation representations, higher-order spectral iterations, and dense prediction as settings in which the method could serve as a drop-in normalization replacement but has not yet been evaluated.

The authors specifically note that the spectral fitting interval may need to be re-estimated in these settings because covariance dimensions and spectral shapes can differ. The paper therefore leaves empirical validation of the method across these application classes as future work.