Large-scale cross-entropy experiments for Sven

Conduct larger-scale cross-entropy classification experiments—such as training ResNet architectures on CIFAR-10 and CIFAR-100—to evaluate Sven’s performance and training dynamics at scale.

Background

The cross-entropy analysis in the paper focuses on MNIST with small MLPs and highlights differences in singular value spectra and training dynamics compared to label regression.

To properly assess Sven for classification objectives, the authors point to the need for larger-scale experiments with modern architectures and datasets.

They explicitly defer these experiments due to resource requirements.

References

We leave larger-scale CE experiments (e.g. ResNets on CIFAR-10/100) for future work, as this will require significantly more computational resources.

— Sven: Singular Value Descent as a Computationally Efficient Natural Gradient Method  (2604.01279 - Bright-Thonney et al., 1 Apr 2026) in Appendix: Classification with Cross-Entropy