Causal relationship between KAG and neural collapse

Determine whether Kolmogorov-Arnold geometry in early layers of neural networks enables neural collapse in later layers or, conversely, whether the formation of collapsed final-layer representations induces Kolmogorov-Arnold geometry in earlier layers, by analyzing the joint evolution of Jacobian-based geometric structure across all layers during training.

Background

Neural collapse is a phenomenon observed late in training where class means align into a symmetric simplex and within-class variability diminishes. This paper studies Kolmogorov-Arnold geometry (KAG) through Jacobians of hidden layers in shallow MLPs trained on MNIST and finds that KAG emerges robustly and is scale-agnostic. The authors raise the possibility that KAG and neural collapse could be connected and explicitly state that clarifying this relationship remains open.

The open question concerns the directionality and mechanism linking the emergence of KAG in early layers with neural collapse in later layers. Resolving this requires careful, layer-wise, temporal analysis of the geometry during training to establish causal or correlational pathways.