Intermediate layer ordering using NCC mismatch

Determine whether, during the Terminal Phase of Training of a supervised deep neural network classifier, the Nearest Class-Center (NCC) mismatch computed at intermediate layer j—defined as the fraction of samples whose predicted class differs from the class whose train class-mean is nearest to the layer-j activation—is nonincreasing with depth, i.e., for all consecutive layers j and j+1, the training-set and test-set NCC mismatches at layer j are greater than or equal to those at layer j+1.

Background

The paper investigates geometric phenomena associated with Neural Collapse during the Terminal Phase of Training (TPT), focusing on the Nearest Class-Center (NCC) behavior across intermediate layers in deep neural networks. NCC mismatch at a given layer is defined as the fraction of samples for which the network’s predicted class does not match the class whose train class-mean is nearest to that layer’s activation.

This conjecture posits a monotonic ordering across layers: deeper layers exhibit lower NCC mismatch than shallower layers on both the training and test sets. If true, it would formalize a depth-related simplification trend in representation geometry, linking intermediate-layer clustering quality to classifier agreement throughout the network.