Flatness–generalization relation among minimizers at fixed dataset size

Establish whether the phase-conditioned relationship between Hessian-based flatness measures and generalization observed across varying dataset-size-to-input-dimension ratios also holds among distinct empirical-risk minimizers within a single fixed-data teacher–student tree committee machine landscape at fixed ratio \(\alpha\).

Background

The paper studies the relationship between Hessian-based flatness and generalization by varying the dataset-size-to-input-dimension ratio $\alpha=P/N$, thereby moving across replica-symmetric, replica-symmetry-breaking, and recovery phases. This differs from the usual empirical setting, in which a fixed training dataset and loss landscape are held constant while different minimizers are compared.

The authors note that determining whether their phase-dependent conclusions transfer to the fixed-data setting would require analyzing distinct minimizers of one landscape at fixed $\alpha$. This question is left unresolved and is explicitly deferred to future work.