Probe set design and hierarchical probe-set construction for 2RDM-based detection

Develop principled methods for designing probe sets for the 2-datapoint reduced density matrix used to detect phase transitions during neural network training; in particular, construct hierarchical families of probe sets, analogous to basis set hierarchies in quantum chemistry, that systematically improve detection of reorganizations across tasks and ensure that relevant transitions are resolved by the chosen samples.

Background

The paper introduces the 2-datapoint reduced density matrix (2RDM), defined as the covariance of per-sample losses over a selected probe set, and shows its spectral statistics (Spectral Heat Capacity and Participation Ratio) can detect and interpret phase transitions during training across diverse settings.

A core limitation acknowledged by the authors is that the 2RDM can only detect transitions that are resolved by the chosen probe samples; thus, probe set selection critically determines what dynamics are observable. The authors draw an analogy to basis set selection in quantum chemistry, where hierarchical basis families enable systematic improvement and coverage.

They explicitly identify probe set design as the most pressing open problem and highlight that constructing principled, hierarchical probe-set families for deep learning remains open, despite its potential to substantially improve practical detection.