Evolution of relaxation spectra during training

Determine how the intrinsic state-space, input-conditioned effective, and full-block collective relaxation spectra of Mamba develop throughout optimization, thereby extending the analysis beyond trained-model snapshots.

Background

The paper analyzes pretrained Mamba-370M models and characterizes three related but distinct dynamical objects: the intrinsic relaxation spectrum of the learned state-space generator, the input-conditioned spectrum produced by selective temporal rescaling, and the collective time-scale density of states obtained from the complete block Jacobian.

Because the measurements are performed on trained models rather than across training checkpoints, they do not reveal how these microscopic and collective spectral structures emerge or reorganize during optimization. The authors identify checkpoint-based measurements as a direct extension that could track this evolution simultaneously across all three dynamical levels.