Reversal at the published refresh density

Establish whether the learning-rate-dependent reversal in the preference between earlier and later refresh placement persists for the budget-59 refresh schedules with every refresh gap between 5 and 15 steps, corresponding to the published Mousse refresh density.

Background

The primary experiments use sparse four-refresh schedules whose middle refreshes are moved while the refresh count and endpoints are fixed. The authors also test a budget-59 pair at the published refresh density, where all gaps are short and range from 5 to 15 steps. Placement still affects endpoint loss, but the highest-learning-rate cells are seed-inconsistent, so the available data do not determine whether the reversal itself remains at this density.

References

The two highest-rate cells were seed-inconsistent, so at this density the data show a placement effect and leave the reversal unresolved.

— A Refresh Interval Does Not Fully Specify a Refresh Schedule: Learning-Rate-Dependent Placement in Kronecker-Preconditioned Language Model Training  (2610.18695 - Wang et al., 13 Oct 2026) in Section 4, subsection “Learning Rate Reverses Placement Preference,” paragraph “Published refresh density”

At eight times the horizon every tested cell favored later placement or tied, which leaves two readings open: the boundary moved above the grid, or the contrast flattened (App.~\ref{app:ladder}).

— A Refresh Interval Does Not Fully Specify a Refresh Schedule: Learning-Rate-Dependent Placement in Kronecker-Preconditioned Language Model Training  (2610.18695 - Wang et al., 13 Oct 2026) in Section 5, subsection “The Boundary Varies with Scale and Horizon,” paragraph “Training horizon”

Whether such rotations occur in our runs, and whether they relate to the reversal, is open (App.~\ref{app:mechanism}).

— A Refresh Interval Does Not Fully Specify a Refresh Schedule: Learning-Rate-Dependent Placement in Kronecker-Preconditioned Language Model Training  (2610.18695 - Wang et al., 13 Oct 2026) in Appendix A, Section “Derivations,” paragraph “Spectral width and eigenvector orientation”