Dice Question Streamline Icon: https://streamlinehq.com

Phase Ib extension to α < 1/4

Prove the Phase Ib loss-curve characterization for the PLRF model—that P(r) \asymp Fpp(r) + F0(r)—in the regime 2β < 1, α < 1/4, and 2(α + β) > 1 by deriving the necessary kernel-function estimates to extend the result below α = 1/4.

Information Square Streamline Icon: https://streamlinehq.com

Background

Phase Ib (2β < 1, 1/4 < α < 1/2, 2(α + β) > 1) is characterized by loss dynamics dominated by the pure-point forcing and the capacity term, with SGD noise negligible. The authors prove this behavior for α > 1/4.

They note that extending the proof to α < 1/4 is hindered by missing kernel-function bounds; resolving these would generalize the phase characterization and unify the analysis across the boundary at α = 1/4.

References

Although we did not prove the statement for \alpha < \tfrac{1}{4} as we do not have estimates for the kernel function, we believe that statement still holds.

4+3 Phases of Compute-Optimal Neural Scaling Laws (2405.15074 - Paquette et al., 23 May 2024) in Proposition “Phase Ib: 2 β < 1, 1/4 < α < 1/2, 2(α + β) > 1” within Section “Below the high-dimensional line (Phases IVa, IVb, Ib, Ic)”