Dice Question Streamline Icon: https://streamlinehq.com

Phase Ic proof for 0 < α < 1/4

Prove the Phase Ic loss-curve characterization for the PLRF model—that P(r) \asymp Fpp(r) + F0(r) and the associated compute-optimal tradeoff—in the regime 2β > 1 and 0 < α < 1/4 by developing sharp bounds on the kernel function in this region.

Information Square Streamline Icon: https://streamlinehq.com

Background

Phase Ic mirrors the simple dynamics of Phases Ia and Ib with dominance of the pure-point forcing and capacity terms. The authors present the Phase Ic result contingent on an assumption (Theorem 5.1 holding for α < 1/4) and note that kernel-function behavior changes in this regime.

They explicitly state they cannot prove the Phase Ic statement due to the lack of sharp kernel bounds when α < 1/4. Establishing such bounds would complete the analysis and confirm the phase behavior and compute-optimal exponents in this region.

References

We can not prove this statement as we do not have sharp bounds on the kernel function in this region.

4+3 Phases of Compute-Optimal Neural Scaling Laws (2405.15074 - Paquette et al., 23 May 2024) in Proposition “Phase Ic: 2 β > 1, 0 < α < 1/4” within Section “Below the high-dimensional line (Phases IVa, IVb, Ib, Ic)”