Theoretical derivation of the phase boundary

Derive the critical phase boundary at $\lambda^*$ from a PAC-Bayes or minimum-description-length argument based on the relative complexity of memorizing and generalizing solutions.

Background

The empirical phase diagram identifies a sharp transition in grokking frequency as AdamW weight decay changes, with a proposed critical value near 1.0. The paper currently characterizes this boundary through experiments rather than a formal complexity-based theory.

A PAC-Bayes or minimum-description-length derivation could explain why regularization favors the generalizing solution and could predict the location of the boundary rather than merely observing it.

References

Can the phase boundary at $\lambda*$ be derived from a PAC-Bayes or minimum description length argument about the relative complexity of the memorizing vs. generalizing solutions?

— Quantifying the Memorization-to-Generalization Transition: Scaling Laws and Phase Structure in Grokking  (2609.10657 - Kataria, 9 Sep 2026) in Paragraph "Open questions," Section 8 (Discussion)