Optimizer dependence of the three-term scaling law

Investigate how changing the optimization algorithm (for example, replacing AdamW with Muon) affects the fitted parameters and predictions of the three-term scaling law L(N,M,K) = E + A/N^alpha + B/M^beta + C/K^gamma, including the resulting optimal batch-size scaling.

Background

All analyses in the paper use training runs based on AdamW, and the proposed three-term law is fitted under that optimizer. Prior work suggests that alternative optimizers such as Muon can support substantially larger batch sizes, which could alter the scaling behavior the law aims to capture.

The authors explicitly note that assessing optimizer effects on the three-term law is left for future work, highlighting a concrete open direction to test the robustness of the scaling law under different optimization algorithms.