Scale dependence of algorithmic progress parameters

Investigate whether algorithmic progress in the relation C_m = k log F_m + b is scale-dependent by determining if the slope parameter k varies across training compute regimes, and quantify how compute-efficiency reductions differ at different FLOP scales (e.g., 10^21 versus 10^25 FLOP).

Background

To estimate algorithmic progress, the paper models capability C_m as a log-linear function of training compute F_m with parameters k (compute scaling slope) and b (algorithmic quality offset). The primary analysis assumes progress occurs via increases in b over time with a fixed k.

The authors caution that algorithmic progress could also change k, implying scale dependence (e.g., larger gains at higher compute scales). They explicitly note insufficient data to test this, leaving a gap in understanding whether efficiency improvements vary by compute scale—an issue with practical implications for forecasting and policy.