Predicting the scale at which emergent capabilities appear in large language models

Determine how to reliably predict, as a function of model scale (e.g., parameter count, training data size, and training compute), the thresholds at which new qualitative “breakthrough” capabilities emerge in large language models, in order to anticipate capability onset and associated risks.

Background

The paper documents that LLMs sometimes exhibit sudden qualitative improvements—so‑called "breakthrough" capabilities—as model scale increases. While these have been observed empirically, the authors note that current understanding does not allow forecasting when such changes will occur. Reliable prediction would be valuable for research planning, safety, and governance, but requires connecting scaling variables (parameters, data, compute) to capability emergence.