Translating scaling-law forecasts into actionable development guidance

Develop principled, prescriptive methodologies that translate pretraining scaling-law forecasts—relating model size, dataset size, and pretraining compute to training loss—into actionable guidance for future language-model development, such as reliably mapping a fixed pretraining compute budget to attainable downstream benchmark performance after contemporary post-training procedures.

Background

Classical scaling laws establish relationships between model size, data, and compute and pretraining loss under controlled settings, but extending these forecasts to inform practical decisions for building future models remains challenging. The downstream accuracy of LLMs is influenced by heterogeneous post-training pipelines, data curation, and temporal effects, which complicates direct use of pretraining loss-based forecasts for deployment.

This paper proposes prescriptive scaling via high-quantile capability boundaries to bridge this gap, aiming to provide a compute-to-performance map that is temporally reliable and robust to recipe variation. The explicit open challenge motivates the need for such methodologies that convert scaling forecasts into actionable planning tools for model development.