Capacity for large-scale compute expansion over the next five years

Ascertain whether sufficient capacity exists to expand both training and inference compute by many orders of magnitude over the next five years for frontier language model development.

Background

The authors relate observed capability trends to historical growth in training compute, noting past exponential increases and recent rises in inference-time compute for models like o1 and o3. They highlight potential limits to future compute scaling and the role of algorithmic efficiency as a substitute.

Whether the AI ecosystem can continue scaling compute by many orders of magnitude in the near term critically affects projections of time horizon growth; this capacity question remains unresolved.