Returns to data scale beyond the public internet “data wall”

Determine whether diminishing performance returns to increasing dataset size persist beyond approximately 15 trillion tokens of public internet text, characterize how specialized or high‑quality datasets alter return behavior, and ascertain whether technical scaling laws translate into economic returns when quality effects, network dynamics, and contamination risks are incorporated.

Background

The paper notes empirical scaling laws showing diminishing marginal performance returns with increasing dataset size, based on regimes constrained by a “data wall” around 15 trillion tokens of public text. It emphasizes that evidence beyond this boundary is sparse and the relationship to economic returns is not established.

Resolving this uncertainty would clarify whether expanding or improving data supplies yields sustained productivity gains and how high-quality or specialized data modifies scaling behavior.