Uncertainty about GPT-4 training data contents and numerical rounding behavior

Determine the composition of GPT-4’s training data and quantify the extent to which large language models round or otherwise transform continuous numerical values during generation.

Background

When shifting from categorical outcomes (e.g., award winners) to continuous macroeconomic variables (e.g., monthly inflation), the authors note that model behavior with numeric outputs could be sensitive to details such as rounding and the nature of the training data—details that are opaque due to proprietary constraints.

This opacity limits interpretation of the model’s predictions and motivates a technical open question regarding the training corpus and numerical generation behavior.