Dice Question Streamline Icon: https://streamlinehq.com

Ability of GPT-4 to Produce Accurate Numerical Forecasts

Determine whether OpenAI's GPT-4, a transformer-based large language model, can produce accurate numerical forecasts, given that large language models are not explicitly trained for numerical prediction tasks.

Information Square Streamline Icon: https://streamlinehq.com

Background

The paper contrasts traditional machine learning models, which are commonly trained specifically to generate numerical forecasts, with LLMs such as GPT-4 that are primarily optimized for general-purpose language processing. This raises a fundamental uncertainty about GPT-4's capacity to deliver accurate numerical predictions.

Within the finance and accounting context, the authors highlight recent evidence on GPT's strengths in certain analytical tasks (e.g., directional trend detection) but note that its numerical forecasting ability has not been established. This motivates their empirical evaluation of GPT-4’s performance on earnings forecasting using press releases and financial statements.

References

While prior research has predominantly focused on traditional machine learning models trained to generate numerical forecasts, LLMs like GPT-4 are not explicitly trained for this task. The ability of GPT-4 to produce accurate forecasts remains an open question.

— The Promise and Peril of Generative AI: Evidence from GPT-4 as Sell-Side Analysts (2412.01069 - Li et al., 2 Dec 2024) in Section 2.2.2, Comparing Traditional Machine Learning, GPT, and Human Analysts