Generalizability of ChatGPT-derived economic properties across LLMs

Determine whether the economic properties identified in studies using ChatGPT for economics and finance are idiosyncratic to ChatGPT or generalizable across Large Language Models of varying architectures and training regimes, to assess if such properties are fundamental to LLMs as a class rather than specific to a single model.

Background

The paper situates its contribution within a fast-growing literature that has largely focused on ChatGPT for economic and financial applications. The authors note that while this research is valuable, it has raised a broader concern about whether findings derived from one model reflect general properties of LLMs or are specific to ChatGPT.

To address this concern empirically in the context of risk preferences, the authors analyze 30 different LLMs and report coherent patterns across architectures. While they present evidence toward generality in risk-related behavior, they explicitly highlight that the overarching question of model-specific versus general economic properties remains open in the literature.