Trustworthiness of LLM answers under fake knowledge cutoffs

Determine principled criteria for evaluating the trustworthiness of GPT-4o’s answers to economic forecasting questions when the model is instructed via system and/or user prompts to ignore information beyond an artificial knowledge cutoff but has memorized the true outcomes for the target periods; specifically, establish whether such answers should be treated as genuine forecasts or as contaminated retrieval of memorized data.

Background

The paper demonstrates that GPT-4o can produce implausibly accurate historical forecasts even when explicitly instructed to respect a fake knowledge cutoff, suggesting it anchors outputs to memorized outcomes rather than deriving predictions from provided context. This raises a fundamental interpretive challenge: when a model “pretends not to know” post-cutoff information but has memorized true outcomes, it is unclear how to assess the credibility of its answers.

The authors’ empirical results show minimal differences in accuracy before and after imposed cutoffs under certain prompting conditions, implying that model responses may reflect motivated reasoning tied to memorized data. A rigorous framework is needed to decide how such answers should be interpreted in forecasting evaluations.