Extent of memorization effects in LLM-based embedding forecasts

Determine the extent to which memorization affects forecasts that rely on embeddings generated by large language models such as GPT-4o in financial prediction tasks, by quantifying how much embedding representations encode training-period outcomes or lookahead information that contaminates forecasting evaluations.

Background

The paper surveys approaches that use LLM-derived embeddings combined with supervised steps for financial prediction. While embeddings have been proposed as a powerful representation for return forecasting, the authors point out that the degree to which memorization contaminates such forecasts during the training period is not established.

Given the broader evidence of selective memorization across economic indicators, indices, and textual data, the unknown extent of contamination in embedding-based forecasts poses a risk to interpreting apparent predictive performance as genuine insight rather than recall.