Assessing a Model’s Knowledge Remains Open

Develop reliable methodologies to assess what factual knowledge a large language model possesses and to evaluate the quality of these methods despite the absence of ground truth about the model’s internal knowledge.

Background

The authors propose a four-category taxonomy derived from sampling-based estimates of how frequently a model outputs correct answers under different prompting and decoding conditions. This approach aims to label facts as ClearlyKnown, MaybeKnown, WeaklyKnown, or Unknown with respect to the pre-trained model.

Despite this contribution, they emphasize that rigorously assessing what a model truly knows is challenging because there is no direct ground truth about internal knowledge, rendering the general problem open and motivating further methodological advances.

References

Assessing a model's knowledge remains an open problem, particularly since evaluating the quality of such methods is challenging due to the lack of ground truth about what the model truly knows.

— Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?  (2405.05904 - Gekhman et al., 2024) in Section 6 (Knowledge Categories Analysis)

This evaluation therefore serves as an external retention check rather than a test of new-knowledge acquisition; constructing date-aligned probes that postdate the model cutoff is left to future work.

— When to Review: Spaced Repetition for Continual Pre-Training of Language Models  (2608.17530 - Atreya et al., 18 Aug 2026) in Appendix, Section External Validation on TemporalWiki