Epistemology of Language Models: Do Language Models Have Holistic Knowledge? (2403.12862v1)

Abstract: This paper investigates the inherent knowledge in LLMs from the perspective of epistemological holism. The purpose of this paper is to explore whether LLMs exhibit characteristics consistent with epistemological holism. These characteristics suggest that core knowledge, such as general scientific knowledge, each plays a specific role, serving as the foundation of our knowledge system and being difficult to revise. To assess these traits related to holism, we created a scientific reasoning dataset and examined the epistemology of LLMs through three tasks: Abduction, Revision, and Argument Generation. In the abduction task, the LLMs explained situations while avoiding revising the core knowledge. However, in other tasks, the LLMs were revealed not to distinguish between core and peripheral knowledge, showing an incomplete alignment with holistic knowledge principles.