The Mercurial Top-Level Ontology of Large Language Models (2405.01581v1)
Abstract: In our work, we systematize and analyze implicit ontological commitments in the responses generated by LLMs, focusing on ChatGPT 3.5 as a case study. We investigate how LLMs, despite having no explicit ontology, exhibit implicit ontological categorizations that are reflected in the texts they generate. The paper proposes an approach to understanding the ontological commitments of LLMs by defining ontology as a theory that provides a systematic account of the ontological commitments of some text. We investigate the ontological assumptions of ChatGPT and present a systematized account, i.e., GPT's top-level ontology. This includes a taxonomy, which is available as an OWL file, as well as a discussion about ontological assumptions (e.g., about its mereology or presentism). We show that in some aspects GPT's top-level ontology is quite similar to existing top-level ontologies. However, there are significant challenges arising from the flexible nature of LLM-generated texts, including ontological overload, ambiguity, and inconsistency.
- Large language models as oracles for instantiating ontologies with domain-specific knowledge, arXiv preprint arXiv:2404.04108 (2024).
- Sweetening ontologies with dolce, in: International conference on knowledge engineering and knowledge management, Springer, 2002, pp. 166–181.
- The generalized upper model 2.0, 1995.
- Towards ontological foundations for conceptual modeling: The unified foundational ontology (ufo) story, Applied ontology 10 (2015) 259–271.
- H. Herre, General formal ontology (gfo): A foundational ontology for conceptual modelling, in: R. Poli, M. Healy, A. Kameas (Eds.), Theory and applications of ontology: computer applications, Springer, 2010, pp. 297–345.
- I. Niles, A. Pease, Towards a standard upper ontology, in: Proceedings of the international conference on Formal Ontology in Information Systems-Volume 2001, 2001, pp. 2–9.
- OpenAI, 2024, URL: https://platform.openai.com/docs/introduction.
- J. Spataro, 2023, URL: https://blogs.microsoft.com/blog/2023/03/16/introducing-microsoft-365-copilot-your-copilot-for-work/.
- S. Pichai, 2023, URL: https://blog.google/technology/ai/bard-google-ai-search-updates/.
- Anthropic, 2023, URL: http://www.anthropic.com/index/introducing-claude.
- Using terms and informal definitions to classify domain entities into top-level ontology concepts: An approach based on language models, Knowledge-Based Systems 265 (2023) 110385.
- P. Mateiu, A. Groza, Ontology engineering with large language models, arXiv preprint arXiv:2307.16699 (2023).
- From human experts to machines: An llm supported approach to ontology and knowledge graph construction, arXiv preprint arXiv:2403.08345 (2024).
- Unifying large language models and knowledge graphs: A roadmap, arXiv preprint arXiv:2306.08302 (2023).
- Llms4ol: Large language models for ontology learning, in: T. R. Payne, V. Presutti, G. Qi, M. Poveda-Villalón, G. Stoilos, L. Hollink, Z. Kaoudi, G. Cheng, J. Li (Eds.), The Semantic Web – ISWC 2023, Springer Nature Switzerland, Cham, 2023, pp. 408–427.
- Large language models in biomedical natural language processing: benchmarks, baselines, and recommendations, arXiv preprint arXiv:2305.16326 (2023).
- S. Hertling, H. Paulheim, Olala: Ontology matching with large language models, in: Proceedings of the 12th Knowledge Capture Conference 2023, K-CAP ’23, Association for Computing Machinery, New York, NY, USA, 2023, p. 131–139. URL: https://doi.org/10.1145/3587259.3627571. doi:10.1145/3587259.3627571.
- N. Guarino, Formal ontology and information systems, in: N. Guarino (Ed.), Formal ontology in information systems: Proceedings of the first international conference (FOIS’98), June 6-8, Trento, Italy, volume 46, IOS press, 1998, pp. 3–15.
- B. Smith, Beyond concepts: Ontology as reality representation, in: Proceedings of the third international conference on formal ontology in information systems (FOIS 2004), IOS Press, 2004, pp. 73–84.
- On the dangers of stochastic parrots: Can language models be too big?, in: Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, 2021, pp. 610–623.
- F. Neuhaus, Ontologies in the era of large language models–a perspective, Applied Ontology 18 (2023) 399–407.
- P. Bricker, Ontological Commitment, in: E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy, Winter 2016 ed., Metaphysics Research Lab, Stanford University, 2016.