CORE-GPT: Combining Open Access research and large language models for credible, trustworthy question answering (2307.04683v1)
Abstract: In this paper, we present CORE-GPT, a novel question-answering platform that combines GPT-based LLMs and more than 32 million full-text open access scientific articles from CORE. We first demonstrate that GPT3.5 and GPT4 cannot be relied upon to provide references or citations for generated text. We then introduce CORE-GPT which delivers evidence-based answers to questions, along with citations and links to the cited papers, greatly increasing the trustworthiness of the answers and reducing the risk of hallucinations. CORE-GPT's performance was evaluated on a dataset of 100 questions covering the top 20 scientific domains in CORE, resulting in 100 answers and links to 500 relevant articles. The quality of the provided answers and and relevance of the links were assessed by two annotators. Our results demonstrate that CORE-GPT can produce comprehensive and trustworthy answers across the majority of scientific domains, complete with links to genuine, relevant scientific articles.
- LSE. LSE, editor. New AI tools that can write student essays require educators to rethink teaching and assessment. The London School of Economics and Political Science; 2022. Accessed: 2023-05-18. https://blogs.lse.ac.uk/impactofsocialsciences/2022/05/17/new-ai-tools-that-can-write-student-essays-require-educators-to- rethink-teaching-and-assessment/.
- SciBERT: A pretrained language model for scientific text. arXiv preprint arXiv:190310676. 2019.
- Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:190711692. 2019.
- Language models are unsupervised multitask learners. OpenAI blog. 2019;1(8):9.
- Language models are few-shot learners. Advances in neural information processing systems. 2020;33:1877–1901.
- OpenAI. OpenAI, editor. GPT-4 Techincal Report. OpenAI; 2023. Accessed: 2023-04-24. Available from: https://cdn.openai.com/papers/gpt-4.pdf.
- A Bibliometric Review of Large Language Models Research from 2017 to 2023. arXiv preprint arXiv:230402020. 2023.
- A survey of large language models. arXiv preprint arXiv:230318223. 2023.
- ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences. 2023;103:102274.
- Shen Yea. Moy L, editor. ChatGPT and other large language models are double-edged swords. Radiological Society of North America; 2023.
- Armstrong K. BBC, editor. ChatGPT: US lawyer admits using AI for case research. BBC; 2023. Accessed: 2023-05-30. Available from: https://www.bbc.co.uk/news/world-us-canada-65735769.
- Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. bioRxiv. 2022:2022–12.
- Alkaissi H, McFarlane SI. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus. 2023;15(2).
- McMichael J. SMU, editor. Artificial Intelligence and the Research Paper: A Librarian’s Perspective. SMU; 2023. Accessed: 2023-05-30. Available from: https://blog.smu.edu/smulibraries/2023/01/20/artificial-intelligence-and-the-research-paper-a-librarians- perspective/.
- Knoth P, et al. CORE: A Global Aggregation Service for Open Access Papers. Nature Scientific Data. Publication due June 2023.
- Gusenbauer M. Search where you will find most: Comparing the disciplinary coverage of 56 bibliographic databases. Scientometrics. 2022;127(5):2683–2745.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.