Papers
Topics
Authors
Recent
Search
2000 character limit reached

Speculative Exploration on the Concept of Artificial Agents Conducting Autonomous Research

Published 6 Dec 2023 in cs.AI and cs.LG | (2312.03497v1)

Abstract: This paper engages in a speculative exploration of the concept of an artificial agent capable of conducting research. Initially, it examines how the act of research can be conceptually characterized, aiming to provide a starting point for discussions about what it means to create such agents. The focus then shifts to the core components of research: question formulation, hypothesis generation, and hypothesis verification. This discussion includes a consideration of the potential and challenges associated with enabling machines to autonomously perform these tasks. Subsequently, this paper briefly considers the overlapping themes and interconnections that underlie them. Finally, the paper presents preliminary thoughts on prototyping as an initial step towards uncovering the challenges involved in developing these research-capable agents.

Authors (1)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (166)
  1. The future of fundamental science led by generative closed-loop artificial intelligence. arXiv preprint arXiv:2307.07522, 2023.
  2. Hiroaki Kitano. Nobel turing challenge: creating the engine for scientific discovery. npj Systems Biology and Applications, 7(1):29, 2021.
  3. Dendral: a case study of the first expert system for scientific hypothesis formation. Artificial intelligence, 61(2):209–261, 1993.
  4. Pat Langley. Scientific discovery: Computational explorations of the creative processes. MIT press, 1987.
  5. Functional genomic hypothesis generation and experimentation by a robot scientist. Nature, 427(6971):247–252, 2004.
  6. The Automated AI-driven Future of Scientific Discovery, pages 679–691. World Scientific, 2023.
  7. Scientific discovery in the age of artificial intelligence. Nature, 620(7972):47–60, 2023.
  8. Artificial intelligence: A powerful paradigm for scientific research. The Innovation, 2(4):100179, 2021.
  9. Artificial intelligence for science in quantum, atomistic, and continuum systems. arXiv preprint arXiv:2307.08423, 2023.
  10. The impact of large language models on scientific discovery: a preliminary study using gpt-4. arXiv preprint arXiv:2311.07361, 2023.
  11. Autonomous discovery in the chemical sciences part ii: outlook. Angewandte Chemie International Edition, 59(52):23414–23436, 2020.
  12. Autonomous discovery in the chemical sciences part i: Progress. Angewandte Chemie International Edition, 59(51):22858–22893, 2020.
  13. A computational inflection for scientific discovery. arXiv preprint arXiv:2205.02007, 2022.
  14. Automated Research Workflows for Accelerated Discovery: Closing the Knowledge Discovery Loop. The National Academies Press, 2022.
  15. Alan F Chalmers. What is this thing called science? Hackett Publishing, 2013.
  16. Scientific Method. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Summer 2021 edition, 2021.
  17. The craft of research. University of Chicago press, 2003.
  18. Frascati Manual et al. Guidelines for collecting and reporting data on research and experimental development. URL: http://www. oecd. org/sti/frascati-manual-2015-9789264239012-en. htm, 2015.
  19. Distributed science-the scientific process as multi-scale active inference. OSF Preprints, 2023.
  20. Epistemology. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Fall 2020 edition, 2020.
  21. Edmund L Gettier. Is justified true belief knowledge? analysis, 23(6):121–123, 1963.
  22. Jun Otsuka. Thinking About Statistics: The Philosophical Foundations. Taylor & Francis, 2022.
  23. A search engine for discovery of scientific challenges and directions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11982–11990, 2022.
  24. Mapping the challenges of hci: An application and evaluation of chatgpt and gpt-4 for cost-efficient question answering. arXiv preprint arXiv:2306.05036, 2023.
  25. Can questions summarize a corpus? using question generation for characterizing covid-19 research. arXiv preprint arXiv:2009.09290, 2020.
  26. Paperrobot: Incremental draft generation of scientific ideas. arXiv preprint arXiv:1905.07870, 2019.
  27. Predicting research trends with semantic and neural networks with an application in quantum physics. Proceedings of the National Academy of Sciences, 117(4):1910–1916, 2020.
  28. Predicting the future of ai with ai: High-quality link prediction in an exponentially growing knowledge network. arXiv preprint arXiv:2210.00881, 2022.
  29. Recent advances in neural question generation. arXiv preprint arXiv:1905.08949, 2019.
  30. A review on question generation from natural language text. ACM Transactions on Information Systems (TOIS), 40(1):1–43, 2021.
  31. Jürgen Schmidhuber. A possibility for implementing curiosity and boredom in model-building neural controllers. In Proc. of the international conference on simulation of adaptive behavior: From animals to animats, pages 222–227, 1991.
  32. Creative research question generation for human-computer interaction research. In Joint Proceedings of the ACM IUI Workshop, 2023.
  33. Evaluating the use of large language model in identifying top research questions in gastroenterology. Scientific reports, 13(1):4164, 2023.
  34. Lani Watson. What is a question. Royal Institute of Philosophy Supplements, 89:273–297, 2021.
  35. Robert S Taylor. The process of asking questions. American documentation, 13(4):391–396, 1962.
  36. Tom D Wilson. Information behaviour: an interdisciplinary perspective. Information processing & management, 33(4):551–572, 1997.
  37. Looking for information: A survey of research on information seeking, needs, and behavior. Emerald Group Publishing, 2016.
  38. The psychology and neuroscience of curiosity. Neuron, 88(3):449–460, 2015.
  39. A survey on intrinsic motivation in reinforcement learning. arXiv preprint arXiv:1908.06976, 2019.
  40. Constructing research questions: Doing interesting research. Sage, 2013.
  41. Stephen B Hulley. Designing clinical research. Lippincott Williams & Wilkins, 2007.
  42. Uri Alon. How to choose a good scientific problem. Molecular cell, 35(6):726–728, 2009.
  43. Nick Huntington-Klein. The effect: An introduction to research design and causality. CRC Press, 2021.
  44. Pierre-Yves Oudeyer. Computational theories of curiosity-driven learning. arXiv preprint arXiv:1802.10546, 2018.
  45. Jutta Schickore. Scientific Discovery. In Edward N. Zalta and Uri Nodelman, editors, The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Winter 2022 edition, 2022.
  46. Norwood Russell Hanson. Patterns of discovery: An inquiry into the conceptual foundations of science. CUP Archive, 1965.
  47. Lorenzo Magnani. Abduction, reason and science: Processes of discovery and explanation. Springer Science & Business Media, 2011.
  48. Dedre Gentner. Analogy in scientific discovery: The case of johannes kepler. In Model-based reasoning, pages 21–39. Springer, 2002.
  49. Where do hypotheses come from? Cognitive psychology, 96:1–25, 2017.
  50. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583–589, 2021.
  51. Scaling deep learning for materials discovery. Nature, 2023.
  52. Automated scientific discovery: From equation discovery to autonomous discovery systems. arXiv preprint arXiv:2305.02251, 2023.
  53. Augmenting scientific creativity with an analogical search engine. ACM Transactions on Computer-Human Interaction, 2022.
  54. Solvent: A mixed initiative system for finding analogies between research papers. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW):1–21, 2018.
  55. Learning to generate novel scientific directions with contextualized literature-based discovery. arXiv preprint arXiv:2305.14259, 2023.
  56. Exploring and verbalizing academic ideas by concept co-occurrence. arXiv preprint arXiv:2306.02282, 2023.
  57. Large language models for automated open-domain scientific hypotheses discovery. arXiv preprint arXiv:2309.02726, 2023.
  58. Can chatgpt be used to generate scientific hypotheses? arXiv preprint arXiv:2304.12208, 2023.
  59. On calibration of modern neural networks. In International conference on machine learning, pages 1321–1330. PMLR, 2017.
  60. On faithfulness and factuality in abstractive summarization. arXiv preprint arXiv:2005.00661, 2020.
  61. M Burton David. The history of mathematics an introduction. McGraw-Hill Professional, 2010.
  62. The role of mathematics in the rise of science. American Journal of Physics, 36(6):564–565, 1968.
  63. Werner Heisenberg. Abstraction in modern science. Nishina Memorial Lectures, pages 1–16, 2008.
  64. Towards the automatic mathematician. In Automated Deduction–CADE 28: 28th International Conference on Automated Deduction, Virtual Event, July 12–15, 2021, Proceedings 28, pages 25–37. Springer International Publishing, 2021.
  65. Mathprompter: Mathematical reasoning using large language models. arXiv preprint arXiv:2303.05398, 2023.
  66. Inductive biases for deep learning of higher-level cognition. Proceedings of the Royal Society A, 478(2266):20210068, 2022.
  67. Bayesian experimental design: A review. Statistical science, pages 273–304, 1995.
  68. N Baker et al. Basic research needs workshop for scientific machine learning: Core technologies for artificial intelligence. Document prepared for Department of Energy Advanced Scientific Computing Research, USA, 10, 2019.
  69. Fact or fiction: Verifying scientific claims. arXiv preprint arXiv:2004.14974, 2020.
  70. A survey on automated fact-checking. Transactions of the Association for Computational Linguistics, 10:178–206, 2022.
  71. Can large language models discern evidence for scientific hypotheses? case studies in the social sciences. arXiv preprint arXiv:2309.06578, 2023.
  72. Chain-of-verification reduces hallucination in large language models. arXiv preprint arXiv:2309.11495, 2023.
  73. Artificial intelligence technologies to support research assessment: A review. arXiv preprint arXiv:2212.06574, 2022.
  74. Automated scholarly paper review: possibility and challenges. arXiv preprint arXiv:2111.07533, 2021.
  75. Hans Radder. The philosophy of scientific experimentation: a review. Automated experimentation, 1(1):1–8, 2009.
  76. Automation in the life science research laboratory. Frontiers in Bioengineering and Biotechnology, 8:571777, 2020.
  77. The rise of self-driving labs in chemical and materials sciences. Nature Synthesis, pages 1–10, 2023.
  78. A mobile robotic chemist. Nature, 583(7815):237–241, 2020.
  79. Robotic crowd biology with maholo labdroids. Nature biotechnology, 35(4):310–312, 2017.
  80. Emergent autonomous scientific research capabilities of large language models. arXiv preprint arXiv:2304.05332, 2023.
  81. Gpt-lab: Next generation of optimal chemistry discovery by gpt driven robotic lab. arXiv preprint arXiv:2309.16721, 2023.
  82. Generation next: Experimentation with ai. Technical report, National Bureau of Economic Research, 2023.
  83. Physics-informed machine learning: A survey on problems, methods and applications. arXiv preprint arXiv:2211.08064, 2022.
  84. Physics-informed machine learning. Nature Reviews Physics, 3(6):422–440, 2021.
  85. A hypothesis is a liability, 2020.
  86. On The Origin of Evolution: Tracing ‘Darwin’s Dangerous Idea’from Aristotle to DNA. Rowman & Littlefield, 2022.
  87. Derek Thomas Whiteside. Before the principia: The maturing of newton’s thoughts on dynamical astronomy, 1664–1684. Journal for the History of Astronomy, 1(1):5–19, 1970.
  88. Bruno Latour. Science in action: How to follow scientists and engineers through society. Harvard university press, 1987.
  89. Yolanda Gil. Will ai write scientific papers in the future? AI Magazine, 42(4):3–15, 2022.
  90. Kyle Stanford. Underdetermination of Scientific Theory. In Edward N. Zalta and Uri Nodelman, editors, The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Summer 2023 edition, 2023.
  91. Autonomous experiments using active learning and ai. Nature Reviews Materials, 8(9):563–564, 2023.
  92. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023.
  93. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  94. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  95. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020.
  96. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.
  97. Scibert: A pretrained language model for scientific text. arXiv preprint arXiv:1903.10676, 2019.
  98. Specter: Document-level representation learning using citation-informed transformers. arXiv preprint arXiv:2004.07180, 2020.
  99. Scirepeval: A multi-format benchmark for scientific document representations. arXiv preprint arXiv:2211.13308, 2022.
  100. Scientific language models for biomedical knowledge base completion: an empirical study. arXiv preprint arXiv:2106.09700, 2021.
  101. Matscibert: A materials domain language model for text mining and information extraction. npj Computational Materials, 8(1):102, 2022.
  102. Galactica: A large language model for science. arXiv preprint arXiv:2211.09085, 2022.
  103. Llemma: An open language model for mathematics. arXiv preprint arXiv:2310.06786, 2023.
  104. Darwin series: Domain specific large language models for natural science. arXiv preprint arXiv:2308.13565, 2023.
  105. Biogpt: generative pre-trained transformer for biomedical text generation and mining. Briefings in Bioinformatics, 23(6):bbac409, 2022.
  106. Gatortron: A large clinical language model to unlock patient information from unstructured electronic health records. arXiv preprint arXiv:2203.03540, 2022.
  107. Learning a foundation language model for geoscience knowledge understanding and utilization. arXiv preprint arXiv:2306.05064, 2023.
  108. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. arXiv preprint arXiv:2306.00890, 2023.
  109. Towards generalist biomedical ai. arXiv preprint arXiv:2307.14334, 2023.
  110. Foundation model for material science. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 15376–15383, 2023.
  111. Climax: A foundation model for weather and climate. arXiv preprint arXiv:2301.10343, 2023.
  112. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
  113. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
  114. OpenAI. Gpt-4. https://openai.com/research/gpt-4, 2023. Version of the Generative Pre-trained Transformer.
  115. OpenAI. Chatgpt. https://openai.com/chatgpt, 2023. Software available from https://openai.com/chatgpt.
  116. Chemcrow: Augmenting large-language models with chemistry tools. arXiv preprint arXiv:2304.05376, 2023.
  117. Do large language models know chemistry? ChemRxiv, 2022.
  118. Prompt engineering of gpt-4 for chemical research: what can/cannot be done? ChemRxiv, 2023.
  119. 14 examples of how llms can transform materials science and chemistry: a reflection on a large language model hackathon. Digital Discovery, 2(5):1233–1250, 2023.
  120. What can large language models do in chemistry? a comprehensive benchmark on eight tasks. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
  121. Large language models for scientific synthesis, inference and explanation. arXiv preprint arXiv:2310.07984, 2023.
  122. Can large language models empower molecular property prediction? arXiv preprint arXiv:2307.07443, 2023.
  123. Large language models, scientific knowledge and factuality: A systematic analysis in antibiotic discovery. arXiv preprint arXiv:2305.17819, 2023.
  124. Capabilities of gpt-4 on medical challenge problems. arXiv preprint arXiv:2303.13375, 2023.
  125. Are large language models ready for healthcare? a comparative study on clinical language understanding. arXiv preprint arXiv:2304.05368, 2023.
  126. Large language models encode clinical knowledge. Nature, 620(7972):172–180, 2023.
  127. Sebastian Bordt and Ulrike von Luxburg. Chatgpt participates in a computer science exam. arXiv preprint arXiv:2303.09461, 2023.
  128. An empirical study on challenging math problem solving with gpt-4. arXiv preprint arXiv:2306.01337, 2023.
  129. Performance of chatgpt on the us fundamentals of engineering exam: Comprehensive assessment of proficiency and potential implications for professional environmental engineering practice. Computers and Education: Artificial Intelligence, page 100183, 2023.
  130. Can gpt-4 perform neural architecture search? arXiv preprint arXiv:2304.10970, 2023.
  131. Automl-gpt: Automatic machine learning with gpt. arXiv preprint arXiv:2305.02499, 2023.
  132. Prompt2model: Generating deployable models from natural language instructions. arXiv preprint arXiv:2308.12261, 2023.
  133. A survey on large language model based autonomous agents. arXiv preprint arXiv:2308.11432, 2023.
  134. Christopher A Bail. Can generative ai improve social science? SocArXiv, 2023.
  135. Can large language models transform computational social science? arXiv preprint arXiv:2305.03514, 2023.
  136. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442, 2023.
  137. John J Horton. Large language models as simulated economic agents: What can we learn from homo silicus? Technical report, National Bureau of Economic Research, 2023.
  138. Anton Korinek. Generative ai for economic research: Use cases and implications for economists. Journal of Economic Literature, 61(4), 2023.
  139. Using large language models to simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning, pages 337–371. PMLR, 2023.
  140. Chatgpt applications in academic research: A review of benefits, concerns, and recommendations. bioRxiv, pages 2023–08, 2023.
  141. Elicit. https://elicit.org/. Accessed on 2023-04-06.
  142. SCISPACE. https://scispace.com/. Accessed on 2023-04-06.
  143. Can gpt-3 write an academic paper on itself, with minimal human input? HAL Open Science, 2022.
  144. Comparing scientific abstracts generated by chatgpt to real abstracts with detectors and blinded human reviewers. NPJ Digital Medicine, 6(1):75, 2023.
  145. Openai chatgpt generated literature review: Digital twin in healthcare. Available at SSRN 4308687, 2022.
  146. Can large language models provide useful feedback on research papers? A large-scale empirical analysis. In arXiv preprint arXiv:2310.01783, 2023.
  147. Reviewergpt? an exploratory study on using large language models for paper reviewing. arXiv preprint arXiv:2306.00622, 2023.
  148. Zachary Robertson. Gpt4 is slightly helpful for peer-review assistance: A pilot study. arXiv preprint arXiv:2307.05492, 2023.
  149. Fighting reviewer fatigue or amplifying bias? considerations and recommendations for use of chatgpt and other large language models in scholarly peer review. Research Integrity and Peer Review, 8(1):4, 2023.
  150. On scientific understanding with artificial intelligence. Nature Reviews Physics, 4(12):761–769, 2022.
  151. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information fusion, 58:82–115, 2020.
  152. KillianLucas. Open interpreter, 2023. Accessed: 2023-09-24, License: MIT.
  153. OpenAI. Chatgpt plugins - code interpreter. https://openai.com/blog/chatgpt-plugins#code-interpreter, 2023. Accessed: 2023-12-03.
  154. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021.
  155. Adept. Act-1: Transformer for actions, 2022. Accessed: 2023-09-25.
  156. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864, 2023.
  157. Automated machine learning: methods, systems, challenges. Springer Nature, 2019.
  158. Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 13(2):e1484, 2023.
  159. Best practices for scientific research on neural architecture search. The Journal of Machine Learning Research, 21(1):9820–9837, 2020.
  160. Neural architecture search: Insights from 1000 papers. arXiv preprint arXiv:2301.08727, 2023.
  161. Machine learning operations (mlops): Overview, definition, and architecture. IEEE Access, 2023.
  162. Can we automate scientific reviewing? Journal of Artificial Intelligence Research, 75:171–212, 2022.
  163. Kid-review: Knowledge-guided scientific review generation with oracle pre-training. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11639–11647, 2022.
  164. Reviewrobot: Explainable paper review generation based on knowledge synthesis. arXiv preprint arXiv:2010.06119, 2020.
  165. Is the future of peer review automated? BMC Research Notes, 15(1):1–5, 2022.
  166. Reviewer assignment algorithms for peer review automation: A survey. Information Processing & Management, 59(5):103028, 2022.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 2 tweets with 0 likes about this paper.

Sign Up for Free