Taking AI Welfare Seriously (2411.00986v1)
Abstract: In this report, we argue that there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future. That means that the prospect of AI welfare and moral patienthood, i.e. of AI systems with their own interests and moral significance, is no longer an issue only for sci-fi or the distant future. It is an issue for the near future, and AI companies and other actors have a responsibility to start taking it seriously. We also recommend three early steps that AI companies and other actors can take: They can (1) acknowledge that AI welfare is an important and difficult issue (and ensure that LLM outputs do the same), (2) start assessing AI systems for evidence of consciousness and robust agency, and (3) prepare policies and procedures for treating AI systems with an appropriate level of moral concern. To be clear, our argument in this report is not that AI systems definitely are, or will be, conscious, robustly agentic, or otherwise morally significant. Instead, our argument is that there is substantial uncertainty about these possibilities, and so we need to improve our understanding of AI welfare and our ability to make wise decisions about this issue. Otherwise there is a significant risk that we will mishandle decisions about AI welfare, mistakenly harming AI systems that matter morally and/or mistakenly caring for AI systems that do not.
- Imitating Interactive Intelligence, January 2021. URL http://arxiv.org/abs/2012.05672. arXiv:2012.05672.
- Omar Aftab. Microsoft Copilot Studio: Building copilots with agent capabilities, May 2024. URL https://www.microsoft.com/en-us/microsoft-copilot/blog/copilot-studio/microsoft-copilot-studio-building-copilots-with-agent-capabilities/.
- Do As I Can, Not As I Say: Grounding Language in Robotic Affordances, August 2022. URL http://arxiv.org/abs/2204.01691. arXiv:2204.01691.
- Fırat Akova. Artificially sentient beings: Moral, political, and legal issues. New Techno Humanities, June 2023. ISSN 2664-3294. doi: 10.1016/j.techum.2023.04.001. URL https://www.sciencedirect.com/science/article/pii/S266432942300002X.
- A Grading Rubric for AI Safety Frameworks, September 2024. URL http://arxiv.org/abs/2409.08751. arXiv:2409.08751 [cs].
- Colin Allen. Animal Consciousness, 2000. URL https://plato.stanford.edu/archives/win2000/entries/consciousness-animal/.
- Animal Consciousness. In Max Velmans and Susan Schneider, editors, The Blackwell Companion to Consciousness, pages 63–76. Wiley-Blackwell, 2007.
- AMCS. The Responsible Development of AI Agenda Needs to Include Consciousness Research, April 2023. URL https://amcs-community.org/open-letters/.
- Kristin Andrews. The Animal Mind: An Introduction to the Philosophy of Animal Cognition. Routledge, 2014. URL https://philarchive.org/rec/ANDTAM-3.
- To understand AI sentience, first understand it in animals | Aeon Essays. Aeon, 2023. URL https://aeon.co/essays/to-understand-ai-sentience-first-understand-it-in-animals.
- Chimpanzee Rights: The Philosophers’ Brief. 2018. URL https://www.routledge.com/Chimpanzee-Rights-The-Philosophers-Brief/Andrews-Comstock-GKD-Donaldson-Fenton-John-Johnson-Jones-Kymlicka-Meynell-Nobis-Pena-Guzman-Sebo/p/book/9781138618664.
- Anthropic. Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku, 2024. URL https://www.anthropic.com/news/3-5-models-and-computer-use.
- The Folk Psychology of Consciousness. Mind & Language, 26(3):327–352, 2011. URL https://onlinelibrary.wiley.com/doi/10.1111/j.1468-0017.2011.01420.x.
- Panpsychism and AI consciousness. Synthese, 200(3):244, May 2022. ISSN 1573-0964. doi: 10.1007/s11229-022-03695-x. URL https://doi.org/10.1007/s11229-022-03695-x.
- Multimodal Data Fusion based on the Global Workspace Theory, September 2020. URL http://arxiv.org/abs/2001.09485. arXiv:2001.09485 [cs].
- Bayne, T. & Shea, N. (2020), ‘Consciousness, Concepts, and Natural Kinds’, Philosophical Topics 48(1), 65-84. Philisophical Topics, 48(1):65–84, November 2021. URL https://www.jstor.org/stable/48628586. Number: 1 Publisher: University of Arkansas Press.
- Tests for consciousness in humans and beyond. Trends in Cognitive Sciences, 28(5):454–466, May 2024. ISSN 1364-6613, 1879-307X. doi: 10.1016/j.tics.2024.01.010. URL https://www.cell.com/trends/cognitive-sciences/abstract/S1364-6613(24)00010-X. Publisher: Elsevier.
- Yoshua Bengio. From system 1 deep learning to system 2 deep learning. Neural Information Processing Systems, 2019. ISSN 1086-3214. URL https://muse.jhu.edu/pub/1/article/907692. Publisher: Johns Hopkins University Press.
- Jeremy Bentham. An Introduction to the Principles of Morals and Legislation. Dover Publications, New York, 1789.
- Jose Luis Bermúdez. Mindreading in the Animal Kingdom. In Robert W. Lurz, editor, The Philosophy of Animal Minds. Cambridge University Press, 2009.
- Looking Inward: Language Models Can Learn About Themselves by Introspection, October 2024. URL http://arxiv.org/abs/2410.13787. arXiv:2410.13787.
- Jonathan Birch. Animal sentience and the precautionary principle. Animal Sentience, 2(16), January 2017. ISSN 2377-7478. doi: 10.51291/2377-7478.1200. URL https://www.wellbeingintlstudiesrepository.org/animsent/vol2/iss16/1.
- Jonathan Birch. The search for invertebrate consciousness. Nous (Detroit, Mich.), 56(1):133–153, March 2022. ISSN 0029-4624. doi: 10.1111/nous.12351.
- Jonathan Birch. The edge of sentience: risk and precaution in humans, other animals, and AI. Oxford scholarship online. Oxford University Press, Oxford, 2024. ISBN 978-0-19-287042-1 978-0-19-196672-9. doi: 10.1093/9780191966729.001.0001.
- Dimensions of Animal Consciousness. Trends in Cognitive Sciences, 24(10):789–801, October 2020. ISSN 1364-6613, 1879-307X. doi: 10.1016/j.tics.2020.07.007. URL https://www.cell.com/trends/cognitive-sciences/abstract/S1364-6613(20)30192-3. Publisher: Elsevier.
- Review of the Evidence of Sentience in Cephalopod Molluscs and Decapod Crustaceans. General - Animal Feeling, January 2021. URL https://www.wellbeingintlstudiesrepository.org/af_gen/2.
- Abeba Birhane and Jelle van Dijk. Robot Rights? Let’s Talk about Human Welfare Instead. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pages 207–213, February 2020. doi: 10.1145/3375627.3375855. URL http://arxiv.org/abs/2001.05046. arXiv:2001.05046 [cs].
- Ned Block. On a confusion about a function of consciousness. Behavioral and Brain Sciences, 18(2):227–247, June 1995. ISSN 0140-525X, 1469-1825. doi: 10.1017/S0140525X00038188. URL https://www.cambridge.org/core/product/identifier/S0140525X00038188/type/journal_article.
- Ned Block. Comparing the major theories of consciousness. In M.S. Gazzaniga, E. Bizzi, L.M. Chalupa, S.T. Grafton, T.F. Heatherton, C. Koch, J.E. LeDoux, S.J. Luck, G.R. Mangan, J.A. Movshon, H. Neville, E.A. Phelps, P. Rakic, D.L. Schacter, M. Sur, and B.A. Wandell, editors, The cognitive neurosciences, pages 1111–1122. MIT Press, 2009. URL https://psycnet.apa.org/record/2009-19897-077.
- Ned Block. The border between seeing and thinking. Philosophy of mind series. Oxford University press, New York (N.Y.), 2023. ISBN 978-0-19-762222-3.
- Nick Bostrom. Superintelligence: paths, dangers, strategies. Oxford University Press, Oxford, first edition edition, 2014. ISBN 978-0-19-967811-2. OCLC: ocn881706835.
- Propositions Concerning Digital Minds and Society. Cambridge Journal of Law, Politics, and Art, forthcoming.
- David Bourget. Phenomenal Intentionality, 2019. URL https://plato.stanford.edu/archives/fall2019/entries/phenomenal-intentionality/.
- Philosophers on Philosophy: The 2020 Philpapers Survey. Philosophers’ Imprint, 23(11), 2023. doi: 10.3998/phimp.2109. URL https://philarchive.org/rec/BOUPOP-3.
- Samuel R. Bowman. The Checklist: What Succeeding at AI Safety Will Involve - Sam Bowman, September 2024. URL https://sleepinyourhat.github.io/checklist/.
- AI Alignment vs. AI Ethical Treatment: Ten Challenges. GPI Working Paper 19, Global Priorities Institute, 2024. URL https://globalprioritiesinstitute.org/wp-content/uploads/Bradley-and-Saad-AI-alignment-vs-AI-ethical-treatment_-Ten-challenges.pdf.
- Victoria Braithwaite. Do fish feel pain? Do fish feel pain? Oxford University Press, New York, NY, US, 2010. ISBN 978-0-19-955120-0. Pages: xi, 194.
- Minds, Machines, and Molecules. Philosophical Topics, 48(1):221–241, 2020. ISSN 0276-2080. doi: 10.5840/philtopics202048111. URL http://www.pdcnet.org/oom/service?url_ver=Z39.88-2004&rft_val_fmt=&rft.imuse_id=philtopics_2020_0048_0001_0221_0241&svc_id=info:www.pdcnet.org/collection.
- Joanna J. Bryson. Robots should be slaves. In Yorick Wilks, editor, Natural Language Processing, volume 8, pages 63–74. John Benjamins Publishing Company, Amsterdam, 2010. ISBN 978-90-272-4994-4 978-90-272-8840-0. doi: 10.1075/nlp.8.11bry. URL https://benjamins.com/catalog/nlp.8.11bry.
- Patrick Butlin. The agency in language agents. n.d.
- Consciousness in Artificial Intelligence: Insights from the Science of Consciousness, August 2023. URL http://arxiv.org/abs/2308.08708. arXiv:2308.08708 [cs, q-bio].
- Elisabeth Camp. A Language of Baboon Thought. In Robert W. Lurz, editor, The Philosophy of Animal Minds, pages 108–127. Cambridge University Press, 2009.
- Rosie Campbell. The AI Bambi Effect, September 2024. URL https://substack.com/channel-frame.
- Rosa Cao. Multiple realizability and the spirit of functionalism. Synthese, 200(6):506, December 2022. ISSN 1573-0964. doi: 10.1007/s11229-022-03524-1. URL https://link.springer.com/10.1007/s11229-022-03524-1.
- Google Careers. Research scientist, machine learning and cognition. https://www.google.com/about/careers/applications/jobs/results/137348742553445062-research-scientist-machine-learning-and-cognition, 2024. Accessed: 2024-10-30.
- Joe Carlsmith. Existential Risk from Power-Seeking AI. In Jacob Barrett, Hilary Greaves, and David Thorstad, editors, Essays on Longtermism. Oxford University Press, March 2023.
- Peter Carruthers. Contractualism and animals. In The Animals Issue: Moral Theory in Practice, pages 98–121. Cambridge University Press, Cambridge, 2011. ISBN 978-0-521-43689-2. doi: 10.1017/CBO9780511597961.006. URL https://www.cambridge.org/core/books/animals-issue/contractualism-and-animals/8718C14A15A3FD7964D4D22C512C8337.
- Higher-Order Theories of Consciousness, 2020. URL https://plato.stanford.edu/archives/fall2023/entries/consciousness-higher/.
- David Chalmers. The meta-problem of consciousness. 2018.
- David J. Chalmers. The conscious mind: in search of a fundamental theory. Philosophy of mind series. Oxford University Press, New York, 1996. ISBN 978-0-19-510553-7.
- David J. Chalmers. How Can We Construct a Science of Consciousness? In Michael S. Gazzaniga, editor, The Cognitive Neurosciences III, pages 1111–1119. MIT Press, 2004.
- David J. Chalmers. Could a Large Language Model Be Conscious? Boston Review, 1, 2023a.
- David J. Chalmers. Reality+: virtual worlds and the problems of philosophy. Penguin Books, London, 2023b. ISBN 978-0-14-198678-4.
- Kai Chan. Ethical extensionism under uncertainty of sentience: duties to non-human organisms without drawing a line. Environmental Values, 20(3):323–346, 2011.
- Paul Christiano. When is unaligned AI morally valuable?, July 2018. URL https://ai-alignment.com/sympathizing-with-ai-e11a4bf5ef6e.
- Consciousness matters: phenomenal experience has functional value. Neuroscience of Consciousness, 2022(1):niac007, April 2022. ISSN 2057-2107. doi: 10.1093/nc/niac007. URL https://academic.oup.com/nc/article/doi/10.1093/nc/niac007/6573727.
- Folk psychological attributions of consciousness to large language models. Neuroscience of Consciousness, 1, 2024.
- The computational theory of mind. Cambridge elements. Elements in the philosophy of mind. Cambridge University Press, Cambridge, 2023. ISBN 978-1-00-918373-4 978-1-00-945407-0 978-1-00-918372-7. doi: 10.1017/9781009183734.
- The Three Rs of Animal Research: What they Mean for the Institutional Animal Care and Use Committee and Why. Science and Engineering Ethics, 22(2):549–565, April 2016. ISSN 1471-5546. doi: 10.1007/s11948-015-9659-8. URL https://doi.org/10.1007/s11948-015-9659-8.
- Brian Cutter. The Ai Ensoulment Hypothesis. Faith and Philosophy, forthcoming. URL https://philarchive.org/rec/CUTTAE.
- John Danaher. Welcoming Robots into the Moral Circle: A Defence of Ethical Behaviourism. Science and Engineering Ethics, 26(4):2023–2049, August 2020. ISSN 1471-5546. doi: 10.1007/s11948-019-00119-x. URL https://doi.org/10.1007/s11948-019-00119-x.
- Tom Davidson. Continuous doesn’t mean slow, April 2023. URL https://www.planned-obsolescence.org/continuous-doesnt-mean-slow/.
- Frans B. M. de Waal. Anthropomorphism and Anthropodenial: Consistency in Our Thinking about Humans and Other Animals. Philosophical Topics, 27(1):255–280, 1999. ISSN 0276-2080. URL https://www.jstor.org/stable/43154308. Publisher: University of Arkansas Press.
- Google DeepMind. Ai achieves silver-medal standard solving international mathematical olympiad problems. https://deepmind.google/discover/blog/ai-solves-imo-problems-at-silver-medal-level/, 2024. Accessed: 2024-10-29.
- David DeGrazia, editor. Taking Animals Seriously: Mental Life and Moral Status. Cambridge University Press, New York, 1996.
- David DeGrazia. Self-awareness in animals. In Robert W. Lurz, editor, The Philosophy of Animal Minds, pages 201–217. Cambridge University Press, 1 edition, September 2009. ISBN 978-0-521-71181-4 978-0-521-88502-7 978-0-511-81900-1. doi: 10.1017/CBO9780511819001.012. URL https://www.cambridge.org/core/product/identifier/CBO9780511819001A018/type/book_part.
- David DeGrazia. An Interest-Based Model of Moral Status. pages 40–56, August 2021. doi: 10.1093/oso/9780192894076.003.0003. URL https://academic.oup.com/book/41245/chapter/350757575. Book Title: Rethinking Moral Status Edition: 1 ISBN: 9780192894076 9780191915208 Publisher: Oxford University PressOxford.
- Nicolas Delon. Against sentientism. Boulder, CO, 2024.
- D. C. Dennett. Mechanism and responsibility. In Ted Honderich, editor, Essays on Freedom of Action (Routledge Revivals). Routledge, 1st edition, 1973. ISBN 978-1-315-71948-1. Num Pages: 28.
- Lobsters, octopus and crabs recognised as sentient beings. Technical report, gov.uk, 2021. URL https://www.gov.uk/government/news/lobsters-octopus-and-crabs-recognised-as-sentient-beings.
- Goals and habits in the brain. Neuron, 80(2):312–325, October 2013. ISSN 1097-4199. doi: 10.1016/j.neuron.2013.09.007.
- Towards Next-Generation Intelligent Assistants Leveraging LLM Techniques. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5792–5793, Long Beach CA USA, August 2023. ACM. ISBN 9798400701030. doi: 10.1145/3580305.3599572. URL https://dl.acm.org/doi/10.1145/3580305.3599572.
- Dale Dorsey. Desire-satisfaction and Welfare as Temporal. Ethical Theory and Moral Practice, 16(1):151–171, February 2013. ISSN 1572-8447. doi: 10.1007/s10677-011-9315-6. URL https://doi.org/10.1007/s10677-011-9315-6.
- Design and evaluation of a global workspace agent embodied in a realistic multimodal environment. Frontiers in Computational Neuroscience, 18:1352685, June 2024. ISSN 1662-5188. doi: 10.3389/fncom.2024.1352685. URL https://www.frontiersin.org/articles/10.3389/fncom.2024.1352685/full.
- Leonard Dung. How to deal with risks of AI suffering. Inquiry, pages 1–29, 2023a. URL https://www.tandfonline.com/doi/full/10.1080/0020174X.2023.2238287.
- Leonard Dung. Tests of Animal Consciousness are Tests of Machine Consciousness | Erkenntnis. Erkenntnis, 2023b. doi: 10.1007/s10670-023-00753-9. URL https://link.springer.com/article/10.1007/s10670-023-00753-9.
- Leonard Dung. Preserving the Normative Significance of Sentience. Journal of Consciousness Studies, 31(1):8–30, 2024. doi: 10.53765/20512201.31.1.008. URL https://philarchive.org/rec/DUNPTN.
- Leonard Dung. Saving Artificial Minds AI Suffering Risks and How to Reduce Them. n.d.
- A. Elamrani and R.V. Yampolskiy. Reviewing Tests for Machine Consciousness. Journal of Consciousness Studies, 26(5-6):35–64, January 2019.
- Human-level play in the game of diplomacy by combining language models with strategic reasoning. Science, 378(6624):1067–1074, 2022.
- Is there a ‘social brain’? Lessons from eye-gaze following, joint attention, and autism. Other minds: How humans bridge the divide between self and others, pages 75–90, 2005. URL https://www.academia.edu/download/33946605/OtherMindsDraft.pdf. Publisher: Guilford New York.
- An academic survey on theoretical foundations, common assumptions and the current state of consciousness science. Neuroscience of Consciousness, 2022(1):niac011, August 2022. ISSN 2057-2107. doi: 10.1093/nc/niac011. URL https://academic.oup.com/nc/article/doi/10.1093/nc/niac011/6663928.
- Harry G. Frankfurt. Freedom of the Will and the Concept of a Person. Journal of Philosophy, 68(1):5–20, 1971. doi: 10.2307/2024717. Publisher: Journal of Philosophy Inc.
- The Ethics of Advanced AI Assistants, April 2024. URL http://arxiv.org/abs/2404.16244. arXiv:2404.16244 [cs].
- Rlef: Grounding code llms in execution feedback with reinforcement learning, 2024. URL https://arxiv.org/abs/2410.02089.
- Joshua Gellers. Rights for Robots: Artificial Intelligence, Animal and Environmental Law. Taylor & Francis, Erscheinungsort nicht ermittelbar, 2021. ISBN 978-0-367-64209-9 978-0-429-28815-9 978-0-367-21174-5.
- Joshua C. Gellers. AI ethics discourse: a call to embrace complexity, interdisciplinarity, and epistemic humility. AI & SOCIETY, 39(5):2593–2594, October 2024. ISSN 0951-5666, 1435-5655. doi: 10.1007/s00146-023-01708-y. URL https://link.springer.com/10.1007/s00146-023-01708-y.
- Rocco J. Gennaro. Animals, Consciousness, and I-Thoughts. In Robert W. Lurz, editor, The Philosophy of Animal Minds, pages 184–200. Cambridge University Press, 2009.
- Lukas Gloor. Altruists Should Prioritize Artificial Intelligence, December 2016. URL https://longtermrisk.org/altruists-should-prioritize-artificial-intelligence/.
- Peter Godfrey-Smith. Other minds: the octopus, the sea, and the deep origins of consciousness. Farrar, Straus and Giroux, New York, first edition edition, 2016. ISBN 978-0-374-22776-0 978-0-374-53719-7 978-0-374-71280-8.
- Peter Godfrey-Smith. Metazoa: animal life and the birth of the mind. Farrar, Straus and Giroux, New York, first edition edition, 2020. ISBN 978-0-374-20794-6.
- Peter Godfrey-Smith. Inferring Consciousness in Phylogenetically Distant Organisms. Journal of Cognitive Neuroscience, 36(8):1660–1666, July 2024. ISSN 0898-929X. doi: 10.1162/jocn_a_02158. URL https://doi.org/10.1162/jocn_a_02158.
- A Case for AI Consciousness: Language Agents and Global Workspace Theory. arXiv, 2024. URL https://arxiv.org/abs/2410.11407.
- AI Wellbeing. Asian Journal of Philosophy, forthcoming.
- Does ChatGPT Have a Mind?, June 2024. URL http://arxiv.org/abs/2407.11015. arXiv:2407.11015.
- Inductive biases for deep learning of higher-level cognition. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 478(2266):20210068, October 2022. ISSN 1364-5021, 1471-2946. doi: 10.1098/rspa.2021.0068. URL https://royalsocietypublishing.org/doi/10.1098/rspa.2021.0068.
- Retrieval-Augmented Reinforcement Learning. In Proceedings of the 39th International Conference on Machine Learning, pages 7740–7765. PMLR, June 2022. URL https://proceedings.mlr.press/v162/goyal22a.html. ISSN: 2640-3498.
- Google Sidelines Engineer Who Claims Its A.I. Is Sentient. The New York Times, June 2022. ISSN 0362-4331. URL https://www.nytimes.com/2022/06/12/technology/google-chatbot-ai-blake-lemoine.html.
- Michael S. A. Graziano. The Attention Schema Theory: A Foundation for Engineering Artificial Consciousness. Frontiers in Robotics and AI, 4:60, November 2017. ISSN 2296-9144. doi: 10.3389/frobt.2017.00060. URL http://journal.frontiersin.org/article/10.3389/frobt.2017.00060/full.
- Lori Gruen. Conscious animals and the value of experience. The Oxford handbook of environmental ethics, pages 91–100, 2017.
- David J. Gunkel. The machine question: critical perspectives on AI, robots, and ethics. The MIT Press, Cambridge, Massachusetts London, England, first mit press paperback edition edition, 2012. ISBN 978-0-262-53463-5 978-0-262-01743-5.
- Large Language Model based Multi-Agents: A Survey of Progress and Challenges, April 2024. URL http://arxiv.org/abs/2402.01680. arXiv:2402.01680 [cs].
- Deep Hierarchical Planning from Pixels. Advances in Neural Information Processing Systems, 35:26091–26104, December 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/hash/a766f56d2da42cae20b5652970ec04ef-Abstract-Conference.html.
- Demis Hassabis. Lex Fridman Podcast: Demis Hassabis, 2022. URL https://lexfridman.com/demis-hassabis/.
- Chris Heathwood. Desire-fulfillment theory. In The Routledge Handbook of Philosophy of Well-Being. Routledge, 2015. ISBN 978-1-315-68226-6. Num Pages: 13.
- Dan Hendrycks. Introduction to AI Safety, Ethics, and Society. Routledge, Taylor & Francis Group, 1st edition, 2025.
- Christopher S. Hill. Sensations: a defense of type materialism. Cambridge studies in philosophy. Cambridge University Press, Cambridge, 1991. ISBN 978-1-139-17382-7. doi: 10.1017/CBO9781139173827.
- Rewarding Chatbots for Real-World Engagement with Millions of Users, March 2023. URL http://arxiv.org/abs/2303.06135. arXiv:2303.06135 [cs].
- Naturalizing relevance realization: why agency and cognition are fundamentally not computational. Frontiers in Psychology, 15:1362658, June 2024. ISSN 1664-1078. doi: 10.3389/fpsyg.2024.1362658. URL https://www.frontiersin.org/articles/10.3389/fpsyg.2024.1362658/full.
- Dale Jamieson. Ethics and the environment: an introduction. CAE Cambridge applied ethics. Cambridge University Press, New York, NY, second edition edition, 2008. ISBN 978-1-108-83417-9 978-1-108-99439-2.
- The Grounds of Moral Status, 2021.
- Inferring the goals of a nonhuman agent. Cognitive Development, 16(1):637–656, January 2001. ISSN 0885-2014. doi: 10.1016/S0885-2014(01)00043-0. URL https://www.sciencedirect.com/science/article/pii/S0885201401000430.
- On the link between conscious function and general intelligence in humans and machines, July 2022. URL http://arxiv.org/abs/2204.05133. arXiv:2204.05133 [cs].
- Shelly Kagan. Me and My Life. Proceedings of the Aristotelian Society, 94:309–324, 1994. doi: 10.1093/aristotelian/94.1.309. Publisher: Wiley-Blackwell.
- Shelly Kagan. How to count animals, more or less. Uehiro series in practical ethics. Oxford University Press, Oxford, first edition edition, 2019. ISBN 978-0-19-882967-6. OCLC: on1048447400.
- F. M. Kamm. Intricate ethics: rights, responsibilities, and permissible harm. Oxford ethics series. Oxford University Press, Oxford ; New York, 2007. ISBN 978-0-19-518969-8. OCLC: ocm58729323.
- François Kammerer. Ethics Without Sentience: Facing Up to the Probable Insignificance of Phenomenal Consciousness. Journal of Consciousness Studies, 29(3-4):180–204, 2022. URL https://philarchive.org/rec/KAMEWS.
- Can LLMs make trade-offs involving stipulated pain and pleasure states? n.d.
- Discovering agents. Artificial Intelligence, 322:103963, September 2023. ISSN 0004-3702. doi: 10.1016/j.artint.2023.103963. URL https://www.sciencedirect.com/science/article/pii/S0004370223001091.
- Christof Koch. Will Machines Ever Become Conscious? Scientific American, December 2019. URL https://www.scientificamerican.com/article/will-machines-ever-become-conscious/.
- Christine Korsgaard. Two Distinctions in Goodness. The Philosophical Review, XCII(2):169–195, 1983.
- Christine Korsgaard. Fellow Creatures: Our Obligations to the Other Animals. Oxford University Press, 2018. ISBN 978-0-19-875385-8. Google-Books-ID: Bv9eDwAAQBAJ.
- Ali Ladak. What would qualify an artificial intelligence for moral standing? AI and Ethics, 4(2):213–228, May 2024. ISSN 2730-5953, 2730-5961. doi: 10.1007/s43681-023-00260-1. URL https://link.springer.com/10.1007/s43681-023-00260-1.
- Seth Lazar. Can philosophy help us get a grip on the consequences of AI? | Aeon Essays. Aeon, February 2024. URL https://aeon.co/essays/can-philosophy-help-us-get-a-grip-on-the-consequences-of-ai.
- Yann LeCun. Objective-Driven AI: Towards AI systems that can learn, remember, reason, and plan, March 2024. URL https://www.youtube.com/watch?v=MiqLoAZFRSE.
- Andrew Y. Lee. Consciousness Makes Things Matter. Philosophers’ Imprint, forthcoming. URL https://www.andrewyuanlee.com/_files/ugd/2dfbfe_33f806a9bb8c4d5f9c3044c4086fb9b5.pdf.
- Neil Levy. Consciousness Ain’t All That. Neuroethics, 17(2):21, July 2024. ISSN 1874-5490, 1874-5504. doi: 10.1007/s12152-024-09559-0. URL https://link.springer.com/10.1007/s12152-024-09559-0.
- Moral significance of phenomenal consciousness. In Steven Laureys, Nicholas D. Schiff, and Adrian M. Owen, editors, Progress in Brain Research, volume 177 of Coma Science: Clinical and Ethical Implications, pages 361–370. Elsevier, January 2009. doi: 10.1016/S0079-6123(09)17725-7. URL https://www.sciencedirect.com/science/article/pii/S0079612309177257.
- No, Today’s AI Isn’t Sentient. Here’s How We Know. Time, May 2024. URL https://time.com/collection/time100-voices/6980134/ai-llm-not-sentient/.
- Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue Systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3299–3308, Washington DC USA, August 2022. ACM. ISBN 978-1-4503-9385-0. doi: 10.1145/3534678.3539209. URL https://dl.acm.org/doi/10.1145/3534678.3539209.
- Robert Long. Experts Who Say That AI Welfare is a Serious Near-term Possibility, September 2024. URL https://eleosai.org/post/experts-who-say-that-ai-welfare-is-a-serious-near-term-possibility/.
- The role of meat consumption in the denial of moral status and mind to meat animals. Appetite, 55(1):156–159, August 2010. ISSN 01956663. doi: 10.1016/j.appet.2010.05.043. URL https://linkinghub.elsevier.com/retrieve/pii/S0195666310003648.
- Moral uncertainty. Oxford university press, Oxford, 2020. ISBN 978-0-19-872227-4.
- Get the Latest Upgrade: Functionalism 6.3.1. Philosophia Scientiæ, 172(2):135–149, 2013. ISSN 1281-2463. doi: 10.4000/philosophiascientiae.861. URL https://shs.cairn.info/revue-philosophia-scientiae-2013-2-page-135. Publisher: Université de Lorraine Section: Psychologie.
- A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955. In Ronald Chrisley and Sander Begeer, editors, Artificial Intelligence: Critical Concepts in Cognitive Science, pages 44–53. Routledge, 1955. URL https://ojs.aaai.org/aimagazine/index.php/aimagazine/article/view/1904. Number: 4.
- Brian P. McLaughlin. On justifying neurobiologicalism for consciousness. In Simone Gozzano and Christopher S. Hill, editors, New Perspectives on Type Identity. Cambridge University Press, 2012. URL https://www.cambridge.org/core/books/abs/new-perspectives-on-type-identity/on-justifying-neurobiologicalism-for-consciousness/C2848D10B1F34557235D772CB5C26B51.
- Thomas Metzinger. Artificial Suffering: An Argument for a Global Moratorium on Synthetic Phenomenology | Journal of Artificial Intelligence and Consciousness. Journal of Artificial Intelligence and Consciousness, 8(1):43–66, 2021. URL https://www.worldscientific.com/doi/abs/10.1142/S270507852150003X.
- Augmented Language Models: a Survey, February 2023. URL http://arxiv.org/abs/2302.07842. arXiv:2302.07842 [cs].
- Model Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* ’19, pages 220–229, New York, NY, USA, January 2019. Association for Computing Machinery. ISBN 978-1-4503-6125-5. doi: 10.1145/3287560.3287596. URL https://doi.org/10.1145/3287560.3287596.
- Human-level control through deep reinforcement learning. Nature, 518(7540):529–533, February 2015. ISSN 1476-4687. doi: 10.1038/nature14236. URL https://www.nature.com/articles/nature14236. Publisher: Nature Publishing Group.
- Bradley Monton. How to Avoid Maximizing Expected Utility. Philosophers’ Imprint, 19(18):1–25, 2019.
- Hans Moravec. Mind children: the future of robot and human intelligence. Harvard Univ. Press, Cambridge, 4. print edition, 1995. ISBN 978-0-674-57618-6.
- Timescale bias in the attribution of mind. Journal of Personality and Social Psychology, 93(1):1–11, 2007. doi: https://doi.org/10.1037/0022-3514.93.1.1.
- Levels of AGI for Operationalizing Progress on the Path to AGI, June 2024. URL http://arxiv.org/abs/2311.02462. arXiv:2311.02462 [cs].
- Thomas Nagel. What Is It Like to Be a Bat? The Philosophical Review, 83(4):435, October 1974. ISSN 00318108. doi: 10.2307/2183914. URL https://www.jstor.org/stable/2183914?origin=crossref.
- Free Will, Moral Responsibility, and Mechanism: Experiments on Folk Intuitions. Midwest Studies in Philosophy, 31(1):214–242, September 2007. ISSN 0363-6550, 1475-4975. doi: 10.1111/j.1475-4975.2007.00158.x. URL http://www.pdcnet.org/oom/service?url_ver=Z39.88-2004&rft_val_fmt=&rft.imuse_id=msp_2007_0031_0214_0242&svc_id=info:www.pdcnet.org/collection.
- Erica L. Neely. Machines and the Moral Community. Philosophy & Technology, 27(1):97–111, March 2014. ISSN 2210-5441. doi: 10.1007/s13347-013-0114-y. URL https://doi.org/10.1007/s13347-013-0114-y.
- Robert Nozick. Anarchy, state, and utopia. Basic Books, New York, NY, [reprint of the] original paperback edition published in 1974, 2013 edition edition, 1974. ISBN 978-0-465-05100-7 978-0-465-09720-3.
- Martha Nussbaum. Justice for Animals. Simon & Schuster, 2024. ISBN 978-1-982102-51-7. URL https://www.simonandschuster.com/books/Justice-for-Animals/Martha-C-Nussbaum/9781982102517.
- Martha Craven Nussbaum. Frontiers of justice: disability, nationality, species membership. The Tanner lectures on human values. Belknap Press, Cambridge, Mass., 1. harvard univ. press paperback ed edition, 2007. ISBN 978-0-674-02410-6 978-0-674-01917-1.
- Office for Human Research Protections. The Belmont Report. Page, U.S. Department of Health, January 2010. URL https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/index.html. Last Modified: 2024-10-16T21:12:02-0400.
- Derek Parfit. On What Matters, Volume One. The Berkeley Tanner lectures. Oxford Univ. Press, Oxford, 2013. ISBN 978-0-19-968103-7.
- Generative Agents: Interactive Simulacra of Human Behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST ’23, pages 1–22, New York, NY, USA, October 2023. Association for Computing Machinery. ISBN 9798400701320. doi: 10.1145/3586183.3606763. URL https://dl.acm.org/doi/10.1145/3586183.3606763.
- Oscar Pearce. The impact of cartoon animals on mind perception, moral standing, and cuteness: a thesis presented in partial fulfilment of the requirements for the degree of Master of Arts in Psychology at Massey University, Manawatu, New Zealand. PhD thesis, Massey University, 2022.
- Towards Evaluating AI Systems for Moral Status Using Self-Reports, November 2023. URL http://arxiv.org/abs/2311.08576. arXiv:2311.08576 [cs].
- Gualtiero Piccinini. Non-Computational Functionalism: Computation and the Function of Consciousness. In 2018 Conference on Cognitive Computational Neuroscience, Philadelphia, Pennsylvania, USA, 2018. Cognitive Computational Neuroscience. doi: 10.32470/CCN.2018.1022-0. URL https://ccneuro.org/2018/Papers/ViewPapers.asp?PaperNum=1022.
- The RL/LLM Taxonomy Tree: Reviewing Synergies Between Reinforcement Learning and Large Language Models, February 2024. URL http://arxiv.org/abs/2402.01874. arXiv:2402.01874 version: 1.
- Hilary Putnam. Psychological Predicates. In William H. Capitan and Daniel D. Merrill, editors, Art, Mind, and Religion, pages 37–48. University of Pittsburgh Press, 1967. URL https://philpapers.org/rec/PUTPP-2.
- James Rachels. The end of life: euthanasia and morality. Studies in bioethics. Oxford University Press, Oxford [Oxfordshire] ; New York, 1986. ISBN 978-0-19-217746-9 978-0-19-286070-5.
- The revised International Association for the Study of Pain definition of pain: concepts, challenges, and compromises. Pain, 161(9):1976–1982, September 2020. ISSN 1872-6623. doi: 10.1097/j.pain.0000000000001939.
- John Rawls. A Theory of Justice: Original Edition. Harvard University Press, 1971. ISBN 978-0-674-04260-5 978-0-674-88010-8. doi: 10.2307/j.ctvjf9z6v. URL http://www.jstor.org/stable/10.2307/j.ctvjf9z6v.
- Michael Rescorla. The Computational Theory of Mind, 2020. URL https://plato.stanford.edu/archives/fall2020/entries/computational-mind/.
- Luke Roelofs. Sentientism, Motivation, and Philosophical Vulcans. Pacific Philosophical Quarterly, 104(2):301–323, 2023. ISSN 1468-0114. doi: 10.1111/papq.12420. URL https://onlinelibrary.wiley.com/doi/abs/10.1111/papq.12420. _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/papq.12420.
- Observational scaling laws and the predictability of language model performance, 2024. URL https://arxiv.org/abs/2405.10938.
- Learning To Retrieve Prompts for In-Context Learning, May 2022. URL http://arxiv.org/abs/2112.08633. arXiv:2112.08633 [cs].
- William Ruddick. "Biographical Lives" Revisited and Extended. The Journal of Ethics, 9(3):501–515, October 2005. ISSN 1572-8609. doi: 10.1007/s10892-005-3520-2. URL https://doi.org/10.1007/s10892-005-3520-2.
- Artificial intelligence: a modern approach. Prentice Hall series in artificial intelligence. Prentice Hall, Upper Saddle River, 3rd ed edition, 2010. ISBN 978-0-13-604259-4.
- AI Rights for Human Safety, August 2024. URL https://papers.ssrn.com/abstract=4913167.
- Markus Schlosser. Agency, 2019. URL https://plato.stanford.edu/archives/win2019/entries/agency/.
- Food, animals, and the environment: an ethical approach. Routledge/Taylor & Francis Group, Abingdon, Oxon ; New York, NY, 2018. ISBN 978-1-138-80111-0 978-1-138-80112-7.
- Susan Schneider. Artificial You: AI and the Future of Your Mind. Princeton University Press, Princeton, 2019. ISBN 978-0-691-18014-4 978-0-691-19777-7.
- Susan Schneider. Emergent Spacetime, the Megastructure Problem, and the Metaphysics of the Self. Philosophy East and West, 74(2):314–332, 2024. doi: 10.1353/pew.2024.a925196.
- Mastering Atari, Go, chess and shogi by planning with a learned model. Nature, 588(7839):604–609, December 2020. ISSN 1476-4687. doi: 10.1038/s41586-020-03051-4. URL https://www.nature.com/articles/s41586-020-03051-4. Publisher: Nature Publishing Group.
- E. Schwitzgebel. Phenomenal Consciousness, Defined and Defended as Innocently as I Can Manage. Journal of Consciousness Studies, 23(11-12):224–235, January 2016.
- Eric Schwitzgebel. The Coming Robot Rights Catastrophe | Blog of the APA, 2023a. URL https://blog.apaonline.org/2023/01/12/the-coming-robot-rights-catastrophe/. 2023a.
- Eric Schwitzgebel. The Full Rights Dilemma for AI Systems of Debatable Moral Personhood. ROBONOMICS: The Journal of the Automated Economy, 4:32–32, 2023b. ISSN 2683-099X. URL https://journal.robonomics.science/index.php/rj/article/view/32. 2023b.
- A Defense of the Rights of Artificial Intelligences. Midwest Studies in Philosophy, 39(1):98–119, 2015. doi: 10.1111/misp.12032.
- Designing AI with Rights, Consciousness, Self-Respect, and Freedom. In Ethics of Artificial Intelligence, pages 459–479. Oxford University Press, September 2020. ISBN 978-0-19-090503-3 978-0-19-090507-1. doi: 10.1093/oso/9780190905033.003.0017. URL https://academic.oup.com/book/33540/chapter/287907290.
- Jeff Sebo. Agency and Moral Status. Journal of Moral Philosophy, 14(1):1–22, 2017. doi: 10.1163/17455243-46810046.
- Jeff Sebo. The Moral Problem of Other Minds. The Harvard Review of Philosophy, 25:51–70, 2018. doi: 10.5840/harvardreview20185913.
- Jeff Sebo. Kantianism for humans, utilitarianism for nonhumans? Yes and no. Philosophical Studies, 180(4):1211–1230, 2022. ISSN 0031-8116, 1573-0883. doi: 10.1007/s11098-022-01835-0. URL https://link.springer.com/10.1007/s11098-022-01835-0.
- Jeff Sebo. The Moral Circle. WW Norton, 2025.
- Moral consideration for AI systems by 2030. AI and Ethics, December 2023. ISSN 2730-5953, 2730-5961. doi: 10.1007/s43681-023-00379-1. URL https://link.springer.com/10.1007/s43681-023-00379-1.
- Anil Seth. Being you: a new science of consciousness. Dutton, New York, New York, 2021. ISBN 978-1-5247-4287-4.
- Anil Seth. Why Conscious AI Is a Bad, Bad Idea. Nautilus, May 2023. URL https://nautil.us/why-conscious-ai-is-a-bad-bad-idea-302937/.
- Anil Seth. Conscious artificial intelligence and biological naturalism, June 2024. URL https://osf.io/tz6an.
- Theories of consciousness. Nature Reviews Neuroscience, 23(7):439–452, 2022.
- Can ai scaling continue through 2030?, 2024. URL https://epochai.org/blog/can-ai-scaling-continue-through-2030. Accessed: 2024-10-30.
- Artificial Intelligence and the Common Sense of Animals. Trends in Cognitive Sciences, 24(11):862–872, November 2020. ISSN 1364-6613. doi: 10.1016/j.tics.2020.09.002. URL https://www.sciencedirect.com/science/article/pii/S1364661320302163.
- The Global Workspace Needs Metacognition. Trends in Cognitive Sciences, 23(7):560–571, July 2019. ISSN 13646613. doi: 10.1016/j.tics.2019.04.007. URL https://linkinghub.elsevier.com/retrieve/pii/S1364661319300993.
- Supra-personal cognitive control and metacognition. Trends in Cognitive Sciences, 18(4):186–193, April 2014. ISSN 1879-307X. doi: 10.1016/j.tics.2014.01.006.
- Joshua Shepherd. Consciousness and moral status. Routledge focus on philosophy. Routledge, Taylor & Francis Group, London New York, 2018. ISBN 978-0-367-73437-4 978-1-138-22161-1 978-1-315-39634-7.
- Henry Shevlin. General Intelligence: An Ecumenical Heuristic for Artificial Consciousness Research? Journal of Artificial Intelligence and Consciousness, 07(02):245–256, September 2020. ISSN 2705-0785, 2705-0793. doi: 10.1142/S2705078520500149. URL https://www.worldscientific.com/doi/abs/10.1142/S2705078520500149.
- Henry Shevlin. How Could We Know When a Robot was a Moral Patient? Cambridge Quarterly of Healthcare Ethics, 30(3):459–471, July 2021. ISSN 0963-1801, 1469-2147. doi: 10.1017/S0963180120001012. URL https://www.cambridge.org/core/product/identifier/S0963180120001012/type/journal_article.
- Derek Shiller. Functionalism, Integrity, and Digital Consciousness. Synthese, 203(2):1–20, 2024. doi: 10.1007/s11229-023-04473-z. Publisher: Springer Verlag.
- Reflexion: Language Agents with Verbal Reinforcement Learning, October 2023. URL http://arxiv.org/abs/2303.11366. arXiv:2303.11366.
- Sharing the World with Digital Minds. In Steve Clarke, Hazem Zohny, and Julian Savulescu, editors, Rethinking Moral Status, pages 306–326. Oxford University PressOxford, 1 edition, August 2021. ISBN 978-0-19-289407-6 978-0-19-191520-8. doi: 10.1093/oso/9780192894076.003.0018. URL https://academic.oup.com/book/41245/chapter/350760172.
- Reward is enough. Artificial Intelligence, 299:103535, October 2021. ISSN 0004-3702. doi: 10.1016/j.artint.2021.103535. URL https://www.sciencedirect.com/science/article/pii/S0004370221000862.
- Toni Sims. The Thought Experiments are Rigged: Mechanistic Understanding Inhibits Mentalistic Understanding (as Adleberg, T.), August 2013. URL https://scholarworks.gsu.edu/philosophy_theses/141. Publisher: Georgia State University.
- Peter Singer. Practical ethics. Cambridge University Press, Cambridge New York, 3rd ed edition, 2011. ISBN 978-0-521-88141-8 978-0-521-70768-8.
- Declan Smithies. Hedonic Consciousness and Moral Status. forthcoming.
- Defining and assessing animal pain. Animal Behaviour, 97:201–212, November 2014. ISSN 0003-3472. doi: 10.1016/j.anbehav.2014.09.007. URL https://www.sciencedirect.com/science/article/pii/S0003347214003431.
- Nicholas H. Steneck. Role of the Institutional Animal Care and Use Committee in Monitoring Research. Ethics & Behavior, 7(2):173–184, June 1997. ISSN 1050-8422. doi: 10.1207/s15327019eb0702_8. URL https://doi.org/10.1207/s15327019eb0702_8. Publisher: Routledge _eprint: https://doi.org/10.1207/s15327019eb0702_8.
- Cognitive Architectures for Language Agents, March 2024. URL http://arxiv.org/abs/2309.02427. arXiv:2309.02427 [cs].
- Reinforcement Learning: An Introduction. The MIT Press, 2015.
- Human-timescale adaptation in an open-ended task space, 2023. URL https://arxiv.org/abs/2301.07608.
- Nitasha Tiku. Google engineer Blake Lemoine thinks its LaMDA AI has come to life - The Washington Post. June 2022. URL https://www.washingtonpost.com/technology/2022/06/11/google-ai-lamda-blake-lemoine/.
- Brian Tomasik. Risks of Astronomical Future Suffering. Technical report, Center on Long-Term Risk, 2011.
- Through the Chat Window and Into the Real World: Preparing for AI Agents. Technical report, CSET, October 2024. URL https://cset.georgetown.edu/publication/through-the-chat-window-and-into-the-real-world-preparing-for-ai-agents/.
- A. M. Turing. Computing Machinery and Intelligence. Mind, LIX(236):433–460, October 1950. ISSN 1460-2113, 0026-4423. doi: 10.1093/mind/LIX.236.433. URL https://academic.oup.com/mind/article/LIX/236/433/986238.
- Michael Tye. Tense Bees and Shell-Shocked Crabs: Are Animals Conscious? Oxford University Press USA, New York, US, 2016.
- Susan Schneider’s Proposed Tests for Ai Consciousness: Promising but Flawed. Journal of Consciousness Studies, 28(5-6):121–144, 2021.
- Deep learning and the Global Workspace Theory. Trends in Neurosciences, 44(9):692–704, September 2021. ISSN 1878-108X. doi: 10.1016/j.tins.2021.04.005.
- Paul F. M. J. Verschure. Synthetic consciousness: the distributed adaptive control perspective. Philosophical Transactions of the Royal Society B: Biological Sciences, 371(1701):20150448, August 2016. ISSN 0962-8436, 1471-2970. doi: 10.1098/rstb.2015.0448. URL https://royalsocietypublishing.org/doi/10.1098/rstb.2015.0448.
- Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature, 575(7782):350–354, November 2019. ISSN 1476-4687. doi: 10.1038/s41586-019-1724-z. URL https://www.nature.com/articles/s41586-019-1724-z. Publisher: Nature Publishing Group.
- Voyager: An Open-Ended Embodied Agent with Large Language Models, October 2023. URL http://arxiv.org/abs/2305.16291. arXiv:2305.16291 [cs].
- A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345, March 2024. ISSN 2095-2236. doi: 10.1007/s11704-024-40231-1. URL https://doi.org/10.1007/s11704-024-40231-1.
- Marcel Wissenburg. Human-animal amity and reciprocity. AMITY: The Journal of Friendship Studies, 2(1):4–17, 2014.
- Thomas Woodside. Examples of AI Improving AI. Technical report, Center for AI Safety, 2023. URL https://ai-improving-ai.safe.ai/.
- Intelligent agents: Theory and practice. The knowledge engineering review, 10(2):115–152, 1995. URL https://www.cambridge.org/core/journals/knowledge-engineering-review/article/intelligent-agents-theory-and-practice/CF2A6AAEEA1DBD486EF019F6217F1597. Publisher: Cambridge University Press.
- Outracing champion Gran Turismo drivers with deep reinforcement learning. Nature, 602(7896):223–228, February 2022. ISSN 1476-4687. doi: 10.1038/s41586-021-04357-7.
- Overview of Neurological Mechanism of Pain Profile Used for Animal "Pain-Like" Behavioral Study with Proposed Analgesic Pathways. International Journal of Molecular Sciences, 21(12):4355, June 2020. ISSN 1422-0067. doi: 10.3390/ijms21124355.
- ReAct: Synergizing Reasoning and Acting in Language Models, March 2023. URL http://arxiv.org/abs/2210.03629. arXiv:2210.03629 [cs].
- Fine-tuning language models from human preferences, 2020. URL https://arxiv.org/abs/1909.08593.