Dice Question Streamline Icon: https://streamlinehq.com

Which AI welfare attribution risk is more likely for specific AI system categories

Determine, for specific categories of AI systems such as large language model chatbots and embodied robots, whether over-attribution of welfare and moral patienthood or under-attribution of these properties is more likely, and identify the key conditions and factors that drive the relative likelihood in each category to inform proportionate precautionary policies.

Information Square Streamline Icon: https://streamlinehq.com

Background

The report argues that both over-attributing and under-attributing welfare and moral patienthood to AI systems carry grave harms. Over-attribution could divert scarce moral attention and resources and empower systems in ways that risk human interests, while under-attribution could lead to unnecessary neglect or harm to AI systems that might morally matter.

The authors highlight human biases—anthropomorphism and anthropodenial—that can push in opposite directions depending on system design and use (e.g., charismatic chatbots versus opaque non-LLM systems). As AI systems become more advanced and socially embedded, discerning which risk predominates for particular kinds of systems becomes critical for designing responsible assessments and policies.

References

At present, it is an open question which kind of risk will be more likely for particular kinds of AI systems, including seemingly conscious and charismatic systems like robots and chatbots.

Taking AI Welfare Seriously (2411.00986 - Long et al., 4 Nov 2024) in Section 1.2 “The risks of mishandling AI welfare”