Necessary and sufficient conditions for robust agency in AI

Characterize the necessary and sufficient computational states and processes for intentional, reflective, and rational agency in artificial systems, specifying which cognitive capacities (such as planning, memory, introspection, situational awareness, and abstract reasoning) must be present and how they must interact to realize each level of robust agency, so that assessments of moral patienthood based on agency can be principled and consistent.

Background

The report defines robust agency in three tiers—intentional, reflective, and rational—and argues that various cognitive capacities plausibly contribute to each tier. It connects robust agency to moral patienthood, noting that agent-centric bases for moral standing may apply even without consciousness on some views.

Despite active research in reinforcement learning and language agents that pursue goals, plan, and reflect, the authors emphasize that precise criteria for what constitutes each level of robust agency remain unsettled, making it difficult to evaluate when AI systems meet morally significant thresholds.