Necessary and sufficient conditions for robust agency in AI

Characterize the necessary and sufficient computational states and processes for intentional, reflective, and rational agency in artificial systems, specifying which cognitive capacities (such as planning, memory, introspection, situational awareness, and abstract reasoning) must be present and how they must interact to realize each level of robust agency, so that assessments of moral patienthood based on agency can be principled and consistent.

Background

The report defines robust agency in three tiers—intentional, reflective, and rational—and argues that various cognitive capacities plausibly contribute to each tier. It connects robust agency to moral patienthood, noting that agent-centric bases for moral standing may apply even without consciousness on some views.

Despite active research in reinforcement learning and language agents that pursue goals, plan, and reflect, the authors emphasize that precise criteria for what constitutes each level of robust agency remain unsettled, making it difficult to evaluate when AI systems meet morally significant thresholds.

References

What it takes to be an intentional, reflective, and/or rational agent is not clear.

— Taking AI Welfare Seriously  (2411.00986 - Long et al., 2024) in Subsection “Robust agency in near-future AI,” subsubsection “Will some AI systems be robustly agentic in the near future?”

Although GenAI systems can produce content, evaluate the quality of the content they generate, and adapt their generated content based on the results of their evaluations without direct human involvement, the question exists as to whether machines can be considered as acting with agency at the same level as humans (Riemer & Peter, 2024).

— Reclaiming Epistemic Agency: A Critical Framework for Human-Generative AI Co-Agency in Education  (2608.26937 - Poudyal, 27 Aug 2026) in Section 1, Introduction