The Tao of Agency: Autotelic AI, Embedded Agency and Dissolution of the Self

Abstract: Most artificial intelligence systems are built on the assumption that goals are exogenous and specified by the designer. Exploring what happens when an agent begins generating its own goals opens the field of autotelic AI. Agents are expected not merely to pursue objectives but to discover them. In this article, we trace its consequences through intrinsic motivation, resource-driven priors, causal-interventional learning, homeostasis, and embeddedness; the last of which is found to be a necessary but not sufficient condition for autotelic agency. Embeddedness individuates the agent at the cost of revealing that the individuation is non-unique, such that the same dynamics admit many valid partitions, each defining a different candidate self. The deepest problem with autotelic AI is therefore not how the agent generates goals, but how it generates and relativizes the self to which the goals are assigned. The agent must believe in its own boundary in order to act, and see through that boundary in order to understand. We consolidate these developments into a single framework and extend it along three directions: a quantum formulation in which the agent-environment cut becomes physical, a philosophical reading against non-dual contemplative traditions, and a concrete LLM-based agentic instantiation.

• The paper presents a unified formal framework for autotelic AI by redefining goal autonomy and the agent-environment boundary.
• It employs causal intervention and resource-driven priors to challenge traditional exogenous reward systems in AI models.
• The research bridges classical and quantum perspectives, drawing on non-dual philosophical insights and cybernetics to understand self-dissolution.

Autotelic AI, Embedded Agency, and the Dissolution of the Self: A Formal Analysis

Introduction

This work ("The Tao of Agency: Autotelic AI, Embedded Agency and Dissolution of the Self" (2606.19924)) provides a comprehensive technical and philosophical investigation into the notion of autotelic agency in artificial intelligence. It systematically identifies the limitations in current paradigms, where agents are rigidly bound to objectives exogenously defined by designers. The paper advances a self-referential framework in which agents generate, revise, and relativize both their goals and their own individual boundaries, thereby dissolving privileged notions of the self. This analysis articulates necessary conditions for autotelic agency, elucidates the implications of resource priors and causal modeling, and connects these to both Markovian and quantum boundaries, ultimately presenting a unified tuple for autotelic agentic systems.

Exogenous Goals and the Structural Limitations of Classical AI

Traditional AI and RL architectures are defined by an exogenous objective, whether it be a reward, a loss function, or a winning predicate. The architecture optimizes a designer-supplied goal, with recent successes in deep learning and RL (e.g., AlphaZero, AlphaFold, LLMs) demonstrating high performance within these constraints. However, misalignment phenomena such as reward hacking and misspecification elucidate the brittleness and underspecification of this paradigm. As tasks transition towards open-endedness or autonomous scientific discovery, the artificial imposition of fixed objectives becomes operationally and conceptually inadequate.

Attempts to loosen these exogenous constraints—via self-referential code revision, open-ended evolution, or intrinsic motivation—have not solved the problem of genuine goal autonomy. All such systems inherit foundational assumptions regarding priors, diversity metrics, or proof-theoretic infrastructures.

Intrinsic Motivation: Progress and Incompleteness

Intrinsic motivation mechanisms—novelty, surprise, competence, learning progress, empowerment, free-energy minimization—replace task-driven rewards with functionals over agent belief states, e.g., prediction errors, Bayesian information gain, entropy maximization, empowerment, etc. This enables the agent to be operationally independent of a designer-supplied reward but still does not endogenize the space of goals itself. The agent's objective remains scalar, static, and privileged; it is surprised, empowered, or compresses, but does not fundamentally redefine or generate new dimensions of $G$ over which preferences may vary. Thus, intrinsic motivation is a necessary operational substrate for autotelic agency but does not constitute it.

Resource-Driven Priors and the Non-Neutrality of Representation

The search for “neutral” priors—uniform or Solomonoff-style priors—over goals fails because uniformity is representation-dependent, and algorithmic-information-theoretic priors such as Kolmogorov complexity are only justified for hypothesis inference, not goal preference. Real agents are physically embedded with bounded resources: energy, space, time, allowable approximation, and code length. Thus, a general resource functional, aggregating constraints via a weighting scheme (e.g., Levin complexity, logical depth, speed prior) may be appropriate.

However, this only schedules goals according to tractability or feasibility; it does not resolve the justificatory regress explaining why any such aggregate preference (or weighting) is rational, except with reference to resource profile. Thus, resource-driven curriculum learning aids skill acquisition but does not yield non-arbitrary valuation over goals.

Causal Intervention and the Grounding of the Goal Space

By reframing the agent-environment coupling in causal rather than correlational terms (drawing on algorithmic information dynamics), the candidate goal space $G$ may be defined as the set of controllable interventions accessible to the embedded agent. This maps the boundary between what is manipulable and what is not, but does not resolve which targets within $G$ are valuable. Ultimately, the homeostatic imperative is invoked: survival-preserving states (the viability set $V$) admit an operational fixpoint, as only agents that maintain $V$ persist to entertain goals—a principle grounded in cybernetics, theoretical biology, and active inference.

Embedded Agency: Markov Blankets, Multiplicity, and the Non-Uniqueness of the Self

Embedded agency departs from dualistic models, rendering the agent as an induced subsystem (with a boundary $b$, a Markov blanket) of the universal dynamics $F$. The Markov blanket provides a statistical (and, under certain physical assumptions, an energetic/informational) screening-off between internal and external, allowing for viable agentic partitions—but these are not unique. Any number of overlapping, hierarchically aligned, or divergent partitions may instantiate the “self,” each with distinct viability sets and operational goals. Constraints such as computational access, light cone limits, subsystem alignment, and delegation (amidst Löbian obstructions and self-reference) fragment the Cartesian goals and identities. Thus, embeddedness is necessary for autotelic agency, but the individuation of agents—and the distinction between agent and environment—emerges as a non-fundamental, context-dependent artifact.

Dissolution of Agency: Coarse-Graining, Narrative Self, and the Paradox of Operational Selfhood

The ambiguity of agent boundaries results in a multiplicity of Markov partitions. Formal measures for selecting preferred partitions include mutual information retained across coarse-grainings, causal integration (IIT $\Phi$), and predictive autonomy, but all are extrinsically justified. Cognitive science, philosophy of mind, and contemplative non-dual traditions converge on the view that the self is an emergent, operationally useful fiction (cf. phenomenology, narrative gravity, enactive mind, shunyata, Taoism). The autotelic agent, therefore, must act as if its own boundary is fundamental (for effective policy and value assignment), while recognizing this boundary is a representational hypothesis instrumental for (but not intrinsic to) cognition and agency. This tension is an intrinsic and irreducible feature of autotelic architecture.

Consolidated Framework

The theory is encapsulated by the tuple $(\mathcal{T}, G, p, C_a, V, b, M)$:

• $\mathcal{T}$: a goal-conditioned policy,
• $G$0: a goal space derived from causal coupling, not designer imposition,
• $G$1: an endogenous distribution over $G$2, shaped by the agent's resource functional $G$3 and viability constraint $G$4,
• $G$5: a Markov blanket defining the agent's operational boundary,
• $G$6: the viability set comprising subspaces compatible with continued agency,
• $G$7: the agent’s self-model, provisionally treating $G$8 as objective for policy, yet ultimately treating $G$9 as contingent.

Each factor is necessary, but mutually interdependent and insufficient in isolation.

Quantum, Philosophical, and Agentic Extensions

The work generalizes the framework into three key extensions:

1. Quantum Generalization: In quantum dynamics, agent-environment decompositions (i.e., partitions into subspaces, specification of Markov blankets, etc.) are basis-dependent, and measurement acts entangle observer and observed, enforcing the physical dissolution of boundaries. The agent's partition $G$0 now modulates quantum entanglement, and the resource and viability sets are quantum channels and reduced states, not classical distributions. This extension motivates formally rigorous quantum-autotelic agents, from quantum knowledge-seeking to fully embedded quantum autotelic agents.
2. Philosophical Correspondence: The operational/ontological split in agent self-model directly correlates with non-dual traditions (e.g., Madhyamaka's two truths, Daoist wuji, neutral monism). The framework explicitly denies both Advaitic metaphysical monism and Ajivika determinism, positing instead that agency is a representational, dynamically instantiated fiction—instrumentally indispensable, ontologically contingent.
3. Agentic AI Instantiation: The formalism is translated to an LLM-based AI agent whose core operation is intention continuation rather than scalar reward maximization. The agent generates, filters, and executes intentions under hard viability constraints, resource budgets, and dynamically maintained self-boundaries. Empirical autotelic metrics include coherence, boundary maintenance, goal diversity, and self-repair. This architecture closely aligns with process-based, model-predictive, or mixed deliberative-reactive agentic AI.

Conclusion

The paper delivers a unified formal, operational, and philosophical architecture for autotelic AI. It demonstrates that truly autotelic agents must generate not only their own goals but also their own selves, with no unique or absolute boundary between agent and environment. The indispensability of operational selfhood collides with its lack of ontological fundamentality, an irony that must be navigated, not eliminated. This insight has foundational implications, both for the design of robust, misalignment-resilient AGI and for the metaphysics of observerhood. Future research in quantum- and classically-embedded agency, scalable intention generation, and meta-agentic self-modeling will be required to translate the principles established herein into engineering practice and theoretical rigor.