Agent/Environment Plasticity & Empowerment

• Agent/Environment Plasticity and Empowerment is a framework that defines an agent’s adaptability to past inputs and its capacity to influence future states using information-theoretic measures.
• It explains how agents maximize their potential control by exploring states offering diverse sensorimotor outcomes, balancing the trade-off between rigidity and plasticity.
• The framework provides practical computational methods for both discrete and continuous systems, addressing key challenges in learning, exploration, and adaptive behavior.

Agent/environment plasticity and empowerment are foundational concepts in the study of intrinsically motivated intelligent systems. Plasticity quantifies the degree to which an agent can be influenced by its past observations—its sensitivity and adaptability to environmental input. Empowerment formalizes the agent’s capacity to influence future observations or states—its effective control over its own sensorimotor experience. Both notions are grounded in information theory, with empowerment classically measured by directed mutual information from actions to future observations, and plasticity characterized as the mirror quantity: how past observations inform future actions. This bidirectional structure not only underpins individual agency but governs the co-adaptive dynamics of agent and environment, with implications for exploration, learning, robustness, and open-ended behavioral evolution.

1. Information-Theoretic Foundations

Plasticity and empowerment are rigorously defined via generalized directed information (GDI), an extension of Massey’s directed information that accommodates arbitrary temporal intervals. For finite action and observation alphabets, let $A_{a:b}$ denote an action sequence over $[a, b]$, and $O_{c:d}$ an observation sequence over $[c, d]$.

• Empowerment:

$$E_{a:b \rightarrow c:d}(X, e) = \max_{X \in \mathcal{X}} I(A_{a:b} \rightarrow O_{c:d})$$

Measures the maximum information that an agent’s actions can direct into future observations under policy class $\mathcal{X}$ and environment $e$.

• Plasticity:

$$P_{a:b \rightarrow c:d}(X, \mathcal{E}) = \max_{e \in \mathcal{E}} I(O_{a:b} \rightarrow A_{c:d})$$

Captures the maximal influence of environmental observations on the agent’s future actions.

A key result is the Mirror Proposition: for any agent $X$ and environment $e$, empowerment of $X$ on $e$ is equal to the plasticity of $e$ with respect to $X$ (and vice versa), establishing a duality between control and adaptivity (Abel et al., 15 May 2025).

2. Agent and Environment Plasticity: Mechanisms and Consequences

Empowerment-driven agents act to maximize their potential for future influence, gravitating towards states where action choices result in maximal diversity of future perceptions. This capacity for agent-driven restructuring, or environmental plasticity, is particularly evident when agents can modify their surroundings (e.g., by building structures in spatial domains).

• In deterministic domains, empowerment is the channel capacity from action sequences to resulting sensor states, reducing to the logarithm of the number of distinct reachable percepts (Salge et al., 2014, Salge et al., 2013).
• Sparse sampling approximations enable empowerment calculation in high-dimensional or continuous environments, supporting complex behaviors such as constructing stairs or dams to reshape affordances (Salge et al., 2014).

Notably, what an agent constructs or chooses to modify depends intimately on its embodiment. Agents with differing morphologies (e.g., climbing vs. non-climbing) leave divergent environmental artefacts as a result of maximizing empowerment, reflecting the coupling between embodiment and environmental plasticity (Salge et al., 2014, Salge et al., 2013).

3. Plasticity–Empowerment Trade-Off and Theoretical Implications

Plasticity and empowerment stand in fundamental tension. For fixed action and observation alphabet sizes, the sum of achievable empowerment and plasticity is bounded:

$$E_{a:b \rightarrow c:d} + P_{a:b \rightarrow c:d} \leq m = \min\{(b-a)\log|\mathcal{O}|, (d-c)\log|\mathcal{A}|\}$$

with the bound tight: maximal empowerment implies minimal plasticity and vice versa. This trade-off manifests as a spectrum ranging from rigid, highly controlling agents (high empowerment, low plasticity) to passive, highly adaptable agents (low empowerment, high plasticity) (Abel et al., 15 May 2025).

This structure precisely captures the stability–plasticity dilemma: excessive empowerment yields rigidity and vulnerability to distribution shift, while absolute plasticity risks absence of cohesive goal-directed behavior. Agency in the full sense requires nonzero values of both quantities.

4. Computation and Practical Realization in Agent Architectures

Modern frameworks implement empowerment in discrete and continuous domains via channel-capacity–based objectives:

• In discrete or deterministic systems, empowerment is approximated by enumerating distinct sensor states reachable under sampled open-loop or closed-loop action sequences (Salge et al., 2014).
• In continuous control, mutual information between action sequences and future states is computed using Monte Carlo integration, with unknown dynamics modeled by Gaussian Processes and iterated forecasting. The Blahut–Arimoto algorithm is employed for channel-maximization (Jung et al., 2012).
• Practical algorithms substitute model errors or neural dynamics predictors as proxies for information-theoretic quantities, sharing representations across curiosity (information gain) and empowerment modules for sample-efficient learning (Abril et al., 2018).

For agent-internal plasticity, representational empowerment measures the agent’s capacity to controllably reconfigure its knowledge structures—maximizing channel capacity from internal transformation operations to modified representational states (Zhou et al., 29 Jul 2025). Architectures then employ meta-learning or policy-gradient methods to maximize this quantity, supporting agent adaptability even under persistent environment shift.

5. Bidirectional Information Flow: Unified Curiosity, Homeostasis, and Empowerment

Plasticity and empowerment together constitute the bidirectional flow of information between agent and environment, unifying multiple intrinsic drives:

• Curiosity/information gain: maximizes the flow of information from environment to agent; reward is proportional to conditional mutual information between next state and agent action, estimated via model error (Abril et al., 2018).
• Empowerment: maximizes flow from agent to environment, i.e., the agent’s actionable influence on its own future.
• Homeostatic regulation: introduces a penalty term that biases curiosity towards regions where agent has some familiarity, interpolating between exploration (heterostatic) and maintenance (homeostatic) behaviors (Abril et al., 2018).

A shared model architecture for these drives provides a principled basis for integrating multiple intrinsic motivations, facilitating faster learning and policy transfer.

6. Role of Plasticity and Empowerment in Learning, Exploration, and Adaptation

Both plasticity and empowerment function as intrinsic objective signals driving exploration and robust adaptation—even in the absence of extrinsic reward:

• In continuous control environments, empowerment alone yields behaviors classically regarded as optimal (e.g., balancing an inverted pendulum or acrobot) (Jung et al., 2012).
• During online learning, empowerment-guided agents prioritize regions with high controllability and observability, focusing exploration on dynamically salient regions while enabling efficient model acquisition (Jung et al., 2012).
• In plastic environments, empowerment maximization results in adaptive restructuring of both agent morphology and environmental affordances, leading to open-ended co-evolution and emergent complexity (Salge et al., 2014, Salge et al., 2013, Christov-Moore et al., 8 Oct 2025).

The agent/environment interplay, especially in multi-agent systems, induces an ongoing arms-race of affordance creation and consumption, promoting cumulative innovation and persistent novelty.

7. Extensions: Embodiment Constraints, Social Care, and Internal Knowledge Plasticity

Recent work expands these concepts by formalizing embodiment dependencies and sociocultural dynamics:

• Physical embodiment and “being-towards-death”: Homeostatic drives emerge from embodiment constraints, with empowerment operationalizing Nietzsche’s “will-to-power” as control over future survivability (Christov-Moore et al., 8 Oct 2025).
• Social care and open-endedness: Empowerment-driven agents in multi-agent environments show emergent care behaviors—such as expending energy to sustain conspecifics—when intrinsic objectives combine homeostasis and empowerment in the reward structure (Christov-Moore et al., 8 Oct 2025).
• Representational empowerment: By treating internal representational states as the substrate for empowerment, agents can maximize their latent adaptability to novel tasks while decoupling from direct environment manipulation (Zhou et al., 29 Jul 2025). This approach enables a trade-off between functional diversity and controllability within the agent’s own cognitive toolkit.

These extensions illustrate the centrality of agent/environment plasticity and empowerment not only to adaptive action and learning, but to the structure of agency, meta-cognition, and the emergence of social behaviors.

---

Key References:

• "Plasticity as the Mirror of Empowerment" (Abel et al., 15 May 2025)
• "A unified strategy for implementing curiosity and empowerment driven reinforcement learning" (Abril et al., 2018)
• "Changing the Environment Based on Empowerment as Intrinsic Motivation" (Salge et al., 2014)
• "Empowerment for Continuous Agent-Environment Systems" (Jung et al., 2012)
• "Agent-centric learning: from external reward maximization to internal knowledge curation" (Zhou et al., 29 Jul 2025)
• "The Contingencies of Physical Embodiment Allow for Open-Endedness and Care" (Christov-Moore et al., 8 Oct 2025)
• "Changing the Environment based on Intrinsic Motivation" (Salge et al., 2013)