Empowerment Objective in Agent Control

• Empowerment is an information-theoretic measure that quantifies how an agent’s choices influence its observable environmental states.
• Estimation techniques such as Monte Carlo methods, variational bounds, and Gaussian process models enable empowerment computation in complex, continuous systems.
• Applications span reinforcement learning, neural cellular automata, and assistive technologies, where empowerment drives exploration, stability, and adaptive representation learning.

The empowerment objective is an information-theoretic, task-independent or domain-independent objective that quantifies how much influence an agent can exert on its environment through its actions, restricted to effects that are available to its own sensors. Across the cited literature, empowerment appears as a channel-capacity functional, a mutual-information objective over action–outcome trajectories, an auxiliary evolutionary objective, an intrinsic reward for reinforcement learning, and, more recently, a principle for learning control-relevant representations and even curating internal knowledge structures (Salge et al., 2013).

1. Information-theoretic formulation

A standard formulation defines empowerment as the channel capacity from actions to future sensed states. In the notation used in the overview chapter, this is

$\mathfrak{E} := C(A\to S) = \max_{p(a)} I(A; S),$

and, in a context-dependent form,

$\mathfrak{E}(r) = \max_{p(a)} I(S; A \mid r).$

In continuous agent–environment systems, the corresponding state-dependent quantity is written as

$$C(\mathbf{x}) := \max_{p(\vec{a})} I(\mathcal{X}'; \mathcal{A} \mid \mathbf{x}),$$

where the maximization is over distributions on action sequences and the successor state is conditioned on the current state $\mathbf{x}$ (Jung et al., 2012).

This formulation makes empowerment a generalization of controllability and observability. The 2012 continuous-systems treatment states that empowerment measures how much influence an agent has on its environment, but only that influence that can be sensed by the agent sensors; it is therefore an information-theoretic generalization of joint controllability and observability. In deterministic discrete systems, the 2013 introduction further notes that empowerment reduces to the logarithm of the number of distinct sensor states that can be reached by an action sequence, which gives the concept an immediate operational interpretation in terms of distinguishable options (Salge et al., 2013).

The same formalism reappears in distributed systems. In neural cellular automata (NCA), empowerment is defined as maximum time-lagged mutual information between agents’ actions and future sensations. The 2023 NCA study writes

$\mathfrak{E}(k) = I(A_0^{k}, S_{N-k}^{N}),$

with $k$ as the time horizon or lag between action and sensation. This shifts empowerment from a single embodied agent to a distributed sensorimotor system in which each cell is treated as an agent (Grasso et al., 2023).

2. Estimation and approximation

The core computational difficulty is that empowerment is a maximized mutual information over action-conditioned future-state distributions. In small discrete settings, this can be handled directly, but in continuous and unknown-transition settings the cited literature relies on approximation. The continuous-systems formulation uses Monte-Carlo approximation for the integral form of empowerment and Gaussian process regression with iterated forecasting to address initially unknown state transition probabilities; it also uses the Blahut–Arimoto algorithm for channel-capacity maximization (Jung et al., 2012).

A major line of work replaces exact computation with variational bounds. The real-time control formulation introduces a variational lower bound based on a planning distribution $q(\mathbf{u}\mid \mathbf{z}',\mathbf{z})$ and a source distribution $\omega(\mathbf{u}\mid\mathbf{z})$, so that empowerment can be optimized jointly with a policy. This permits amortized learning of empowerment-maximizing policies in nonlinear continuous spaces, including when system dynamics are learned from raw data through a Deep Variational Bayes Filter (Karl et al., 2017).

Other estimators target sample efficiency or high-dimensional observations. Empowerment-driven exploration with MINE estimates mutual information by combining a forward dynamics model with Mutual Information Neural Estimator, using the empowerment estimate as an intrinsic reward in sparse-reward settings (Kumar, 2018). “Efficient Empowerment Estimation for Unsupervised Stabilization” replaces variational lower bounds with a learned Gaussian channel representation,

$O = G(s)\cdot A + \eta,$

so that empowerment can be computed by convex optimization through a water-filling solution; the paper reports lower sample complexity, more stable training, and empowerment estimation from images (Zhao et al., 2020). “Latent-Predictive Empowerment” replaces the usual mutual information between skills and states with a latent-predictive objective that only requires a simpler latent-predictive model rather than a full simulator of the environment, and empirically learns similar-sized skillsets as the leading empowerment algorithm that assumes access to a model of the transition dynamics (Levy et al., 2024).

3. Distributed morphogenesis and neural cellular automata

The NCA literature gives a particularly clear example of empowerment as an auxiliary objective rather than a terminal task objective. “Empowered Neural Cellular Automata” extends empowerment to a distributed, multi-agent sensorimotor system embodied by an NCA and incorporates it into Age-Fitness Pareto Optimization. In that setting, each cell’s action is its signaling output, each cell’s sensor is the average signal of its Von Neumann neighbors, and combined optimization of shape matching and empowerment produces higher fitness than evolving for morphogenesis alone; the paper further reports a synergistic relationship between morphogenesis and empowerment (Grasso et al., 2022).

The 2023 follow-up isolates the effect of time horizon. Its NCA empowerment is

$$\mathfrak{E}(k)= - \sum_{a_{ijn} \in A_0^{k},\,s_{ijn} \in S_{N-k}^{N}} p(a_{ijn}, s_{ijn}) \log_2 \frac{p(a_{ijn}, s_{ijn})}{p(a_{ijn})p(s_{ijn})},$$

where action at time $\mathfrak{E}(r) = \max_{p(a)} I(S; A \mid r).$0 is paired with the sensed value at time $\mathfrak{E}(r) = \max_{p(a)} I(S; A \mid r).$1. The study evaluates three treatments: Bi-loss (Age + Loss), Tri-loss-empowerment (Age + Loss + Empowerment $\mathfrak{E}(r) = \max_{p(a)} I(S; A \mid r).$2), and Tri-loss (Age + Loss in the early phase + Loss in the late phase) (Grasso et al., 2023).

Its main empirical result is that shorter time delays result in marked improvements over empowerment with longer delays, when compared to evolutionary selection only for homeostasis. Short-term empowerment with $\mathfrak{E}(r) = \max_{p(a)} I(S; A \mid r).$3 leads to lower loss, whereas the benefit declines gradually with longer time horizons and approaches that of the controls. The same study further reports that short-term empowered NCA are more stable, show lower cell transiency, typically produce cohesive, single-component shapes, and generalize better to unseen homeostatic challenges; when used as seeds for further evolution on new targets, they adapt faster and achieve lower loss than controls (Grasso et al., 2023).

These results make the time horizon itself part of the objective design. A plausible implication is that empowerment is not merely a generic regularizer in distributed morphogenesis; the temporal scale at which action–sensation coupling is optimized can determine whether the objective accelerates or attenuates the discovery of homeostatic local rules.

4. Reinforcement learning, exploration, and control

In reinforcement learning, empowerment is typically used as an intrinsic motivation signal. The continuous-control literature argues that greedily maximizing local empowerment drives agents toward states where they have more options and tend to avoid deaths, traps, or dead-ends. In well-known control tasks such as inverted pendulum, bicycle, and acrobot, empowerment-maximizing policies often lead to balancing or swing-up without explicit extrinsic rewards (Jung et al., 2012).

This connection is made explicit in unsupervised control. “Unsupervised Real-Time Control through Variational Empowerment” uses negative empowerment as the cost in a stochastic optimal-control framework, so that the policy is trained to maximize cumulative empowerment over trajectories; the reported behaviors include pendulum swing-up and balancing, movement toward the center in a ball-in-a-box task, and unsupervised balance in a bipedal system (Karl et al., 2017). “Efficient Empowerment Estimation for Unsupervised Stabilization” makes the same point more narrowly: maximizing mutual information between agent actuators and future states coincides with stabilization at upright positions in several dynamical systems (Zhao et al., 2020).

In meta-RL, empowerment can be directed toward task identification rather than environment control per se. MetaCURE defines an exploration objective

$\mathfrak{E}(r) = \max_{p(a)} I(S; A \mid r).$4

the mutual information between exploration experiences and the latent task variable. The paper separates exploration and exploitation policies and reports significant outperformance over state-of-the-art baselines on sparse-reward MuJoCo locomotion tasks and Meta-World tasks (Zhang et al., 2020).

More recent work uses empowerment as a pre-training signal. “Information-Theoretic Policy Pre-Training with Empowerment” introduces discounted empowerment,

$\mathfrak{E}(r) = \max_{p(a)} I(S; A \mid r).$5

to balance control across short and long horizons. The paper argues that empowerment-based pre-training yields data-efficient downstream task adaptation and improved adaptability, especially with long horizons (Schneider et al., 7 Oct 2025).

5. Representation learning and controllable abstractions

A recurring criticism of standard visual or latent-state learning is that it does not distinguish functionally relevant factors from distractors. The empowerment literature answers this by tying representation quality to controllability. In visual model-based RL, “INFOrmation Prioritization through EmPOWERment” adds the conditional mutual information term

$\mathfrak{E}(r) = \max_{p(a)} I(S; A \mid r).$6

to prioritize information that is correlated with action. Under a bottleneck, this causes the latent state to capture controllable aspects of the environment first, while also promoting faster exploration in sparse-reward tasks (Bharadhwaj et al., 2022).

A stronger theoretical claim appears in “Learning to Perceive the World Through Control: Empowerment-Based Representation Learning”. That paper states that empowerment agents induce two distinct representations—forward and backward—that capture complementary aspects of the state, and both of which are invariant to control-irrelevant features. It further argues that interaction aimed at maximizing control is essential for learning useful invariance properties, aligning the approach with causal learning rather than passive representation learning from fixed datasets (Bastankhah et al., 28 May 2026).

This line of work also motivates latent or internal variants of the objective. “Latent-Predictive Empowerment” moves the skill-diversity objective into latent predictive space, while preserving a principled connection to the usual mutual information between skills and terminal states (Levy et al., 2024). “Agent-centric learning: from external reward maximization to internal knowledge curation” goes further by defining representational empowerment as

$\mathfrak{E}(r) = \max_{p(a)} I(S; A \mid r).$7

which measures an agent’s ability to controllably maintain and diversify its own knowledge structures rather than its ability to control external state (Zhou et al., 29 Jul 2025).

Taken together, these works suggest a shift from empowerment as a property of reachable world states to empowerment as a property of controllable abstraction. This suggests that the objective can function not only as an exploration bonus but also as a criterion for what information should be retained in a learned model.

6. Assistance, causality, and broader human-centered formulations

In assistive settings, empowerment has been proposed as a goal-agnostic objective for increasing a human’s control rather than inferring the human’s latent goal. “AvE: Assistance via Empowerment” formalizes human empowerment as

$\mathfrak{E}(r) = \max_{p(a)} I(S; A \mid r).$8

and augments an RL reward with an estimate of the human’s empowerment. Because exact estimation is computationally hard in continuous domains, the paper also proposes an empowerment-inspired proxy based on the variance of final states reached by sampled action sequences; in a simulated teleoperation study, empowerment-based assistance improves success rate and preserves the person’s autonomy and ability to achieve any eventual state (Du et al., 2020).

Empowerment has also been coupled directly to causal structure learning. “Towards Empowerment Gain through Causal Structure Learning in Model-Based RL” defines

$\mathfrak{E}(r) = \max_{p(a)} I(S; A \mid r).$9

with $$C(\mathbf{x}) := \max_{p(\vec{a})} I(\mathcal{X}'; \mathcal{A} \mid \mathbf{x}),$$0 the learned causal structure, and then maximizes the gain between empowerment under the causal model and empowerment under a dense model. The proposed ECL framework alternates causal model learning, empowerment-driven exploration, and downstream policy learning with an intrinsic curiosity reward, and is reported to improve causal discovery, sample efficiency, and asymptotic performance across six environments (Cao et al., 14 Feb 2025).

The same action–outcome logic appears in work on human causal learning. “Empowerment Gain and Causal Model Construction” defines empowerment as

$$C(\mathbf{x}) := \max_{p(\vec{a})} I(\mathcal{X}'; \mathcal{A} \mid \mathbf{x}),$$1

and argues that children and adults are sensitive to controllability and variability in their causal interventions. The empirical studies summarized in the paper report preferences for controllable and variable machines rather than machines that are merely variable or merely controllable, linking empowerment to effective causal intervention design (Yiu et al., 9 Dec 2025).

Outside agent-centered AI, the term is also used in broader sociotechnical assessment. “Metrics for Assessing Inclusivity and Empowerment of People for Supporting the Design of Inclusive Product Lifecycles” defines empowerment as a context-dependent, iterative process and proposes candidate metrics such as

$$C(\mathbf{x}) := \max_{p(\vec{a})} I(\mathcal{X}'; \mathcal{A} \mid \mathbf{x}),$$2

with $$C(\mathbf{x}) := \max_{p(\vec{a})} I(\mathcal{X}'; \mathcal{A} \mid \mathbf{x}),$$3 and the inclusivity metric $$C(\mathbf{x}) := \max_{p(\vec{a})} I(\mathcal{X}'; \mathcal{A} \mid \mathbf{x}),$$4 showing the highest positive, significant correlation across ten case studies. This marks a different usage of “empowerment”: not channel capacity, but stakeholder power and agency within lifecycle processes (Yaldiz et al., 2024).

7. Limitations, trade-offs, and emerging directions

Despite its generality, empowerment remains difficult to compute and easy to mis-specify. The continuous-control and stabilization literature repeatedly notes that exact or sample-based mutual-information estimation is challenging in nonlinear continuous spaces, that variational lower bounds can be biased and unstable in training, and that many skill-based methods require either a simulator or a model of the transition dynamics (Zhao et al., 2020). The NCA literature adds a different sensitivity: the action–sensation lag $$C(\mathbf{x}) := \max_{p(\vec{a})} I(\mathcal{X}'; \mathcal{A} \mid \mathbf{x}),$$5 is not an innocuous hyperparameter, and longer horizons can reduce the efficacy of empowerment as an auxiliary objective (Grasso et al., 2023).

A second limitation is normative rather than computational. “When Empowerment Disempowers” shows that, in multi-human settings, assistive RL agents optimizing for one human’s empowerment can significantly reduce another human’s environmental influence and rewards. The paper formalizes this as disempowerment, introduces the Disempower-Grid test suite, and reports that joint empowerment mitigates disempowerment at the cost of the user’s reward (Yang et al., 6 Nov 2025). This directly challenges the assumption that a goal-agnostic objective that appears aligned in a single-agent setting remains aligned in multi-agent settings.

A third open direction concerns the locus of control. Classical empowerment is environment-centric: it measures control over future sensed states. Representational empowerment shifts the objective inward to the ability to shape one’s own internal knowledge structures, with preparedness rather than direct environmental influence as the target (Zhou et al., 29 Jul 2025). This suggests an unresolved question: whether the most useful long-horizon form of empowerment for generally adaptive systems is external, internal, or a coupling of both.

Across these strands, empowerment remains a unifying objective for controllability, exploration, and option-preserving behavior. The literature does not present it as a single algorithmic recipe, but as a family of mutual-information objectives whose empirical behavior depends on embodiment, temporal horizon, estimator design, and social context.