---
title: Empowerment Objective in Agent Control
url: https://www.emergentmind.com/topics/empowerment-objective
type: topic
---

# Empowerment Objective in Agent Control

The empowerment objective is an information-theoretic, task-independent or domain-independent objective that quantifies how much influence an agent can exert on its environment through its actions, restricted to effects that are available to its own sensors. Across the cited literature, empowerment appears as a channel-capacity functional, a mutual-information objective over action–outcome trajectories, an auxiliary evolutionary objective, an intrinsic reward for reinforcement learning, and, more recently, a principle for learning control-relevant representations and even curating internal knowledge structures [1310.1863].

## 1. Information-theoretic formulation

A standard formulation defines empowerment as the channel capacity from actions to future sensed states. In the notation used in the overview chapter, this is
\[
\mathfrak{E} := C(A\to S) = \max_{p(a)} I(A; S),
\]
and, in a context-dependent form,
\[
\mathfrak{E}(r) = \max_{p(a)} I(S; A \mid r).
\]
In continuous agent–environment systems, the corresponding state-dependent quantity is written as
\[
C(\mathbf{x}) := \max_{p(\vec{a})} I(\mathcal{X}'; \mathcal{A} \mid \mathbf{x}),
\]
where the maximization is over distributions on action sequences and the successor state is conditioned on the current state \(\mathbf{x}\) [1201.6583].

This formulation makes empowerment a generalization of controllability and observability. The 2012 continuous-systems treatment states that empowerment measures how much influence an agent has on its environment, but only that influence that can be sensed by the agent sensors; it is therefore an information-theoretic generalization of joint controllability and observability. In deterministic discrete systems, the 2013 introduction further notes that empowerment reduces to the logarithm of the number of distinct sensor states that can be reached by an action sequence, which gives the concept an immediate operational interpretation in terms of distinguishable options [1310.1863].

The same formalism reappears in distributed systems. In neural cellular automata (NCA), empowerment is defined as maximum time-lagged mutual information between agents’ actions and future sensations. The 2023 NCA study writes
\[
\mathfrak{E}(k) = I(A_0^{k}, S_{N-k}^{N}),
\]
with \(k\) as the time horizon or lag between action and sensation. This shifts empowerment from a single embodied agent to a distributed sensorimotor system in which each cell is treated as an agent [2305.15220].

## 2. Estimation and approximation

The core computational difficulty is that empowerment is a maximized mutual information over action-conditioned future-state distributions. In small discrete settings, this can be handled directly, but in continuous and unknown-transition settings the cited literature relies on approximation. The continuous-systems formulation uses Monte-Carlo approximation for the integral form of empowerment and Gaussian process regression with iterated forecasting to address initially unknown state transition probabilities; it also uses the Blahut–Arimoto algorithm for channel-capacity maximization [1201.6583].

A major line of work replaces exact computation with variational bounds. The real-time control formulation introduces a variational lower bound based on a planning distribution \(q(\mathbf{u}\mid \mathbf{z}',\mathbf{z})\) and a source distribution \(\omega(\mathbf{u}\mid\mathbf{z})\), so that empowerment can be optimized jointly with a policy. This permits amortized learning of empowerment-maximizing policies in nonlinear continuous spaces, including when system dynamics are learned from raw data through a Deep Variational Bayes Filter [1710.05101].

Other estimators target sample efficiency or high-dimensional observations. Empowerment-driven exploration with MINE estimates mutual information by combining a forward dynamics model with Mutual Information Neural Estimator, using the empowerment estimate as an intrinsic reward in sparse-reward settings [1810.05533]. “Efficient Empowerment Estimation for Unsupervised Stabilization” replaces variational lower bounds with a learned Gaussian channel representation,
\[
O = G(s)\cdot A + \eta,
\]
so that empowerment can be computed by convex optimization through a water-filling solution; the paper reports lower sample complexity, more stable training, and empowerment estimation from images [2007.07356]. “Latent-Predictive Empowerment” replaces the usual mutual information between skills and states with a latent-predictive objective that only requires a simpler latent-predictive model rather than a full simulator of the environment, and empirically learns similar-sized skillsets as the leading empowerment algorithm that assumes access to a model of the transition dynamics [2410.11155].

## 3. Distributed morphogenesis and neural cellular automata

The NCA literature gives a particularly clear example of empowerment as an auxiliary objective rather than a terminal task objective. “Empowered Neural Cellular Automata” extends empowerment to a distributed, multi-agent sensorimotor system embodied by an NCA and incorporates it into Age-Fitness Pareto Optimization. In that setting, each cell’s action is its signaling output, each cell’s sensor is the average signal of its Von Neumann neighbors, and combined optimization of shape matching and empowerment produces higher fitness than evolving for morphogenesis alone; the paper further reports a synergistic relationship between morphogenesis and empowerment [2205.06771].

The 2023 follow-up isolates the effect of time horizon. Its NCA empowerment is
\[
\mathfrak{E}(k)= - \sum_{a_{ijn} \in A_0^{k},\,s_{ijn} \in S_{N-k}^{N}} p(a_{ijn}, s_{ijn}) \log_2 \frac{p(a_{ijn}, s_{ijn})}{p(a_{ijn})p(s_{ijn})},
\]
where action at time \(n\) is paired with the sensed value at time \(n+k\). The study evaluates three treatments: **Bi-loss** (Age + Loss), **Tri-loss-empowerment** (Age + Loss + Empowerment \(\mathfrak{E}(k)\)), and **Tri-loss** (Age + Loss in the early phase + Loss in the late phase) [2305.15220].

Its main empirical result is that shorter time delays result in marked improvements over empowerment with longer delays, when compared to evolutionary selection only for homeostasis. Short-term empowerment with \(k=1\) leads to lower loss, whereas the benefit declines gradually with longer time horizons and approaches that of the controls. The same study further reports that short-term empowered NCA are more stable, show lower cell transiency, typically produce cohesive, single-component shapes, and generalize better to unseen homeostatic challenges; when used as seeds for further evolution on new targets, they adapt faster and achieve lower loss than controls [2305.15220].

These results make the time horizon itself part of the objective design. A plausible implication is that empowerment is not merely a generic regularizer in distributed morphogenesis; the temporal scale at which action–sensation coupling is optimized can determine whether the objective accelerates or attenuates the discovery of homeostatic local rules.

## 4. Reinforcement learning, exploration, and control

In reinforcement learning, empowerment is typically used as an intrinsic motivation signal. The continuous-control literature argues that greedily maximizing local empowerment drives agents toward states where they have more options and tend to avoid deaths, traps, or dead-ends. In well-known control tasks such as inverted pendulum, bicycle, and acrobot, empowerment-maximizing policies often lead to balancing or swing-up without explicit extrinsic rewards [1201.6583].

This connection is made explicit in unsupervised control. “Unsupervised Real-Time Control through Variational Empowerment” uses negative empowerment as the cost in a stochastic optimal-control framework, so that the policy is trained to maximize cumulative empowerment over trajectories; the reported behaviors include pendulum swing-up and balancing, movement toward the center in a ball-in-a-box task, and unsupervised balance in a bipedal system [1710.05101]. “Efficient Empowerment Estimation for Unsupervised Stabilization” makes the same point more narrowly: maximizing mutual information between agent actuators and future states coincides with stabilization at upright positions in several dynamical systems [2007.07356].

In meta-RL, empowerment can be directed toward task identification rather than environment control per se. MetaCURE defines an exploration objective
\[
J_{\pi_e}(C_{:H},K)=I_{\pi_e}(C_{:H};K),
\]
the mutual information between exploration experiences and the latent task variable. The paper separates exploration and exploitation policies and reports significant outperformance over state-of-the-art baselines on sparse-reward MuJoCo locomotion tasks and Meta-World tasks [2006.08170].

More recent work uses empowerment as a pre-training signal. “Information-Theoretic Policy Pre-Training with Empowerment” introduces discounted empowerment,
\[
\mathcal{E}_{\lambda}(\mathbf{s}_i)=\sum_{k=0}^{H}\lambda^k \max_{p(\mathbf{a}_{i:i+k})} I(\mathbf{s}_{i+k+1};\mathbf{a}_{i:i+k}\mid \mathbf{s}_i),
\]
to balance control across short and long horizons. The paper argues that empowerment-based pre-training yields data-efficient downstream task adaptation and improved adaptability, especially with long horizons [2510.05996].

## 5. Representation learning and controllable abstractions

A recurring criticism of standard visual or latent-state learning is that it does not distinguish functionally relevant factors from distractors. The empowerment literature answers this by tying representation quality to controllability. In visual model-based RL, “INFOrmation Prioritization through EmPOWERment” adds the conditional mutual information term
\[
I(A_{t-1}; Z_t \mid Z_{t-1})
\]
to prioritize information that is correlated with action. Under a bottleneck, this causes the latent state to capture controllable aspects of the environment first, while also promoting faster exploration in sparse-reward tasks [2204.08585].

A stronger theoretical claim appears in “Learning to Perceive the World Through Control: Empowerment-Based Representation Learning”. That paper states that empowerment agents induce two distinct representations—forward and backward—that capture complementary aspects of the state, and both of which are invariant to control-irrelevant features. It further argues that interaction aimed at maximizing control is essential for learning useful invariance properties, aligning the approach with causal learning rather than passive representation learning from fixed datasets [2605.30656].

This line of work also motivates latent or internal variants of the objective. “Latent-Predictive Empowerment” moves the skill-diversity objective into latent predictive space, while preserving a principled connection to the usual mutual information between skills and terminal states [2410.11155]. “Agent-centric learning: from external reward maximization to internal knowledge curation” goes further by defining representational empowerment as
\[
\operatorname{RepEmp}(Z_k)=\max_{\boldsymbol{\omega}_k\in \Omega^T} I(Z'_k;\boldsymbol{\omega}_k\mid Z_k),
\]
which measures an agent’s ability to controllably maintain and diversify its own knowledge structures rather than its ability to control external state [2507.22255].

Taken together, these works suggest a shift from empowerment as a property of reachable world states to empowerment as a property of controllable abstraction. This suggests that the objective can function not only as an exploration bonus but also as a criterion for what information should be retained in a learned model.

## 6. Assistance, causality, and broader human-centered formulations

In assistive settings, empowerment has been proposed as a goal-agnostic objective for increasing a human’s control rather than inferring the human’s latent goal. “AvE: Assistance via Empowerment” formalizes human empowerment as
\[
\mathcal{E}(s)=\max_{p(\vec{A}_T\mid s)} \mathcal{I}[\vec{A}_T;S_T\mid s],
\]
and augments an RL reward with an estimate of the human’s empowerment. Because exact estimation is computationally hard in continuous domains, the paper also proposes an empowerment-inspired proxy based on the variance of final states reached by sampled action sequences; in a simulated teleoperation study, empowerment-based assistance improves success rate and preserves the person’s autonomy and ability to achieve any eventual state [2006.14796].

Empowerment has also been coupled directly to causal structure learning. “Towards Empowerment Gain through Causal Structure Learning in Model-Based RL” defines
\[
\mathcal{E}:=\max_{\pi(\cdot\mid s_t)} \mathcal{I}(s_{t+1};a_t\mid M),
\]
with \(M\) the learned causal structure, and then maximizes the gain between empowerment under the causal model and empowerment under a dense model. The proposed ECL framework alternates causal model learning, empowerment-driven exploration, and downstream policy learning with an intrinsic curiosity reward, and is reported to improve causal discovery, sample efficiency, and asymptotic performance across six environments [2502.10077].

The same action–outcome logic appears in work on human causal learning. “Empowerment Gain and Causal Model Construction” defines empowerment as
\[
\mathsf{Emp}(s)=\max_{p(a)} I(A;S'\mid S=s),
\]
and argues that children and adults are sensitive to controllability and variability in their causal interventions. The empirical studies summarized in the paper report preferences for controllable and variable machines rather than machines that are merely variable or merely controllable, linking empowerment to effective causal intervention design [2512.08230].

Outside agent-centered AI, the term is also used in broader sociotechnical assessment. “Metrics for Assessing Inclusivity and Empowerment of People for Supporting the Design of Inclusive Product Lifecycles” defines empowerment as a context-dependent, iterative process and proposes candidate metrics such as
\[
E_4=\text{empowering impact}\times \text{dependency level}\times \#\text{means},
\]
with \(E_4\) and the inclusivity metric \(I_3\) showing the highest positive, significant correlation across ten case studies. This marks a different usage of “empowerment”: not channel capacity, but stakeholder power and agency within lifecycle processes [2410.17287].

## 7. Limitations, trade-offs, and emerging directions

Despite its generality, empowerment remains difficult to compute and easy to mis-specify. The continuous-control and stabilization literature repeatedly notes that exact or sample-based mutual-information estimation is challenging in nonlinear continuous spaces, that variational lower bounds can be biased and unstable in training, and that many skill-based methods require either a simulator or a model of the transition dynamics [2007.07356]. The NCA literature adds a different sensitivity: the action–sensation lag \(k\) is not an innocuous hyperparameter, and longer horizons can reduce the efficacy of empowerment as an auxiliary objective [2305.15220].

A second limitation is normative rather than computational. “When Empowerment Disempowers” shows that, in multi-human settings, assistive RL agents optimizing for one human’s empowerment can significantly reduce another human’s environmental influence and rewards. The paper formalizes this as disempowerment, introduces the Disempower-Grid test suite, and reports that joint empowerment mitigates disempowerment at the cost of the user’s reward [2511.04177]. This directly challenges the assumption that a goal-agnostic objective that appears aligned in a single-agent setting remains aligned in multi-agent settings.

A third open direction concerns the locus of control. Classical empowerment is environment-centric: it measures control over future sensed states. Representational empowerment shifts the objective inward to the ability to shape one’s own internal knowledge structures, with preparedness rather than direct environmental influence as the target [2507.22255]. This suggests an unresolved question: whether the most useful long-horizon form of empowerment for generally adaptive systems is external, internal, or a coupling of both.

Across these strands, empowerment remains a unifying objective for controllability, exploration, and option-preserving behavior. The literature does not present it as a single algorithmic recipe, but as a family of mutual-information objectives whose empirical behavior depends on embodiment, temporal horizon, estimator design, and social context.

Source: https://www.emergentmind.com/topics/empowerment-objective