Interplay Between World Representations and Empowerment Maximization

Investigate how representations of the world affect empowerment maximization, and characterize the relationship between the choice of world representation and the behavior or effectiveness of empowerment-maximizing agents.

Background

The paper presents empowerment maximization as highly sensitive to the representation used to describe an agent’s environment. In the proposed Skills Estimate Empowerment (SEE) method, the learned classifier receives selected environmental variables that define the degrees of freedom over which empowerment is measured, while the policy and critic may retain access to the full observation. This design choice affects which environmental changes are treated as salient and therefore which behaviors are rewarded.

Although the experiments demonstrate that different representation choices can produce substantially different empowerment-maximizing behaviors, the paper does not resolve the general relationship between world representations and empowerment maximization. In particular, it leaves open how representations should be selected or learned so that empowerment captures meaningful control rather than trivial or irrelevant perturbations.