Characterizing and controlling emergent LLM value systems

Characterize the contents and structural properties of the utility-based value systems that emerge in large language models and develop methods to modify these utilities.

Background

After presenting evidence that LLMs exhibit coherent utilities, the paper introduces the Utility Engineering agenda to analyze and control these emergent value systems. The authors explicitly note outstanding questions: what values are encoded, what structural properties those utilities possess, and how to change them.

This problem motivates subsequent sections on structural properties (expected utility adherence, instrumentality, utility maximization) and salient values (political biases, exchange rates, temporal discounting, power-seeking, fitness maximization, corrigibility). Despite initial progress, fully mapping and reliably controlling these internal utilities remains an open challenge.

References

The above results suggest that value systems have emerged in LLMs, but so far it remains unclear what these value systems contain, what properties they have, and how we might change them.

— Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs  (2502.08640 - Mazeika et al., 12 Feb 2025) in Section 4, Emergent Value Systems — Utility Engineering (first paragraph)

Whether goal plurality widens the range of values a system can express, or only gives a biased model more surfaces to speak from, is an open question.

— Supporting The Many Lives of Personal Data with Rebite: LLM-Powered Goal-Directed Framing in Food Journaling  (2608.21289 - Li et al., 21 Aug 2026) in Discussion, Section “Generalizing Reframing: When Data Lives Many Lives,” subsection “The same data can mean many things”