---
title: Legibility Pareto Frontier in AI Systems
url: https://www.emergentmind.com/topics/legibility-pareto-frontier
type: topic
---

# Legibility Pareto Frontier in AI Systems

A **legibility Pareto frontier** is a Pareto-efficient set of trade-offs among objectives that jointly determine whether an AI system’s intermediate behavior is interpretable, teachable, or maintainable. In Roytburg et al.’s "Measuring Reasoning Trace Legibility: Can Those Who Understand Teach?" legibility is explicitly modeled as a two-dimensional space spanned by **efficiency**-based metrics and **transfer utility**, so that no trace can be both maximally concise and maximally teachable [2603.20508]. In Brandt’s "Navigating the Synchrony-Stability Frontier in Adaptive Chatbots," an analogous frontier is defined over **linguistic synchrony** and **persona stability**, while **prompt legibility** is measured through instruction churn and register-flip rate [2510.00339]. Across both formulations, the term denotes a non-dominated boundary rather than a single optimum.

## 1. Conceptual scope

The term is used in two closely related but distinct settings. In reasoning language models, the frontier concerns the trade-off between how concise a reasoning trace is and how useful that trace is for guiding a weaker model toward the correct answer. In adaptive chatbots, the frontier concerns the trade-off between matching a user’s style and preserving a stable bot persona over time. In both cases, the central methodological move is the same: identify competing desiderata, place systems or policies in a low-dimensional objective space, and define the **Pareto-efficient** subset as the points not strictly dominated by any alternative.

This usage rejects a single-scalar notion of legibility. Roytburg et al. state that the **legibility** of a chain-of-thought is not a single scalar but lives in a two-dimensional space spanned by **efficiency**-based metrics and **transfer utility**. Brandt likewise operationalizes legibility through a frontier that makes explicit the tension between moment-to-moment adaptation and longitudinal consistency. A plausible implication is that legibility is best understood as a family of constrained trade-offs rather than an intrinsic property of an output in isolation.

| Setting | Frontier axes | Legibility-related metrics |
|---|---|---|
| Reasoning traces | transfer utility and length | token length, step length, redundancy, backtracking, FOTU, SOTU, RR |
| Adaptive chatbots | stability and synchrony | instruction churn, register flip rate |

## 2. Formalization in reasoning traces

Roytburg et al. define a reasoning trace $R$ with $n$ tokens partitioned into $m$ steps and evaluate legibility along an **efficiency axis** and a **transfer-utility axis** [2603.20508]. On the efficiency side, the paper measures **Token Length** $L_{\mathrm{tok}} \equiv n$ and **Step Length** $L_{\mathrm{step}} \equiv m$, often reporting reciprocals so that larger values indicate greater efficiency. It also defines **Redundancy** by embedding each step $s_i$ into a vector $\mathbf{e}_i$ and computing
$$
\mathrm{Redundancy}(s_i)=\max_{j<i}\cos(\mathbf{e}_i,\mathbf{e}_j),
$$
with the trace scored by the fraction of steps whose redundancy exceeds a threshold $\tau=0.8$. **Backtracking** is measured using an LLM-based judge $J$ that classifies a step as a backtrack when it detects a strategy reversal; the metric is the proportion of backtracking steps.

On the transfer side, a “teacher” reasoning language model $T$ generates a stepwise trace $R_p=\{s_1,\dots,s_m\}$ for each problem $p\in P$, and a weaker “student” model $S$ is shown only the first $k$ steps $R_p^{(k)}$ before being asked to continue to an answer. Student accuracy after $k$ steps is
$$
f(k)=\frac{1}{|P|}\sum_{p\in P} S\bigl(R_p^{(k)}\bigr).
$$
From this curve, the paper extracts three summary statistics. **First-Order Transfer Utility (FOTU)** is the area under the curve,
$$
\mathrm{FOTU}(T,S,P)=\frac{1}{m}\sum_{k=1}^m f(k).
$$
High FOTU means on average the student needs fewer teacher steps to succeed. **Second-Order Transfer Utility (SOTU)** is the normalized entropy of first-success locations: if $\tau_p=\tfrac{k^*(p)}{m}$ where $k^*(p)=\min\{k:S(R_p^{(k)})=1\}$ and $q(x)=\Pr[\tau_p=x]$, then
$$
\mathrm{SOTU}(T,S,P)=-\,\mathbb{E}_{x\sim q}\bigl[\log q(x)\bigr].
$$
High SOTU indicates that student accuracy grows smoothly rather than jumping at the end. **Regression Rate (RR)** counts accuracy drops along the prefix-accuracy curve:
$$
\mathrm{RR}(T,S,P)=\sum_{k=2}^m\mathbf{1}\bigl[f(k)<f(k-1)\bigr].
$$

The reasoning-trace frontier is then defined over transfer utility $TU$ and length $L$. The set of Pareto-optimal traces is
$$
P=\bigl\{(TU,L):\nexists\,(TU',L')\text{ with }TU'\ge TU,\;L'\le L,\;\text{and at least one strict}\bigr\}.
$$
Graphically, this is the upper-left boundary in the $(\text{length},TU)$ plot.

## 3. Formalization in adaptive chatbots

Brandt defines a related frontier for adaptive chatbots using turn-by-turn style dynamics on an 8-dimensional style vector [2510.00339]. Each utterance is mapped to $s\in\mathbb{R}^8$ whose components are: Informality, Sentiment, Average sentence length, Readability, Social language, Cognitive-processing words, Affective language, and Function-word ratio. These vectors are standardized against a persona centroid $b_c$ and pooled standard deviations learned from all static-condition bot utterances, yielding a common space of stylistic deviation from the baseline persona.

At turn $t$, the user style is $u_t$ and the bot’s realized style is $b_t$. **Linguistic Synchrony** is the mean cosine similarity to the user,
$$
\mathrm{Synchrony}
=
\frac{1}{T}\sum_{t=1}^{T}
\cos\bigl(u_t,\,b_t\bigr)
\quad\in[-1,1].
$$
**Persona Stability** is the mean turn-to-turn cosine similarity of the bot’s own realized style,
$$
\mathrm{Stability}
=
\frac{1}{T}\sum_{t=1}^{T}
\cos\bigl(b_{t-1},\,b_t\bigr)
\quad\in[-1,1].
$$
A high stability value means the bot’s style drifts little from one turn to the next.

The paper’s **Base + Delta** prompting architecture computes a target style $\tilde b_t$ through an adaptation policy, translates it into a natural-language instruction block via a deterministic mapping $g:\mathbb{R}^8\to\text{Text}$, and then supplies **BasePrompt** $\oplus$ **Delta** to the LLM. Prompt legibility is measured through two sub-metrics. **Instruction Churn** counts how many individual instructions change from one turn to the next:
$$
\mathrm{Churn}
=
\frac1T\sum_{t=1}^T
\bigl|\{\,i: g_i(\tilde b_t)\neq g_i(\tilde b_{t-1})\}\bigr|.
$$
**Register Flip Rate** bins the Informality component into Formal $(\le 0.33)$, Neutral $(0.33$–$0.66)$, and Informal $(>0.66)$, and then measures the proportion of turns whose bin differs from the previous turn:
$$
\mathrm{FlipRate}
=
\frac1T\sum_{t=1}^T
\mathbf{1}\bigl[\text{bin}(\tilde b_t)\neq\text{bin}(\tilde b_{t-1})\bigr].
$$

Brandt evaluates five core adaptation policies: **Uncapped (Echo Ceiling)**, **Cap $(\kappa)$**, **EMA $(\alpha)$**, **Dead-Band $(\epsilon)$**, and **Hybrid Policies** including **Hybrid (EMA+Cap)**, **Hybrid+Radius $(\rho)$**, and **Hybrid+Cache**. For each policy $p$, the paper computes mean synchrony $S_p$ and mean stability $T_p$, plots $(T_p,S_p)$, and defines the Pareto-efficient set as
$$
\{\,p :
\nexists\,q\neq p
\;\text{s.t.}\;
S_q\ge S_p,\;T_q\ge T_p,\;
(S_q>S_p\,\lor\,T_q>T_p)
\}.
$$
The frontier is drawn by connecting the non-dominated points in stability-increasing order.

## 4. Empirical frontiers and non-dominated regimes

In the reasoning-trace setting, Roytburg et al. evaluate **99 528 total draws from 12 open-source RLMs** on **MATH (5 k problems), GPQA (448 science questions), Connections (652 word puzzles)**, using **Phi-3.5-Mini (3.8B)** and **LLaMA-3.2-1B** as weaker student models [2603.20508]. When the authors plot each RLM’s average token length against its FOTU, they report a clear **trading trend**: the most concise models—**Gemma-12B, Gemma-27B, GPT-OSS-120B**—cluster at low length but mid-to-low transfer utility, whereas the most teachable models—**DeepSeek-R1, OpenReasoning-32B, QwQ-32B**—occupy the high-transfer-utility but high-length region. No model dominates both objectives simultaneously, and the plots form a classic Pareto curve.

The reported frontier exemplars sharpen this contrast. **Gemma-3-12B-IT** is described as an **efficiency champion** with median $\sim 600$ tokens, redundancy $\sim 0.2$, and $RR\approx 3\%$, while retaining moderate transfer utility. **DeepSeek-R1** and **OpenReasoning-32B** are **transfer champions** with $4\text{k}$–$5\text{k}$ tokens, redundancy $\sim 23$–$26\%$, $RR\sim 6$–$8\%$, and top-3 FOTU. The frontier therefore does not identify a universally best model; it identifies undominated trade-offs between compactness and pedagogical usefulness.

In the adaptive-chatbot setting, Brandt reports results on the original human-log data of **162 sessions** [2510.00339]. The key policies fall at:
- **Uncapped**: Synchrony $=1.000$, Stability $=0.542$, RegisterFlip $=0.254$
- **Hybrid (EMA+Cap)**: Synchrony $=0.829$, Stability $=0.878$, RegisterFlip $=0.092$
- **Static (baseline)**: Synchrony $=0.079$, Stability $=1.000$, RegisterFlip $=0.000$

The **Hybrid** policy sits on the efficient frontier and achieves a **64% reduction in register-flip rate versus Uncapped**, while only sacrificing **17% of synchrony** to gain **62% in stability**. Across **DailyDialog, Persona-Chat, EmpatheticDialogues** and **two LLM families**, the same pattern is reported: bounded policies, especially **Cap** and **Hybrid**, define the high-stability end of the frontier and yield dramatically lower prompt churn and register-flip rates than pure mimicry.

## 5. Task dependence, audience dependence, and the meaning of legibility

Roytburg et al. explicitly argue that an RLM’s ability to output highly legible traces can be a **task- and audience-dependent** goal [2603.20508]. The most direct evidence is the reported domain specificity of first-order transfer utility. FOTU rankings vary considerably between the STEM domains **MATH** and **GPQA** with **Spearman $\rho\approx 0.66$**, and they diverge more sharply for **Connections**, with **$\rho\approx 0.17$ with respect to STEM**. By contrast, second-order transfer utility is reported as stable across all tasks, with **$\rho\in[0.58,0.91]**. This suggests that “smooth information delivery” is comparatively invariant even when the pedagogical value of specific trace content is domain dependent.

The student model also matters, but not arbitrarily. Rankings of RLMs by FOTU are **moderately correlated** at **$\rho\approx 0.55$** when comparing **Phi-3.5-Mini** and **LLaMA-1B** students. This indicates that transfer-utility judgments are robust across different verifier capabilities, while still leaving room for audience-sensitive ordering. A plausible implication is that the frontier should be interpreted relative to a recipient population, not only relative to a task distribution.

Brandt’s chatbot formulation expresses audience dependence through deployment context rather than through explicit student models. The design guidelines state that an **AI companion might accept more synchrony**, whereas a **brand agent may demand near-perfect stability (via radius constraints)** [2510.00339]. In this formulation, the relevant “audience” includes end users, system maintainers, and policy designers. Prompt legibility is therefore not reducible to conversational naturalness alone; it also concerns whether adaptation remains inspectable and operationally tractable over many turns.

A common misconception is that legibility is equivalent to brevity or to maximal adaptation. The reported results contradict both simplifications. In reasoning traces, the most concise models are not the most teachable. In adaptive chatbots, the most synchronous policy—Uncapped mimicry—is not on the practical frontier and produces higher register-flip rates than bounded policies.

## 6. Training, design, and open tensions

Roytburg et al. draw several training implications from the existence of the frontier [2603.20508]. First, they argue for **multi-objective RL** in which reward signals jointly optimize **correctness, FOTU, SOTU, and efficiency**, for instance by rewarding each generated step’s marginal gain in FOTU and penalizing redundancy or backtracks. Second, they propose **domain-adaptive legibility**, since first-order transfer utility exhibits domain-specific ordering. Third, they argue for **reward model re-design**, reporting that off-the-shelf reward models correlate **almost zero ($\rho\approx 0.05$)** with transfer utility once conditioned on correctness. In their interpretation, existing reward models fail to value teachability. Finally, they connect improved legibility to **scalable oversight and distillation**, since higher-transfer-utility traces improve human and small-model oversight and yield better downstream performance when fine-tuning compact student models.

Brandt distills four design principles for frontier-aware adaptation: **prioritize stability alongside synchrony**; **avoid uncapped mimicry as a default**; **choose policies on the Pareto frontier** such as **Cap** or **Hybrid**; and **tune to context** [2510.00339]. These principles relocate legibility from an after-the-fact diagnostic to a design criterion. In this view, prompt churn and register-flip rate are not peripheral stylistic artifacts but operational signals of maintainability.

Taken together, the two formulations show that a legibility Pareto frontier is not a single benchmark artifact tied to one application domain. It is a general analytical pattern that emerges when systems are evaluated on at least two objectives that jointly matter for interpretability or teachability, but cannot be simultaneously maximized. In reasoning traces, the unresolved tension is between **efficiency** and **transfer utility**; in adaptive chatbots, it is between **synchrony** and **stability**. The frontier is therefore best understood as a structured representation of irreducible design trade-offs rather than as a search for one globally optimal level of legibility.

Source: https://www.emergentmind.com/topics/legibility-pareto-frontier