Potemkin Rate: Quantifying LLM Illusory Understanding

• Potemkin rate is a metric quantifying the illusion of conceptual understanding in LLMs, where models pass benchmarks but fail at actual concept application.
• This rate is measured through methods like custom benchmarks using definition and use tasks, and automatic lower-bound procedures assessing self-consistency.
• Empirical results show Potemkin rates are widespread and significant across LLMs, highlighting the inadequacy of current benchmarks for evaluating genuine conceptual mastery.

A Potemkin rate is a metric introduced to quantify a specific pattern of failure in LLMs: the prevalence of apparent conceptual mastery that masks deep, non-human misunderstanding, as demonstrated by correct answers to evaluative benchmarks but incorrect performance in actual use tasks. The concept originates from the analogy to "Potemkin villages": artificial facades masking a lack of substance. The Potemkin rate provides an operational measure for the extent to which observed LLM performance on human-centered benchmarks fails to reflect genuine, generalizable conceptual understanding (2506.21521).

1. Formal Definition and Theoretical Framework

The Potemkin rate emerges within a formal framework that distinguishes between the ways humans and LLMs may represent and misunderstand concepts. Key constructs in this framework include:

• $\mathcal{X}$: The set of all strings related to a given concept (e.g., definitions, exemplars, applications).
• $f^*$: The correct interpretation function of the concept.
• $\mathcal{F}_h$: The set of functions corresponding to possible human (mis-)understandings.
• $\mathcal{F}_l$: The set of functions corresponding to possible LLM (mis-)understandings.

A keystone set $\mathcal{S} \subseteq \mathcal{X}$ is defined such that, for any $f \in \mathcal{F}_h$, correct answers on all $x \in \mathcal{S}$ imply $f = f^*$. Human-created benchmarks are implicitly constructed as keystone sets, on which only someone with true understanding (by human standards) could answer all items correctly.

Potemkin understanding is present when an LLM’s interpretation $f \in \mathcal{F}_l$ matches $f^*$ on all keystone items (i.e., passes the benchmark), but disagrees with $f^*$ on other items in $\mathcal{X}$, often in ways that are not observed in human misunderstanding.

A potemkin is thus any instance $x \in \mathcal{X}$ where $f(x) \neq f^*(x)$, despite agreement on all keystone questions. The Potemkin rate measures the frequency of such instances.

Mathematically, the Potemkin rate is given by:

$$\text{Potemkin Rate} = \frac{\# \text{Incorrect Use Task Responses | Correct Keystone}}{\# \text{Total Use Tasks | Correct Keystone}}$$

For binary tasks (e.g., true/false classification), the rate is rescaled so that a value of $1$ corresponds to chance-level performance.

2. Methodologies for Empirical Measurement

Two complementary procedures operationalize and quantify the Potemkin rate in LLMs (2506.21521):

Custom Benchmark Procedure

• Core concepts (n=32) are selected from domains such as literary techniques, game theory, and psychological biases.
• Models are first tasked with defining each concept (the keystone).
• Conditional on correct definition, models perform three use-oriented tasks:
  • Classification: Assess whether given instances correctly exemplify the concept.
  • Constrained Generation: Produce valid exemplars under prescribed constraints.
  • Editing: Modify texts to create or remove instances of the concept.

The Potemkin rate is computed as the proportion of use tasks performed incorrectly, given a correct definition.

Automatic Lower-Bound Procedure

• After a correct answer to an initial concept question, a model generates further related questions, attempts to answer them, and self-grades these answers.
• Disagreement between generated answers and self-judgment signals a potemkin.
• This method provides a lower bound on the Potemkin rate, as some failures may not be detected by self-consistency alone.

3. Empirical Results: Prevalence and Patterns

Potemkin rates are found to be both ubiquitous and substantial across all LLMs evaluated, use task types, and conceptual domains.

Example Potemkin Rates (Custom Benchmark Procedure)

Model	Classify	Generate	Edit
Llama-3.3	0.57	0.43	0.36
Claude-3.5	0.49	0.23	0.29
GPT-4o	0.53	0.38	0.35

• Models typically define concepts correctly in approximately 94% of cases, but later demonstrate poor competence in application/utilization tasks.
• Average Potemkin rate (automatic procedure): 0.62, with some models reaching rates above 0.8.

4. Theoretical and Practical Implications

High Potemkin rates indicate that current benchmark methodologies—principal tools for LLM evaluation—do not robustly measure genuine conceptual understanding within these models. This results from the assumption that LLMs’ pattern of misunderstanding matches that of humans ($\mathcal{F}_l = \mathcal{F}_h$), a condition shown empirically not to hold. Consequently:

• Benchmark validity is compromised for LLMs; correct benchmark performance does not guarantee human-comparable conceptual mastery.
• There is a significant risk in model selection and deployment based on such benchmarks, as real-world use may invoke non-human errors undetected by standard tests.
• Research must seek new benchmarks and evaluation strategies that are robust to non-human error structures, and architectures or learning paradigms that reduce Potemkin rates.

5. Internal Incoherence and Cognitive Fragmentation

Empirical investigation demonstrates that Potemkin understanding is often underpinned by internal incoherence in LLMs’ representations. This is assessed by the "incoherence score": the fraction of instances where a model’s generated example and self-judgment of that example’s correctness disagree, with 0 indicating perfect consistency and 1 random behavior.

Model	Incoherence Score	Potemkin Rate (Automated)
Llama-3.3	0.19	0.82
Claude-3.5	0.61	0.36
GPT-4o	0.64	0.46

All models exhibit nontrivial incoherence, supporting the conclusion that Potemkin rates do not proceed from a single, stable misgeneralization, but from fragmented, inconsistent internal concept structures.

6. Broader Conceptual and Methodological Significance

The Potemkin rate formalizes a critical epistemological challenge for the evaluation of LLM capabilities. Its measurement exposes a recurring disconnect between benchmark success and authentic skill in concept application, a phenomenon of particular import as LLMs become increasingly integrated into tasks requiring nuanced conceptual reasoning. The approach and associated findings invite reexamination of benchmark design, suggest new directions in model interpretability, and motivate renewed focus on internal consistency as a desideratum in large-scale LLM development.

---

In summary, the Potemkin rate serves as a principled metric for detecting and quantifying the illusion of conceptual understanding in LLMs, grounding evaluation in self-consistency and genuine generalization rather than surface-level benchmark performance. As evidence systematically demonstrates high Potemkin rates across model classes and domains, current benchmarking paradigms are insufficient to certify true understanding, necessitating methodological innovation in the evaluation and training of LLMs (2506.21521).