---
title: Unfaithful Perception Rate
url: https://www.emergentmind.com/topics/unfaithful-perception-rate
type: topic
---

# Unfaithful Perception Rate

Unfaithful perception rate quantifies the frequency or proportion of model outputs, reasoning steps, communication acts, or perceptual judgments that do not accurately reflect the available evidence, ground truth, or perceptual input. The term spans information theory (e.g. rate-distortion-perception tradeoff), machine learning (hallucination in LLMs/VLMs), psychophysics (conditioned hallucinations), causal cognition (disagreement in causal structure), and human–AI or human–environment settings. Across domains, unfaithful perception is operationalized as the complement of a rigorously defined faithfulness or perceptual fidelity metric. This article surveys its formalizations, measurement protocols, computational models, empirical findings, and the implications for trust, system design, and interpretability.

## 1. Formal Definitions Across Domains

Within information theory and communication, the unfaithful perception rate is typically defined via a constraint on the divergence between the marginal distribution of reconstructions and that of the true source. Let $X$ denote the source, $\hat X$ its reconstruction, and $d$ a divergence (often total variation, an $f$-divergence, or Wasserstein). For distortion $D$ and perception threshold $P$, the rate–distortion–perception function is
\[
R(D, P) = \min_{p_{\hat X|X}} I(X; \hat X) \quad \text{s.t.} \quad \mathbb{E}[\Delta(X, \hat X)] \le D,\, d(p_X, p_{\hat X}) \le P
\]
with the unfaithful perception rate indexed by $P$ (the maximum permissible divergence) [1901.07821, 2305.04604, 2204.06049].

In large language model (LLM) evaluation, unfaithful perception rate generalizes the hallucination rate. For output facts $F = \{f_1, \dots, f_n\}$ with reference dataset $S$, monofact rate $M$ (probability that a fact appears exactly once in $S$), and calibration error $C$ (total variation or KL between model output and reference frequencies), Kalai & Vempala prove
\[
H \ge M - C
\]
where $H$ is the probability of an unfaithful (hallucinated) generation [2502.08666].

Faithfulness in multimodal models is measured as the fraction of atomic or chain-level statements visually supported by an image, yielding an unfaithful perception rate as $1$ minus the proportion of faithful objects, steps, or atomic facts [2511.08409, 2512.12218, 2311.01477].

Psychophysically, as in conditioned hallucinations, the unfaithful perception rate is the conditional probability of a subject reporting a percept ("voice") where none was present: $\text{CH} = P(\text{report percept} \mid \text{no actual stimulus})$ [2306.13832].

In causal reasoning, if decision-makers $R_1, ..., R_N$ hold structural causal models (SCMs) $G_{R_i}$ over variables $X$, then the unfaithful perception rate is the fraction of pairs $(i, j)$ with $G_{R_i} \neq G_{R_j}$ on at least one directed edge; equivalently, the indicator of edge disagreement averaged over pairs [2401.13408].

## 2. Mathematical Characterizations and Metrics

Information theory systematically characterizes the trade-off between rate, distortion, and perception:
\[
R(D, P) = \inf_{p_{\hat X|X}} I(X;\hat X) \quad \text{s.t.} \quad \mathbb{E}[\Delta(X, \hat X)]\le D,\, d(p_X, p_{\hat X}) \le P
\]
where $d$ may be total variation, $f$-divergence, or Wasserstein [1901.07821, 2408.15015]. $\Delta R(D, P) = R(D, P) - R(D, \infty)$ quantifies the rate penalty relative to unconstrained distortion coding.

In LLM frameworks, unfaithful perception rate employs bin-wise calibration measures and monofact statistics:
\[
\widehat{H} \gtrsim \widehat{M} - \widehat{C}
\]
where $\widehat{M}$ is the empirical monofact rate, and $\widehat{C}$ is a calibration error (often binned KL divergence or TV distance). Intervention via selective upweighting reduces $\widehat{H}$ by increasing $\widehat{C}$ at fixed $\widehat{M}$ [2502.08666].

In multimodal and VLM contexts, atomic-level and sentence-level unfaithful perception rates are computed as:
\[
\mathrm{UPR} = 1 - \mathrm{FaithScore} = \frac{\#\text{hallucinated atomic facts}}{\#\text{total atomic facts}}
\]
or for reasoning-chains,
\[
\mathrm{UPR} = \frac{\text{\# unfaithful perception steps}}{\text{\# perception steps}}
\]
with variants for chain-level and sentence-level aggregation [2512.12218, 2511.08409, 2311.01477].

For causal graphs, the metric is:
\[
\mathrm{UPR} = \frac{1}{N(N-1)/2} \sum_{1 \le i < j \le N} \mathbf{1}\{G_{R_i} \neq G_{R_j}\}
\]
[2401.13408].

## 3. Robust Measurement Protocols

Protocols precisely specify what is labeled "unfaithful" or "hallucinated" depending on the context:

- In information theory, algorithms compute $R(D,P)$ via parametric Lagrangian minimization with alternating or relaxed Blahut–Arimoto–style iterations, yielding globally optimal solutions under convexity for a prescribed $P$ [2408.15015, 2305.04604].
- LLM hallucination rates are measured by (a) sampling training data to control empirical monofact rates, (b) training models under controlled miscalibration, and (c) validating predicted hallucination rates with bin-wise metric computations [2502.08666].
- Faithfulness in VLMs is operationalized via LLM or VEM-based pipelines for sub-sentence identification, atomic fact extraction, and fact–image entailment. Sentence- and atomic-level UPRs are reported globally, by fact type (entities, colors, relations), and by prompt structure [2311.01477].
- In explicit chain-of-thought tasks, unfaithful recovery is annotated as present when the model produces the correct answer via an unsupported, hallucinatory, or inconsistent chain (no explicit correction of an injected error) [2405.15092].
- Causal disagreement rates are measured over a population of SCMs, quantifying pairwise or reference-based divergence in graph structure [2401.13408].
- Human studies employ forced-choice or evidence-localization (e.g., in image forensics), reporting unfaithful perception as the false negative rate in detecting alterations [1509.05301].
- Conditioned hallucination in psychophysics uses the proportion of false-positive reports in no-stimulus trials; model-based inferences (e.g., HGF) relate unfaithful perception rate to latent parameters (prior/sensory weighting) [2306.13832].

## 4. Empirical Rates, Determinants, and Bounds

Empirically, unfaithful perception rates span domains and model types:

- In lossy coding, enforcing perfect perceptual fidelity increases minimal rate relative to unconstrained distortion coding; the rate penalty $\Delta R(D,0)$ obeys known upper bounds (e.g., by at most a 3 dB distortion penalty under MSE) [1901.07821].
- LLMs: Hallucination rates approach the monofact rate at perfect calibration; reductions of up to 40% are achieved via targeted miscalibration [2502.08666]. Empirical values for modern SOTA models are $\sim$6% for GPT-5 (summarization, VeriGray), 14–46% (VLMs, visual atomic facts), and up to 60% (LLM in-context under high monofact rate) [2510.21118, 2311.01477].
- FaithAct and other visual reasoning systems report relative gains in perceptual faithfulness up to 26%—translating into lowered UPR—without loss of task accuracy [2511.08409, 2512.12218].
- In chain-of-thought error recovery, unfaithful rates for GPT-4 reach 35–40% on copying errors and remain non-trivial even with explicit error-detection prompts [2405.15092].
- Human image forensics show UPR = 0.535 (53.5% of altered digital images were missed), independent of confidence or attention [1509.05301].
- Conditioned hallucination (CH) rates correlate with prior-overweighting in perception models (HGF) and with clinically assessed hallucination proneness [2306.13832].
- Empirical rate surfaces (e.g., $R(D,P)$) are monotonic and convex in both distortion and perception tolerance; "free" zero-rate communication arises when sufficient decoder side information is present [2312.05437].

## 5. Applications and Implications

Unfaithful perception rate is an operational measure for:

- **Benchmarking**: Global and fine-grained quantification of hallucination and error modes in LLMs, VLMs, and reasoning chains.
- **Algorithm design**: Explicit optimization of perceptual fidelity in information-theoretic coding, informed by convexity and parametric algorithms [2305.04604].
- **Intervention analysis**: Tuning model calibration or training procedures (e.g., selective upweighting, self-reflection and regeneration in VLMs, error-detecting prompts) to manage UPR and its tradeoff against coverage and accuracy [2502.08666, 2512.12218, 2511.08409].
- **Fairness, trust, and interpretability**: In causal settings, high UPR signals stakeholder disagreement over causal structure, critically affecting fairness analyses. In chain-of-thought reasoning, persistent unfaithfulness undermines the transparency and reliability of intermediate steps—even when final answers are correct [2401.13408, 2405.15092].
- **Perception science**: CH rates in clinical and non-clinical populations link behavioral unfaithful perception to mechanistic model parameters (e.g., prior/sensory weighting, decision noise), supporting theoretical models of hallucination [2306.13832].

## 6. Open Challenges and Future Directions

Core methodological challenges include:

- **Disambiguating annotation boundaries:** Recent frameworks (VeriGray) make explicit the "Out-Dependent" zone, disentangling hallucinations from cases requiring external knowledge or ambiguous references. This is crucial for reducing annotation noise and improving detector evaluation [2510.21118].
- **Consistency across domains:** Definitions and thresholds for faithfulness and unfaithfulness (and therefore UPR) depend on rigorous operationalizations—factual entailment vs. perceptual grounding vs. statistical divergence—necessitating careful specification and agreement across benchmarks [2311.01477, 2512.12218].
- **Algorithmic efficiency:** Numerical evaluation of $R(D,P)$ under general f-divergence constraints is nontrivial, demanding sophisticated alternating minimization and root-finding schemes; convergence guarantees and computational overhead are subject to the regularity of the metric [2408.15015, 2305.04604].
- **Human–AI interaction:** High UPR persists in humans for subtle digital forgeries; scaling human–AI ensembles or hybrid workflows introduces additional layers of perceptual (un)faithfulness [1509.05301, 2401.13408].
- **Uncertainty and abstention:** Selective prediction and classifier abstention (on Out-Dependent or ambiguous cases) reduces hallucination risk but poses coverage–fidelity trade-offs.

Ongoing work seeks to generalize perceptual faithfulness constraints to more complex knowledge domains, leverage explicit machine annotation and external retrieval, and integrate perceptual constraints directly into model training. The unfaithful perception rate serves as both a theoretical lever and a practical diagnostic for advancing the reliability and interpretability of autonomous and human-in-the-loop systems.

Source: https://www.emergentmind.com/topics/unfaithful-perception-rate