Can LLMs Introspect? A Reality Check

Abstract: Can LLMs detect and report their own internal states? A number of studies have argued that the answer to this question is yes. We argue, based on lessons from human metacognition research, that this conclusion may be premature: to be convinced of this conclusion we need to distinguish genuine introspection from pattern matching based on surface-level cues. Furthermore, we argue that behavioral evidence alone is inherently insufficient to establish strong introspective claims. We re-examine two recently introduced evaluation paradigms in light of this consideration. In the first paradigm, models are expected to detect whether their internal states have been tampered with. We find that models cannot reliably distinguish such interventions on their internal states from manipulations of the input, suggesting that their success in the original studies reflects their ability to detect anomalies more generally, as opposed to interventions on their internal states in particular. In the second paradigm we examine, models are tasked with predicting labels derived from their own hidden states. Here, we find that classifiers that only have access to the input achieve equivalent performance to the model's own in-context predictions, indicating that the original results do not conclusively demonstrate that the model has privileged access to its internal representations. We further introduce a relabeled control setting, where models cannot rely on the semantics of the task to solve it, and instead must rely on the internal representation; models perform closer to chance on this better-controlled version of the task. Taken together, these results indicate that current evidence is insufficient to establish that LLMs display metacognitive monitoring.

• The paper demonstrates that LLM introspective claims rely on input-driven shortcuts, challenging evidence for genuine metacognition.
• It employs rigorous controls in both biofeedback and steering paradigms to reveal that models use superficial cues overshadowing real internal monitoring.
• The study advocates for mechanistic probing and calibration-based designs to more accurately diagnose authentic introspective processes in LLMs.

Critical Analysis of "Can LLMs Introspect? A Reality Check" (2605.26242)

Motivation and Context

The capacity of LLMs for self-monitoring, or metacognitive abilities, has attracted increased attention in recent literature. Several recent works posit that LLMs can introspect on their internal states, reportedly exhibiting emergent properties akin to human metacognition. This paper systematically interrogates two dominant paradigms used to support such claims: (1) biofeedback paradigms where models report labels derived from internal activations, and (2) steering-based paradigms where models are asked to detect injected concepts via activation interventions. The authors argue that these paradigms do not escape input-driven shortcuts and fail to provide sufficient evidence for genuine introspective or metacognitive monitoring.

Construct Validity of Introspection Evaluations

The paper hinges its critique on two primary axes: (a) empirical—are existing results actually indicative of privileged introspective access, or do they conflate surface-level input cues with internal state monitoring? (b) principled—even if input-based confounds are eliminated, is behavioral accuracy alone enough to establish the presence of dissociable second-order introspective computation?

Drawing from human metacognition literature, where above-chance correlations between confidence and accuracy often stem from shallow heuristics rather than genuine second-order monitoring, the authors argue behavioral evidence in LLMs is susceptible to similar confounds. They formalize privileged access as requiring task labels to be unpredictable from input alone while remaining predictable from internal representations. However, even satisfaction of this criterion is insufficient; introspection, in its strong sense, requires evidence of a computational step distinct from ordinary forward pass.

Figure 1: Input-driven confounds in introspective paradigms, contrasting biofeedback and steering-awareness conditions.

Empirical Examination of Biofeedback and Metacognitive Paradigms

Biofeedback Paradigm

The authors replicate and extend the methodology of "biofeedback" paradigms, in which LLMs must predict labels derived from probes (e.g., logistic regression, PCA components) trained on hidden states. They introduce critical controls:

• Random Relabeling: Semantic correlation between input and probe label is broken; model performance drops to chance, indicating original task success is input-driven.
• Input Probes: Probes trained on layer-0 token embeddings achieve comparable or superior accuracy in predicting hidden state labels compared to in-context LLM predictions.

Figure 2: Llama-3.1-8B-Instruct logistic regression probe reveals decorrelation-induced collapse in accuracy.

These results decisively demonstrate that purported metacognitive monitoring in "biofeedback" is reducible to first-order semantic classification or shallow input features.

Belief Dominance Paradigm

The "Belief Dominance" framework (as in (Yalon et al., 2 Feb 2026)) uses cluster labels derived from hidden representations capturing the model's conflict resolution between parametric (base) and context-provided (counterfactual) knowledge. The authors show that probes trained solely on entity input embeddings—without access to hidden states or prompt context—match or exceed LLM performance in predicting Belief Dominance clusters, indicating the paradigm also falls prey to input-based shortcuts.

Steering-Based Self-Report Paradigm

The steering paradigm evaluates whether models can detect activation-level interventions ("thought injection"). The original two-way design (intervention vs. control) does not discriminate between input-based anomalies and genuine introspective access. The authors augment the paradigm with a third "gaslight" condition, representing input-level interventions. Performance patterns reveal that models fail to reliably distinguish activation-level from input-level interventions, especially in the three-way discrimination task.

Figure 3: Two-way condition—model discriminates control from activation-level intervention with above-chance accuracy.

Figure 4: Three-way condition—model fails to reliably separate input from activation interventions, suggesting generic anomaly detection.

These findings undermine the introspection hypothesis and suggest that models rely on generic anomaly detection, not privileged self-access to their internal activations.

Mechanistic and Philosophical Implications

The paper reinforces that behavioral paradigms alone—whether biofeedback or steering-based—cannot establish strong introspective self-monitoring in LLMs. The information-theoretic notion of privileged access is necessary but insufficient. Genuine introspection requires mechanistic evidence of a dissociable second-order process, such as selective impairment via causal intervention or signature dissociation between first-order and second-order computations (e.g., confidence calibration failures). They cite concurrent work ([macar2026mechanisms]) that begins to address this evidentiary bar but does not settle the question.

Practical and Theoretical Implications

The authors' critique has significant practical ramifications for model interpretability, reliability, and claims regarding emergent cognitive faculties in AI. The demonstrated prevalence of input-driven shortcuts in self-reporting paradigms means that trustworthiness, uncertainty calibration, and explanations of model behavior cannot be assumed to reflect metacognitive awareness. Theoretically, the paper advocates for a shift toward paradigms informed by the human metacognition literature, with stronger mechanistic controls and dissociation-based diagnostic criteria.

Future Directions for AI Self-Reflection

The work suggests several promising avenues:

• Mechanistic Probing: Causal interventions targeting distinct circuits hypothesized to underpin introspective computation.
• Calibration-Based Designs: Evaluating dissociations between task accuracy and second-order confidence or self-report.
• Cross-Model Comparisons: Designs where privileged self-access is tested against equally informed external models, but with absence of confounding input cues.

These directions are critical for advancing robust, theory-driven evaluations of metacognitive monitoring in LLMs and for establishing any claim to introspective awareness.

Conclusion

This paper provides a rigorous, methodologically grounded critique of current introspection paradigms applied to LLMs, demonstrating that apparent metacognitive abilities are largely explained by input-level shortcuts or generic anomaly detection. Behavioral evidence, in isolation, cannot substantiate strong claims about dissociable introspective computation. This work raises the evidentiary bar for future research, emphasizing the necessity of mechanistic controls and construct-valid paradigms informed by cognitive science. The implications extend to interpretability, trustworthiness, and the theoretical study of emergent AI cognition.