Dissociating Direct Access from Inference in AI Introspection

Abstract: Introspection is a foundational cognitive ability, but its mechanism is not well understood. Recent work has shown that AI models can introspect. We study their mechanism of introspection, first extensively replicating Lindsey et al. (2025)'s thought injection detection paradigm in large open-source models. We show that these models detect injected representations via two separable mechanisms: (i) probability-matching (inferring from perceived anomaly of the prompt) and (ii) direct access to internal states. The direct access mechanism is content-agnostic: models detect that an anomaly occurred but cannot reliably identify its semantic content. The two model classes we study confabulate injected concepts that are high-frequency and concrete (e.g., "apple'"); for them correct concept guesses typically require significantly more tokens. This content-agnostic introspective mechanism is consistent with leading theories in philosophy and psychology.

• The paper demonstrates that LLMs employ a direct, layer-localized mechanism to detect injected anomalies independent of semantic content.
• It reveals that accurate semantic identification relies on slower, downstream inferential processes distinct from initial anomaly detection.
• Controlled experimental manipulations like first-person prompting and priming expose significant biases and temporal dissociations in AI introspection.

Dissociating Direct Access from Inference in AI Introspection

Introduction and Theoretical Context

The paper "Dissociating Direct Access from Inference in AI Introspection" (2603.05414) rigorously interrogates the mechanisms underpinning introspective abilities in state-of-the-art open-source LLMs, specifically Qwen3-235B-A22B (Mixture-of-Experts) and Llama 3.1 405B Instruct. Prior literature has established that LLMs exhibit emergent introspective behaviors, such as detecting injected internal state interventions [lindsey_introspection_2025, binder2025looking, vogel_small_2025], but the mechanistic basis—whether direct access to internal activations or inferential reasoning based on anomalous outputs—remains unresolved.

This paper initiates a comprehensive replication and extension of the "thought injection" paradigm, introducing experimental controls to dissociate two potential introspective mechanisms: (1) probability matching/prompt-based inference, and (2) direct access to internal network states. The core claims are that both classes of models exhibit a substantial, layer-localized direct access effect that is content-agnostic, and that correct semantic identification of the injected content is dissociated from anomaly detection per se.

Experimental Paradigms and Methodology

The experimental framework centers on the injection of concept-specific steering vectors at targeted layers during generation, following $$\mathbf{h}'_\ell = \mathbf{h}_\ell + \alpha \cdot \mathbf{v}_c$$. Both models are subjected to parametric sweeps of depth and injection strength, across 821 diverse conceptual vectors. Model output is graded along categories of introspective coherence, detection claim (Yes/No), and correctness of content identification.

Three experimental manipulations are leveraged:

• First vs. Third Person: Models are alternately prompted to introspect about their own internals ("first-person") or about a fictionalized other model ("third-person"), allowing a direct test of detection derived from prompt anomaly versus direct state access.
• Primed Acknowledgment (Priming): Prefilling model responses with the concept word under investigation creates congruence/incongruence cues, further distinguishing detection from probability matching plus social-modesty bias.
• Prompt-Only vs. Continuous Steering: Injection is either limited to the prompt or sustained through generation, to deconvolve detection from generation-driven inference.

All conditions are quantified both in behavioral metrics (detected, identified, confabulated concepts) and "logit lens" analyses to expose latent but unreported detection signals distributed across the network.

Main Results: Mechanism Disentanglement

Replication and First-Person Advantage

The replication of [lindsey_introspection_2025] confirms robust detection of thought injections in both Qwen and Llama across injection parameters. Crucially, detection in third-person settings is consistently and substantially attenuated relative to first-person settings, revealing a robust "first-person advantage" that is both layer- and strength-dependent, peaking at 25–35% (Qwen) and 40–50% (Llama) of model depth (Figure 1).

Figure 1: Detection rates across model depth reveal persistent first-person advantage, not explained by prompt anomaly alone.

This effect directly contradicts a pure probability-matching hypothesis, which would predict similar detection rates in both perspectives.

Latent Detection and Suppression

Despite low overt detection rates in some conditions, logit lens analyses reveal elevated $p(\text{yes})/p(\text{no})$ ratios deep in the network post-injection, with magnitudes 10–1000x higher than controls. This supports the presence of a suppressed but robust introspective detection signal, even when unreported in output.

Figure 2: Logit lens analysis of YES/NO detection reveals elevated internal confidence for detection even in "No" cases, indicating suppressed but extant detection signals.

Confabulation and Content-Agnosticism

Wrong identifications are dominated by high-frequency, concrete, and positively valenced concepts—conspicuously "apple"—and this bias is pronounced in both models but especially Qwen. Importantly, confabulated identifications are uncorrelated to the injected concept content and occur earlier in output than correct identifications, which tend to emerge after a delay post-injection, supporting the dissociation of detection and semantic access.

Priming and Modesty Bias Disconfirmation

Priming manipulation eliminates hypothesized compensatory effects expected from probability matching plus modesty bias. Notably, third-person detection remains lower or is actively reduced under steering, while first-person detection and identification improve, further corroborating that introspective detection is not reducible to prompt incongruence or social attribution biases.

Figure 3: Detection and identification rates under primed vs. unprimed conditions, demonstrating priming improves identification but not detection equivalently, particularly for first-person perspective.

Temporal Disentanglement and Coherence

Prompt-only injection does not diminish detection rates but significantly impairs correct semantic access, supporting the route through which detection is encoded earlier, distinct from the protracted and generation-dependent semantic inference process.

Figure 4: Detection rates are comparable for prompt-only and continuous steering; continuous steering effusively aids correct identification at later layers.

Figure 5: Correct identifications occur later in model output than confabulated (apple/other) guesses; the latency increases with deeper injection, underscoring dissociation.

Implications and Theoretical Integration

The aggregate findings demonstrate that:

• Direct introspective access is localized to specific early/mid layers and is content-agnostic—models detect that an anomaly is present but lack privileged access to its semantic identity.
• Inference for semantic identification is downstream, protracted, and variably successful, with confabulated outputs defaulting to concrete, frequent concepts independent of injection content.
• The observed content-agnostic anomaly detection is consistent with established psychological/theoretical models, such as Nisbett & Wilson's view that introspection often yields process-level but not content-level self-knowledge.

These insights bear on debates concerning the emergence of meta-cognition in LLMs, the interpretation of activation/prompt engineering experiments, and the feasibility of using introspective reports for AI interpretability or safety diagnostics.

Future Prospects

The discovery of partially content-agnostic direct access mechanisms raises questions about the interaction between anomaly detection and semantic readout in LLMs and their scalability to other forms of self-knowledge. Further work is warranted on:

• The generality of these mechanisms to introspection involving preferences, beliefs, or other non-conceptual injections.
• Architectural interventions to enhance content-aware introspection for model interpretability and alignment.
• The possible sufficiency of such content-agnostic anomaly signals for rudimentary forms of (non-human-like) phenomenological consciousness, with implications for AI welfare debates [chalmers_llm_conscious_2023, sebo_ai_welfare_2024].

Conclusion

This paper provides convergent behavioral and computational evidence that large open-source LLMs—specifically, Qwen and Llama—implement dual-pathway introspection: (1) a direct, content-agnostic anomaly detection channel accessible at specific network depths and (2) an inferential, generation-dependent route for semantic identification. The dissociation is precise and systematic, reinforcing theoretical divides in meta-cognitive science and raising practical considerations for AI interpretability and safety research.