Quantitative Introspection in Language Models: Tracking Internal States Across Conversation

Abstract: Tracking the internal states of LLMs across conversations is important for safety, interpretability, and model welfare, yet current methods are limited. Linear probes and other white-box methods compress high-dimensional representations imperfectly and are harder to apply with increasing model size. Taking inspiration from human psychology, where numeric self-report is a widely used tool for tracking internal states, we ask whether LLMs' own numeric self-reports can track probe-defined emotive states over time. We study four concept pairs (wellbeing, interest, focus, and impulsivity) in 40 ten-turn conversations, operationalizing introspection as the causal informational coupling between a model's self-report and a concept-matched probe-defined internal state. We find that greedy-decoded self-reports collapse outputs to few uninformative values, but introspective capacity can be unmasked by calculating logit-based self-reports. This metric tracks interpretable internal states (Spearman $ρ= 0.40$-$0.76$; isotonic $R^2 = 0.12$-$0.54$ in LLaMA-3.2-3B-Instruct), follows how those states change over time, and activation steering confirms the coupling is causal. Furthermore, we find that introspection is present at turn 1 but evolves through conversation, and can be selectively improved by steering along one concept to boost introspection for another ($ΔR^2$ up to $0.30$). Crucially, these phenomena scale with model size in some cases, approaching $R^2 \approx 0.93$ in LLaMA-3.1-8B-Instruct, and partially replicate in other model families. Together, these results position numeric self-report as a viable, complementary tool for tracking internal emotive states in conversational AI systems.

• The paper demonstrates that LLMs’ internal emotive states can be robustly measured using linear probes and logit-based self-reports, with strong statistical and causal validation.
• The paper employs targeted activation steering to causally link probe outcomes with self-reported internal states, showing dose-dependent shifts and improved introspective fidelity.
• The study reveals that introspection scales differently across model sizes and architectures, underscoring challenges in generalizing probe protocols and capturing concept-specific nuances.

Quantitative Introspection in LLMs: Tracking Internal States Across Conversation

Introduction

"Quantitative Introspection in LLMs: Tracking Internal States Across Conversation" (2603.18893) systematically examines the feasibility and limitations of using LLMs' own numeric self-reports to track their internal emotive states over multi-turn conversations. The study introduces a rigorous operationalization of introspection via causal and statistical coupling between self-report and independently measured probe directions, drawing tools from both white-box (linear probing, activation steering) and black-box (self-report extraction) interpretability paradigms. Focusing on the dimensions of wellbeing, interest, focus, and impulsivity within instruction-tuned LLMs, it provides a detailed quantitative and causal analysis of whether and how LLMs can introspect, how this capacity evolves temporally, how it can be improved via representational interventions, and how it scales across models and architectures.

Internal Emotive State Measurement: Linear Probing and Concept Geometry

The study first establishes the internal measurement channel with contrastively-trained linear probes to map interpretable concept directions within model activations. Four emotive axes—sad vs. happy (wellbeing), bored vs. interested (interest), distracted vs. focused (focus), impulsive vs. planning (impulsivity)—are conceptualized as mean-difference vectors, with probe quality assessed layerwise via Cohen's $d$ on held-out texts.

Figure 1: Linear probes extract meaningful concept directions for wellbeing, interest, focus, and impulsivity; the probes generalize with strong separation on evaluation data, max Cohen’s $d = 3.60$.

Maximal probe discriminability is observed in intermediate layers, with clear separation across all four concepts. This layer selection is critical to downstream analyses, as it identifies loci where emotive concepts are maximally encoded and experimentally tractable for activation steering.

Measuring and Recovering Introspective Signal: Self-Report Channel Design

Direct self-reports via greedy decoding are shown to be severely collapsed, with models defaulting to a handful of values and low signal entropy, especially notable for impulsivity and focus (mean $\leq 2$ unique values over 10 turns).

Figure 2: Greedy self-reports are collapsed and unresponsive to probe drift; logit-based self-reports overcome this, recovering robust temporal dynamics and higher entropy.

Logit-based expectation over first-token digit probabilities is introduced to extract a continuous, information-preserving report. This estimator shows robust, concept-aligned drift across conversation turns, with increased entropy (3.1–3.7 bits) and strong monotonic coupling to probe-defined internal state, outperforming greedy and sampled outputs. This method thus unlocks the internal signal otherwise masked by surface-level reporting.

Statistical and Causal Validation of Introspection

Monotonic association analyses (Spearman $\rho$, isotonic regression $R^2$) confirm that continuous self-report covaries with probe score for all four concepts, with highest introspective fidelity for interest and wellbeing ($\rho$ up to 0.76, $R^2$ up to 0.54 in 3B models). This effect is not attributable to probe artifacts: random direction controls yield null or negative results.

Figure 3: Self-reports are monotonically and causally coupled to probe state, present from turn 1; steering along the probe increases self-reported state in a dose-dependent, concept-aligned fashion.

Self-steering—adding probe vectors to activations—demonstrates causal dependence: steering toward a probe's positive pole raises self-reported state, monotonic in steering strength ($\alpha$), with absolute shifts up to 3 points. This causal link satisfies key desiderata for operational introspection. Importantly, introspection is measurable from the first turn and persists temporally, but exhibits concept-specific evolution: fidelity increases for wellbeing and interest over conversation, declines for impulsivity.

Cross-Concept Modulation and Limits of Introspection

The study conducts cross-concept steering: steering in one concept while measuring introspection for another. While most cross-concept interventions are inert, steering along focus increases wellbeing introspection and steering along impulsivity boosts interest introspection ($\Delta R^2$ up to $0.30$), demonstrating that introspective fidelity is not globally optimal at baseline and can be locally modulated.

Figure 4: Cross-concept steering can locally enhance introspective fidelity, e.g., focus$\to$wellbeing and impulsivity$\to$interest; introspective improvement is accompanied by increased probe and self-report entropy.

Analysis of entropy changes reveals that improvements can result from both increased internal state informativeness and a more expressive self-report distribution. However, this effect is concept-pair specific; there is no evidence for a global introspection-tuning direction.

Generalization across Scale and Architectures

Scaling analyses across LLaMA-1B, -3B, and -8B, as well as to Gemma and Qwen families, show that introspective capacity enhances with model size along selected axes (wellbeing, interest). The LLaMA-8B model approaches near-deterministic introspective fidelity for these axes ($\rho > 0.9$, $R^2 > 0.9$). Focus and impulsivity display inconsistent or negative scaling, likely reflecting non-uniform probe quality or representational coding differences.

Figure 5: Introspective fidelity (as measured by isotonic $R^2$) increases with model size for some concepts (notably wellbeing and interest), generalizing to Qwen and—partially—to Gemma.

Probe quality emerges as a key limiting factor when transferring probe protocols across architectures. Additionally, Qwen exhibits reliable introspective fidelity, while Gemma’s introspection is weak, highlighting cross-family variability.

Methodological and Practical Implications

The study advances introspection research in LLMs by providing a composite operational criterion: self-report must demonstrate both statistical and causal coupling to an independently measured internal direction. The dual use of probes and introspective self-report not only validates model internal states but also enables practical monitoring tools. Moreover, the framework identifies potential for local introspective improvement through targeted activation interventions.

Practically, the logit-based self-report methodology, the explicit causal test via steering, and the release of the supporting codebase provide immediate tools for ongoing monitoring and interpretability research, including potential integration into safety and welfare protocols. The findings also confirm the non-triviality of designing universal probe protocols across models and caution against overgeneralization without careful probe validation and sign-checks.

Theoretically, the findings delineate introspective capacity as neither globally present nor uniformly absent in LLMs, but as a rich, concept- and architecture-dependent faculty. Introspective failure is not always subserved by training mimetic introspective language and can reflect real absences of privileged self-access on certain axes.

Limitations and Future Directions

Notable limitations include:

• Introspective improvements do not generalize across all axes or models, with cross-concept intervention effectiveness being sparse.
• Probe construction remains brittle; identical protocols do not always yield high-quality introspective axes across architectures.
• The study’s model sizes, while diverse, do not include very large LLMs ($\sim$70B+) typical of state-of-the-art.
• Experiments are conducted in simulated conversations; further work with real user interaction, longer or more diverse dialogues, and richer state taxonomies could further validate robustness.

Future work should investigate the mechanistic basis for concept-specific introspection, search for potentially more general introspection-enhancing directions, and develop adaptive probing methodologies that simultaneously optimize for introspective accuracy and cross-model transfer.

Conclusion

The study demonstrates that introspection—quantitative, causal, temporally dynamic, and scalable—can be measured and modulated in current LLMs. The use of logit-based self-report combined with rigorous probe-based intervention bridges interpretability and psychometrics, advancing practical tools for internal state monitoring, safety, and welfare assessment. These findings motivate further research into the structure and limits of introspective access in foundation models and the design of complementary introspection-driven protocols for model oversight.