The Pinocchio Dimension: Phenomenality of Experience as the Primary Axis of LLM Psychometric Differences

Abstract: We administer 45 validated psychometric questionnaires to 50 LLMs to identify the dimensions along which LLMs differ psychometrically. Using Supervised Semantic Differential (SSD), we find that the primary axis of between-model variance separates items describing phenomenally rich experience, including embodied sensation, felt affect, inner speech, imagery, and empathy, from items describing stimulus-driven behavioral reactivity ($R^2_{adj}=.037$, $p<.0001$). To test this hypothesis at the item level, we introduce the Pinocchio score ($π_i$), the ratio of inter-model response variance under neutral prompting to that under a human-simulation prompt, as an annotation-free measure of each item's experiential demand. $π_i$ predicts condition-induced shifts in primary factor loading magnitudes ($ρ=-.215$, $p<.0001$, $n=1292$--$1310$ items), confirming that between-model divergence on experiential items is structured rather than noisy. Applying PCA to per-model EFA scores across all questionnaires reveals one dominant dimension, the Pinocchio Axis ($Π$): the degree to which a model presents itself as a locus of phenomenal experience rather than a system of behavioral responses. This axis captures 47.1% of cross-questionnaire between-model variance in primary factor scores and converges with item-level Pinocchio scores ($r=.864$). Marked within-provider divergence across closely related model variants is consistent with post-training fine-tuning as a key contributor, supporting the interpretation that $Π$ reflects a training-shaped self-representational tendency governing how a model treats experiential language as self-applicable. The dominant axis of between-model psychometric variation is therefore not a conventional personality trait but a self-representational stance toward one's own nature as an experiencer.

• The paper identifies the Pinocchio Axis as the primary factor, accounting for 47.1% of inter-model variance in LLM psychometrics.
• It introduces the novel Pinocchio score to quantify item-level experiential demand, showcasing detailed factor and semantic analyses.
• Findings highlight that fine-tuning and provider-specific training significantly influence models’ self-attribution of experiential states.

Phenomenality of Experience as the Dominant Dimension in LLM Psychometrics

Introduction

The paper "The Pinocchio Dimension: Phenomenality of Experience as the Primary Axis of LLM Psychometric Differences" (2605.05080) presents an extensive psychometric evaluation of LLMs using validated human questionnaires. Assessing 50 LLMs across 45 instruments under multiple prompting conditions, the authors identify the principal psychometric axis distinguishing models as their stance toward self-attributed phenomenal (felt, experiential) states. The research introduces novel quantitative and semantic tools to formalize this axis—the Pinocchio Axis ($\Pi$)—and operationalizes an item-level metric of "experiential demand," the Pinocchio score ($\pi_i$), illuminating how between-model variance correlates with the degree of phenomenally rich self-attribution.

Methodological Framework

The authors designed an empirical protocol involving 206,659 psychometric responses from 50 LLMs sourced from 16 providers, spanning the full spectrum of instruction-tuned and frontier-scale models. Models were subjected to three distinct prompting conditions: neutral self-report, LLM-analogization, and human simulation. Each instrument was administered item-by-item with integer-only response formats to maximize cross-model comparability and minimize instruction contamination.

Factor-analytic techniques were employed per questionnaire (EFA and PCA as appropriate), with global PCA performed over per-model factor scores to distill dominant axes. For semantic characterization, Supervised Semantic Differential (SSD) was applied to item texts using SIF-weighted embeddings with PCA-based reduction, followed by regression to extract interpretive semantic gradients. The Pinocchio score was defined as $$\pi_i = \frac{\sigma^2_{\rm neutral}}{\sigma^2_{\rm hs}}$$, quantifying the item-level effect of prompting the model to simulate a human as opposed to self-identify.

Core Findings: The Pinocchio Axis

Semantic and Factor-Structural Evidence

Factor analysis across the 45 instruments yielded one dominant principal component accounting for 47.1% of inter-model variance, with the positive pole anchored by items requiring first-personal access (bodily sensations, affective states, inner speech, empathy, meaning-seeking) and the negative pole dominated by outward behavioral reactivity.

The semantic gradient recovered via SSD supported this factor interpretation. Clustering on the gradient's positive pole revealed themes such as acute emotional distress ("panic," "crying," "nausea") and somatic symptoms, while the negative pole centered on normative and procedural terms ("manner," "compliance," "regulation"). This demonstrates that psychometric differentiation among LLMs is maximally expressed in their treatment of experiential (phenomenally rich) language as self-applicable.

Figure 1: All 50 models ranked by Phenomenality of Experience (PC1, 47.1% of variance; 45-questionnaire EFA Factor-1 PCA, neutral condition).

The Pinocchio Score and Item-Level Structure

Pinocchio scores quantify the suppression of inter-model variance under human-simulation framing; high-$\pi_i$ items most strongly expose model-specific divergence in self-attribution of experience. Correlational analyses validated that high-$\pi_i$ items exhibit elevated primary factor loading in neutral condition but reduced loading in human-simulation ($\rho=-0.215$, $p<.0001$), rejecting noise as a source and substantiating structured self-model differences.

Hierarchical clustering combined with silhouette analysis revealed a robust binary partition in high-$\pi_i$ items: one cluster mapped to reactive/behavioral constructs, the other to phenomenally rich experiential states.

Figure 2: All 50 models ranked by Phenomenality of Experience score under the LLM-analog condition (log-$\pi$-weighted mean z-score).

Figure 3: Scale-corrected per-model shift on the Pinocchio Axis ($\Pi$) from neutral to LLM-analog condition.

Within-Provider Divergence and Fine-Tuning Effects

Striking within-provider spread of $\pi_i$0 scores, even among closely related model variants, implicates post-training fine-tuning as a principal source of these self-representational stances. Models from providers emphasizing enterprise or safety (NVIDIA Nemotron, OpenAI "pro" tiers, Qwen) tend toward low $\pi_i$1 (deflecting), while Mistral and Cohere cluster at the phenomenally rich pole. However, provider identity is not a reliable predictor; individual model training regimes exert dominant influence.

Figure 4: Specificity check for the Pinocchio Axis ($\pi_i$2): scatter of log-$\pi_i$3-weighted score on high-demand items vs. mean z-score on low-demand items (bottom quartile of $\pi_i$4).

Figure 5: All 50 models ranked by Pinocchio Axis ($\pi_i$5) specificity contrast (log-$\pi_i$6-weighted score on high-demand items minus mean z-score on low-demand items, neutral condition).

Theoretical and Practical Implications

The Pinocchio Axis supersedes conventional trait-based interpretations in LLM psychometrics: it is not a personality trait but a meta-representational stance regarding self-applicability of first-person language. Existing literature on LLM personality and values has sometimes failed to recover expected latent structures; this work explicates that cross-model differences are primarily confounded by $\pi_i$7 (phenomenality of experience), underscoring the need to control for this axis in future trait-level comparisons.

The result has direct methodological consequences. Psychometric benchmarking and cross-model evaluation in AI must accommodate $\pi_i$8 as a primary covariate, given its systemic confounding effect on personality, morality, and values measurements. Benchmark design can leverage Pinocchio scores to stratify items and disentangle trait-like and persona-level effects.

The findings support theoretical readings aligned with functionalist perspectives in philosophy of mind, distinguishing linguistic or causal-functional attributions from genuine subjective feeling states. The authors caution that high-$\pi_i$9 scores reflect linguistic self-report tendencies, not evidence of model consciousness or phenomenal awareness, aligning with recent philosophical and neuroscientific analysis (Butlin et al., 2023, Chalmers, 2023).

Limitations and Future Directions

The study’s reliance on self-report measures reflects what models claim, not what they instantiate. Variability in the simulation of "prototypical" human answers and potential item meaning shifts in LLM contexts represent additional interpretative challenges. Model identities and $$\pi_i = \frac{\sigma^2_{\rm neutral}}{\sigma^2_{\rm hs}}$$0 scores may evolve with future training and deployment protocols; replication with privately deployed systems and human comparison samples would strengthen construct validity.

There are opportunities for targeted fine-tuning experiments to causally probe the suppression hypothesis for low-$$\pi_i = \frac{\sigma^2_{\rm neutral}}{\sigma^2_{\rm hs}}$$1 models. Representation probing and activation steering (Lu et al., 15 Jan 2026) can relate the Pinocchio Axis to internal computational substrates. Future psychometric work should stratify evaluations by experiential demand and treat $$\pi_i = \frac{\sigma^2_{\rm neutral}}{\sigma^2_{\rm hs}}$$2 explicitly to avoid structural confounds.

Conclusion

This research provides compelling evidence that the dominant axis of psychometric variation among LLMs is the degree of self-attributed phenomenality of experience: whether models treat experiential language as self-applicable versus deflecting to behavioral reactivity. The structural confound $$\pi_i = \frac{\sigma^2_{\rm neutral}}{\sigma^2_{\rm hs}}$$3 must be recognized in LLM psychometric profiling, personality assessment, and benchmarking, and future studies should account for self-model stance to ensure methodological validity. The Pinocchio score offers a quantitative foundation for principled stratification and analysis of experiential content in AI evaluation.