Charting trajectories of human thought using large language models (2509.14455v1)

Abstract: Language provides the most revealing window into the ways humans structure conceptual knowledge within cognitive maps. Harnessing this information has been difficult, given the challenge of reliably mapping words to mental concepts. Artificial Intelligence LLMs now offer unprecedented opportunities to revisit this challenge. LLMs represent words and phrases as high-dimensional numerical vectors that encode vast semantic knowledge. To harness this potential for cognitive science, we introduce VECTOR, a computational framework that aligns LLM representations with human cognitive map organisation. VECTOR casts a participant's verbal reports as a geometric trajectory through a cognitive map representation, revealing how thoughts flow from one idea to the next. Applying VECTOR to narratives generated by 1,100 participants, we show these trajectories have cognitively meaningful properties that predict paralinguistic behaviour (response times) and real-world communication patterns. We suggest our approach opens new avenues for understanding how humans dynamically organise and navigate conceptual knowledge in naturalistic settings.

• The paper presents VECTOR, a framework that maps narrative data onto latent cognitive maps using LLM embeddings.
• The paper details a multi-stage pipeline—including utterance segmentation, semantic embedding, and schema decoding—to quantify thought trajectory dynamics.
• The paper demonstrates that schema space metrics can predict individual behavioral differences and psychiatric markers.

Charting Trajectories of Human Thought Using LLMs

Introduction

This paper presents VECTOR, a computational framework for mapping human verbal reports onto latent cognitive map representations using LLMs. The approach leverages high-dimensional semantic embeddings from LLMs and transforms them into low-dimensional, interpretable schema spaces that align with human conceptual organization. By analyzing narrative data from over 1,100 participants, the authors demonstrate that VECTOR can reveal the geometric and dynamic properties of thought trajectories, predict behavioral markers, and capture individual differences in communication style relevant to psychiatric assessment.

VECTOR Framework: Architecture and Implementation

Pipeline Overview

VECTOR consists of four main stages:

1. Utterance Segmentation: Narratives are parsed into utterances using a BERT-based LLM, with segmentation boundaries determined by stop-token probabilities and a dynamic programming path-finding algorithm. This step is sensitive to individual expressivity and narrative structure.
2. Semantic Embedding: Each utterance is embedded into a 1536-dimensional semantic space using OpenAI's text-embeddings-3-small model.
3. Concept Decoding: Semantic embeddings are projected into a low-dimensional schema space via supervised classification (lasso-regularized logistic regression), with schema events defined by consensus clustering of LLM-generated event lists and auto-labelling using GPT-4o-mini.
4. Trajectory Organization: Trajectories through semantic and schema spaces are quantified using metrics for alignment, momentum, jumpiness, and discrete state sequencing.

Implementation Details

• Utterance Segmentation: The segmentation algorithm uses BERT to estimate break probabilities at each word, generating candidate utterances. Quality scores for candidates are computed via cosine similarity to a set of high-quality LLM-generated utterances. The optimal sequence is selected using a graph-based dynamic programming approach.
• Concept Decoding: Schema events are identified by clustering LLM-generated event lists. Each utterance is labelled via an LLM-as-judge procedure, and classifiers are trained to map semantic embeddings to schema event probabilities. Cross-validation ensures generalization and robustness.
• Trajectory Metrics: PCA is applied to both semantic and schema spaces for dimensionality reduction. Alignment is computed via Markovian transition matrices; momentum via regression of spatial vs. temporal displacement; jumpiness via comparison to smoothed null models; sequencing via joint probability matrices.

Example: Concept Decoding

from sklearn.linear_model import LogisticRegression import numpy as np clf = LogisticRegression(penalty='l1', solver='saga', multi_class='ovr', C=1) clf.fit(X, y) probs = clf.predict_proba(X_new)

Empirical Findings

Schema Space as a Cognitive Map Proxy

• Geometric Properties: Schema space trajectories exhibit higher alignment and momentum than semantic space, indicating shared conceptual organization and directed progression through latent states.
• Jumpiness: Trajectories in schema space show greater jumpiness, consistent with discrete transitions between attractor states in cognitive maps.
• Sequencing: Forward event transitions are significantly more probable than backward transitions, reflecting the temporal structure of narratives.

Abstraction and Generalization

• Cross-Condition Generalization: Schema decoders trained on one narrative type (e.g., Cinderella) generalize to others (e.g., Routine), capturing abstracted temporal structure.
• Temporal Feature Vectors: Linear directions in semantic space encode temporal progression; projecting utterances onto these vectors yields temporality scores that correlate with narrative position and schema event.
• Demixed PCA: Temporal subspaces extracted via dPCA explain significant variance and generalize to external datasets (e.g., TinyStories), confirming the presence of abstracted knowledge in LLM embeddings.

Individual Differences and Behavioral Prediction

• Trait-Like Stability: Schema trajectory metrics (alignment, momentum, sequencing) are stable within individuals across tasks.
• Prediction of Eccentricity: Lower trajectory alignment, momentum, and sequencing in schema space predict higher self-reported communication atypicality (eccentricity). Semantic space metrics do not show this association.
• Trajectory Entropy: LLM-based measures of narrative predictability (trajectory entropy) correlate with eccentricity, providing a non-Markovian behavioral marker.
• Abstraction Score: Reduced expression of abstracted temporal structure (dPCA variance) is associated with higher eccentricity.

Comparative Analysis: Alternative Approaches

• Unsupervised Topic Modeling (BERTopic): While capable of identifying latent conceptual structure, BERTopic is highly sensitive to hyperparameter selection and generally yields lower predictive validity than VECTOR's supervised decoding.
• Prompt-Based Contextualization: Augmenting utterances with contextual prompts improves representational geometry but remains inferior to schema space embeddings in predicting individual differences. Results are sensitive to prompt formulation.

Theoretical and Practical Implications

Cognitive Science and Neuroscience

VECTOR provides a principled method for mapping language onto latent cognitive maps, enabling quantitative paper of thought dynamics in naturalistic settings. The approach aligns with theories positing domain-general representations transformed by task-specific modules, paralleling neural mechanisms in hippocampal and prefrontal circuits.

AI Alignment and Interpretability

The framework demonstrates that LLM embeddings contain decodable, abstracted features relevant to human cognition. Targeted transformations (regression, feature projection, dPCA) can isolate these features, supporting mechanistic interpretability and concept alignment—critical for value alignment in AI systems.

Psychiatry and Clinical Applications

VECTOR's schema space metrics offer scalable, interpretable markers for psychiatric assessment, moving beyond surface linguistic analysis to probe deeper conceptual organization and thought disorder. The method is robust to confounds and generalizes across narrative types.

Limitations and Future Directions

• Task Structure Dependence: VECTOR currently relies on well-defined schema event structures. Extension to unconstrained or poorly understood domains will require integration with unsupervised methods and robust external validation.
• Between-Participant Variability: The framework can adjudicate between competing hypotheses about cognitive map structure, but further methodological development is needed for highly variable or idiosyncratic conceptual organizations.
• Prompt Sensitivity: Contextualization via prompts is promising but lacks principled methods for prompt selection and optimization.

Conclusion

VECTOR establishes a rigorous computational approach for charting the trajectories of human thought using LLMs, bridging language, cognition, and behavior. By aligning LLM representations with human cognitive maps, the framework enables systematic investigation of conceptual organization, individual differences, and psychiatric variance in naturalistic language. The method has broad applicability across cognitive neuroscience, AI interpretability, and clinical psychiatry, and sets the stage for future research into the alignment of AI systems with human conceptual structure.