ThoughtTrace: User Cognition in Conversational AI

• ThoughtTrace is a dataset that pairs real-world chat transcripts with explicit thought annotations, revealing latent motivations and satisfaction criteria in human-AI interactions.
• It employs a rigorous annotation protocol with defined reason and reaction categories, ensuring high-quality cognitive data through comprehensive user training and multi-step quality controls.
• Empirical analyses show that incorporating thought data significantly improves user-behavior prediction and assistant alignment, yielding over 40% relative performance gains compared to history-only models.

ThoughtTrace is a large-scale dataset designed to enable the empirical study of user cognition in human–AI conversational interactions by providing paired real-world chat transcripts and users’ self-reported motivations and reactions. Unlike prior conversational datasets, which capture only overt message exchange, ThoughtTrace introduces explicit user “thought” annotations that reveal latent drivers and satisfaction criteria underlying dialogue behavior. The dataset is intended to facilitate research on cognitive dynamics, user simulation, and assistant alignment by providing actionable signals for model training and evaluation (Jin et al., 19 May 2026).

1. Dataset Construction and Annotation Protocol

ThoughtTrace recruited 1,058 English-speaking participants via Prolific, operating under an IRB-approved protocol. Each participant—following informed consent—completed a tutorial and comprehension quiz to ensure understanding of the annotation interface and the concept of “thoughts.” Participants each performed two self-defined, “everyday” task-oriented conversations across 20 different LMs, with a 10-minute interaction window per task. During each turn, users provided private thought annotations: at user turns, they clicked “+ Reasons” to indicate the motivation for their prompt; at assistant turns, they clicked “+ Reactions” to record their feelings or judgments about the AI’s response. Post-task, users summarized their completed/expected outcomes and filled a demographic survey (age, gender, education, occupation, AI-use frequency, primary use cases).

Annotation guidelines defined seven reason categories:

• task_motivation
• task_continuation
• task_reorientation
• content_expectation
• style_expectation
• context_grounding_and_constraints
• social_and_others

Five reaction categories were specified:

• explicit_affirmation
• partial_satisfaction
• presentation_style
• scope_fit
• content_relevance

Quality control involved a combined approach: initial comprehension ensured by tutorial and quiz, automated filtering to exclude disengaged participants, and subsequent LLM-based quality filtering (only thoughts rated ≥4/5 retained for downstream tasks). As self-reported thoughts lack classical inter-annotator agreement, annotation consistency was instead monitored by multi-step LLM-based classification, rule-based normalization, and random human spot checks.

2. Corpus Statistics and Structural Characteristics

ThoughtTrace comprises 1,058 users, 2,155 conversations, and 17,058 turns, with 10,174 thought annotations (mean ≈0.60 thoughts/turn). Data were collected from 20 distinct LMs, which include proprietary and open-weight models such as GPT-5.4, Gemini 3.1, Claude Opus 4.6, xAI Grok 4.20, among others. Each model received 30–162 users, contributing to 60–337 conversations and 200–2,600 messages per model.

Topical coverage spans seven broad domains—Lifestyle, Business & Society, Education, Health, Technology, etc.—and 36 fine-grained topics (Travel 9.0%, Finance 9.3%, Education 7.7%, Dining 8.4%). No single topic accounts for over 10% of conversation volume, evidencing broad topical heterogeneity. The median conversation consists of 8 turns, markedly exceeding the 2-turn median observed in WildChat and LMSYS-Chat-1M. Token-length distributions range from 2,000–5,000, with a substantial long tail extending beyond 10,000 tokens.

Multi-turn user behaviors are classified into five types:

1. First request (25.2%)
2. Completely new request (12.5%)
3. Re-attempt/revision (2.9%)
4. New variation (2.3%)
5. Extend/deepen prior task (57.0%)

Formal notation for the dataset sets $U = \{u_1, \ldots, u_{1058}\}$ (users), $M = \{m_1, \ldots, m_{20}\}$ (models), $C = \{c_1, \ldots, c_{2155}\}$ (conversations), $T = \{t_1, \ldots, t_{10174}\}$ (thoughts), with each conversation $c_i$ as a sequence of messages, and a subset annotated with reason/reaction-type thoughts from $T$.

3. Semantic Analysis of Thoughts Versus Messages

Analytical experiments demonstrate that thoughts are semantically and distributionally distinct from conversational utterances. Embedding-level analysis using text-embedding-3-large revealed greater Euclidean shifts for message→reason and reaction→next message pairs than for consecutive message pairs. Representative metrics (centroid shift, maximum mean discrepancy (MMD), linear-probe AUC) are:

Paired Types	Centroid Δ	MMD	AUC
Current→Next message	0.120	0.096	0.721
Message→Reason	0.225	0.182	0.977
Reaction→Next message	0.320	0.257	0.988

LLM-based semantic coverage (1–5 scale) found message-to-reason overlap at 3.22 (“partial overlap, core missing”), and message-to-reaction at 2.00 (“minimal overlap”). When three frontier LMs (GPT-5.4, Gemini 3.1, Claude Opus) were prompted to infer reasons and reactions, held-out LLMs rated semantic similarity at 2.93/5 (reasons) and 2.54/5 (reactions). This indicates that thoughts are not reliably reconstructible from chat context alone, confirming their value as an independent modality for cognitive annotation.

4. Downstream Modeling and Empirical Utility

Experiments demonstrate the practical benefits of incorporating thought annotations in conversational modeling tasks.

a) User-Behavior Prediction

In next user-message prediction, augmenting dialogue history with both reason and reaction annotations improved model performance for all tested LMs. Semantic similarity was evaluated (0–100 scale) via a third-party model, with results:

Setting	GPT-5.4	Gemini 3.1	Opus 4.6	Avg
History only	21.4	22.1	21.3	21.6
Thoughts-augmented	27.4	28.9	35.5	30.6

The average relative gain was approximately +41.7%. This shows that thoughts provide actionable signals for simulating user trajectories and anticipating utterances.

b) Thought-Guided Rewrites for Assistant Alignment

Assistant response rewrites were evaluated under two paradigms: message-guided (using follow-up user messages) and thought-guided (using private “dissatisfaction” reactions and their labels). Qwen3.5-4B was fine-tuned on each variant with evaluation via the Arena-Hard benchmark (98.6% human correlation, judged by GPT-4o):

Method	Win (%)	SC Win (%)
Base Qwen3.5-4B	24.6	22.5
+ WildChat (msg-guided)	41.8	41.5
+ ThoughtTrace (msg-guided)	44.0	43.6
+ ThoughtTrace (thought-guided)	47.9	48.1

Thought-guided alignment yielded a +4.5% SC win over message-guided on ThoughtTrace and surfaced 2.2× more dissatisfaction instances than messages alone, substantiating the fine-grained alignment utility of thought data.

5. Implications, Limitations, and Research Outlook

ThoughtTrace establishes user thoughts as a novel data modality, empirically shown to: (1) capture latent motivations and satisfaction factors absent from observable chat, (2) remain semantically distinct and generally irrecoverable from surface messages, (3) span diverse cognitive categories that evolve across conversational stages, and (4) enhance downstream predictive and alignment outcomes. It enables the creation of more realistic user simulators that model latent cognition, supports novel training paradigms for assistants that anticipate implicit user goals, and motivates evaluation metrics centered on actual user satisfaction rather than surface-form correctness.

Limitations of the dataset include the reliance on elicited, self-reported reasoning (distinct from unconscious thought), potential reactivity effects due to annotation, and controlled recruitment (Prolific) rather than fully in-the-wild sampling. The absence of traditional inter-annotator agreement statistics is mitigated through interface training, automated/LLM-based quality screening, and post hoc validation.

A plausible implication is that, by augmenting observable chat with explicitly annotated cognitive context, future conversational AI systems can be more finely attuned to genuine user objectives, preferences, and sources of dissatisfaction. Thought-centered approaches may facilitate new lines of inquiry in user simulation, alignment, and adaptive personalization within large-scale dialogue datasets (Jin et al., 19 May 2026).