WildChat-4.8M: Dialogue Dataset for Personalized RLHI

• WildChat-4.8M is a large-scale two-party dialogue dataset derived from ChatGPT logs, designed for research in RLHI and personalized model alignment.
• It employs robust filtering and data curation techniques to retain high-quality dialogues for coherent interaction modeling and persona inference.
• The dataset’s comprehensive structure and metadata enable novel RLHI methods, including persona-conditioned DPO and reward-based optimization.

WildChat refers to a large-scale dataset of real-world, two-party dialogues collected from user interactions with ChatGPT in production environments. Comprising approximately 1,000,000 distinct conversations, WildChat serves as a foundational resource for research in personalized model alignment, particularly supporting novel reinforcement learning from human interaction (RLHI) paradigms. All data referenced in major RLHI experiments and evaluations, including those pertaining to user-guided rewrites and persona-driven reward modeling, are derived from this ∼1M-conversation corpus, which spans a broad range of user intents and interaction contexts (Jin et al., 29 Sep 2025).

1. Collection Methodology and Filtering

WildChat consists of user-assistant dialogue logs sourced directly from in-the-wild sessions on deployed ChatGPT systems. No explicit user demographic or temporal metadata is made available in the published descriptions, and the precise timespan of collection is unspecified. Data curation involves a multi-step filtering pipeline:

• Exclusion of non-English prompts via language detection annotations.
• Removal of all instructions referencing Midjourney (i.e., image-generation requests).
• User retention criteria are set such that only users with at least 3 but fewer than 100 distinct conversations are retained, facilitating persona inference while avoiding low-quality or botscripted behavior.
• Conversations exceeding 10 turns are discarded to maintain interaction coherence.
• Only users providing "meaningful feedback" (filtered via an LLM classifier) are retained.
• No explicit profanity filtering or detailed privacy anonymization is reported beyond the above protocol.

A plausible implication is that these filtering steps aim to maximize the quality and coherence of the dialogue corpus, with sufficient diversity for downstream preference modeling and personalized alignment.

2. Dataset Composition and Metadata

Each WildChat entry represents a two-party (user, assistant) dialogue with comprehensive turn-level and dialogue-level metadata:

• Dialogue-level: Implicit dialogue_id, anonymized user_id, and ordered sequence of turns.
• Turn-level: Speaker identifier ("user" or "assistant"), raw message text, and turn index within the dialogue. Timestamps and per-turn token counts are not published.
• Persona metadata: For each user $u$, a natural-language persona $p_u$ is synthesized by prompting an LLM on the user’s cumulative message history. Personas comprise up to five bullet points that describe stable preferences, typically mapping to 3–5 preference dimensions (e.g., stylistic tendencies, detail preference, etc.). Persona embeddings (via OpenAI’s text-embedding-3-small) are used post hoc to analyze diversity, but are not part of training workflows.

This structure underpins RLHI methods that condition on explicit human context, linking persistent user traits to turn-level behavioral signals.

3. Statistical Properties and Data Splits

The dataset contains approximately 1,000,000 dialogues, with a reported mean dialogue length of 2.54 turns ($\approx$2.54 million total turns). Total token count, as well as the median and standard deviation of dialogue lengths, are not reported.

Message-type distribution based on a random sample of user messages is as follows:

Message Type	Proportion (%)
Initial request	27.07
New request/topic shift	40.40
Re-attempt w/ feedback	26.51
Re-attempt w/o feedback	4.77
Positive feedback	1.25

Topical coverage is qualitatively broad, spanning creative writing, programming, Q&A, reasoning/math, and personal advice. The dataset is shown to cover a wider contextual diversity (average cosine distance = 0.865) than HH-RLHF or HelpSteer2. No per-domain percentages or finer-grained topical distributions are provided.

Training and evaluation splits are as follows:

• 80% of dialogues are used for RLHI training, with 20% reserved for evaluation.
• For WildChat UserEval, 100 users with at least 10 conversations are selected; all but the last five are designated as reference history, with the final five forming the held-out set per user.
• No additional stratification or random seed specification is described.

4. Use in Personalized RL and Optimization Objectives

WildChat enables two core RLHI approaches for model alignment:

User-Guided Rewrites employs a persona-conditioned DPO (Direct Preference Optimization) objective: $$L_{\text{persona-DPO}} = E_{u,i}\left[ \log \sigma \left( \beta \cdot \Big( \log \frac{\pi_\theta(y^+_{u,i} \mid x_{u,i}, p_u)}{\pi_{\text{ref}}(y^+_{u,i} \mid x_{u,i}, p_u)} - \log \frac{\pi_\theta(y^-_{u,i} \mid x_{u,i}, p_u)}{\pi_{\text{ref}}(y^-_{u,i} \mid x_{u,i}, p_u)} \Big) \right) \right]$$ where $\pi_\theta$ is the current policy, $\pi_{\text{ref}}$ is a frozen reference model, $y^+$ represents user-guided rewrites, $y^-$ is the original response, $x_{u,i}$ is the multi-turn prompt, and $p_u$ is the persona. $p_u$0 governs loss sharpness.

User-Based Rewards: For each $p_u$1, $p_u$2 candidate responses are sampled conditioned on the prompt and persona. Each is scored by a reward model $p_u$3. $p_u$4 and $p_u$5 are selected as the pair with maximal and minimal reward, and DPO optimization proceeds as above.

These approaches operationalize direct persona-level preference learning and are directly tied to the structure and metadata furnished by WildChat.

5. Evaluation Metrics

Key model evaluation metrics rely on both subjective and benchmarked outcomes:

• Personalization win-rate (%): Percentage where the RLHI model's answer is judged more aligned with the user's persona compared to baseline.
• Instruction-following win-rate (%): Percentage where the answer better obeys the instruction.
• UserEval win-rate (%): Holistic preference metric combining personalization and instruction-following.
• Standard benchmarks: AlpacaEval2 (length-controlled and raw win %), Arena-Hard score (%), and reasoning accuracy (averaged over Minerva, OlympiadBench, GPQA, MMLU-Pro).

WildChat enables not only model training on organic conversational preferences but also realistic testbeds for user-centric evaluation of alignment and generalization.

6. Availability, Licensing, and Policy Constraints

No public release URL or explicit open-source license for WildChat is specified. All data derive from Meta internal logs and are thus subject to Meta's proprietary data use and privacy policies, which are not detailed in the publication. There is no indication of a permissive or Creative Commons license.

This lack of explicit public access constrains third-party reproducibility and independent analysis, restricting WildChat's direct utility to projects internal to or partnered with Meta, absent further licensing disclosures (Jin et al., 29 Sep 2025).