PersonaAtlas: Synthetic Dataset for LLM Personalization

• PersonaAtlas is a synthetic dataset for large-scale, multi-turn personalized LLM interactions that captures dynamic user preferences and realistic query noise.
• It leverages the PersonaGym framework to simulate dialogues using specialized agents, incorporating latent user personas and controlled noise injection.
• PPOpt applies a reason-then-optimize approach to enhance personalization while preserving task accuracy, as confirmed by rigorous empirical evaluations.

PersonaAtlas denotes, in the large-language-model personalization literature, a large-scale synthetic dataset of high-fidelity multi-turn personalized interaction trajectories released together with the PersonaGym data-generation framework and the Personalized Prompt Optimization (PPOpt) method. It is designed to address two stated bottlenecks in personalized prompting: the absence of high-quality, privacy-sensitive data that capture personalized user–LLM interactions at scale, and the lack of robust reward signals for individual preferences. The dataset is organized around latent user personas, partial observability of those personas, dynamic preference revelation over multiple turns, and explicit modeling of realistic noise patterns in queries and feedback (Ma et al., 12 Feb 2026).

1. Definition and formalization

The central object underlying PersonaAtlas is a latent persona $p_u$ for each user $u$, represented as a set of categorical feature–value pairs,

$p_u = \{ (f_i, v_i) \}_{i=1}^d,$

where $f_i$ denotes a dimension and $v_i$ its value. The feature space includes examples such as verbosity, role, expertise, risk posture, time sensitivity, learning style, length, formatting, and citation style, and the formulation explicitly distinguishes between hard constraints and soft constraints. Hard constraints are must-satisfy, whereas soft constraints are optional or partially observed (Ma et al., 12 Feb 2026).

PersonaAtlas is motivated by the claim that task-level prompt optimization typically optimizes an aggregate objective and therefore neglects individual differences in preferred style, structure, verbosity, and latent constraints such as privacy, domain conventions, and risk posture. The formulation also assumes that real users rarely state all preferences explicitly. Instead, they reveal them gradually through multi-turn interactions, for example by rephrasing, requesting bullet points, or correcting tone. Within this framework, only an observed subset $o_u \subseteq p_u$ is initially available, sampled as

$$o_u \sim \mathrm{Sample}(p_u; \pi_{\mathrm{mask}}),$$

and compiled into a natural-language system prompt

$s_u = \mathrm{Compile}(o_u; \Psi).$

This explicit partial-observability model is a defining characteristic of PersonaAtlas and differentiates it from static persona–preference pair constructions (Ma et al., 12 Feb 2026).

2. PersonaGym and dynamic trajectory synthesis

PersonaAtlas is generated by PersonaGym, an agentic LLM system with three specialized agents: a User agent $M_{\mathrm{user}}$, an Assistant agent $M_{\mathrm{asst}}$, and a Distractor agent $u$0. The User agent simulates goal-directed queries and feedback conditioned on the compiled specification $u$1; the Assistant agent produces answers; and the Distractor agent injects controlled, semantic-aware noise into queries and feedback to model typos, missing constraints, or ambiguous requests (Ma et al., 12 Feb 2026).

For each synthetic user $u$2, PersonaGym generates a multi-turn dialogue

$u$3

where $u$4 is the user query, $u$5 is the assistant response, and $u$6 is the feedback. A follow-up $u$7 is issued if and only if the user is not satisfied; otherwise the trajectory terminates. Turn-level supervision is derived from the binary signal

$u$8

so that $u$9 denotes satisfaction. The resulting data model ties satisfaction labels to dialogue continuation behavior rather than to an externally defined scalar score (Ma et al., 12 Feb 2026).

PersonaGym also injects explicit preference signals and structured corruption. Query stylization is sampled as

$p_u = \{ (f_i, v_i) \}_{i=1}^d,$0

and the Distractor applies noise according to

$p_u = \{ (f_i, v_i) \}_{i=1}^d,$1

The corruption mechanism has three layers: syntactic noise, incomplete info, and semantic ambiguity. This design encodes the paper’s rejection of the common simplification that personalization can be treated as a static mapping from persona attributes to ideal outputs; instead, preference expression is treated as a dynamic process unfolding over interaction history (Ma et al., 12 Feb 2026).

The generation pipeline is specified in six steps: sampling a persona from a PersonaBank $p_u = \{ (f_i, v_i) \}_{i=1}^d,$2; masking features; compiling the system specification; sampling a seed query from a mixture including QA, summarization, and code; optionally injecting noise into the query; and then iterating over dialogue turns up to $p_u = \{ (f_i, v_i) \}_{i=1}^d,$3, with answer generation, feedback generation, optional feedback corruption, termination or follow-up decisions, label assignment, and history updates (Ma et al., 12 Feb 2026).

3. Dataset composition, scale, and schema

PersonaAtlas is described as a large-scale, high-quality, and diverse synthetic dataset of high-fidelity multi-turn personalized interaction trajectories. Its reported scale is approximately 2,000 distinct synthetic personas, more than 10,000 multi-turn conversations, and roughly 30,000 turn-level samples, with each turn constituting one training instance (Ma et al., 12 Feb 2026).

The personas are drawn from approximately 20–30 feature dimensions spanning three groups: basic profile, behavioral traits, and output constraints. The dataset parameters include noise application rates $p_u = \{ (f_i, v_i) \}_{i=1}^d,$4 each and stylization probability $p_u = \{ (f_i, v_i) \}_{i=1}^d,$5. Diversity is quantified by Self-BLEU, INGF, and TTR, while human evaluation is reported through Align@5, Plaus@5, and AgreeRate (Ma et al., 12 Feb 2026).

Aspect	Value	Note
Distinct synthetic personas	$p_u = \{ (f_i, v_i) \}_{i=1}^d,$6	Multi-feature latent personas
Multi-turn conversations	$p_u = \{ (f_i, v_i) \}_{i=1}^d,$7	Personalized trajectories
Turn-level samples	$p_u = \{ (f_i, v_i) \}_{i=1}^d,$8	One sample per turn
Self-BLEU $p_u = \{ (f_i, v_i) \}_{i=1}^d,$9	$f_i$0	Compared with $f_i$1–$f_i$2
INGF $f_i$3	$f_i$4	Diversity metric
TTR $f_i$5	$f_i$6	Diversity metric
Align@5	$f_i$7	Human evaluation
Plaus@5	$f_i$8	Human evaluation
AgreeRate	$f_i$9	Human evaluation

At the schema level, each JSON record stores the user identifier, turn index, observed persona, compiled preference specification, dialogue history, input query, assistant response, feedback, and the satisfaction label. In the notation of the release, a turn-level instance contains "user_id", "turn_index", "observed_persona", "preference_spec", "history", "input_query", "assistant_response", "feedback", and "label". The presence of both "observed_persona" and "preference_spec" preserves the distinction between structured latent features and their natural-language compilation, which is consequential for downstream prompt-optimization pipelines (Ma et al., 12 Feb 2026).

4. PPOpt: reason-then-optimize personalization

PersonaAtlas is paired with PPOpt, a scalable and model-agnostic framework that optimizes user prompts based on interaction histories without modifying the deployed LLM. PPOpt adopts a reason-then-optimize paradigm. Given state $v_i$0, the prompt optimizer $v_i$1 first generates a natural-language profile summary $v_i$2, and then rewrites the initial prompt $v_i$3 conditioned on $v_i$4. The stated purpose of this factorization is to avoid shortcut rewrites that ignore user preferences and thereby to avoid reward hacking (Ma et al., 12 Feb 2026).

The framework explicitly compares the unoptimized and optimized responses. The base response is

$v_i$5

and the optimized response is

$v_i$6

Optimization is driven by the multi-objective reward

$v_i$7

The profile inference reward is

$v_i$8

scored by an LLM-as-judge according to how well $v_i$9 matches ground-truth $o_u \subseteq p_u$0. The task outcome reward is

$o_u \subseteq p_u$1

based on pairwise preference judgment over correctness, clarity, and preference satisfaction (Ma et al., 12 Feb 2026).

Policy optimization is given in PPO-style form:

$o_u \subseteq p_u$2

where $o_u \subseteq p_u$3 and $o_u \subseteq p_u$4 is the advantage. The training algorithm has two phases. First, a cold-start SFT stage trains $o_u \subseteq p_u$5 on pseudo targets $o_u \subseteq p_u$6 generated by a strong teacher, GPT-5.2, using token-level cross-entropy. Second, an RL stage repeatedly samples $o_u \subseteq p_u$7 prompt candidates, obtains $o_u \subseteq p_u$8 and $o_u \subseteq p_u$9, computes $$o_u \sim \mathrm{Sample}(p_u; \pi_{\mathrm{mask}}),$$0 and $$o_u \sim \mathrm{Sample}(p_u; \pi_{\mathrm{mask}}),$$1, updates $$o_u \sim \mathrm{Sample}(p_u; \pi_{\mathrm{mask}}),$$2 with policy gradient or PPO, and anneals $$o_u \sim \mathrm{Sample}(p_u; \pi_{\mathrm{mask}}),$$3 to balance profile versus task emphasis (Ma et al., 12 Feb 2026).

5. Training configuration and implementation

The synthetic generation stack and the optimization stack use different model roles. The preference spec compiler $$o_u \sim \mathrm{Sample}(p_u; \pi_{\mathrm{mask}}),$$4, the User agent, and the Distractor agent use GPT-5.2, while the Assistant agent uses Llama-3.3-70B-instruct. The masking policy $$o_u \sim \mathrm{Sample}(p_u; \pi_{\mathrm{mask}}),$$5 applies random per-feature drop at 50%, noise strengths are $$o_u \sim \mathrm{Sample}(p_u; \pi_{\mathrm{mask}}),$$6, and the maximum number of turns is $$o_u \sim \mathrm{Sample}(p_u; \pi_{\mathrm{mask}}),$$7 (Ma et al., 12 Feb 2026).

For PPOpt supervised fine-tuning, the base architecture is Llama-3-8B-instruct, or alternatively Qwen3-8B or GPT-oss-20B, with LoRA rank $$o_u \sim \mathrm{Sample}(p_u; \pi_{\mathrm{mask}}),$$8. Optimization uses AdamW with peak learning rate $$o_u \sim \mathrm{Sample}(p_u; \pi_{\mathrm{mask}}),$$9, minimum learning rate $s_u = \mathrm{Compile}(o_u; \Psi).$0, 50 warm-up steps, cosine decay, batch size 128, and 3 epochs. The RL algorithm is Group Relative Policy Optimization, described as a variant of PPO, with clip $s_u = \mathrm{Compile}(o_u; \Psi).$1 (Ma et al., 12 Feb 2026).

Sampling and reward evaluation are also specified. PPOpt samples $s_u = \mathrm{Compile}(o_u; \Psi).$2 completions per state with temperature $s_u = \mathrm{Compile}(o_u; \Psi).$3 and top-$s_u = \mathrm{Compile}(o_u; \Psi).$4. GPT-4o-mini is used for $s_u = \mathrm{Compile}(o_u; \Psi).$5, while GPT-4o-mini or GPT-5.2 is used for $s_u = \mathrm{Compile}(o_u; \Psi).$6. Reward weights are $s_u = \mathrm{Compile}(o_u; \Psi).$7 and $s_u = \mathrm{Compile}(o_u; \Psi).$8, tuned via grid search, and training runs for 10,000 steps. These details show that PersonaAtlas is not merely a data release but part of an end-to-end personalization pipeline with explicit synthetic supervision, structured reward modeling, and lightweight adaptation via LoRA rather than modification of the deployed assistant model (Ma et al., 12 Feb 2026).

6. Empirical behavior, limitations, and future directions

Evaluation uses two LLM-as-judge metrics: a personalization score that rates alignment of output to $s_u = \mathrm{Compile}(o_u; \Psi).$9 on a 0–10 scale, normalized to $M_{\mathrm{user}}$0, and a task completion score that rates correctness and completeness against ground truth. On synthetic benchmarks over five held-out domains, PPOpt increases personalization from 5.41 to 7.20, a gain of $M_{\mathrm{user}}$1 or $M_{\mathrm{user}}$2, while task completion changes by only $M_{\mathrm{user}}$3 or $M_{\mathrm{user}}$4. Across backbones, reported personalization gains range from $M_{\mathrm{user}}$5 to $M_{\mathrm{user}}$6 on GPT-4o-mini, GPT-5.2, Llama-3.3-70B, Qwen3-32B, and Claude-Sonnet, with task completion remaining within $M_{\mathrm{user}}$7. On a real-world test set of human-edited interactions, personalization improves by $M_{\mathrm{user}}$8 to $M_{\mathrm{user}}$9 across three optimizers, while task completion changes by at most $M_{\mathrm{asst}}$0 (Ma et al., 12 Feb 2026).

The ablation results are especially important for interpreting PersonaAtlas as a training substrate rather than just a benchmark. SFT alone yields a strong personalization boost over vanilla prompting. RL without $M_{\mathrm{asst}}$1 is reported as unstable and can regress for some models. Full PPO with both rewards yields the highest and most stable gains, which the paper presents as confirmation that explicit profile inference guidance is necessary. This directly counters the misconception that prompt rewriting alone suffices for personalization if a task-level reward is available (Ma et al., 12 Feb 2026).

The reported limitations are equally explicit. Synthetic personas may not capture all real-user idiosyncrasies, and the judge models are LLM-based and may inherit biases. The future directions named in the work are fine-tuning on real user logs or human-edited trajectories to close the sim2real gap, extending the approach to multimodal or dialog-state settings, incorporating richer latent-state dynamics such as evolving preferences over thousands of turns, and exploring human-in-the-loop RLHF with real feedback signals (Ma et al., 12 Feb 2026).

7. Unrelated neuroimaging usage of the term

In an unrelated neuroimaging usage presented in a summary of data-driven probabilistic atlas construction, “PersonaAtlas” denotes a personalized whole-brain probabilistic atlas assembled from a dictionary of regional structural atlases and probabilistic label maps. In that setting, a point distribution model is learned for each of 132 anatomical regions, regional shapes are clustered into anatomical phenotypes, and a new subject is matched to region-specific dictionary entries by Pearson correlation after affine registration to MNI space. The resulting region-wise probabilistic maps are normalized to form a personalized whole-brain atlas, and evaluation is carried out by Dice similarity against ground-truth segmentation (Huo et al., 2018).

That neuroimaging usage is methodologically distinct from the LLM-personalization PersonaAtlas. The former concerns subject-specific spatial priors for MRI interpretation and processing; the latter concerns synthetic multi-turn interaction trajectories for personalized prompting and prompt optimization. The shared label therefore does not indicate a shared research lineage, but rather two separate notions of personalization organized around the broader idea of an atlas tailored to an individual instance (Huo et al., 2018).