---
title: 'PrimeX: Dataset for Personalized Opinion Modeling'
url: https://www.emergentmind.com/topics/primex
type: topic
---

# PrimeX: Dataset for Personalized Opinion Modeling

Searching arXiv for “PrimeX” to confirm the exact topic and any closely related similarly named works.
Search query: PrimeX arXiv
PrimeX is a dataset introduced to study whether richer representations of an individual’s belief system can help language models better personalize to that person. It combines public-opinion responses, first-person explanations, and Primal World Beliefs from the same respondents, and is designed to support both NLP personalization research and psychological research on links between worldview, opinion, and explanatory language. The dataset contains data from **858 anonymous U.S. residents** and centers on the hypothesis that opinion prediction may improve when models use not only demographics or a few prior answers, but also written explanations and broader worldview information operationalized through Primal World Beliefs (“Primals”) [2510.00174].

## 1. Scope and motivating problem

PrimeX addresses a limitation in prior persona and opinion datasets for language-model personalization: they often contain only demographics or isolated opinions, usually not **multiple topics from the same individual**, and almost never richer cognitive signals like explanations or worldview [2510.00174]. Its guiding question is whether worldview and human-written explanations can serve as compact, useful user representations for opinion prediction and alignment.

The dataset is therefore structured around three linked layers of person-level information. The first is a set of public-opinion responses drawn from multiple Pew American Trends Panel (ATP) surveys. The second is a set of free-text explanations in which respondents justify selected answers in their own words. The third is a psychometrically grounded worldview survey, the **18-item Primal World Beliefs Inventory (PI-18)**. This combination makes PrimeX a dataset in which the same individual is represented through cross-topic opinions, explicit reasons, and an abstract worldview measure.

A plausible implication is that PrimeX shifts personalization research away from coarse demographic stereotyping toward user models that encode belief structure more directly. The paper explicitly frames this as relevant to **personalized language models (PA-LMs)** and to psychological analysis of how worldview relates to expressed opinion and explanatory language [2510.00174].

## 2. Corpus design, sample, and data collection

PrimeX contains four principal data components [2510.00174].

| Component | Size | Notes |
|---|---:|---|
| Public opinion questions | 30 | From three Pew ATP surveys |
| Written explanations | 9 | 3 explained items from each ATP survey |
| PI-18 worldview items | 18 | 6-point Likert responses |
| Demographic attributes | 11 | Self-reported participant metadata |

Each participant completed **30 public opinion questions**, **9 written explanations**, **18 Primal World Belief survey items**, and **11 demographic attributes**. The opinion questions were drawn from **three Pew American Trends Panel surveys**, with **10 questions selected from each**: **Wave 34** on biomedical and food issues, **Wave 41** on expectations about America in 2050, and **Wave 54** on economic inequality [2510.00174]. The authors manually chose items that ask about **personal opinions**, not factual knowledge, and that have relatively **high-entropy response distributions**. Questions were multiple-choice, some binary and some ordinal with 3–4 substantive options, and every question also included **“Prefer not to answer.”**

The explanation layer is one of the dataset’s main novelties. For **3 questions from each of the 3 ATP surveys**, respondents wrote **2–4 sentences** explaining why they held that opinion, drawing on “any aspect of your personal history, social life, experiences, thoughts, feelings, beliefs, or values” [2510.00174]. These explanations are attached to specific opinion choices rather than collected as general autobiographical text.

The sample comprises **858 anonymous U.S. residents**, all English-speaking and recruited through **User Research International**. Participants were selected to be relatively balanced across **male/female ratio, age range, and U.S. geographic region**, although the final sample retains a **female bias** and other demographic skews [2510.00174]. The appendix reports regional counts of South 269, West 263, Midwest 183, and Northeast 143; age groups of 18–29: 200, 30–49: 231, 50–64: 214, and 65+: 212; and gender counts of 489 female, 348 male, 18 non-binary, and 3 prefer not to say. Education is skewed upward relative to the U.S. average, and the sample underrepresents Spanish/Hispanic/Latino respondents.

The survey session was designed to take about **30 minutes**, with the PI-18 usually taking under 10 minutes. Participants gave informed consent, could skip any question, and could withdraw entirely after reviewing their responses; opt-out data were removed. The process was guided by an institutional review board [2510.00174].

Question order was fixed for all respondents: ATP Wave 34, ATP Wave 54, ATP Wave 41, PI-18, and then additional demographics. Within each survey section, the **3 opinion questions that required explanations came first**, followed by the remaining 7 questions. For modeling experiments, the paper uses a **user-level train/test split** of **430 train users** and **428 test users** [2510.00174].

## 3. Worldview representation through Primal World Beliefs

PrimeX represents worldview using the **18-item Primal World Beliefs Inventory (PI-18)**, a Likert-style instrument answered on a **6-point scale** from **Strongly disagree (0)** to **Strongly agree (5)**, with some items reverse-scored [2510.00174]. The PI-18 measures four high-level worldview dimensions:

- **Good**
- **Safe**
- **Alive**
- **Enticing**

The paper gives the scoring equations exactly as follows [2510.00174]:
\[
\text{Good} = (sd1+sd2x+sd3x+sd4x+sd5+sd6+ed1+ed2x+ed3x+ed4+ed5+ed6x+ed7+am1+am4)/15
\]

\[
\text{Safe} = (sd1+sd2x+sd3x+sd4x+sd5+sd6)/6
\]

\[
\text{Enticing} = (ed1+ed2x+ed3x+ed4+ed5+ed6x+ed7)/7
\]

\[
\text{Alive} = (am1+am2+am3x+am4+am5)/5
\]

Worldview is represented in two forms depending on the experiment: either as the raw **PI-18 item responses**, or as the derived continuous scores for **Good, Safe, Alive, and Enticing**, often accompanied by contextual descriptions of what each dimension means [2510.00174]. This dual representation is important because the prompting format differs by model: for **GPT-4o**, Primals are given as scores plus natural-language contextual descriptions, whereas for **Mistral 7B Instruct v0.3**, they are represented as PI-18 question/response pairs because of context constraints.

The sample’s worldview statistics were close to reported U.S. averages. Mean scores were **Good 3.09**, **Safe 2.50**, **Alive 2.66**, and **Enticing 3.73** [2510.00174]. This suggests that the worldview portion of the corpus was not dramatically atypical relative to prior U.S. findings, even though the respondent pool is demographically skewed in other respects.

Preprocessing of this component is straightforward and largely deterministic. The main choices were mapping PI-18 responses to numerical values 0–5, reverse-scoring selected items, computing the four Primal summary scores, ignoring “Prefer not to answer” when computing opinion–worldview correlations, and treating binary opinion questions as binary and non-binary questions as ordinal for correlation analysis [2510.00174].

## 4. Modeling tasks and personalization protocol

The paper evaluates PrimeX primarily through **opinion prediction** and **Primal prediction** [2510.00174]. In the opinion-prediction task, for user \(u\), the seed set is defined as
\[
S^u = \{(a_i^u,e_i^u) \mid q_i \in S\},
\]
and the unseen target answers as
\[
T^u = \{a_j \mid q_j \in T\}.
\]
The model predicts each unseen answer one at a time from a text-formatted persona:
\[
\hat{a}_j = LM_\mathcal{T}(q_j,U),
\]
where \(q_j \in T\) is the test question and \(U\) is the user representation. Output is constrained to a **single token** corresponding to one answer choice, and accuracy is the proportion of exact matches \(\hat{a}_j=a_j^u\) over users and held-out questions [2510.00174].

Two personalization settings are used. In **All Topics**, the seed set is the **9 explained opinions** and the test set is the remaining **21 opinions**, so \(|S|=9\) and \(|T|=21\). In **Cross Topic**, the seed set is **10 questions from Wave 34**, including the 3 explained ones, and the test set is **20 questions from Waves 41 and 54**, so \(|S|=10\) and \(|T|=20\) [2510.00174]. The second setting is explicitly harder because it requires generalization across topic areas.

The evaluated models are **GPT-4o** and **Mistral 7B Instruct v0.3**. The principal user representations are **Demographics**, **Demographics + Opinions**, **+ Primals**, **+ Explanations**, **+ Generated Explanations**, and **PrimeX persona**, where the last is defined as demographics + seed opinions + explanations + Primals [2510.00174]. Human explanations are inserted as “Reason:” fields attached to seed opinions. In the **Generated Explanations** condition, a fine-tuned GPT-4o generates synthetic explanations for each seed opinion conditioned on demographics and that opinion.

The paper also examines whether worldview can be inferred from responses and explanations. In that task, a model predicts PI-18 responses and the derived Primal scores are compared to the true ones using **MSE** [2510.00174]. This makes PrimeX not only a dataset for using worldview as conditioning information, but also for attempting to recover worldview from behavior and language.

## 5. Empirical structure and benchmark findings

Before personalization experiments, the paper shows that Primals correlate meaningfully with opinions [2510.00174]. It computes Pearson \(r\) for binary questions and Spearman \(\rho\) for ordinal ones. Among the larger reported associations are **Child Standard of Living in the future** with **Good: \(r=0.338\)** and **Safe: \(r=0.326\)**, **Evolution** with **Alive: \(r=0.322\)**, and **Gas Prices impact view of economy** with **Alive: \(r=0.310\)**. Using the paper’s cited psychological benchmarks, \(r=0.1\) is small, \(r=0.2\) typical, and \(r=0.3\) relatively large. The strongest associations were especially common in the future-oriented Wave 41 questions.

The paper also studies whether explanations reveal worldview. Using **zero-shot GPT-4o as a judge** on explanations from the 50 highest- and 50 lowest-scoring respondents per Primal, it finds that explanations from high-Primal users more often exhibit evidence of high belief, and vice versa [2510.00174]. This analysis is presented as exploratory rather than as ground-truth annotation. Explanation length also differed by worldview, with the striking result that lower-Alive respondents wrote much longer explanations.

The central personalization results are opinion-prediction accuracies. For **GPT-4o**, the reported results are [2510.00174]:

| User representation | All Topics | Cross Topic |
|---|---:|---:|
| Demographics | 42.36 | 39.24 |
| Demo + Opinions | 45.15 | 39.82 |
| + Primals | 46.21 | 40.42 |
| + Explanations | 48.12 | 40.21 |
| + Generated Explanations | 46.30 | 40.02 |
| PrimeX persona | 48.31 | 40.68 |

For **Mistral 7B Instruct v0.3**, the reported results are [2510.00174]:

| User representation | All Topics | Cross Topic |
|---|---:|---:|
| Demographics | 42.40 | 39.72 |
| Demo + Opinions | 44.37 | 40.12 |
| + Primals | 41.92 | 41.03 |
| + Explanations | 46.20 | 40.40 |
| + Generated Explanations | 45.13 | 40.37 |
| PrimeX persona | 45.44 | 40.78 |

These results support a differentiated interpretation. In **All Topics**, **human explanations** are the strongest auxiliary signal for both models, and the best GPT-4o result is **48.31** with the full **PrimeX persona** [2510.00174]. In **Cross Topic**, gains are smaller overall, and worldview appears more robust than explanations: **Mistral** significantly benefits from **Primals**, while **GPT-4o** shows a statistically significant improvement only with the full **PrimeX persona**. The paper’s synthesis is that explanations help most when the model can exploit same-topic or mixed-topic seed opinions, whereas worldview is especially useful for cross-topic personalization.

PrimeX is also used to examine **default-bias effects**. With only **Demographics**, model accuracy correlates with some Primals, suggesting that model defaults align better with certain worldview types. For GPT-4o, these correlations are **Good 0.15**, **Safe 0.13**, **Enticing 0.10**, and **Alive -0.04**; with the **PrimeX persona**, they weaken to **0.06**, **0.05**, **0.03**, and **-0.07** respectively [2510.00174]. For Mistral, the same pattern of attenuation is reported. The authors interpret this as evidence that richer persona information helps models move away from generic or default alignment tendencies.

Finally, the paper tests whether worldview can be predicted from behavior and language. For **Good**, **Safe**, **Alive**, and **Enticing**, the reported MSEs are **0.56, 1.13, 1.46, 0.61** for **Demo + Opinions**; **0.55, 1.40, 1.07, 0.62** for **+ Explanations**; and **0.46, 0.69, 1.22, 0.65** for the **trained GPT-4o** predictor [2510.00174]. The trained predictor significantly improves over the zero-shot methods for **Good** and **Safe**, but not for **Alive** or **Enticing**.

## 6. Explanation utility, ethics, limitations, and significance

A distinctive technical contribution of the paper is its formalization of explanation usefulness for personalization [2510.00174]. For a user \(u\), baseline conditioning uses demographics \(D\) plus a single seed question-answer pair \((q_i,a_i^u)\), denoted
\[
U = D + (q_i,a_i^u).
\]
The utility of an explanation \(e_i^u\) is defined as the expected log-likelihood gain on held-out opinions:
\[
\mathcal{M}(e_i^u) =
\mathbb{E}_{q_j \in T,\, a_j^u \in T^u}
\left[
\log(\mathcal{P}(a_j^u \mid q_j, U+e_i^u))
-
\log(\mathcal{P}(a_j^u \mid q_j, U))
\right].
\]
The paper states that this is equivalent to the reduction in cross-entropy on the user’s held-out opinions.

Empirically, utility scores ranged from **-0.503 to 0.317**, with **mean = 0.024** and **IQR = 0.054** [2510.00174]. High-utility explanations were much **longer**—**333 vs 144** average length—and more semantically similar to both seed and test question-response pairs. Their similarity to the seed question was **0.609 vs 0.484**, and to test questions **0.182 vs 0.151**. The question eliciting the highest-utility explanations was **Church Econ**; the lowest was **Medical Costs**. Low-utility explanations were more likely to reveal worldview according to the LLM judge, and the paper notes that this suggests current PA-LMs may not yet fully exploit worldview-rich text.

The ethical framing is conventional but explicit. Participants gave **informed consent**, could skip any question, could withdraw after completing the survey, and data collection was guided by an internal review board [2510.00174]. The paper also notes that the dataset contains subjective opinions that may be offensive to some users. It frames richer personalization as potentially reducing stereotyped model behavior, while acknowledging that persona-conditioned models can exhibit unfair bias.

Several limitations are emphasized. PrimeX contains only **English-speaking U.S. residents**; the sample underrepresents **Spanish/Hispanic/Latino** and **male** respondents; education is skewed upward; and the dataset size—**858 users**—is relatively small for large-scale NLP training [2510.00174]. The online survey format may disadvantage participants with lower technical comfort or weaker typing ability, especially for free-text explanations. Some experiments rely on **GPT-4o**, a proprietary model that may change over time, affecting reproducibility.

Taken together, PrimeX’s main contribution is a dataset that combines **opinions, human-written explanations, and validated worldview measures from the same individuals** [2510.00174]. The empirical record in the paper indicates that this combination is useful but unevenly so: explanations and worldview both help opinion prediction, yet their utility depends on topic transfer, model family, and whether the signal is expressed directly as structured worldview scores or indirectly through text. This suggests that PrimeX is best understood as a resource for more psychologically grounded personalization research rather than as a closed or saturated benchmark.

Source: https://www.emergentmind.com/topics/primex