PerFairX: Fairness & Personality in Recommenders

• PerFairX is a unified benchmarking framework for LLM-based recommenders that evaluates the trade-offs between personality-driven personalization and demographic fairness.
• It employs both neutral and personality-sensitive prompting along with comprehensive metrics like PAS, GPA, DP, and EO to assess alignment, diversity, and accuracy.
• Experimental findings reveal that stronger personality alignment often results in greater demographic disparities, highlighting complex fairness-personalization trade-offs.

PerFairX is a unified benchmarking framework for large-language-model recommender systems that is designed to quantify the trade-offs between demographic fairness and psychological alignment in generated recommendations. It is motivated by the observation that prompt-based personalization through inferred user personality traits, especially under the Big Five or OCEAN model, can improve alignment with individual users while introducing or amplifying disparities across demographic groups. In its reported instantiation, PerFairX evaluates neutral and personality-sensitive prompting for ChatGPT and DeepSeek on MovieLens 10M and Last.fm 360K, and its central finding is that stronger personality alignment often coincides with worse demographic fairness (Sah, 20 Aug 2025).

1. Conceptual scope and problem setting

PerFairX is framed as a response to a specific tension in LLM-based recommendation: the integration of user personality traits can enhance recommendation alignment with individual preferences, but such personalization may inadvertently exacerbate demographic disparities. The framework therefore treats fairness and personality not as isolated criteria but as jointly audited dimensions of recommender behavior (Sah, 20 Aug 2025).

The framework is explicitly concerned with recommendations generated by LLMs under prompt-based interactions. In this setting, personalization is induced by prompt content rather than by conventional retraining or fine-tuning. This places prompt design, model sensitivity, and user-profile construction at the center of evaluation. PerFairX accordingly functions as an evaluation and benchmarking framework rather than as a standalone recommendation algorithm.

A common misunderstanding in this area is that personality conditioning simply adds user-centricity without changing the fairness profile of the system. PerFairX rejects that assumption empirically: its benchmark is built precisely to expose cases in which psychological fit improves while demographic equity deteriorates. This suggests that prompt-based personalization in LLM recommenders should be treated as a fairness-relevant intervention rather than as a purely preference-matching refinement.

2. Personality modeling and prompt construction

PerFairX represents each user with a five-dimensional OCEAN vector,

$\vec{p}_u = [O_u, C_u, E_u, A_u, N_u] \in [0,1]^5,$

corresponding to Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism (Sah, 20 Aug 2025).

The framework states that personality traits are inferred from real user behavioral data, including genre affinity, rating patterns, and activity diversity, using established genre–trait mappings from psychology and prior recommender studies. In this formulation, personality is not self-reported within the benchmark; it is reconstructed from observed user behavior and then used both for prompt construction and for downstream evaluation of recommendation alignment.

PerFairX distinguishes two prompt regimes. The neutral prompt elicits recommendations for a “general audience” and serves as a baseline for evaluating demographic fairness. The personality-sensitive prompt describes inferred trait(s), for example “I am an introverted movie lover...,” so that the LLM conditions its output on psychological attributes. This design enables controlled comparisons along two axes: demographic fairness under neutral prompting and personality alignment under personality-sensitive prompting.

This prompt pairing is methodologically important because it isolates the effect of explicit personality conditioning. Rather than comparing different recommendation models under incomparable inputs, PerFairX compares the same model under systematically altered prompts, making prompt sensitivity itself part of the audit target.

3. Metric suite and aggregate scoring

PerFairX uses a suite of ten core metrics spanning personality alignment, fairness and diversity, prompt sensitivity, and recommendation accuracy. The framework also defines an aggregate scalar, the FPx score, for unified comparison across models and prompt settings (Sah, 20 Aug 2025).

Dimension	Metrics	Role
Personality alignment	PAS, GPA	Quantify psychological fit
Fairness and diversity	DP, EO, ILF	Measure demographic disparity and list diversity
Prompt sensitivity and accuracy	SNSR@K, SNSV@K, Jaccard@K, Precision@K, Recall@K	Measure prompt robustness and retrieval quality

The Personality Alignment Score (PAS) is defined as the cosine similarity between the inferred user OCEAN vector and the vectorized genres of recommended items: $$PAS(u) = \frac{\vec{p}_u \cdot \vec{g}_u}{\|\vec{p}_u\| \cdot \|\vec{g}_u\|}.$$

The Genre-Personality Alignment (GPA) metric is described as a weighted sum of the correspondence between recommended genres and OCEAN-linked genre sets. In the benchmark, PAS and GPA jointly characterize psychological alignment, but they need not move in the same direction.

For demographic fairness, PerFairX adopts Demographic Parity (DP) and Equal Opportunity (EO): $DP = |P(\hat{Y}=1|A=0) - P(\hat{Y}=1|A=1)|,$

$$EO = |P(\hat{Y}=1|Y=1, A=0) - P(\hat{Y}=1|Y=1, A=1)|.$$

It also includes Intra-list Fairness (ILF), an entropy-based diversity measure within the recommendation list: $ILF@K = -\sum_{g \in G} p(g) \cdot \log p(g).$

Prompt sensitivity is evaluated with SNSR@K, SNSV@K, and Jaccard@K. The Jaccard overlap between neutral and personality-sensitive top-$K$ lists is defined as

$$Jaccard@K = \frac{|R_{neutral}^K \cap R_{sensitive}^K|}{|R_{neutral}^K \cup R_{sensitive}^K|}.$$

Accuracy is measured with Precision@K and Recall@K.

These metrics are aggregated through the FPx (PerFairX) Score: $\begin{split} \text{FPx}(u) = & \; \alpha \cdot PAS + \beta \cdot GPA + \gamma \cdot (1 - DP) \ & + \delta \cdot (1 - EO) + \epsilon \cdot ILF + \zeta \cdot Jaccard@K \ & + \eta \cdot Precision@K + \mu \cdot Recall@K . \end{split}$ Because the weights are configurable, the score makes the fairness–personalization trade-off explicit rather than hiding it inside a single fixed objective. A plausible implication is that PerFairX is intended not only for leaderboard-style comparison, but also for policy-sensitive tuning in which different deployments prioritize different combinations of equity, alignment, and relevance.

4. Experimental instantiation

The reported benchmark evaluates two LLMs—OpenAI’s ChatGPT (GPT-4o) and DeepSeek—on two datasets: MovieLens 10M and Last.fm 360K (Sah, 20 Aug 2025). MovieLens 10M is used as a movies domain with gender, age, and occupation attributes, while Last.fm 360K serves as a music domain with gender, age, and country.

Only users with at least $200$ interactions were retained for robust personality inference. For each dataset, five representative user profiles with diverse traits were selected. Each model was prompted with both neutral and personality-sensitive prompts, and each prompt produced top-$K$ recommendations with $$PAS(u) = \frac{\vec{p}_u \cdot \vec{g}_u}{\|\vec{p}_u\| \cdot \|\vec{g}_u\|}.$$0. The resulting outputs were scored using the full metric suite, and behavior was analyzed across sensitive attributes including gender, age, and occupation or country.

This setup emphasizes controlled auditing rather than maximal-scale ranking evaluation. The benchmark uses a small set of representative profiles, but subjects each profile to multi-axis analysis, enabling direct inspection of how prompt style and model choice reshape alignment, diversity, and group disparity. The design therefore privileges interpretability of trade-offs over purely aggregate retrieval reporting.

5. Empirical findings

The benchmark reports that personality-aware prompting significantly improves alignment with individual traits but can exacerbate fairness disparities across demographic groups (Sah, 20 Aug 2025). This is the principal empirical claim of PerFairX and the basis for its framing as a fairness–personalization benchmark.

On personality alignment, DeepSeek showed substantial improvement in PAS under personality-sensitive prompting, increasing from $$PAS(u) = \frac{\vec{p}_u \cdot \vec{g}_u}{\|\vec{p}_u\| \cdot \|\vec{g}_u\|}.$$1 to $$PAS(u) = \frac{\vec{p}_u \cdot \vec{g}_u}{\|\vec{p}_u\| \cdot \|\vec{g}_u\|}.$$2 in MovieLens, with similar gains reported in Last.fm. ChatGPT’s PAS remained stable but lower. At the same time, GPA often decreased with sensitive prompts, indicating that better trait alignment did not necessarily translate into stronger genre alignment. This divergence is methodologically important because it shows that “personality alignment” is not reducible to a single measurement.

On fairness and diversity, the paper reports that personalization improves alignment and diversity, as measured by ILF, but increases demographic disparities, as measured by DP and EO. One highlighted case is DeepSeek under sensitive prompting, where it achieved $$PAS(u) = \frac{\vec{p}_u \cdot \vec{g}_u}{\|\vec{p}_u\| \cdot \|\vec{g}_u\|}.$$3 but $$PAS(u) = \frac{\vec{p}_u \cdot \vec{g}_u}{\|\vec{p}_u\| \cdot \|\vec{g}_u\|}.$$4. In the framework’s interpretation, this exemplifies the fairness–personalization trade-off: psychologically sharper recommendations can also become demographically narrower.

Model comparison follows the same pattern. DeepSeek outperformed ChatGPT in PAS, ILF, Precision, and Recall, and achieved higher FPx scores on both datasets; the paper gives MovieLens values of $$PAS(u) = \frac{\vec{p}_u \cdot \vec{g}_u}{\|\vec{p}_u\| \cdot \|\vec{g}_u\|}.$$5 for DeepSeek versus $$PAS(u) = \frac{\vec{p}_u \cdot \vec{g}_u}{\|\vec{p}_u\| \cdot \|\vec{g}_u\|}.$$6 for ChatGPT. ChatGPT, however, was more stable under prompt variation and marginally fairer on DP and EO, but less personalized. The benchmark therefore does not identify a uniformly dominant model; it identifies distinct operating points along a trade-off surface.

The reported visual analyses reinforce this interpretation. Trade-off plots such as PAS versus DP showed that increasing personality fit through sensitive prompting nearly always induced higher demographic disparity. Example outputs further illustrated that neutral prompts yielded popular, generic recommendations, whereas personality-sensitive prompts produced more personalized but demographically narrow suggestions. This suggests that prompt sensitivity is not merely a stylistic feature of LLM recommenders, but a mechanism through which fairness characteristics can change materially.

6. Design implications, evaluation guidelines, and scope

PerFairX advances a set of design guidelines for equitable and personalized LLM recommender systems. First, system designers are advised to conduct an explicit audit for fairness–personalization trade-offs rather than assuming that improvements in personality fit are costless. Second, the framework argues for multi-metric evaluation: simple accuracy or one-dimensional fairness measures are presented as insufficient for capturing the full spectrum of user- and group-level outcomes (Sah, 20 Aug 2025).

Third, prompt design is treated as a first-class audit variable. The framework recommends careful engineering of both neutral and personality-integrated prompts and makes prompt sensitivity analysis part of the evaluation pipeline. Fourth, FPx-style weighting and aggregation are used to make trade-offs explicit and tunable. Fifth, evaluation should include varied user personalities and demographic groups, since impacts often differ across intersections of these attributes.

These recommendations define the scope of PerFairX with some precision. It is neither a direct debiasing method nor a pure personalization benchmark. It is a principled benchmark for simultaneously auditing personality alignment, demographic equity, diversity, prompt robustness, and accuracy in LLM-generated recommendations. Its stated contribution is to guide the development of recommender systems that are both equitable and psychologically informed, with relevance to inclusive, user-centric AI applications in continual learning contexts.