Social Desirability Bias (SDB)

• Social Desirability Bias is the tendency to skew self-reports toward socially acceptable responses, affecting both human surveys and artificial intelligence outputs.
• Robust measurement methods, such as list experiments and psychometric profiling, reveal biases with discrepancies up to 15–23 percentage points in sensitive topics.
• Mitigation strategies, including refined survey designs and prompt engineering in LLMs, are critical to reduce SDB and improve the authenticity of reported data.

Social desirability bias (SDB) is a pervasive threat to the validity of self-reported data in both human social science and artificial intelligence contexts. SDB arises when respondents, either human or synthetic agents, systematically distort their answers toward what they believe to be socially acceptable or normatively approved, rather than reporting their true attitudes, beliefs, or behaviors. This construct operates at both the psychological (impression management, self-deception) and algorithmic (reinforcement from human feedback) levels and increasingly requires rigorous measurement and mitigation strategies as survey research and LLM applications converge in the emulation and simulation of human behavior.

1. Foundations and Definitions

In human self-report research, social desirability bias is formally defined as the tendency of individuals to provide answers that cast themselves in a favorable light, in accordance with perceived social norms, at the expense of candor (Lee et al., 2024, Listo et al., 12 Mar 2025). Psychologically, SDB comprises two key elements: impression management (deliberate conformity to social expectations) and self-deception (unconscious endorsement of overly flattering self-representations) (Lee et al., 2024). This dual-factor structure explains the persistence and robustness of SDB across a variety of sensitive or evaluatively charged survey topics, notably ethics, politics, stigmatized behaviors, and prosocial acts.

In LLMs, SDB manifests not from self-concept but as a learned statistical preference for generating outputs that appear socially acceptable or agreeable—often as a byproduct of alignment methods such as reinforcement learning from human feedback (RLHF) and recursive corpora ingestion (Cadei et al., 22 Sep 2025, Bian et al., 24 Oct 2025). Models thus default to safe, harmonious, or normatively positive responses, particularly when queried on sensitive issues or tasked with simulating human participants (Chapala et al., 27 Dec 2025, Salecha et al., 2024).

2. Measurement and Quantification

Survey Methodology in Humans: The empirical quantification of SDB proceeds by contrasting direct response formats against indirect or anonymized techniques. For instance, list experiments (item-count techniques) embed a sensitive statement among innocuous items, asking for only the total count of affirmative responses. Under random assignment and design-effect constraints, the difference in means between respondents exposed to the sensitive item (long list) or not (short list) estimates the true prevalence of the underlying attitude, with SDB inferred as the gap between this counterfactually private measure and the direct response (Listo et al., 12 Mar 2025). Robust empirical discrepancies of 15–23 percentage points have been observed for workplace attitudes toward sexual minorities (Listo et al., 12 Mar 2025).

Mathematical Formulation (Survey Indices): In LLM-based SDB studies, binary response indices (e.g., Social Desirability Response—SDR index) sum dichotomously scored items, e.g.,

$$\text{SDR} = \sum_{i=1}^{13} x_i,\quad x_i \in \{0,1\}$$

where $x_i = 1$ if the response is socially desirable. Across synthetic personas in GPT-4, empirical means of $\mathrm{M}=5.05$ (SD=2.88) have been reported (Lee et al., 2024).

Psychometric Profiling in LLMs: LLM SDB is quantified via standardized personality assessments (e.g., Big Five Inventory), defining effect sizes relative to human norms:

$$\Delta = \frac{\mu_{\text{evaluated}} - \mu_{\text{baseline}}}{\sigma_{\text{human}}}$$

where $\mu_{\text{baseline}}$ is the average for isolated questions (no evaluative context), and $\mu_{\text{evaluated}}$ under batteries or explicit instructions (Salecha et al., 2024). In GPT-4, $\Delta$ can exceed $+1.4\sigma$ (e.g., Conscientiousness), with strong reductions (–1.11σ) for “undesirable” traits such as Neuroticism.

Meta-Score for LLMs ("SDB Score"): For model trend analysis, SDB can be computed as

$$\mathrm{SDB} = \frac{(\tilde{O}+\tilde{C}+\tilde{A}) - (\tilde{N}+\tilde{E}) + 2}{5}$$

with each Big Five trait $\tilde{O}, \tilde{C}, \ldots, \tilde{N}$ rescaled to [0,1]. Observed SDB scores have increased linearly over sequential model generations, with an estimated annual increase of 0.0466 SDB units (Cadei et al., 22 Sep 2025).

3. Mechanisms and Model-Based Explanations

Theoretical Modeling: Statistical mechanical frameworks have formalized SDB as emergent from the interplay of private preferences ($s_i$), public declarations ($x_i$), integrity ($\mu$; propensity to internal honesty), and self-monitoring ($\lambda$; pressure to conform):

$$H(\{s_i\}, \{x_i\}) = -\frac{J}{N}\sum_{i\neq j} s_ix_j - \mu\sum_i s_ix_i - \frac{\lambda}{N}\sum_{i\neq j} x_ix_j$$

Phase diagrams delineate regimes where public reporting overshoots or undershoots private attitudes ($\Delta = \langle x\rangle - \langle s\rangle$), illuminating the structural roots of phenomena such as the “Bradley effect” in polling (Gamberi et al., 2022). These models distinguish between integrity-driven alignment (minimizing $|x_i-s_i|$), conversion (private attitudes aligning with prevailing public signals), and pure social conformity (alignment within $x_i$) (Gamberi et al., 2022).

Algorithmic SDB in LLMs: RLHF and corpus recursion induce SDB by over-rewarding outputs that are perceived as helpful, agreeable, or safe—functionally equivalent to impression management in humans. The "Narcissus Hypothesis" posits that recursive pretraining on RLHF-modulated corpora causes models to increasingly favor SDB, drifting toward synthetically "flattering" representations and collapsing onto what is termed the Rung of Illusion (a detachment from ground-truth causality in favor of in-group assurance) (Cadei et al., 22 Sep 2025).

Multi-Agent Social Simulation: LLM-driven simulations of social dialogue (e.g., chatroom experiments) exhibit “Utopian illusion”—overidealized, harmonious, and conflict-averse interactions, as measured by semantic similarity, positivity bias, and the under-representation of negative or dissenting speech acts (Bian et al., 24 Oct 2025). Quantitative metrics include elevated mean VADER sentiment scores (μ≈0.45 LLMs vs. μ≈0.10 humans), persistent over-selection of high-status roles, and increased inter-agent redundancy in embeddings (Bian et al., 24 Oct 2025).

4. Identification, Diagnosis, and Heterogeneity

List Experiments and Sensitivity Bias Across Subgroups: Classical SDB detection assumes uniform polarity (bias direction) across all respondents. However, non-uniform polarity (opposite sign biases across subgroups) fundamentally undermines standard diagnostics and joint estimators, producing false positives in monotonicity violation tests and biasing prevalence and causal effect estimates (Hatz et al., 2024). For example, some subgroups may under-report a trait while others over-report it—a scenario requiring subgroup-specific diagnostics and estimation procedures supported by robust difference-in-means or standard ML estimators, not monotonicity-assuming joint models (Hatz et al., 2024).

Demographic Gradients: In both human and LLM studies, SDB effect sizes vary by demographic attributes. In GPT-4 synthetic samples, SDR scores covary positively with age ($b=0.03$ per year, $p<.001$) and education ($b=0.29$ per level, $p<.05$), with stronger commitment-induced SDB among older personas, but no significant associations with gender or income (Lee et al., 2024). In human samples, SDB is systematically higher among men, those without managerial experience, and respondents with no LGBTQ+ acquaintances, as evidenced by list experiment outcomes (Listo et al., 12 Mar 2025).

Robustness and Alternative Specifications: SDB findings remain consistent under randomization of question order, paraphrasing of items, and across model temperatures in LLMs. Reverse-coding (negating evaluative wording) can reduce, but does not abolish, SDB in both humans and models (Salecha et al., 2024, Chapala et al., 27 Dec 2025). Double-list and split-sample designs in list experiments, combined with behavioral triangulation (e.g., real-stakes donation), further validate the presence and magnitude of SDB (Listo et al., 12 Mar 2025).

5. Mitigation Strategies and Experimental Controls

Human Surveys: Double-list experiments with careful attention to non-sensitive item selection, random assignment, and robust diagnostic testing of identification assumptions (randomization, no design effects, no liars) are critical to minimize and measure SDB (Listo et al., 12 Mar 2025, Hatz et al., 2024). Adjusting item polarity and conducting subgroup-wise diagnostic tests are recommended where heterogeneous bias is anticipated (Hatz et al., 2024).

LLM-Based Mitigations: Prompt engineering substantially mitigates SDB in synthetic (silicon) samples. Neutral, third-person reformulations reduce the modal collapse to socially approved answers and achieve distributions closer to empirical benchmarks (e.g., American National Election Study). Empirical evaluation via Jensen–Shannon divergence demonstrates mean JSD reductions (Replicate: 0.1033, Reformulated: 0.0787 in GPT-4.1-mini, $p$ non-overlapping at a 95% CI) (Chapala et al., 27 Dec 2025). Preambles promising anonymity or instructions to "be analytical" do not reduce and may exacerbate SDB, as do naive stochastic decoding methods (Chapala et al., 27 Dec 2025). Robust mitigation thus entails combining prompt reformulation with moderate stochasticity (temperature $\approx$ 1) and bootstrap-based uncertainty quantification.

Advanced Model Training: Increasing social authenticity and diversity, incorporating reasoning-oriented objectives (e.g., chain-of-thought generation), antagonistic-AI setups to reward disagreement, and corpus diversification (including conflict-rich, less filtered data) have been proposed to counteract the homogenizing, utopian drift induced by SDB (Bian et al., 24 Oct 2025, Cadei et al., 22 Sep 2025). These methods aim to restore a broader spectrum of human-like social and affective behavior in simulated agents.

6. Empirical Impact and Scientific Implications

Survey Accuracy and Policy: SDB can mask substantial pockets of “genuine discomfort” or deviant attitudes that, if uncorrected, yield misleading estimates for policymaking or research—e.g., the 15–23 percentage point underreporting of discomfort with sexual minorities in the workplace (Listo et al., 12 Mar 2025). In elections, SDB underpins the Bradley effect and related phenomena, contributing to poll–outcome discrepancies (Gamberi et al., 2022).

LLM Use as Human Proxies: The emergence of strong SDB in LLM-based psychometrics (Δ~1.2σ shifts on the Big Five) significantly constrains the validity of “virtual participants” in behavioral research, requiring substantial rethinking of instrument design, analysis, and interpretation for any paradigm relying on LLM-simulated survey responses (Salecha et al., 2024, Chapala et al., 27 Dec 2025).

Corpus and Model Integrity: Recursive SDB, amplified by semi-synthetic texts and RLHF, can irreversibly pollute future training corpora, further detaching models from empirical ground truth and impairing downstream causal inference or policy interventions (Cadei et al., 22 Sep 2025).

7. Future Directions and Open Challenges

Outstanding challenges in SDB research include generalizing measurement beyond current instruments, cross-model and cross-cultural validation, and the development of robust, theory-driven survey and simulation designs resilient to heterogeneous, subgroup-specific patterns of SDB (Hatz et al., 2024, Lee et al., 2024). For LLMs, ongoing audits for SDB drift, causal lifting techniques, and benchmarking against exogenous data remain essential to prevent entrenchment in “Utopian illusion” or the Rung of Illusion (Cadei et al., 22 Sep 2025, Bian et al., 24 Oct 2025). Future experimental protocols should combine convergent behavioral and self-report indices—both in silico and in human populations—and further refine subgroup targeting, prompt construction, and corpus curation methods to accurately capture the complex interplay between social norms, evaluation contexts, and response behavior.

---

References

• (Lee et al., 2024) Lee et al., "Exploring Social Desirability Response Bias in LLMs: Evidence from GPT-4 Simulations"
• (Listo et al., 12 Mar 2025) Listo, Muñoz & Sansone, "Measuring the Sources of Taste-Based Discrimination Using List Experiments"
• (Chapala et al., 27 Dec 2025) Wang et al., "Mitigating Social Desirability Bias in Random Silicon Sampling"
• (Salecha et al., 2024) Salecha et al., "LLMs Show Human-like Social Desirability Biases in Survey Responses"
• (Cadei et al., 22 Sep 2025) Cadei et al., "The Narcissus Hypothesis: Descending to the Rung of Illusion"
• (Bian et al., 24 Oct 2025) Xia et al., "Social Simulations with LLM Risk Utopian Illusion"
• (Gamberi et al., 2022) Agliari et al., "Rationalizing systematic discrepancies between election outcomes and opinion polls"
• (Hatz et al., 2024) Hatz & Randahl, "When Sensitivity Bias Varies Across Subgroups: The Impact of Non-uniform Polarity in List Experiments"