Restoring Heterogeneity in LLM-based Social Simulation: An Audience Segmentation Approach

Abstract: LLMs are increasingly used to simulate social attitudes and behaviors, offering scalable "silicon samples" that can approximate human data. However, current simulation practice often collapses diversity into an "average persona," masking subgroup variation that is central to social reality. This study introduces audience segmentation as a systematic approach for restoring heterogeneity in LLM-based social simulation. Using U.S. climate-opinion survey data, we compare six segmentation configurations across two open-weight LLMs (Llama 3.1-70B and Mixtral 8x22B), varying segmentation identifier granularity, parsimony, and selection logic (theory-driven, data-driven, and instrument-based). We evaluate simulation performance with a three-dimensional evaluation framework covering distributional, structural, and predictive fidelity. Results show that increasing identifier granularity does not produce consistent improvement: moderate enrichment can improve performance, but further expansion does not reliably help and can worsen structural and predictive fidelity. Across parsimony comparisons, compact configurations often match or outperform more comprehensive alternatives, especially in structural and predictive fidelity, while distributional fidelity remains metric dependent. Identifier selection logic determines which fidelity dimension benefits most: instrument-based selection best preserves distributional shape, whereas data-driven selection best recovers between-group structure and identifier-outcome associations. Overall, no single configuration dominates all dimensions, and performance gains in one dimension can coincide with losses in another. These findings position audience segmentation as a core methodological approach for valid LLM-based social simulation and highlight the need for heterogeneity-aware evaluation and variance-preserving modeling strategies.

• The paper demonstrates that audience segmentation can effectively restore heterogeneity in LLM social simulations by controlling identifier granularity and parsimony.
• The paper employs rigorous empirical evaluation using US survey data and multiple fidelity metrics to reveal challenges like over-regularization.
• The paper finds that theory-driven, data-driven, and instrument-based segmentation logics yield distinct trade-offs between distributional, structural, and predictive fidelity.

Restoring Heterogeneity in LLM-Based Social Simulation via Audience Segmentation

Problem Formulation and Theoretical Motivation

LLM-based social simulation has gained prominence as a scalable alternative to traditional human data collection, but there is a persistent bottleneck: the collapse of heterogeneity into an "average persona." This phenomenon is underpinned by LLMs’ training objectives (e.g., MLE) and further amplified by methodological practices that fail to explicitly encode subgroup diversity. The erasure of heterogeneity—termed “heterogeneity masking”—is shown to be not merely a technical artifact but a consequence of epistemological typological thinking, contrasting with the population-thinking necessary for nuanced social simulation. Ensuring that simulated data reflects authentic, structured population variance is critical for valid inference and downstream modeling.

Audience Segmentation: Conceptual and Empirical Foundations

The paper proposes formalizing audience segmentation as a core methodological device to restore heterogeneity. Drawing from communication and marketing sciences, segmentation identifies theoretically and empirically meaningful subgroups via combinations of demographic, psychographic, and behavioral identifiers. The utility of this approach hinges not on perfect replication of real-world subgroup distributions, but on recovering theoretically salient, empirically discriminable structures.

Segmentation operates through three methodological axes:

• Granularity: The depth and multidimensionality of identifier specification.
• Parsimony: The trade-off between comprehensive versus compact identifier sets.
• Selection Logic: Justification of identifier inclusion, which may be theory-driven, data-driven, or instrument-based.

Experimental Design and Evaluation Framework

Empirical analysis leverages US climate-opinion survey data (594 human respondents, stratified for representativeness), with "silicon samples" generated using Llama 3.1-70B and Mixtral 8x22B. Six segmentation configurations are evaluated, reflecting different levels of granularity, parsimony, and identifier selection logic (Classic Demo, Theory-augmented at two scales, Data-driven, Instrument-based at two scales). Persona conditioning in prompts is standardized, and output attitudinal responses are direct (Likert-scale integers), facilitating rigorous metric comparison.

A three-dimensional fidelity framework structures evaluation:

• Distributional fidelity: Alignment of generated and empirical response distributions (MAE, KLD, weighted F1, etc.).
• Structural fidelity: Preservation of within-group variance (SD, CV) and between-group differentiation (nEMD, MDS mapping, Procrustes distance).
• Predictive fidelity: Replication of relational associations between identifiers and outcomes (Cramér’s V).

Key Findings

Granularity Does Not Guarantee Improved Fidelity

Moving from demographic-only (Demo) to moderately enriched (Demo+Theory-15) segmentation improves simulation accuracy, especially in distributional metrics (e.g., KLD decreases, F1 increases), but further expansion to highly granular (Demo+Theory-59) configurations yields inconsistent or adverse effects, particularly on structural and predictive fidelity.

Figure 1: Comparison of distributional fidelity metrics across segmentation configurations and LLMs.

Figure 2: Comparison of structural fidelity metrics across segmentation configurations and LLMs.

The empirical evidence highlights that excessive identifier granularity can trigger over-regularization and mask subgroup differentiation—contrary to prevailing assumptions about "more detailed personas” leading to richer simulation outputs.

Informative Parsimony Outperforms Comprehensiveness

Across both theory-driven and instrument-based comparisons, succinct sets of discriminative identifiers (e.g., Theory-15 versus Theory-59, Item-4 versus Item-15) frequently match or outperform comprehensive variants in preserving between-group and predictive relationships. Particularly, compact configurations are associated with better alignment in Cramér’s V with human benchmarks, quantifying the preservation of empirical association strengths.

Figure 3: Comparison of Cramér's V across segmentation configurations and LLMs. Brackets and numbers indicate the differences from the human benchmark.

Notably, parsimonious configurations retain, and occasionally enhance, structural variance, indicating that thoughtful identifier selection (rather than mere volume) best fosters subgroup differentiation.

Identifier Selection Logic Dictates Dimension-Specific Performance

No single configuration simultaneously optimizes distributional, structural, and predictive fidelity. Instead:

• Instrument-based (pre-validated survey instrument, e.g., Item-15) configurations best preserve distributional shape (minimal KLD).
• Data-driven (feature-selected via GBM) configurations yield stronger between-group structure (highest nEMD, lowest Procrustes distance) and most faithfully recover identifier-outcome associations (Cramér’s V).
• Theory-driven selections offer intermediate, less specialized performance profiles.

  Figure 4: MDS maps of empirical and simulated subgroup structures across segmentation configurations and LLMs.

Performance gains in one fidelity dimension are observed to sometimes coincide with drawbacks in another, emphasizing the importance of aligning segmentation configuration with specific research priorities.

Residual Over-Regularization Remains

Despite best practices in segmentation, all tested LLMs demonstrate over-regularization relative to human empirical data—i.e., simulated subgroup differences remain systematically compressed. The implication is that even optimized prompt engineering and identifier design cannot completely overcome model-intrinsic variance suppression.

Implications for Social Simulation and AI Research

These findings have significant implications for both the practice and theory of LLM-based social simulation:

• Methodological specification and reporting: Clearly documenting segmentation logic and parsimony thresholds is vital for reproducibility and interpretability in simulation studies.
• Heterogeneity-aware evaluation: Adopting distributional, structural, and predictive fidelity metrics, rather than only aggregate-level comparisons, is indispensable for rigorous validation of simulated social data.
• Model selection and development: Persistent over-regularization underscores the need for future research into modeling techniques and alignment protocols that maintain or even amplify core axes of population variance during both pre-training and fine-tuning.

Substantively, the study demonstrates that the restoration of heterogeneity in LLM-driven "silicon samples" is not solely a question of technical capability but of deliberate methodological design grounded in population-thinking. As simulation increasingly supplements or replaces survey-based social research, these practices will become central to the validity of AI-enabled social science.

Conclusion

Audience segmentation, when informed by theory, empirical discriminability, and selection logic, provides an effective pathway to mitigating the average persona effect in LLM-based social simulation. However, gains are conditional and dimension-specific: indiscriminate increases in identifier granularity do not guarantee better simulation performance, and parsimony, when aligned with task-relevant information, can outperform comprehensive conditioning. Persistent over-regularization across models flags unresolved challenges in preserving authentic subgroup structure and variance, warranting future innovation in architecture and alignment. Ultimately, this research establishes segmentation design, not mere prompt engineering, as foundational to high-fidelity simulation of social heterogeneity.