Validation methodology for persona/profile alignment to real individuals

Develop robust calibration and validation procedures that establish and assess persona/profile alignment in LLM-based generative agents by matching agent behaviors to those of specific real individuals within the modeled situations.

Background

The paper contends that operational validity in generative ABMs often requires validating that an agent’s behavior aligns with the particular person it is intended to represent, not merely internal consistency. Achieving such alignment would likely necessitate linking to individual-level data (e.g., surveys, experiments, or digital traces) and raises methodological and ethical complexities.

Currently, most studies rely on zero-shot models and prompt personas without empirical calibration to specific individuals. The lack of established procedures for profile alignment undermines claims of realism and external validity in generative ABMs.