Demonstrating social-scientific fidelity of generative ABMs to human actions

Show that LLM-driven generative agent-based models can reproduce human actions with sufficient fidelity to support socially scientific productivity by providing rigorous empirical evidence that agent behaviors align with human behaviors relevant to the modeled phenomena.

Background

A key claim of generative ABMs is their potential to better represent human behavior. The paper emphasizes that, despite this promise, there is no demonstrated evidence yet that such models reproduce human actions at a level that makes them productively useful for social science.

Addressing this gap requires explicit empirical tests and benchmarks that link modeled agent actions to human actions in the relevant contexts, beyond surface-level believability or stylistic similarity of outputs.

References

It has yet to be shown that the models can be made to reproduce human actions in such a way as to make them social scientifically productive.

— Do Large Language Models Solve the Problems of Agent-Based Modeling? A Critical Review of Generative Social Simulations  (2504.03274 - Larooij et al., 4 Apr 2025) in Conclusion (Section 6)

Future work should clarify the effective range of LLM-driven crowd simulation across diverse venues and events, leveraging the growing availability of open urban mobility datasets.

— Distilling Aggregate Mobility Statistics into a Language Model Policy for Post-Event Crowd Simulation  (2608.19778 - Amano et al., 20 Aug 2026) in Conclusion