Demonstrating social-scientific fidelity of generative ABMs to human actions

Show that LLM-driven generative agent-based models can reproduce human actions with sufficient fidelity to support socially scientific productivity by providing rigorous empirical evidence that agent behaviors align with human behaviors relevant to the modeled phenomena.

Background

A key claim of generative ABMs is their potential to better represent human behavior. The paper emphasizes that, despite this promise, there is no demonstrated evidence yet that such models reproduce human actions at a level that makes them productively useful for social science.

Addressing this gap requires explicit empirical tests and benchmarks that link modeled agent actions to human actions in the relevant contexts, beyond surface-level believability or stylistic similarity of outputs.

References

It has yet to be shown that the models can be made to reproduce human actions in such a way as to make them social scientifically productive.

— Do Large Language Models Solve the Problems of Agent-Based Modeling? A Critical Review of Generative Social Simulations  (2504.03274 - Larooij et al., 4 Apr 2025) in Conclusion (Section 6)

Finally, our transfer results show that the model can generalize to new but related simulated settings, but they do not yet demonstrate broad generalization to real-world pedestrian behavior across diverse traffic situations.

— A Human-Like Pedestrian Model for Automated Driving Simulations  (2609.29175 - Wang et al., 24 Sep 2026) in Section 6, Discussion, paragraph on limitations

Future work should clarify the effective range of LLM-driven crowd simulation across diverse venues and events, leveraging the growing availability of open urban mobility datasets.

— Distilling Aggregate Mobility Statistics into a Language Model Policy for Post-Event Crowd Simulation  (2608.19778 - Amano et al., 20 Aug 2026) in Conclusion

More broadly, the extent to which LLM-based social simulations reflect human behavior remains an open question and warrants further investigation.

— Polarization Dynamics under Algorithmic Recommendations in LLM-based Online Social Networks  (2610.13351 - Dehkordi et al., 9 Oct 2026) in Section 7.1, “Limitations and Future Work” (Conclusion and Future Work)

These applications rely on the assumption that LLM agents are sufficiently faithful proxies for human behavior. Whether LLMs can provide such fidelity remains an empirical question.

— Cross-Centaur: Modeling Individual Decision-Making Across Psychological Tasks with Language Models  (2610.23157 - Asano et al., 16 Oct 2026) in Section 2, Related Work, page 3