Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

MetaAgents: Simulating Interactions of Human Behaviors for LLM-based Task-oriented Coordination via Collaborative Generative Agents (2310.06500v1)

Published 10 Oct 2023 in cs.AI

Abstract: Significant advancements have occurred in the application of LLMs for various tasks and social simulations. Despite this, their capacities to coordinate within task-oriented social contexts are under-explored. Such capabilities are crucial if LLMs are to effectively mimic human-like social behavior and produce meaningful results. To bridge this gap, we introduce collaborative generative agents, endowing LLM-based Agents with consistent behavior patterns and task-solving abilities. We situate these agents in a simulated job fair environment as a case study to scrutinize their coordination skills. We propose a novel framework that equips collaborative generative agents with human-like reasoning abilities and specialized skills. Our evaluation demonstrates that these agents show promising performance. However, we also uncover limitations that hinder their effectiveness in more complex coordination tasks. Our work provides valuable insights into the role and evolution of LLMs in task-oriented social simulations.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. AntonOsika. Gpt engineer, 2023. https://github.com/AntonOsika/gpt-engineer.
  2. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861 (2021).
  3. babyagi. Babyagi, 2023. https://github.com/yoheinakajima/babyagi.
  4. Modeling and understanding human routine behavior. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (2016), pp. 248–260.
  5. Bassil, Y. A simulation model for the waterfall software development life cycle. arXiv preprint arXiv:1205.6904 (2012).
  6. Selected models for agent-based simulation of social networks. In 3rd Symposium on social networks and multiagent systems (SNAMAS 2011) (2011), Society for the Study of Artificial Intelligence and the Simulation of Behaviour, pp. 27–32.
  7. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021).
  8. The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates, USA, 1983.
  9. Chateval: Towards better llm-based evaluators through multi-agent debate. arXiv preprint (2023).
  10. The use of technologies in the recruiting, screening, and selection processes for job candidates. International journal of selection and assessment 11, 2-3 (2003), 113–120.
  11. Improving factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305.14325 (2023).
  12. S33{}^{3}start_FLOATSUPERSCRIPT 3 end_FLOATSUPERSCRIPT: Social-network simulation system with large language model-empowered agents. arXiv preprint arXiv:2307.14984 (2023).
  13. Handel, M. J. Skills mismatch in the labor market. Annual Review of Sociology 29, 1 (2003), 135–165.
  14. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352 (2023).
  15. 40 years of cognitive architectures: core cognitive abilities and practical applications. Artificial Intelligence Review 53, 1 (2020), 17–94.
  16. Laird, J. E. It knows what you’re going to do: Adding anticipation to a quakebot. In Proceedings of the fifth international conference on Autonomous agents (2001), pp. 385–392.
  17. Camel: Communicative agents for” mind” exploration of large scale language model society. arXiv preprint arXiv:2303.17760 (2023).
  18. Encouraging divergent thinking in large language models through multi-agent debate. arXiv preprint arXiv:2305.19118 (2023).
  19. Agentsims: An open-source sandbox for large language model evaluation. arXiv preprint arXiv:2308.04026 (2023).
  20. Training socially aligned language models in simulated human society. arXiv preprint arXiv:2305.16960 (2023).
  21. Skills mismatch: Concepts, measurement and policy approaches. Journal of Economic Surveys 32, 4 (2018), 985–1015.
  22. melih unsal. Demogpt, 2023.
  23. Mills, R. How to define a workflow that keeps content production on track, Jul 2016.
  24. How data science workers work with data: Discovery, capture, curation, design, creation. In Proceedings of the 2019 CHI conference on human factors in computing systems (2019), pp. 1–15.
  25. OpenAI. Chatgpt, 2023.
  26. OpenAI. Gpt-4 technical report. ArXiv abs/2303.08774 (2023).
  27. Organization, I. L. What is skills mismatch and why should we care? www.ilo.org (Apr 2020).
  28. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35 (2022), 27730–27744.
  29. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442 (2023).
  30. Social simulacra: Creating populated prototypes for social computing systems. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (2022), pp. 1–18.
  31. Communicative agents for software development. arXiv preprint arXiv:2307.07924 (2023).
  32. Reworkd. Agentgpt, 2023.
  33. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950 (2023).
  34. Learning to collaborate: An instructional approach to promoting collaborative problem solving in computer-mediated settings. The journal of the Learning Sciences 14, 2 (2005), 201–241.
  35. Samsonovich, A. V. Toward a unified catalog of implemented cognitive architectures. BICA 221, 2010 (2010), 195–244.
  36. Neural theory-of-mind? on the limits of social intelligence in large lms. arXiv preprint arXiv:2210.13312 (2022).
  37. Schweyer, A. Talent management systems: Best practices in technology solutions for recruitment, retention and workforce planning. John Wiley & Sons, 2004.
  38. Significant-Gravitas. Autogpt, 2023. https://github.com/Significant-Gravitas/Auto-GPT.
  39. Agent-based modeling: A new approach for theory building in social psychology. Personality and social psychology review 11, 1 (2007), 87–104.
  40. smol ai. Smolmodels, 2023.
  41. Stup, R. Standard operating procedures: A writing guide. State College: Penn State University (2001).
  42. Surowiecki, J. The wisdom of crowds. Anchor, 2005.
  43. team openpm. Workgpt, 2023. https://github.com/team-openpm/workgpt.
  44. Human-ai collaboration in data science: Exploring data scientists’ perceptions of automated ai. Proceedings of the ACM on human-computer interaction 3, CSCW (2019), 1–24.
  45. A survey on large language model based autonomous agents. arXiv preprint arXiv:2308.11432 (2023).
  46. Recagent: A novel simulation paradigm for recommender systems. arXiv preprint arXiv:2306.02552 (2023).
  47. A survey on metaverse: Fundamentals, security, and privacy. IEEE Communications Surveys & Tutorials (2022).
  48. Unleashing cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration. arXiv preprint arXiv:2307.05300 (2023).
  49. Coordination in task-performing groups. Theory and research on small groups (2002), 177–204.
  50. How do data science workers collaborate? roles, workflows, and tools. Proceedings of the ACM on Human-Computer Interaction 4, CSCW1 (2020), 1–23.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Yuan Li (392 papers)
  2. Yixuan Zhang (94 papers)
  3. Lichao Sun (186 papers)
Citations (70)
View on Semantic Scholar