Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

LLM Voting: Human Choices and AI Collective Decision Making (2402.01766v3)

Published 31 Jan 2024 in cs.CL, cs.AI, cs.CY, cs.LG, econ.GN, and q-fin.EC

Abstract: This paper investigates the voting behaviors of LLMs, specifically GPT-4 and LLaMA-2, their biases, and how they align with human voting patterns. Our methodology involved using a dataset from a human voting experiment to establish a baseline for human preferences and conducting a corresponding experiment with LLM agents. We observed that the choice of voting methods and the presentation order influenced LLM voting outcomes. We found that varying the persona can reduce some of these biases and enhance alignment with human choices. While the Chain-of-Thought approach did not improve prediction accuracy, it has potential for AI explainability in the voting process. We also identified a trade-off between preference diversity and alignment accuracy in LLMs, influenced by different temperature settings. Our findings indicate that LLMs may lead to less diverse collective outcomes and biased assumptions when used in voting scenarios, emphasizing the need for cautious integration of LLMs into democratic processes.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies, 2023.
  2. The real dangers of generative ai. Journal of Democracy, 35(1):147–162, Jan 2024.
  3. Leveraging AI for democratic discourse: Chat interventions can improve online political conversations at scale. Proceedings of the National Academy of Sciences, 120(41):e2311627120, October 2023a. doi: 10.1073/pnas.2311627120. Publisher: Proceedings of the National Academy of Sciences.
  4. Out of One, Many: Using Language Models to Simulate Human Samples. Political Analysis, 31(3):337–351, July 2023b. ISSN 1047-1987, 1476-4989. doi: 10.1017/pan.2023.2.
  5. Arrow, K. J. Social Choice and Individual Values. Yale University Press, 1951.
  6. The Evolution of Cooperation. Basic Books, 1981.
  7. The Internal State of an LLM Knows When It’s Lying, 2023.
  8. A polynomial-time algorithm to achieve extended justified representation. ar5iv, 2017. URL https://ar5iv.org/abs/1703.10415.
  9. Becker, G. S. The Economic Approach to Human Behavior. University of Chicago Press, 1976.
  10. A within-subject analysis of other-regarding preferences. Games and Economic Behavior, 72(2):321–338, 2011.
  11. Emergent autonomous scientific research capabilities of large language models, April 2023. URL http://arxiv.org/abs/2304.05332. arXiv:2304.05332 [physics].
  12. Swarm Intelligence: From Natural to Artificial Systems. Oxford University Press, 1999.
  13. Approval Voting. Birkhäuser, Boston, 1983. ISBN 9783764331085. URL https://archive.org/details/approvalvoting00bram.
  14. ChemCrow: Augmenting large-language models with chemistry tools, 2023.
  15. ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate, 2023.
  16. Improving Factuality and Reasoning in Language Models through Multiagent Debate, May 2023. URL http://arxiv.org/abs/2305.14325. arXiv:2305.14325 [cs].
  17. Towards Measuring the Representation of Subjective Global Opinions in Language Models, June 2023. URL http://arxiv.org/abs/2306.16388. arXiv:2306.16388 [cs].
  18. Properties of multiwinner voting rules. Social Choice and Welfare, 48(3):599–632, March 2017. doi: 10.1007/s00355-017-1026-z. URL https://doi.org/10.1007/s00355-017-1026-z.
  19. From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models, July 2023. URL http://arxiv.org/abs/2305.08283. arXiv:2305.08283 [cs].
  20. Ferland, B. A rational or a virtuous citizenry? – The asymmetric impact of biases in votes-seats translation on citizens’ satisfaction with democracy. Electoral Studies, 40:394–408, December 2015. ISSN 0261-3794. doi: 10.1016/j.electstud.2015.03.008. URL https://www.sciencedirect.com/science/article/pii/S0261379415000530.
  21. Floridi, L. AI as agency without intelligence: on ChatGPT, large language models, and other generative models. Philosophy & Technology, 36(1):15, 2023.
  22. Supported democracy: reinventing direct democracy, ai and voting twice, December 2023. URL https://kof.ethz.ch/en/news-and-events/kof-bulletin/kof-bulletin/2023/12/Supported-democracy-reinventing-direct-democracy-AI-and-voting-twice.html. Accessed: 2024-01-24.
  23. ChatLLM Network: More brains, More intelligence, April 2023. arXiv:2304.12998 [cs].
  24. On the legitimacy of voting methods. Available at SSRN 4372245, 2023.
  25. Democracy by design: Perspectives for digitally assisted, participatory upgrades of society. Participatory Upgrades of Society (November 2, 2022), 71, 2022. doi: 10.1016/j.jocs.2023.102061.
  26. Hidalgo, C. Augmented democracy, 2018. URL https://www.peopledemocracy.com/. Accessed: 2024-01-12.
  27. MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework, 2023.
  28. Horton, J. J. Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?, January 2023. URL http://arxiv.org/abs/2301.07543. arXiv:2301.07543 [econ, q-fin].
  29. Strategic voting in multi-winner elections with approval balloting: a theory for large electorates. Social Choice and Welfare, 47(3):559–587, 2016.
  30. CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society, 2023.
  31. Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate, 2023.
  32. Training Socially Aligned Language Models on Simulated Social Interactions, October 2023. URL http://arxiv.org/abs/2305.16960. arXiv:2305.16960 [cs].
  33. Political Turbulence: How Social Media Shape Collective Action. Princeton University Press, 2016. ISBN 9780691159225.
  34. Generative Agents: Interactive Simulacra of Human Behavior, August 2023. URL http://arxiv.org/abs/2304.03442. arXiv:2304.03442 [cs].
  35. Using Personas as a Tool for Teaching Civic Communication. The Journal of General Education, 68(3-4):252–262, July 2019. ISSN 0021-3667. doi: 10.5325/jgeneeduc.68.3-4.0252. URL https://doi.org/10.5325/jgeneeduc.68.3-4.0252.
  36. Sen, A. Rationality and social choice. The American economic review, 85(1):1, 1995.
  37. Personality Traits in Large Language Models, September 2023. URL http://arxiv.org/abs/2307.00184. arXiv:2307.00184 [cs].
  38. Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, 2009.
  39. Axiomatic characterization of committee scoring rules. Journal of Economic Theory, 180:244–273, 2019. ISSN 0022-0531. doi: https://doi.org/10.1016/j.jet.2018.12.011. URL https://www.sciencedirect.com/science/article/pii/S0022053119300043.
  40. Playing Games with Ais: The Limits of GPT-3 and Similar Large Language Models. Minds and Machines, 32(2):341–364, June 2022. ISSN 1572-8641. doi: 10.1007/s11023-022-09602-0. URL https://doi.org/10.1007/s11023-022-09602-0.
  41. Accelerating science with human-aware artificial intelligence. Nature Human Behaviour, 7(10):1682–1696, 2023.
  42. Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents, June 2023. URL http://arxiv.org/abs/2306.03314. arXiv:2306.03314 [cs].
  43. Plurality: The future of collaborative technology and democracy, 2023. URL https://github.com/pluralitybook/plurality/blob/main/contents/english.
  44. Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms, October 2023. URL http://arxiv.org/abs/2310.05984. arXiv:2310.05984 [cs].
  45. Walsh, V. Rationality as self-interest versus rationality as present aims. The American Economic Review, 84(2):401–405, 1994. ISSN 00028282. URL http://www.jstor.org/stable/2117867.
  46. A Survey on Large Language Model based Autonomous Agents, September 2023a. URL http://arxiv.org/abs/2308.11432. arXiv:2308.11432 [cs].
  47. Primacy Effect of ChatGPT, October 2023b. URL http://arxiv.org/abs/2310.13206. arXiv:2310.13206 [cs].
  48. Unleashing Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration, July 2023c. URL http://arxiv.org/abs/2307.05300. arXiv:2307.05300 [cs].
  49. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837, 2022.
  50. AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation, October 2023. URL http://arxiv.org/abs/2308.08155. arXiv:2308.08155 [cs].
  51. Designing digital voting systems for citizens: Achieving fairness and legitimacy in digital participatory budgeting voting mechanism, 2023. Submitted to ACM Digital Government: Research and Practice.
  52. LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples, 2023.
  53. ProAgent: Building Proactive Cooperative AI with Large Language Models, 2023a.
  54. Building Cooperative Embodied Agents Modularly with Large Language Models, 2023b.
  55. Mindstorms in Natural Language-Based Societies of Mind, May 2023. URL http://arxiv.org/abs/2305.17066. arXiv:2305.17066 [cs].
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Joshua C. Yang (5 papers)
  2. Marcin Korecki (5 papers)
  3. Damian Dailisan (6 papers)
  4. Carina I. Hausladen (5 papers)
  5. Dirk Helbing (80 papers)
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now