Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Large Language Models Can Be Used to Estimate the Latent Positions of Politicians (2303.12057v4)

Published 21 Mar 2023 in cs.CY and cs.CL

Abstract: Existing approaches to estimating politicians' latent positions along specific dimensions often fail when relevant data is limited. We leverage the embedded knowledge in generative LLMs to address this challenge and measure lawmakers' positions along specific political or policy dimensions. We prompt an instruction/dialogue-tuned LLM to pairwise compare lawmakers and then scale the resulting graph using the Bradley-Terry model. We estimate novel measures of U.S. senators' positions on liberal-conservative ideology, gun control, and abortion. Our liberal-conservative scale, used to validate LLM-driven scaling, strongly correlates with existing measures and offsets interpretive gaps, suggesting LLMs synthesize relevant data from internet and digitized media rather than memorizing existing measures. Our gun control and abortion measures -- the first of their kind -- differ from the liberal-conservative scale in face-valid ways and predict interest group ratings and legislator votes better than ideology alone. Our findings suggest LLMs hold promise for solving complex social science measurement problems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. Stephen Ansolabehere, James M. Snyder and Charles Stewart “Candidate Positioning in U.S. House Elections” In American Journal of Political Science 45.1 [Midwest Political Science Association, Wiley], 2001, pp. 136–159 URL: http://www.jstor.org/stable/2669364
  2. Elisabeth R. Gerber and Jeffrey B. Lewis “Beyond the Median: Voter Preferences, District Heterogeneity, and Political Representation” In Journal of Political Economy 112.6 The University of Chicago Press, 2004, pp. 1364–1383 URL: http://www.jstor.org/stable/10.1086/424737
  3. Larry M. Bartels “Economic Inequality and Political Representation” In The Unsustainable American State Oxford University Press, 2009 DOI: 10.1093/acprof:oso/9780195392135.003.0007
  4. Danielle M. Thomsen “Opting Out of Congress: Partisan Polarization and the Decline of Moderate Candidates” Cambridge University Press, 2017 DOI: 10.1017/9781316872055
  5. “Policy Preferences and Policy Change: Dynamic Responsiveness in the American States, 1936–2014” In American Political Science Review 112.2 Cambridge University Press, 2018, pp. 249–266 DOI: 10.1017/S0003055417000533
  6. Keith T. Poole and Howard Rosenthal “Ideology and Congress: A Political Economic History of Roll Call Voting” New Haven, CT: Yale University Press, 1997
  7. Michele L. Swers “Are Women More Likely to Vote for Women’s Issue Bills than Their Male Colleagues?” In Legislative Studies Quarterly 23.3 [Wiley, Comparative Legislative Research Center], 1998, pp. 435–448 URL: http://www.jstor.org/stable/440362
  8. Keith Krehbiel “Pivotal Politics: A Theory of U.S. Lawmaking” Chicago: University of Chicago Press, 1998 DOI: doi:10.7208/9780226452739
  9. Joshua Clinton, Simon Jackman and Douglas Rivers “The Statistical Analysis of Roll Call Data” In American Political Science Review 98.2 Cambridge University Press, 2004, pp. 355–370 DOI: 10.1017/S0003055404001194
  10. Gary W. Cox and Mathew D. McCubbins “Setting the Agenda: Responsible Party Government in the U.S. House of Representatives” Cambridge University Press, 2005 DOI: 10.1017/CBO9780511791123
  11. Benjamin Highton and Michael S. Rocca “Beyond the Roll-Call Arena: The Determinants of Position Taking in Congress” In Political Research Quarterly 58.2, 2005, pp. 303–316 DOI: 10.1177/106591290505800210
  12. Cheryl Boudreau, Christopher S. Elmendorf and Scott A. MacKenzie “Racial or Spatial Voting? The Effects of Candidate Ethnicity and Ethnic Group Endorsements in Local Elections” In American Journal of Political Science 63.1, 2019, pp. 5–20 DOI: https://doi.org/10.1111/ajps.12401
  13. Annelise Russell “Minority Opposition and Asymmetric Parties? Senators’ Partisan Rhetoric on Twitter” In Political Research Quarterly 74.3, 2021, pp. 615–627 DOI: 10.1177/1065912920921239
  14. Keith T. Poole and Howard Rosenthal “A Spatial Model for Legislative Roll Call Analysis” In American Journal of Political Science 29.2 [Midwest Political Science Association, Wiley], 1985, pp. 357–384 URL: http://www.jstor.org/stable/2111172
  15. Keith T. Poole “Spatial Models of Parliamentary Voting”, Analytical Methods for Social Research Cambridge University Press, 2005 DOI: 10.1017/CBO9780511614644
  16. “Measuring Bias and Uncertainty in DW-NOMINATE Ideal Point Estimates via the Parametric Bootstrap” In Political Analysis 17.3 [Cambridge University Press, Oxford University Press, Society for Political Methodology], 2009, pp. 261–275 URL: http://www.jstor.org/stable/25791974
  17. “News Sharing on Social Media: Mapping the Ideology of News Media Content, Citizens, and Politicians” OSF Preprints, 2020 DOI: 10.31219/osf.io/ch8gj
  18. Adam Bonica “Mapping the Ideological Marketplace” In American Journal of Political Science 58.2 [Midwest Political Science Association, Wiley], 2014, pp. 367–386 URL: http://www.jstor.org/stable/24363491
  19. “Training language models to follow instructions with human feedback” In Advances in Neural Information Processing Systems 35, 2022, pp. 27730–27744
  20. Ralph Allan Bradley and Milton E. Terry “Rank Analysis of Incomplete Block Designs: The Method of Paired Comparisons” In Biometrika 39.3-4, 1952, pp. 324–345 DOI: 10.1093/biomet/39.3-4.324
  21. Daniel J. Hopkins and Hans Noel “Trump and the Shifting Meaning of “Conservative”: Using Activists’ Pairwise Comparisons to Measure Politicians’ Perceived Ideologies” In American Political Science Review 116.3 Cambridge University Press, 2022, pp. 1133–1140 DOI: 10.1017/S0003055421001416
  22. Nolan McCarty “Measuring Legislative Preferences” Publisher Copyright: © The several contributors 2011. All rights reserved. In The Oxford Handbook of the American Congress United Kingdom: Oxford University Press, 2011 DOI: 10.1093/oxfordhb/9780199559947.003.0004
  23. Peter John Loewen, Daniel Rubenson and Arthur Spirling “Testing the power of arguments in referendums: A Bradley–Terry approach” Special Symposium: Germany’s Federal Election September 2009 In Electoral Studies 31.1, 2012, pp. 212–221 DOI: https://doi.org/10.1016/j.electstud.2011.07.003
  24. David Carlson and Jacob M. Montgomery “A Pairwise Comparison Framework for Fast, Flexible, and Reliable Human Coding of Political Texts” In American Political Science Review 111.4 Cambridge University Press, 2017, pp. 835–843 DOI: 10.1017/S0003055417000302
  25. “Substance and Change in Congressional Ideology: NOMINATE and Its Alternatives” In Studies in American Political Development 30.2 Cambridge University Press, 2016, pp. 128–146 DOI: 10.1017/S0898588X16000092
  26. “Ends Against the Middle: Measuring Latent Traits when Opposites Respond the Same Way for Antithetical Reasons” In Political Analysis Cambridge University Press, 2023, pp. 1–20 DOI: 10.1017/pan.2022.33
  27. Adam Bonica “Database on Ideology, Money in Politics, and Elections”, Available at https://data.stanford.edu/dime, 2016
  28. Seongho Kim “ppcor: An R Package for a Fast Calculation to Semi-partial Correlation Coefficients” In Communications for Statistical Applications and Methods 22.6 The Korean Statistical Society,Korean International Statistical Society, 2015, pp. 665–674 DOI: 10.5351/CSAM.2015.22.6.665
  29. Petter Törnberg “ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning”, 2023 arXiv:2304.06588 [cs.CL]
  30. “GPT is an effective tool for multilingual psychological text analysis” PsyArXiv, 2023 DOI: 10.31234/osf.io/sekf5
  31. “AI Chat Assistants can Improve Conversations about Divisive Topics”, 2023 arXiv:2302.07268 [cs.HC]
  32. “Synthetic Replacements for Human Survey Data? The Perils of Large Language Models” SocArXiv, 2023 DOI: 10.31235/osf.io/5ecfa
  33. James J. Heckman and James M. Snyder “Linear Probability Models of the Demand for Attributes with an Empirical Application to Estimating the Preferences of Legislators” In The RAND Journal of Economics 28 [RAND Corporation, Wiley], 1997, pp. S142–S189 URL: http://www.jstor.org/stable/3087459
  34. Andrew D. Martin and Kevin M. Quinn “Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the U.S. Supreme Court, 1953–1999” In Political Analysis 10.2 [Oxford University Press, Society for Political Methodology], 2002, pp. 134–153 URL: http://www.jstor.org/stable/25791672
  35. Jonathan B. Slapin and Sven-Oliver Proksch “A Scaling Model for Estimating Time-Series Party Positions from Texts” In American Journal of Political Science 52.3, 2008, pp. 705–722 DOI: https://doi.org/10.1111/j.1540-5907.2008.00338.x
  36. “The Ideological Mapping of American Legislatures” In The American Political Science Review 105.3 [American Political Science Association, Cambridge University Press], 2011, pp. 530–551 URL: http://www.jstor.org/stable/41480856
  37. “Validating Estimates of Latent Traits from Textual Data Using Human Judgment as a Benchmark” In Political Analysis 21.3 Cambridge University Press, 2013, pp. 298–313 DOI: 10.1093/pan/mpt002
  38. Pablo Barberá “Birds of the Same Feather Tweet Together: Bayesian Ideal Point Estimation Using Twitter Data” In Political Analysis 23.1 Cambridge University Press, 2015, pp. 76–91 DOI: 10.1093/pan/mpu011
  39. “Ideological Scaling of Social Media Users: A Dynamic Lexicon Approach” In Political Analysis 26.4 Cambridge University Press, 2018, pp. 457–473 DOI: 10.1017/pan.2018.30
  40. “Partisan Associations of Twitter Users Based on Their Self-Descriptions and Word Embeddings” Presented at APSA 2019, 2019
  41. “Word Embeddings for the Analysis of Ideological Placement in Parliamentary Corpora” In Political Analysis 28.1 Cambridge University Press, 2020, pp. 112–133 DOI: 10.1017/pan.2019.26
  42. “Language Models are Few-Shot Learners” arXiv, 2020 DOI: 10.48550/ARXIV.2005.14165
  43. Jay Alammar “The Illustrated GPT-2 (Visualizing Transformer Language Models)” Retrieved from https://jalammar.github.io/illustrated-gpt2/, 2019
  44. “Attention Is All You Need” In Advances in Neural Information Processing Systems 30 Curran Associates, Inc., 2017 URL: https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
  45. “Fine-Tuning Language Models from Human Preferences” arXiv, 2019 DOI: 10.48550/ARXIV.1909.08593
  46. “Proximal Policy Optimization Algorithms” arXiv, 2017 DOI: 10.48550/ARXIV.1707.06347
  47. “Illustrating Reinforcement Learning from Human Feedback (RLHF)” https://huggingface.co/blog/rlhf In Hugging Face Blog, 2022
  48. “Gender and Representation Bias in GPT-3 Generated Stories” In Proceedings of the Third Workshop on Narrative Understanding Virtual: Association for Computational Linguistics, 2021, pp. 48–55 DOI: 10.18653/v1/2021.nuse-1.5
  49. “Out of One, Many: Using Language Models to Simulate Human Samples” In Political Analysis Cambridge University Press, 2023, pp. 1–15 DOI: 10.1017/pan.2023.2
  50. “Voteview: Congressional Roll-Call Votes Database” https://voteview.com/, 2021
  51. “Bradley-Terry Models in R: The BradleyTerry2 Package” In Journal of Statistical Software 48.9, 2012, pp. 1–21 DOI: 10.18637/jss.v048.i09
  52. Lauren R. Johnson, Deon McCray and Jordan M. Ragusa “#NeverTrump: Why Republican members of Congress refused to support their party’s nominee in the 2016 presidential election” In Research & Politics 5.1, 2018 DOI: 10.1177/2053168017749383
Citations (17)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets