Papers
Topics
Authors
Recent
2000 character limit reached

Underspecification in Language Modeling Tasks: A Causality-Informed Study of Gendered Pronoun Resolution (2210.00131v4)

Published 30 Sep 2022 in cs.CL and cs.AI

Abstract: Modern language modeling tasks are often underspecified: for a given token prediction, many words may satisfy the user's intent of producing natural language at inference time, however only one word will minimize the task's loss function at training time. We introduce a simple causal mechanism to describe the role underspecification plays in the generation of spurious correlations. Despite its simplicity, our causal model directly informs the development of two lightweight black-box evaluation methods, that we apply to gendered pronoun resolution tasks on a wide range of LLMs to 1) aid in the detection of inference-time task underspecification by exploiting 2) previously unreported gender vs. time and gender vs. location spurious correlations on LLMs with a range of A) sizes: from BERT-base to GPT-4 Turbo Preview, B) pre-training objectives: from masked & autoregressive language modeling to a mixture of these objectives, and C) training stages: from pre-training only to reinforcement learning from human feedback (RLHF). Code and open-source demos available at https://github.com/2dot71mily/uspec.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. Invariant Risk Minimization.
  2. Controlling Selection Bias in Causal Inference. In Lawrence, N. D.; and Girolami, M., eds., Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, volume 22 of Proceedings of Machine Learning Research, 100–108. La Palma, Canary Islands: PMLR.
  3. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences, 113(27): 7345–7352.
  4. Recovering Causal Effects from Selection Bias. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1).
  5. Recovering from Selection Bias in Causal and Statistical Inference. Proceedings of the AAAI Conference on Artificial Intelligence, 28(1).
  6. Recognition in Terra Incognita.
  7. Language Models are Few-Shot Learners. In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., Advances in Neural Information Processing Systems, volume 33, 1877–1901. Curran Associates, Inc.
  8. Toward Gender-Inclusive Coreference Resolution. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 4568–4595. Online: Association for Computational Linguistics.
  9. Scaling Instruction-Finetuned Language Models.
  10. Illustrating bias due to conditioning on a collider. International Journal of Epidemiology, 39(2): 417–420.
  11. Underspecification Presents Challenges for Credibility in Modern Machine Learning. Journal of Machine Learning Research, 23(226): 1–61.
  12. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. CoRR, abs/1810.04805.
  13. To Adjust or Not to Adjust? Sensitivity Analysis of M-Bias and Butterfly-Bias. Journal of Causal Inference, 3(1): 41–57.
  14. Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond. Transactions of the Association for Computational Linguistics, 10: 1138–1158.
  15. Shortcut Learning in Deep Neural Networks. CoRR, abs/2004.07780.
  16. Collider bias undermines our understanding of COVID-19 disease risk and severity. Nature Communications, 11(1).
  17. Heckman, J. J. 1979. Sample Selection Bias as a Specification Error. Econometrica, 47(1): 153–161.
  18. Hernán, M. A. 2017. Invited Commentary: Selection Bias Without Colliders. American Journal of Epidemiology, 185(11): 1048–1050.
  19. Training Compute-Optimal Large Language Models. arXiv:2203.15556.
  20. Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lens. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 9804–9817. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics.
  21. Diversify and Disambiguate: Learning From Underspecified Data.
  22. TSNLP - Test Suites for Natural Language Processing. In COLING 1996 Volume 2: The 16th International Conference on Computational Linguistics.
  23. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 7871–7880. Online: Association for Computational Linguistics.
  24. RoBERTa: A Robustly Optimized BERT Pretraining Approach. CoRR, abs/1907.11692.
  25. Understanding Stereotypes in Language Models: Towards Robust Measurement and Zero-Shot Debiasing. arXiv preprint arXiv:2212.10678.
  26. Collider scope: when selection bias can substantially influence observed associations. Int. J. Epidemiol., 47(1): 226–235.
  27. OpenAI. 2023a. Model - OpenAI Documentation. https://platform.openai.com/docs/models. (Accessed on 04/17/2024).
  28. OpenAI. 2023b. Model index for researchers - OpenAI API. https://archive.ph/IXxGm. (Accessed on 03/07/2023).
  29. Training language models to follow instructions with human feedback.
  30. Encoding Weights of Irregular Sparsity for Fixed-to-Fixed Model Compression. In International Conference on Learning Representations.
  31. Reducing Gender Bias in Abusive Language Detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2799–2804. Brussels, Belgium: Association for Computational Linguistics.
  32. Pearl, J. 2009. Causality. Cambridge, UK: Cambridge University Press, 2 edition. ISBN 978-0-521-89560-6.
  33. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. J. Mach. Learn. Res., 21(1).
  34. Rudinger, R. 2019. winogender-schemas. https://github.com/rudinger/winogender-schemas. (Accessed on 08/15/2023).
  35. Gender Bias in Coreference Resolution. CoRR, abs/1804.09301.
  36. Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization.
  37. UL2: Unifying Language Learning Paradigms. In The Eleventh International Conference on Learning Representations.
  38. Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests.
  39. Investigating Gender Bias in Language Models Using Causal Mediation Analysis. In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., Advances in Neural Information Processing Systems, volume 33, 12388–12401. Curran Associates, Inc.
  40. Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns. Transactions of the Association for Computational Linguistics, 6: 605–617.
  41. Measuring and Reducing Gendered Correlations in Pre-trained Models.
  42. WEF. 2021. World Economic Forum Global Gender Gap Report. https://www3.weforum.org/docs/WEF˙GGGR˙2021.pdf. (Accessed on 01/10/2022).
  43. A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models. arXiv:2303.10420.
  44. Causal Parrots: Large Language Models May Talk Causality But Are Not Causal. Transactions on Machine Learning Research.
  45. PAWS: Paraphrase Adversaries from Word Scrambling.
  46. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), 15–20. New Orleans, Louisiana: Association for Computational Linguistics.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now

Whiteboard

Paper to Video (Beta)

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Authors (1)

  1. Emily McMilin

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 3 tweets with 4 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free