Papers
Topics
Authors
Recent
Search
2000 character limit reached

The Inner Sentiments of a Thought

Published 4 Jul 2023 in cs.CL and cs.AI | (2307.01784v1)

Abstract: Transformer-based large-scale LLMs are able to generate highly realistic text. They are duly able to express, and at least implicitly represent, a wide range of sentiments and color, from the obvious, such as valence and arousal to the subtle, such as determination and admiration. We provide a first exploration of these representations and how they can be used for understanding the inner sentimental workings of single sentences. We train predictors of the quantiles of the distributions of final sentiments of sentences from the hidden representations of an LLM applied to prefixes of increasing lengths. After showing that predictors of distributions of valence, determination, admiration, anxiety and annoyance are well calibrated, we provide examples of using these predictors for analyzing sentences, illustrating, for instance, how even ordinary conjunctions (e.g., "but") can dramatically alter the emotional trajectory of an utterance. We then show how to exploit the distributional predictions to generate sentences with sentiments in the tails of distributions. We discuss the implications of our results for the inner workings of thoughts, for instance for psychiatric dysfunction.

Authors (2)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. The psychological functions of function words. \JournalTitleSocial communication 1, 343–359 (2007).
  2. Vaswani, A. et al. Attention is all you need. \JournalTitleAdvances in neural information processing systems 30 (2017).
  3. Bert: Pre-training of deep bidirectional transformers for language understanding. \JournalTitlearXiv preprint arXiv:1810.04805 (2018).
  4. Radford, A. et al. Language models are unsupervised multitask learners. \JournalTitleOpenAI blog 1, 9 (2019).
  5. Mteb: Massive text embedding benchmark. \JournalTitlearXiv preprint arXiv:2210.07316 (2022).
  6. Srivastava, A. et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. \JournalTitlearXiv preprint arXiv:2206.04615 (2022).
  7. What does bert look at? an analysis of bert’s attention. \JournalTitlearXiv preprint arXiv:1906.04341 (2019).
  8. Bert rediscovers the classical nlp pipeline. \JournalTitlearXiv preprint arXiv:1905.05950 (2019).
  9. A primer in bertology: What we know about how bert works. \JournalTitleTransactions of the Association for Computational Linguistics 8, 842–866 (2021).
  10. Pavlick, E. Semantic structure in deep learning. \JournalTitleAnnual Review of Linguistics 8, 447–471 (2022).
  11. Systematicity in gpt-3’s interpretation of novel english noun compounds. \JournalTitlearXiv preprint arXiv:2210.09492 (2022).
  12. Using cognitive psychology to understand gpt-3. \JournalTitlearXiv preprint arXiv:2206.14576 (2022).
  13. Dasgupta, I. et al. Language models show human-like content effects on reasoning. \JournalTitlearXiv preprint arXiv:2207.07051 (2022).
  14. Krause, B. et al. Gedi: Generative discriminator guided sequence generation. \JournalTitlearXiv preprint arXiv:2009.06367 (2020).
  15. Holtzman, A. et al. Learning to write with cooperative discriminators. \JournalTitlearXiv preprint arXiv:1805.06087 (2018).
  16. Fudge: Controlled text generation with future discriminators. \JournalTitlearXiv preprint arXiv:2104.05218 (2021).
  17. Context-aware language modeling for goal-oriented dialogue systems. \JournalTitlearXiv preprint arXiv:2204.10198 (2022).
  18. Offline rl for natural language generation with implicit language q learning. \JournalTitlearXiv preprint arXiv:2206.11871 (2022).
  19. Text generation with efficient (soft) q-learning. \JournalTitlearXiv preprint arXiv:2106.07704 (2021).
  20. Leblond, R. et al. Machine translation decoding beyond beam search. \JournalTitlearXiv preprint arXiv:2104.05336 (2021).
  21. Stiennon, N. et al. Learning to summarize with human feedback. \JournalTitleAdvances in Neural Information Processing Systems 33, 3008–3021 (2020).
  22. Ouyang, L. et al. Training language models to follow instructions with human feedback. \JournalTitlearXiv preprint arXiv:2203.02155 (2022).
  23. Cohan, A. et al. Smhd: a large-scale resource for exploring online language usage for multiple mental health conditions. \JournalTitlearXiv preprint arXiv:1806.05258 (2018).
  24. Tweeteval: Unified benchmark and comparative evaluation for tweet classification. \JournalTitlearXiv preprint arXiv:2010.12421 (2020).
  25. Demszky, D. et al. Goemotions: A dataset of fine-grained emotions. \JournalTitlearXiv preprint arXiv:2005.00547 (2020).
  26. Koenker, R. Quantile regression. \JournalTitleCambridge University Press (2015).
  27. Distributional reinforcement learning with quantile regression. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018).
  28. Distributional reinforcement learning (MIT Press, 2023).
  29. Shaping the narrative arc: An information-theoretic approach to collaborative dialogue. \JournalTitlearXiv preprint arXiv:1901.11528 (2019).
  30. Wikipedia contributors. Intensifier. \JournalTitleWikipedia, Wikimedia Foundation (2019).
  31. Reinforcement learning: An introduction (MIT press, 2018).
  32. Hanu, L. Unitary team. detoxify. github (2020).
  33. Ctrl: A conditional transformer language model for controllable generation. \JournalTitlearXiv preprint arXiv:1909.05858 (2019).
  34. Korbak, T. et al. Pretraining language models with human preferences. \JournalTitlearXiv preprint arXiv:2302.08582 (2023).
  35. Lu, X. et al. Quark: Controllable text generation with reinforced unlearning. \JournalTitlearXiv preprint arXiv:2205.13636 (2022).
  36. Reducing conversational agents’ overconfidence through linguistic calibration. \JournalTitleTransactions of the Association for Computational Linguistics 10, 857–872 (2022).
  37. Card, D. et al. Computational analysis of 140 years of us political speeches reveals more positive but increasingly polarized framing of immigration. \JournalTitleProceedings of the National Academy of Sciences 119, e2120510119 (2022).
  38. Miner, A. S. et al. A computational approach to measure the linguistic characteristics of psychotherapy timing, responsiveness, and consistency. \JournalTitlenpj Mental Health Research 1, 19 (2022).
  39. A unified approach to interpreting model predictions. \JournalTitleAdvances in neural information processing systems 30 (2017).
  40. Shapley, L. S. A value for n𝑛nitalic_n-person games. \JournalTitleContributions to the Theory of Games 2, 307–317 (1953).
  41. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 1135–1144 (2016).
  42. Beck, A. T. Cognitive therapy and the emotional disorders (Penguin, 1979).
  43. In an absolute state: Elevated use of absolutist words is a marker specific to anxiety, depression, and suicidal ideation. \JournalTitleClinical Psychological Science 6, 529–542 (2018).
  44. Peril, prudence and planning as risk, avoidance and worry. \JournalTitleJournal of Mathematical Psychology 106, 102617 (2022).
  45. Catastrophic worrying: personal inadequacy and a perseverative iterative style as features of the catastrophizing process. \JournalTitleJournal of Abnormal Psychology 107, 576 (1998).
  46. Hoffmann, J. et al. Training compute-optimal large language models. \JournalTitlearXiv preprint arXiv:2203.15556 (2022).
  47. Thoppilan, R. et al. Lamda: Language models for dialog applications. \JournalTitlearXiv preprint arXiv:2201.08239 (2022).
  48. Social-iq: A question answering benchmark for artificial social intelligence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8807–8817 (2019).
  49. Inducing positive perspectives with text reframing. \JournalTitlearXiv preprint arXiv:2204.02952 (2022).
  50. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing (2017). To appear.
  51. Russell, J. A. A circumplex model of affect. \JournalTitleJournal of personality and social psychology 39, 1161 (1980).
  52. Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. \JournalTitleJournal of personality and social psychology 76, 805 (1999).
  53. Russell, J. A. Core affect and the psychological construction of emotion. \JournalTitlePsychological review 110, 145 (2003).
  54. Barrett, L. F. Feelings or words? understanding the content in self-report ratings of experienced emotion. \JournalTitleJournal of personality and social psychology 87, 266 (2004).
  55. Adam: A method for stochastic optimization. \JournalTitlearXiv preprint arXiv:1412.6980 (2014).
  56. The curious case of neural text degeneration. \JournalTitlearXiv preprint arXiv:1904.09751 (2019).
  57. Xu, J. et al. Learning to break the loop: Analyzing and mitigating repetitions for neural text generation. \JournalTitlearXiv preprint arXiv:2206.02369 (2022).
  58. Mohammad, S. Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 english words. In Proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: Long papers), 174–184 (2018).
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up