Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Survey in Characterization of Semantic Change (2402.19088v3)

Published 29 Feb 2024 in cs.CL and cs.AI

Abstract: Live languages continuously evolve to integrate the cultural change of human societies. This evolution manifests through neologisms (new words) or \textbf{semantic changes} of words (new meaning to existing words). Understanding the meaning of words is vital for interpreting texts coming from different cultures (regionalism or slang), domains (e.g., technical terms), or periods. In computer science, these words are relevant to computational linguistics algorithms such as translation, information retrieval, question answering, etc. Semantic changes can potentially impact the quality of the outcomes of these algorithms. Therefore, it is important to understand and characterize these changes formally. The study of this impact is a recent problem that has attracted the attention of the computational linguistics community. Several approaches propose methods to detect semantic changes with good precision, but more effort is needed to characterize how the meaning of words changes and to reason about how to reduce the impact of semantic change. This survey provides an understandable overview of existing approaches to the \textit{characterization of semantic changes} and also formally defines three classes of characterizations: if the meaning of a word becomes more general or narrow (change in dimension) if the word is used in a more pejorative or positive/ameliorated sense (change in orientation), and if there is a trend to use the word in a, for instance, metaphoric or metonymic context (change in relation). We summarized the main aspects of the selected publications in a table and discussed the needs and trends in the research activities on semantic change characterization.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (116)
  1. Allan, K. 2013. The Oxford handbook of the history of linguistics. OUP Oxford.
  2. Current methods in historical semantics. De Gruyter Mouton Berlin.
  3. Bad seeds: Evaluating lexical methods for bias measurement. In Annual Meeting of the Association for Computational Linguistics.
  4. Apidianaki, M. 2022. From word types to tokens and back: A survey of approaches to word meaning representation and interpretation. Computational Linguistics, 49:465–523.
  5. Ardley, G. 1968. God and the soul. Philosophical Studies, 18:325–326.
  6. Structured sentiment analysis as dependency graph parsing. ArXiv, abs/2105.14504.
  7. Biemann, C. 2006. Chinese whispers - an efficient graph clustering algorithm and its application to natural language processing problems. In Proceedings of TextGraphs: the First Workshop on Graph Based Methods for Natural Language Processing, pp. 73–80, New York City. Association for Computational Linguistics.
  8. Polysemy in the lexicon and in discourse. Polysemy: Flexible patterns of meaning in mind and language.
  9. Neural network algorithm for detection of new word meanings denoting named entities. IEEE Access, PP:1–1.
  10. Bowern, C. 2019. Semantic change and semantic stability: Variation is key. In LChange@ACL.
  11. Brezina, V. 2018. Statistics in Corpus Linguistics: A Practical Guide. Cambridge University Press.
  12. Broad, C. D. 1938. Examination of McTaggart’s Philosophy, volume Vol. II. Cambridge University Press.
  13. Feelings from the past—adapting affective lexicons for historical emotion analysis. In LT4DH@COLING.
  14. Burke, K. 1969. A grammar of motives, volume 177. Univ of California Press.
  15. From word to sense embeddings: A survey on vector representations of meaning. J. Artif. Intell. Res., 63:743–788.
  16. Sensespotting: Never let your parallel data tie you to an old domain. In Annual Meeting of the Association for Computational Linguistics.
  17. From n-gram to skipgram to concgram. International journal of corpus linguistics, 11(4):411–433.
  18. MelBERT: Metaphor detection via contextualized late interaction using metaphorical identification theories. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1763–1773, Online. Association for Computational Linguistics.
  19. A lexicographic appraisal of an automatic approach for detecting new word-senses. In Proceedings of eLex.
  20. Automatically identifying changes in the semantic orientation of words. In International Conference on Language Resources and Evaluation.
  21. A survey of the state of explainable ai for natural language processing. In AACL.
  22. Bert: Pre-training of deep bidirectional transformers for language understanding. ArXiv, abs/1810.04805.
  23. Outta control: Laws of semantic change and inherent biases in word representation models. In Conference on Empirical Methods in Natural Language Processing.
  24. Sense tree: Discovery of new word senses with graph-based scoring. In Lernen, Wissen, Daten, Analysen.
  25. Dynamic generative model for diachronic sense emergence detection. In International Conference on Computational Linguistics.
  26. Ethayarajh, K. 2019. How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings. In Conference on Empirical Methods in Natural Language Processing.
  27. Adjusting scope: A computational approach to case-driven research on semantic change. In Workshop on Computational Humanities Research.
  28. A survey on concept drift adaptation. ACM Computing Surveys (CSUR), 46:1 – 37.
  29. Simcse: Simple contrastive learning of sentence embeddings. ArXiv, abs/2104.08821.
  30. Interpretable word sense representations via definition generation: The case of semantic change analysis. In Annual Meeting of the Association for Computational Linguistics.
  31. Intrinsic evaluations of word embeddings: What can we do better? In Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval@ACL 2016, Berlin, Germany, August 2016, pp. 36–42. Association for Computational Linguistics.
  32. Vector-space models of semantic representation from a cognitive perspective: A discussion of common misconceptions. Perspectives on Psychological Science, 14:1006 – 1033.
  33. An overview on fine-grained text sentiment analysis: Survey and challenges. Journal of Physics: Conference Series, 1757.
  34. Diachronic word embeddings reveal statistical laws of semantic change. ArXiv, abs/1605.09096.
  35. On computational historical linguistics in the 21st century. Theoretical Linguistics, 45:233 – 245.
  36. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. ArXiv, abs/2111.09543.
  37. Automatic extraction of linguistic metaphors with lda topic modeling. In Proceedings of the First Workshop on Metaphor in NLP, pp. 58–66.
  38. Bad company - neighborhoods in neural embedding spaces considered harmful. In Calzolari, N., Matsumoto, Y., and Prasad, R., editors, COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, December 11-16, 2016, Osaka, Japan, pp. 2785–2796. ACL.
  39. Challenges for computational lexical semantic change. ArXiv, abs/2101.07668.
  40. Chapter 7: Semantic change. De Gruyter Mouton, Berlin, Boston.
  41. Identifying metaphorical word use with tree kernels. In Proceedings of the First Workshop on Metaphor in NLP.
  42. Diachronic sense modeling with deep contextualized word embeddings: An ecological view. In Annual Meeting of the Association for Computational Linguistics.
  43. Word-level contextual sentiment analysis with interpretability. In AAAI Conference on Artificial Intelligence.
  44. A framework for analyzing semantic change of words across time. IEEE/ACM Joint Conference on Digital Libraries, pp. 229–238.
  45. The lexical typology of semantic shifts, volume 58. Walter de Gruyter GmbH & Co KG.
  46. Koch, P. 2016. Meaning change and semantic shifts. The lexical typology of semantic shifts, 58:21–66.
  47. Diachronic word embeddings and semantic shifts: a survey. In International Conference on Computational Linguistics.
  48. Contextualized embeddings for semantic change detection: Lessons learned. ArXiv, abs/2209.00154.
  49. Metaphors we live by. University of Chicago press.
  50. Learning word sense distributions, detecting unattested senses and identifying novel senses using topic models. In Annual Meeting of the Association for Computational Linguistics.
  51. Word sense induction for novel sense detection. In Conference of the European Chapter of the Association for Computational Linguistics.
  52. Understanding the downstream instability of word embeddings. CoRR, abs/2003.04983.
  53. Metonymy and text messaging: A framework for understanding creative uses of metonymy. Applied Linguistics, 39(4):481–507.
  54. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. ArXiv, abs/2205.05638.
  55. Topical word embeddings. In AAAI Conference on Artificial Intelligence.
  56. Graph-based dynamic word embeddings. In International Joint Conference on Artificial Intelligence.
  57. Adapting predominant and novel sense discovery algorithms for identifying corpus-specific sense differences. In Proceedings of TextGraphs-11: the Workshop on Graph-based Methods for Natural Language Processing, pp. 11–20, Vancouver, Canada. Association for Computational Linguistics.
  58. Impact of target word and context on end-to-end metonymy detection. ArXiv, abs/2112.03256.
  59. Metaphorical polysemy detection: Conventional metaphor meets word sense disambiguation. ArXiv, abs/2212.08395.
  60. On the downstream performance of compressed word embeddings. CoRR, abs/1909.01264.
  61. Finding predominant word senses in untagged text. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), pp. 279–286, Barcelona, Spain.
  62. McWhorter, J. 2011. The power of Babel: A natural history of language. Random House.
  63. Merriam-Webster 2023. ’sick,’ entry 6. https://www.merriam-webster.com/dictionary/sick. Accessed: 2023-09-15.
  64. What do you mean, BERT? In Proceedings of the Society for Computation in Linguistics 2020, pp. 279–290, New York, New York. Association for Computational Linguistics.
  65. Efficient estimation of word representations in vector space. In International Conference on Learning Representations.
  66. Rethinking the role of demonstrations: What makes in-context learning work? In Conference on Empirical Methods in Natural Language Processing.
  67. Composition in distributional models of semantics. Cognitive science, 34 8:1388–429.
  68. An automatic approach to identify word sense changes in text media across timescales. Natural Language Engineering, 21:773 – 798.
  69. Mohammed, E. T. 2009. Polysemy as a lexical problem in translation. Online Submission, 55:1–19.
  70. A survey on contextualised semantic shift detection. ArXiv, abs/2304.01666.
  71. Moss, A. 2020. Detecting lexical semantic change using probabilistic gaussian word embeddings.
  72. Local and global context for supervised and unsupervised metonymy resolution. In Conference on Empirical Methods in Natural Language Processing.
  73. Efficient non-parametric estimation of multiple embeddings per word in vector space. ArXiv, abs/1504.06654.
  74. Rapid adaptation of neural machine translation to new languages. ArXiv, abs/1808.04189.
  75. The Oxford handbook of the history of English. Oxford University Press.
  76. Universal dependencies v2: An evergrowing multilingual treebank collection. In International Conference on Language Resources and Evaluation.
  77. Don’t invite BERT to drink a bottle: Modeling the interpretation of metonymies using BERT and distributional representations. In Proceedings of the 28th International Conference on Computational Linguistics, pp. 6831–6837, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  78. Deep contextualized word representations. In North American Chapter of the Association for Computational Linguistics.
  79. Pinker, S. 2003. The language instinct: How the mind creates language. Penguin UK.
  80. Natural language and natural selection. Behavioral and Brain Sciences, 13:707 – 727.
  81. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:140:1–140:67.
  82. Sentiment analysis methods for understanding large-scale texts: a case for using continuum-scored words and word shift graphs. EPJ Data Science, 6:1–21.
  83. Sentence-bert: Sentence embeddings using siamese bert-networks. In Conference on Empirical Methods in Natural Language Processing.
  84. Continuity, divergence, and the evolution of brain language pathways. Frontiers in evolutionary neuroscience, 3:11.
  85. Deep neural models of semantic shift. In North American Chapter of the Association for Computational Linguistics.
  86. Where new words are born: Distributional semantic analysis of neologisms and their semantic neighborhoods. ArXiv, abs/2001.07740.
  87. Semantic density analysis: Comparing word meaning across time and phonetic space. In Proceedings of the Workshop on Geometrical Models of Natural Language Semantics, GEMS ’09, 104–111, USA. Association for Computational Linguistics.
  88. German in flux: Detecting metaphoric change via word entropy. In Conference on Computational Natural Language Learning.
  89. A wind of change: Detecting and evaluating lexical semantic change across times and domains. In Annual Meeting of the Association for Computational Linguistics.
  90. SemEval-2020 task 1: Unsupervised lexical semantic change detection. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pp. 1–23, Barcelona (online). International Committee for Computational Linguistics.
  91. A natural history of the human mind: tracing evolutionary changes in brain and cognition. Journal of Anatomy, 212.
  92. Room to glo: A systematic comparison of semantic change detection approaches with word embeddings. In Conference on Empirical Methods in Natural Language Processing.
  93. Unsupervised metaphor identification using hierarchical graph factorization clustering. In North American Chapter of the Association for Computational Linguistics.
  94. Matching the blanks: Distributional similarity for relation learning. In Annual Meeting of the Association for Computational Linguistics.
  95. Recursive deep models for semantic compositionality over a sentiment treebank. In Conference on Empirical Methods in Natural Language Processing.
  96. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631–1642, Seattle, Washington, USA. Association for Computational Linguistics.
  97. Survey of computational approaches to lexical semantic change. arXiv: Computation and Language.
  98. Computational approaches to semantic change. BoD–Books on Demand.
  99. Finding individual word sense changes and their delay in appearance. In Recent Advances in Natural Language Processing.
  100. Analyzing semantic change in japanese loanwords. In Conference of the European Chapter of the Association for Computational Linguistics.
  101. Tang, X. 2018. A state-of-the-art of semantic change computation. Natural Language Engineering, 24:649 – 676.
  102. Semantic change computation: A successive approach. World Wide Web, 19:375–415.
  103. Can word sense distribution detect semantic changes of words? arXiv preprint arXiv:2310.10400.
  104. Teraoka, T. 2016. Metonymy analysis using associative relations between words. In International Conference on Language Resources and Evaluation.
  105. Traugott, E. C. 2017. Semantic change.
  106. Tracing metaphors in time through self-distance in vector spaces. ArXiv, abs/1611.03279.
  107. Cross-lingual laws of semantic change. Computational approaches to semantic change, 6:219.
  108. Knowledge-enhanced temporal word embedding for diachronic semantic change estimation. Soft Computing, 24:12901 – 12918.
  109. Are all good word vector spaces isomorphic? ArXiv, abs/2004.04070.
  110. Word2fun: Modelling words as functions for diachronic word representation. In Neural Information Processing Systems.
  111. A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review, 55:5731 – 5780.
  112. Factors influencing the surprising instability of word embeddings. CoRR, abs/1804.09692.
  113. Semantic change detection with gaussian word embeddings. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:3349–3361.
  114. Frequency-based distortions in contextualized word embeddings. ArXiv, abs/2104.08465.
  115. The finer they get: Combining fine-tuned models for better semantic change detection. In Nordic Conference of Computational Linguistics.
  116. Zipf, G. K. 1949. Human behavior and the principle of least effort. Addison-Wesley Press, Inc.
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now