Simple dynamic word embeddings for mapping perceptions in the public sphere

Abstract: Word embeddings trained on large-scale historical corpora can illuminate human biases and stereotypes that perpetuate social inequalities. These embeddings are often trained in separate vector space models defined according to different attributes of interest. In this paper, we develop a unified dynamic embedding model that learns attribute-specific word embeddings. We apply our model to investigate i) 20th century gender and ethnic occupation biases embedded in the Corpus of Historical American English (COHA), and ii) biases against refugees embedded in a novel corpus of talk radio transcripts containing 119 million words produced over one month across 83 stations and 64 cities. Our results shed preliminary light on scenarios when dynamic embedding models may be more suitable for representing linguistic biases than individual vector space models, and vice-versa.