Learning dynamic word embeddings with drift regularisation

Abstract: Word usage, meaning and connotation change throughout time. Diachronic word embeddings are used to grasp these changes in an unsupervised way. In this paper, we use variants of the Dynamic Bernoulli Embeddings model to learn dynamic word embeddings, in order to identify notable properties of the model. The comparison is made on the New York Times Annotated Corpus in English and a set of articles from the French newspaper Le Monde covering the same period. This allows us to define a pipeline to analyse the evolution of words use across two languages.