2000 character limit reached
Code-Switching Language Modeling using Syntax-Aware Multi-Task Learning (1805.12070v2)
Published 30 May 2018 in cs.CL
Abstract: Lack of text data has been the major issue on code-switching LLMing. In this paper, we introduce multi-task learning based LLM which shares syntax representation of languages to leverage linguistic information and tackle the low resource data issue. Our model jointly learns both LLMing and Part-of-Speech tagging on code-switched utterances. In this way, the model is able to identify the location of code-switching points and improves the prediction of next word. Our approach outperforms standard LSTM based LLM, with an improvement of 9.7% and 7.4% in perplexity on SEAME Phase I and Phase II dataset respectively.