Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Code Switching Language Model Using Monolingual Training Data (2012.12543v2)

Published 23 Dec 2020 in cs.CL, cs.SD, and eess.AS

Abstract: Training a code-switching (CS) LLM using only monolingual data is still an ongoing research problem. In this paper, a CS LLM is trained using only monolingual training data. As recurrent neural network (RNN) models are best suited for predicting sequential data. In this work, an RNN LLM is trained using alternate batches from only monolingual English and Spanish data and the perplexity of the LLM is computed. From the results, it is concluded that using alternate batches of monolingual data in training reduced the perplexity of a CS LLM. The results were consistently improved using mean square error (MSE) in the output embeddings of RNN based LLM. By combining both methods, perplexity is reduced from 299.63 to 80.38. The proposed methods were comparable to the LLM fine tune with code-switch training data.

Citations (1)

Summary

We haven't generated a summary for this paper yet.