MEGA is the first comprehensive benchmarking of generative Large Language Models (LLMs) that evaluates their performance on 8 diverse tasks and 33 typologically diverse languages.
The study compares generative LLMs to state-of-the-art non-autoregressive models and presents a framework for evaluating generative LLMs in the multilingual setting.
Key terms:
Generative AI: Artificial intelligence models capable of generating content such as text, images, or music
MEGA: A comprehensive benchmarking for evaluating generative LLMs on diverse tasks and languages
Non-autoregressive models: AI models that do not generate output sequentially but instead generate all elements simultaneously
Multilingual setting: Evaluating AI models on tasks and languages other than English